Latency Numbers Every Engineer Should Know

The most useful thing you can memorize about computers is not a framework API — it's a table of how long things take. Every performance problem, every scaling debate, every "should we add a cache?" question reduces to matching these numbers against your workload's shape.

Here is the table, ordered by magnitude, on 2026-era hardware:

Read the last line slowly again: one cross-continent round trip is worth about a hundred million L1 cache references. When an engineer says "just call the other service," they are spending a century of L1 time — in one round trip of real time.

The two axes of every answer

All these numbers compress into a mental model with two axes:

Distance/time spent waiting. Cache levels are roughly a decade apart (1 ns → 10 ns → 100 ns). Network hops obey RTT physics: you spend ~10 ms of true delay per thousand kilometers, plus queueing, per hop. Disk is not "slow" — it is 100,000× slower than memory, which is the only number that ever matters for sizing caches.

Amplification. A cache miss costs 1 lookup; a full table scan costs pages × misses; a cache-averse workload (random access on magnetic disk) is 10,000× slower than the same workload held comfortably in memory.

Rules that fall out of the table

1. The order of magnitude is destiny

If a critical path has 4 sequential network hops, no amount of application-side optimization matters next to removing one hop. Design for hop count on latency-critical paths the way you design for complexity class in algorithms.

2. Syscalls are cheap — until they're not

A system call is 100–500 ns alone. Modern Linux does ~500M/s simple syscalls on one core. The cost shows up when you cross the boundary per unit of a high-frequency operation: gettimeofday() in a hot loop, per-packet I/O, per-row logging. The pattern to remember is amortization: the kernel entry cost is constant, so the cost per item falls as you process more items per syscall (buffering, batching, epoll over blocking reads).

3. The cache hierarchy is your free performance policy

A data structure that lives in L2 is ~100× faster to read than one that lives in RAM. On a modern core, a B-tree node is a cache line or two; a skip list node is four pointer-chased cache lines. This is why tiny constant factors in algorithms tables are real again at this scale — and why the "neat" data structure from the algorithms course can lose to the ugly one that's cache-friendly.

The table as a design tool

Take a concrete decision: should the frontend call the backend inline, or should the client fetch and parallelize?

Inline server-side call, cold:
  server logic     1–5 ms
  + DB read       1–10 ms (maybe more on miss)
  + same-DC hop   0.5 ms
  ---------------
  total           ~3–15 ms
 
Client parallelized, cross-continent backend:
  base RTT         40–150 ms
  + server side    ~3–15 ms
  ---------------
  total           ~45–165 ms — two hops saved by clients

Whether parallelism saves you depends entirely on whether you removed serial round trips — that is a direct application of the table, and it's the difference between "this feels slow" and "this is slow by arithmetic."

Numbers drift; the ratio doesn't

The absolute numbers will move (faster memory, faster NVMe, faster links), but the ratios are stable — memory stays ~100,000× faster than disk, network RTTs stay dominated by the speed of light plus queueing, cache hierarchy stays ~10× per level. Engineers who internalize the table once can re-derive it forever.