Context Switch Costs, Measured

When one thread stops running and another starts, the CPU does not "switch tasks" — it performs surgery on its own state. A context switch is a save, a decision, and a restore, executed in hardware and kernel code, and the price shows up in every latency percentile of every service. This is what the machine actually does, and what it actually costs.

What the CPU has to do

A thread's entire CPU existence is: register state, stack, and address space. On x86-64 a switch means:

1. Save general-purpose registers (RAX through R15, ~120 bytes) to the outgoing thread's kernel stack.
2. Save FPU/vector state — XSAVE can be 1–2 KB; this is why AVX-512 workloads make context switches dramatically more expensive.
3. Switch kernel stacks — RSP now points into the incoming task's kernel stack.
4. Switch CR3, the register holding the page table base — unless the two threads are in the same process.
5. Run the scheduler: pick the next runnable task (CFS/EEVDF tree lookup).
6. Restore the incoming thread's registers and return into its user code.

Steps 1–6 alone are fast — sub-microsecond on modern hardware. The lmbench ctx benchmark on a recent x86 server reports ~2–4 µs per switch, and the gap between "fast" and "measured" is entirely steps 4 and the one that follows: memory translation and cache.

The TLB: why process switches cost more than thread switches

The TLB is the CPU's cache of virtual-to-physical translations, and it is per-address- space. When CR3 changes, every TLB entry belongs to a different world. Older CPUs flushed the entire TLB on every process switch; a cold TLB means every instruction fetch, load, and stack access walks the page tables again — page walks cost ~100–200 cycles, and there are thousands of misses after a flush.

Modern x86 has PCID (Process Context Identifiers): each address space gets a tag, and the TLB can hold translations for several processes at once. A process switch must still invalidate the outgoing tag's entries, but the next time that process runs, some entries may survive. Same-process thread switches never touch CR3 at all. That asymmetry is the measurable part of "threads are cheaper to switch than processes."

Cache pollution: the hidden invoice

The TLB is the explicit cost; the cache is the implicit one. Every thread has a working set — hot code and data — living in L1/L2. When another thread runs on the core, its working set evicts the first thread's lines, and when the first thread comes back it finds its cache cold. On a modern server core (48 KB L1d, ~1.25 MB L2) a busy working set can take 5–20 µs to re-warm — several times the cost of the switch itself.

That is why naive benchmarks understate production costs: a microbenchmark switching between two tiny loops shows ~2 µs; a production service switching between 64 threads with real working sets sees 5–10 µs of effective cost per switch.

The arithmetic that kills servers

Take a real number: 8 cores, a service doing 50,000 switches per core per second. At a conservative 3 µs per switch:

50,000 switches/s × 3 µs × 8 cores = 1.2 s of switching per second
= 15% of total CPU spent doing state surgery

Then add the queue. When runnable threads exceed cores (oversubscription), the scheduler runs a lottery: threads that lose are switched out without having done useful work. Oversubscription turns context switch cost from "overhead" into "pure waste."

Why too many threads hurt

• More threads than cores → every thread is a latency victim: it waits its turn, and each turn costs a switch.
• Threads blocked on I/O wake up on a different core → the cache worked for nobody; NUMA hops add ~100+ ns per remote access on top.
• Lock contention grows superlinearly with thread count: each contended lock parks a thread, wakes a thread, and pays two switches.
• Stacks are pinned: 16 KB kernel stack and up to 8 MB user stack per thread. 10,000 threads burn tens of GB of virtual address space and real physical memory for stacks alone.

"Add more threads for throughput" is correct only until cores are saturated. After that, more threads buy latency, not throughput.

What to do with this number

1. Measure: vmstat 1 — the r column vs. core count; perf sched for switch counts.
2. Match threads to cores for CPU-bound work; let I/O threads be cheap (epoll, not thread-per-connection).
3. Prefer user-space switching (goroutines, async) when blocking is the norm — ~100 ns vs 3 µs is a 30× difference, because there's no syscall and no CR3 change.
4. Pin with taskset/cpuset when the working set is the bottleneck.