A context switch is the kernel's sleight of hand: it makes N threads share one CPU as if each owned it. Every switch is pure overhead — the CPU does no useful work while it changes who's running — so the question is what that overhead is and why it's expensive.
Step 1 — the timer interrupt
Something must force the handoff, or a greedy task never yields. With HZ=250, the local APIC fires a timer interrupt every 4ms. On idle CPUs with NOHZ, the tick is disabled entirely — the only "timer" is the scheduler tick needed for the next wakeup deadline.
Step 2 — accounting and need_resched
The interrupt handler runs update_curr(): the current task's vruntime (virtual runtime — actual runtime weighted by nice value, e.g. nice -20 runs 25x faster than nice 19) is advanced, the task is reinserted into the CFS red-black tree keyed by vruntime, and if it's no longer the leftmost (most-starved) node, set_tsk_need_resched() sets a flag in the task's thread_info. Nothing switches yet — the flag just marks the spot.
Step 3 — schedule() actually switches
The switch happens at the next scheduling point: return from interrupt, return from syscall, or an explicit schedule()/sleep. __schedule():
- Takes the runqueue lock (rq->lock).
- Calls
pick_next_task→ CFS picks the leftmost node of the rb-tree — the task with the smallest vruntime, i.e. the one that has run least. - Removes it from the tree, pushes the outgoing task back in (or onto the
expired/sleepingstate). - Context-switch via
switch_to().
Step 4 — the memory switch
If the tasks belong to different processes (different mm), switch_mm() loads the new process's page-table root into CR3 — and that's where the real cost hides. Loading CR3 either flushes the whole TLB (no PCID/ASID) or switches to the new address-space's ASID tag. Then every subsequent memory access misses the TLB and re-walks: tens to hundreds of ns per access for a while. Cache pollution is the bigger tax — the new task warms L1/L2/L3 from scratch, and the old task's data is cold when it returns. This is why two threads of one process switch cheaper (no CR3 reload) — the "same-address-space fast path."
Step 5 — the register switch
switch_to() is the deepest magic: it swaps current (the per-CPU pointer to the running task), swaps the kernel stacks, and saves/restores callee-saved registers + FPU state. The __schedule() call returns in a different task than the one that made it — the same C stack frame continues in whatever task is now current. FPU state is usually lazy: saved only when the new task actually uses the FPU.
Step 6 — return to user mode
The outgoing interrupt frame (or syscall return path) belongs to the new task, so iret/sysret lands in the next task's user space, right where it was preempted. The handoff is complete: on average one task lost the CPU and another gained it in ~1-20µs.
What it costs
- Raw switch cost (registers, stack, rq lock): ~1-5µs.
- TLB flush + cold caches on process switches: pushes real-world cost to ~5-20µs — and in throughput terms, 10,000-100,000 switches/sec can shave 5-20% off a workload.
- Same-address-space (thread) switches skip the CR3 reload: ~2-3x cheaper.
$ perf stat -e context-switches,cpu-migrations,cycles ./myapp
context-switches 3,872,091 # ~19k/sec over 200s
cpu-migrations 9,214 # threads jumping CPUs (expensive: cache loss + rq lock)The scary number is the migration count. A thread moved to another CPU loses its entire cache footprint — hundreds of µs of hidden cost. Context switches are amortizable; migrations are the ones that actually hurt.