How an Operating System Schedules a Thread

When your thread "runs", it runs because the kernel decided it should — right now, on this core, with this priority, sharing that core with forty other threads. The decision is made every few milliseconds, a hundred times per second per core, and almost none of it is visible from your process. This is the article about the invisible part.

The scheduler's job description

The scheduler exists because demand exceeds supply: on an 8-core machine, a typical server has hundreds of runnable threads and 8 seats. The scheduler must:

1. Give every runnable thread a fair share of CPU over time.
2. Keep latency down — a foreground thread should feel instant, not fair.
3. Keep throughput up — context switches are not free.
4. Respect priority and asymmetry (NUMA, power states, cache topology).

The balancing act between 1 and 2 — fairness vs. latency — is the entire history of scheduling policy. Linux answered it with CFS (Completely Fair Scheduler) in 2007, and with EEVDF since 6.6: give each thread a scheduling time slice proportional to its weight, then pick whichever thread has received the least CPU relative to its entitlement.

What actually happens on a context switch

A context switch is not "the kernel flicking a switch". The full cost looks like this:

previous thread's user code
        ▼  (timer interrupt / syscall / I/O completion)
1. kernel entry          — switch to kernel stack, save registers
2. scheduler decision     — pick next runnable thread (CFS red-black tree lookup)
3. TLB invalidation       — the CPU's address-translation cache is per-*process*
4. restore next registers — load the next thread's saved state
5. kernel exit            — return into the next thread's user code
        ▼
next thread's user code

The steps to circle are 3 and the ones not shown: cache and TLB cold. Every process switch starts the new thread against a CPU whose cache lines belong to the old one — a "warm" context switch on the same core still costs on the order of 1–2 µs, and a switch across cores carries the entire NUMA/energy tax. At 100k switches/second, that's 10% of a core spent doing nothing but deciding.

That cost is why modern servers are tuned to avoid switching: busy-polling, NAPI, epoll edge-triggered loops, and the 1:1 thread model replaced by coroutines in Go/Rust async — fewer runnable threads than cores is the real scheduler optimization.

Priorities are advice, not orders

Every thread carries a priority, but the scheduler's job is to translate that into time shares within a constantly moving budget. Linux implements this as:

• nice (-20…19) — a weight multiplier, not a hard rank. A nice-19 thread still gets CPU; it just gets a smaller slice of the same core.
• RT priorities — preemptive: a SCHED_FIFO thread at rt-priority 1 will run before any normal thread, and can starve everything else if it never yields. This is why real-time scheduling is a footgun with a safety label.
• cgroups throttling — the modern currency. cpu.max budgets, burst, and per-group fairness are how containers actually cap CPU: the kernel's entity fair-share counts probe into every cgroup, and the throttle says "your share is exhausted, sleep".

The surprising consequence: your "priority 99" thread can be preempted by nothing, and still not run, because a cgroup parent can deplete the budget first. Priorities shape the queue; budgets shape the floor.

Where your thread actually waits

A thread is runnable or it isn't. When it isn't — waiting on I/O, a lock, a channel — it leaves the run queue entirely and parks on a wait queue. The difference matters:

Waking a thread is itself a scheduling operation: the waking thread (often the kernel's I/O completion path) plucks the sleeper and enqueues it — possibly on a different core, costing a cross-core wakeup and its cache pathologies. Every blocking call is a scheduled event and every timeout you see in a profiler is this machinery, not "the function is slow".

Concurrency models are scheduling models in disguise

• 1:1 (OS threads) — the kernel decides everything. Preemption, priority, fairness is real; overhead per thread is ~µs-scale stack + kernel switch cost.
• M:N (goroutines, tasks) — your runtime multiplexes user threads onto kernel threads. Switching a goroutine is a ~50–200 ns user-space operation — 10–100× cheaper than a kernel switch — because there's no syscall and no TLB flush. The government of preemption moves into your process: Go's GOMAXPROCS is literally "how many kernel threads do I rent to run goroutines on".

That scale gap is the reason "synchronous blocking" in Go or async Rust costs less than the same pattern in C threads — not because kernels are slow, but because user-space switching skipped the invoice.