What happens when a sleeping thread wakes up?

The scheduler's glamorous half is deciding which task runs. The quiet half — which actually determines whether your service feels fast — is waking tasks up. A thread blocked on a mutex, a socket, or a timer is gone from the CPU; how quickly it gets back determines your wake latency, and that path is a pipeline of its own.

Step 1 — going to sleep

The task blocks: mutex_lock, futex_wait, sk_wait_data, or schedule_timeout(). It sets its state to TASK_INTERRUPTIBLE (or TASK_UNINTERRUPTIBLE for uninterruptible I/O waits), removes itself from the runqueue, and calls schedule(). A sleeping task holds no CPU; it's only reachable through the wait queue it parked itself on (a lock's waiter list, a socket's sk_sleep, an hrtimer's rb-tree).

Step 2 — the wakeup event

One of three things happens next:

Another thread calls mutex_unlock / futex_wake (kernel: wake_up_q walks the waiter list and calls try_to_wake_up).
A timer expires: the hrtimer interrupt fires the callback, which wakes the task (this is sleep(1)'s entire mechanism).
An interrupt (e.g. network RX) completes I/O and wakes the waiter — skb completion paths.

Whichever path: someone calls try_to_wake_up(task, ...).

Step 3 — try_to_wake_up: choosing the CPU

try_to_wake_up is schedule()'s mirror. It must decide where the task should run. The ladder of choices: wake-affine (wake_affine — prefers the waker's CPU, then its LLC domain, so the woken task inherits hot caches), NUMA-aware placement for long-running tasks, and finally select_task_rq, which looks for an idle CPU (cheapest: no preemption needed, no cache eviction — running on an empty CPU is strictly better than interrupting someone). The old task's cpus_allowed mask and cgroup constraints are respected throughout.

Step 4 — vruntime placement

CFS must decide when the task runs. The sleeping task's vruntime is frozen while it sleeps; on wakeup it's placed in the runqueue's red-black tree. If the task woke from a long sleep, its vruntime is stale (very small — it hasn't "run" in hours), so it's capped: max_vruntime = min_vruntime - granularity. This prevents a long-sleeper from instantly starving everything for minutes of CPU, while still giving it a latency bonus. The tree now has a new leftmost node.

Step 5 — wakeup preemption

If the woken task's vruntime is lower than the currently-running task's vruntime minus sysctl_sched_wakeup_granularity, CFS declares wakeup preemption: resched_curr() sets need_resched on the target CPU. The running task isn't interrupted mid-instruction — it continues until the next preemption point (timer tick, syscall exit), then schedule() hands over. Wakeup preemption is why a thread waking from pthread_cond_wait typically resumes within ~50µs rather than at the next 4ms tick.

Step 6 — the idle kick

If the target CPU is idle, resched_curr can't rely on a tick (idle CPUs with NOHZ have no ticks!). Instead the kernel sends an IPI (smp_send_reschedule) — an inter-processor interrupt that wakes the idle CPU out of halt/mwait, runs its scheduler, and starts the woken task. An idle-wakeup is the best case: no one gets preempted, and the wake completes in ~5-30µs.

Step 7 — the switch

The actual handover happens in schedule() — the same machinery as a context switch (see that trace): runqueue lock, pick leftmost, switch_mm + switch_to. From the woken task's perspective: one moment it was inside futex_wait; now it's returning from the syscall as if nothing happened.

What it costs

Idle wakeup: ~5-30µs (IPI + switch).
Preemptive wakeup: ~20-100µs (wait for the preemption point + switch).
Timer-based wakeup: bounded by timer slack; nanosleep on CLOCK_MONOTONIC wakes within ~50µs of the deadline.
The hidden cost: cache affinity. A woken task that lands on a cold CPU pays 100s of µs of cache misses — often more than the switch itself.

bash

$ perf sched record -- sleep 1 && perf sched latency
  task                  |   runtime ms  |  switch time (avg)
  ----------------------+---------------+-------------------
  perf:                  |    1.41 ms    |        16.8 us
  sleep:                 |    1.12 ms    |        21.4 us

Roughly 20µs per wake — a few hundred syscalls of work, spent entirely on deciding who runs and handing over. If your service's p99 latency is dominated by "wake someone up," the fix is usually not a faster scheduler but fewer wakeups: batching, epoll event aggregation, or busy-polling the hot path.