The Runtime Theory
KernelDSAscheduling

What actually happens when you call read()?

A step-by-step walk from a libc read() call, through the trap into the kernel, syscall table dispatch, kernel work, and the return path to user space.

The Runtime Theory Team3 min read08 steps

layer stack

Kernel

HWHardware
KKernel
RTRuntime
APPApplication
SYSSystem
CLIClient
NETNetwork
TLSCrypto
SRVServer

adjacent altitudes in this subsystem are still being traced

trace spine

  1. 01 libc wrapper executes the syscall instruction
  2. 02 Hardware trap into kernel mode
  3. 03 Register state saved to pt_regs
  4. 04 Dispatch via sys_call_table
  5. 05 Entry-side checks and seccomp filters
  6. 06 Kernel work: vfs_read and the page cache
  7. 07 Exit path: signals, reschedule, sysret
  8. 08 User space resumes one instruction later
On this page

A syscall is the only sanctioned way for user code to get the kernel to do anything: open a file, read a socket, spawn a process. Everything else — function calls, branches, cache hits — stays inside the process. The boundary crossing itself costs something, and its shape determines the syscall overhead you see in strace and perf.

Step 1 — the libc wrapper

c
ssize_t n = read(fd, buf, 4096);

On x86-64 Linux, glibc's read wrapper loads the syscall number into %rax (0 for read, 1 for write), places arguments in %rdi, %rsi, %rdx, and executes the syscall instruction. This is a fast-entry instruction: it atomically drops the CPU to ring 0 and jumps to the kernel entry point in the entry_SYSCALL_64 trampoline. No permission checks, no gate checks — trust comes later.

Step 2 — trap and register save

The hardware switches to the kernel stack (via TSS). Nothing is saved yet — the kernel now writes the full user register state into pt_regs on the kernel stack. Pure CPU work: ~30-60ns, no allocations, no locks. The CPU also records the user-mode instruction pointer in %rcx so sysret knows where to resume.

Step 3 — dispatch through the syscall table

The kernel validates %rax < NR_syscalls and indexes sys_call_table[%rax] — a single computed jump, ~5ns. read resolves to ksys_read(). This table is the kernel's API surface: ~450 entries on a modern x86-64 kernel.

Step 4 — entry-side checks

syscall_enter_from_user_mode() runs per-syscall security and tracing hooks: audit, tracepoints (sys_enter_*), preemption bookkeeping, and — when a seccomp filter is attached (Docker, Chrome, systemd) — a BPF program evaluated against the syscall number and arguments. That filter is usually tens of ns; with dozens of rules it can dominate entry cost.

Step 5 — the actual kernel work

ksys_readvfs_readfile->f_op->read_iter. For a regular file on ext4: resolve struct file from the fd table, take the file's read lock, look up the page in the page cache, and copy_to_user() into your buffer (a range-validated, page-fault-safe copy). Cached page: roughly 1µs end to end. Cold page: you're headed to storage — see the file-read trace.

Step 6 — the return path

syscall_exit_to_user_mode() mirrors step 4: deliver pending signals, check need_resched (if the scheduler wants the CPU, schedule() runs here instead of resuming your thread), and fire sys_exit_* tracepoints. This is also where deferred signal delivery actually happens — the kernel only runs signal handlers when returning to user mode.

Step 7 — back in user space

sysretq restores ring 3 and resumes one instruction after the syscall. The wrapper converts a negative errno into errno and returns. Total boundary cost: ~50-100ns for an empty syscall like getpid on modern x86-64; the legacy int 0x80 path was ~10x slower and is unused by modern libcs.

What it costs

  • Empty syscall (getpid): ~50-100ns with syscall/sysret, up to 1µs on older entry paths.
  • copy_to_user on a cache hit: ~20-40ns per 4KB — the copy is cheap; the machinery around it isn't.
  • Meltdown mitigation: kernels built with full KPTI pay extra TLB flushes per entry/exit — up to 5-10% on syscall-heavy workloads on pre-2018 CPUs; PCID keeps it near 1-2% on modern hardware.

The revealing number is that syscall cost is mostly fixed — batching 8KB instead of 4KB is nearly free. You pay per call, not per byte. That's exactly why io_uring and the vDSO exist: amortize the boundary away.

bash
$ strace -c -f ./myapp 2>&1 | tail -5
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 48.32    0.420312        1213       346           read
 19.10    0.166201         480       346           write

346 read calls at ~1.2ms each: the app isn't paying for the syscall — it's paying for cache misses and disk behind them. The boundary is 0.1% of that.