What happens during a system call?

This question tests whether you can describe the transition at the hardware level. The interviewer wants the trap, the register convention, and the privilege change — not "the program asks the kernel to do something."

The mental model: the CPU has privilege levels (rings on x86). User code runs in ring 3, where privileged instructions and direct hardware access trap. The kernel runs in ring 0. A system call is the only sanctioned way across that boundary, and the crossing itself — not the work — is the real cost.

Walk through the mechanism on x86-64 Linux. Userspace loads the syscall number into rax and up to six arguments into rdi, rsi, rdx, r10, r8, r9, then executes the syscall instruction. The hardware saves the return address into rcx and rflags into r11, loads the kernel stack pointer, switches to ring 0, and jumps to a fixed entry point. The kernel entry code saves the remaining registers on the kernel stack, checks the syscall number against the bounds of the syscall table, dispatches through sys_call_table[rax], and runs the handler. Arguments that are pointers are validated with copy_from_user/copy_to_user, because userspace can pass any address it wants. The handler does the work, returns an error code or result in rax, and the return path (sysret) restores user registers, ring 3, and control at the instruction after syscall.

Price it: a syscall costs roughly 50–100ns on modern hardware — an order of magnitude more than a 1–2ns function call. The direct cost is the trap, register saves, and kernel work. The indirect cost is larger: the kernel entry runs on a different stack, touches kernel page tables (cold after KPTI-style mitigations), and evicts user cache lines. That's why buffered I/O and read in large chunks matter, and why high-frequency syscalls like gettimeofday are moved into the vDSO — a kernel-mapped page of user-callable code that never traps at all.

Tradeoffs and edge cases worth naming:

• strace attaches at the syscall boundary, which is why it changes behavior and timing.
• Seccomp filters run before the handler, intercepting syscalls at the entry — this is how containers and browsers sandbox.
• EINTR happens when a signal interrupts a syscall, and the handler may need to retry.
• Two entry paths on x86: syscall (fast, AMD64) and int 0x80 (legacy) — they dispatch to the same table.

A strong closing line: a system call is a trap-driven privilege transition with a register contract, and the kernel does the work on the caller's behalf at the price of the crossing itself.