What lives inside a C function's call stack frame, and what happens in memory during a stack overflow?

A strong answer

On x86-64 architectures, the hardware call stack grows downward from high virtual memory addresses toward low addresses, tracked by the stack pointer register (%rsp). When a caller invokes a function with the call instruction, the CPU pushes the 8-byte return address (the instruction pointer %rip of the instruction right after call) onto the stack and jumps to the callee. The callee's prologue optionally pushes the caller's frame pointer (%rbp), sets %rbp = %rsp, and subtracts the frame size from %rsp to reserve space for local automatic variables, register spill slots, and any arguments beyond the first six registers.

When the function executes ret, it restores %rsp and %rbp, pops the 8-byte return address back into %rip, and resumes the caller. Notice that popping a stack frame does not zero out the memory—it simply moves %rsp back up. If a function returns the address of a local variable (return &local;), that pointer now points into unclaimed memory below %rsp that the very next function call (printf, for instance) will immediately overwrite.

The operating system reserves a fixed virtual memory region for each thread's stack (typically 8 MB on Linux) and places an unmapped guard page (PROT_NONE) right below its bottom boundary. If deep recursion or a giant local array (int buf[4000000];) pushes %rsp into the guard page, the CPU's Memory Management Unit raises a hardware page fault that the kernel delivers as SIGSEGV (stack overflow).

What the interviewer is listening for

• Accurately names the contents of a stack frame: return address, saved frame pointer, local variables, and spilled registers.
• Explains that stack allocation/deallocation is just pointer arithmetic on %rsp ($O(1)$ cost), which is why returning &local creates a dangling pointer.
• Describes how stack overflow is caught by the MMU hitting a PROT_NONE guard page at the bottom of the stack segment.

Common mistakes

• Allocating multi-megabyte buffers as local stack arrays (char buf[10 * 1024 * 1024];) instead of on the heap (malloc) or in static storage.
• Confusing a stack overflow (exhausting stack space via deep recursion or huge local arrays) with a stack buffer overflow (writing past the end of a local array and overwriting the saved return address on the stack).