How do you find the hot path with perf and flame graphs?

A strong answer

A strong candidate frames How do you find the hot path with perf and flame graphs? around the exact memory layout, compiler semantics, and hardware execution model covered in Profiling With perf and Reading Flame Graphs. Walk through perf record and report, flame graph reading, and the rerun that proves a layout fix.

Throughput lags and the source offers no clue: loops look flat, atomics look right, yet eight threads stall. Guessing at the cause wastes days. You need a profile that counts where cycles and misses happen, with stacks that point at lines.

The commands below record and summarize a test binary built with debug info plus optimizations. Frame pointers or dwarf call graphs keep stacks intact so report and flame stages can attribute correctly.

In a production interview, always connect the specification rule to its concrete runtime consequence: state what lives on the stack versus the heap or static storage segment, how the compiler or operating system validates alignments, lifetimes, and error codes, and what happens under adversarial or boundary inputs (such as zero-length buffers, integer overflow, concurrent mutation, or partial I/O). Backing your explanation with a small trace of register/memory state transitions immediately separates senior systems engineers from candidates who only memorized syntax.

What the interviewer is listening for

• Mastery of core invariant: Record CPU profiles with perf record and read perf report output.
• Mastery of core invariant: Build and read flame graphs to find hot stacks.
• Mastery of core invariant: Turn profile data into targeted layout or algorithm changes.
• Hardware and ABI awareness: Explaining how sizes, alignments, calling conventions, or kernel boundaries behave on real 64-bit targets rather than relying on implementation-defined assumptions.
• Defensive engineering: Naming the exact compiler flags (-Wall -Wextra -Werror), static analyzers, or runtime sanitizers (-fsanitize=address,undefined,thread) that verify correctness in CI.

Common mistakes

• Explaining only surface syntax without tracing the underlying memory addresses, ownership lifetime, or kernel/user-space state transitions.
• Assuming undefined behavior (such as signed overflow, uninitialized reads, strict-aliasing violations, or dangling pointers) will "just work" because it passed an unoptimized -O0 debug build.
• Ignoring return codes, partial reads/writes, alignment padding, or cleanup paths when an allocation or syscall fails midway through a function.