TRT
Sign in
hardExpertdebugging

Why do naive C++ microbenchmarks often produce misleading or irreproducible numbers, and how do you make a benchmark trustworthy?

Identify compiler dead-code elimination, cold page faults, CPU frequency scaling, and wrong clock choices, and explain how to fix each one.

TRT practice prompt — not a verified question from a named employer.

The Runtime Theory Team2 min read
Learn it firstBenchmarking: Clocks, Warmup and Honest NumbersPro — members onlyTime C++ code with high-resolution clocks, defeat optimizer tricks, warm up caches, and report numbers that survive a rerun.

A strong answer

A naive C++ loop timed with a single start/stop timestamp almost always measures the wrong thing due to four pitfalls:

  1. Compiler dead-code elimination and constant folding: At -O2 or -O3, if the result of the loop is never observed outside the loop, the compiler proves the computation has no side effects and deletes the entire loop (or hoists invariant math outside the loop), reporting a bogus 0 ns per operation. You must feed outputs into an opaque sink (DoNotOptimize via volatile or empty asm volatile("" : : "r,m"(val) : "memory")) and ClobberMemory() so the compiler cannot delete or pre-compute the work.
  2. Cold start vs steady state: The very first iteration triggers OS minor page faults (zero-filling newly mapped 4 KB pages), cold L1/L2/L3 caches, unprimed branch predictors, and idle CPU turbo frequency states. Always run untimed warmup iterations before starting the clock.
  3. Wrong clock: std::chrono::system_clock can jump forward or backward when NTP adjusts system time (And on some standard libraries high_resolution_clock is just an alias for system_clock). Always time intervals with std::chrono::steady_clock, which is guaranteed to be monotonic.
  4. OS scheduling noise and thermal drift: A single run can be skewed by an OS context switch, interrupt, or CPU thermal throttling. Run many trials, interleave competing implementations (A, B, A, B), and report the median and spread (e.g., p10/p90 or IQR) rather than a single number or an outlier-sensitive mean.

What the interviewer is listening for

  • Explains how -O3 eliminates unused work and how DoNotOptimize / volatile sinks prevent it without crippling register allocation inside the timed body.
  • Chooses std::chrono::steady_clock and explains why system_clock is unsafe for duration measurements.
  • Insists on warmup runs, multiple interleaved trials, and distribution metrics (median + spread).

Common mistakes

  • Benchmarking a -O0 debug build, which measures stack-load/store overhead and uninlined STL wrapper calls rather than real production machine code.
  • Including test fixture allocation or random number generation inside the timed region when only the algorithm itself should be measured.

This answer walks

Practice follow-ups

  1. 01Why should you use std::chrono::steady_clock instead of high_resolution_clock or system_clock?
  2. 02Why is reporting the median and interquartile range better than reporting only the arithmetic mean of benchmark runs?

More interviews in this topic

easy · languageWalk through the internal structure of a Go channel (`hchan`), explain the exact table of channel state behaviors (nil vs open vs closed for send, receive.easy · languageHow does Go's compiler decide whether a variable is allocated on the stack or the heap (Escape Analysis), why is returning a value `User` often faster than.easy · languageExplain how Go's Concurrent Tri-Color Mark-and-Sweep GC works—why doesn't Go use a compacting/generational GC like the JVM, how does the Hybrid Write Barrier.

Not started

Sign in to save your learning progress.

Sign in to save