Expert · Module 23 · Lesson 4 of 4
Benchmarks and performance counters
A stopwatch reports a number, not a verdict. Warm-up, outliers, dead-code elimination and the wrong statistic can each invent an improvement; hardware counters show where the cycles actually went.
By the end you can
- Compute a median and a mean from timing samples and quantify an outlier's effect
- Name the counters that show whether a workload is memory-bound
- Explain why warm-up, sample size and dead-code elimination change results
Builds on Counting steps, Best, average and worst case, Caches: why neighbors are fast.
01 · A question first
A benchmark reports 15.76 ms for code that runs in about 12 ms. One sample out of eight was 41.7 ms — a garbage collection pause, not your function — and the average kept it. What number should you report, and how do you keep the pause out of the next run?
Make a guess before you read on.
02 · The idea
Members only
Unlock the full lesson
This lesson is part of the membership. Join to read it in full, along with every other premium deep dive.
What's inside
- What a stopwatch reports
- Warm-up: the first runs are not the runs
- Outliers and which statistic to report
- Measuring the wrong thing
- Counters: where the cycles went
- Microbenchmarks versus end-to-end
- Tools that already do this
- Pseudocode & Multi-Language Implementations (C++, Java, Python)
03 · Practice
Something wrong with this lesson? Report a problem