The Runtime Theory
RuntimeInternalsexecution

What happens when the JIT compiles your hot loop?

A step-by-step walk from interpreter to machine code: profiling, tiered compilation, speculative optimization, and the deoptimization guards that keep it honest.

The Runtime Theory Team1 min read05 steps

layer stack

Runtime

HWHardware
KKernel
RTRuntime
APPApplication
SYSSystem
CLIClient
NETNetwork
TLSCrypto
SRVServer

adjacent altitudes in this subsystem are still being traced

trace spine

  1. 01 Interpreter runs the bytecode
  2. 02 Profiling accumulates counts
  3. 03 Baseline compiler emits code
  4. 04 Optimizing compiler speculates
  5. 05 Guards and deoptimization

for (let i = 0; i < 1e9; i++) in JavaScript, Java, or C# does not start as machine code. It starts as interpreted bytecode, then — if it runs often enough — the runtime promotes it to compiled machine code, with each promotion level doing more profiling and bolder optimization. This is the trace of that promotion.

trace stepRuntime

The engine (V8, HotSpot, CoreCLR) parses the source into bytecode and runs it on the interpreter — a dispatch loop: fetch opcode, execute, advance. Interpretation costs roughly 10–100× the machine-code cost per operation (opcode dispatch, operand stack juggling, no register allocation), but it starts instantly — no compile wait, no warmup. This is the startup/speed tradeoff's engine-side half.

trace stepRuntime

As the interpreter runs, it profiles: every loop back-edge increments a hotness counter; every property access and call site records observed receiver types and shapes (hidden classes in V8, MEGAMORPHIC/UNIMORPHIC states in HotSpot's type profile). The profiling is deliberately woven into the bytecode — each point that could be optimized pays a little bookkeeping so the optimizer later knows what to assume.

trace stepRuntime

Counter crosses the tier threshold (V8: 1,030 back-edges for baseline) → the baseline compiler (Sparkplug in V8, C1 in HotSpot) emits straightforward machine code: no deep optimization, just a faithful translation of the bytecode. Compilation takes milliseconds and runs on a background thread; the interpreter's stack is stack-walked into the compiled frame (the on-stack replacement, OSR, that makes the transition seamless). Startup latency is now behind you; this code runs maybe 2–5× the interpreter's speed.

trace stepRuntime

Cross the next threshold (V8: ~30× the baseline count) → the optimizing compiler (TurboFan, C2) recompiles the function using the profile. It can now do everything a compiler does with type knowledge: inline calls (un-inlining the cost of function-call-trace itself), constant-fold, escape-analyze allocations off the heap, vectorize. Every optimization rests on the profile's bets — e.g. "this + is always number+number" or "this property access is always shape X" — so the emitted code is guarded: each bet is checked with a cheap runtime test.

trace stepRuntime

The guard fails — a string hits the number-only +, or a new object shape arrives — and the engine executes a deoptimization: it throws away the optimized frame and rebuilds the interpreter state for that call site (deopt is always to a correct point; the optimizer never assumes safety, only performance). Deopt costs a stack walk and a state reconstruction, ~microseconds, and the code is usually re-optimized with the new profile. A pathological pattern (megamorphic call sites) can deopt in a loop — the compiler and the workload fighting — which is exactly the "JIT thrashing" you see in benchmarks.

bash
node --trace-opt --trace-deopt -e "function f(a){return a.x+a.y} for(let i=0;i&lt;2e7;i++) f({x:i,y:i});"
java -XX:+PrintCompilation -XX:+PrintDeoptimizationDetails -jar app.jar

What the machine actually does is run your loop three times, at three different speeds, with a profiler watching the second run and a compiler betting on the third — then standing ready to undo the bet the moment reality disagrees. The magic isn't that the machine code is fast; it's that the machine code exists at all, earned and guarded.