The Runtime Theory
easyApplicationDSA#serverless#cold-starts#latency#lambda

Explain serverless cold starts and how to reduce them

Probes whether you know what a cold start actually contains — microVM boot, runtime init, handler wiring — and can order mitigations by effectiveness with real numbers.

The Runtime Theory Team2 min readasked at vercel · cloudflare · amazon · netflix

This question is testing whether you know what a cold start actually contains, not whether you can name the marketing answer. Anyone can say "keep functions warm." A strong answer decomposes the interval into named machine stages, explains why platforms optimize the median while users feel the cold path, and orders the mitigations by effectiveness with numbers attached.

The mental model: a cold start is a boot sequence, not a delay. The interval between "the platform decides to run your function" and "your handler's first line" has four stages: sandbox creation — a Firecracker microVM boot with network wiring, ~100-200 ms; runtime bootstrap — language runtime process and stdlib imports, ~50-300 ms; handler wiring — loading your code and resolving the handler symbol, ~10-100 ms; and your module-level init, which is variable and usually the dominant term. A 128 MB Node.js function with no dependencies cold starts in a few hundred milliseconds; a 1 GB JVM function initializing an SDK client, a connection pool, and a config loader cold starts in 2-4 seconds. The platform fixed the first three stages; your code owns the fourth.

The walkthrough. When a request hits an idle function, the platform provisions a fresh microVM: kernel load, boot, then the runtime process and your code. Warm instances are a cache with a time-to-live — the platform keeps an idle sandbox alive for a few minutes, then reaps it. The failure mode that matters: a burst of 200 requests to an idle function provisions ~200 microVMs in parallel, and every one of those requests pays the full cold path. The aggregate p99 is the cold start, not the warm median — a cache miss during a spike is the expected failure mode, not an anomaly.

Mitigations, in order of effectiveness. Provisioned concurrency is the only mechanism that eliminates cold starts — you pay a per-instance hourly rate and the platform keeps the sandbox warm; the p95 goes flat. SnapStart checkpoints a booted JVM or .NET process after init and restores it in 150-400 ms instead of re-initializing — but sockets die at restore and entropy must be re-seeded. Lazy initialization moves SDK clients and connection pools out of module scope into a memoized first-call pattern. Smaller, purpose-built functions pay less init per cold start. And there is a pricing angle: cold starts bill at full rate, so a 2.3-second cold call costs ~15x a 150 ms warm call — reducing cold starts is a cost optimization, not just a latency one.

Tradeoffs and edge cases. SnapStart snapshots are taken on a cold run and every restore inherits whatever state that first run left behind — anything depending on wall-clock time, external state, or randomness at init is a corruption candidate. And if the SLO can't tolerate the distribution at all, the honest answer is a provisioned container: it has a boot cost, but it boots once, not per request. Measure the cold percentile with your own load generator — the platform's published table is a lie of aggregation.

This answer walks

Follow-ups they'll push on

  1. 01Which mitigation eliminates cold starts, and what does it cost?
  2. 02Why does a burst of requests make the p99 the cold start and not the warm median?
  3. 03Where does SnapStart fail, and what state does it corrupt?

More interviews in this topic

One dispatch a week

The trace behind each question, the tradeoff that explains it, and one technical dispatch per week — no noise.

One technical dispatch per week. No noise.