A cache only helps if it is warm, and the moment it is cold — restart, flush, mass expiry — every request pays origin latency at once. This trace follows one cache key from a miss through population, and shows the stampede that happens when 500 concurrent misses discover the same cold key simultaneously.
1. The cache is cold
The Redis cluster was just restarted, a deploy flushed the namespace, or a hot key's TTL expired a second ago. Every key that a request touches is now absent. The cache's hit rate, which was 97% ten minutes ago, is functionally zero — and the origin is about to learn what that means.
2. The first request misses and reads through
GET /api/feed hits the read-through path. The cache returns nil — one Redis round trip (~0.5ms), which the request pays before it even knows it will miss. The handler decides to populate: query the origin (Postgres or the upstream service) and write the result back. If the handler uses "cache-aside," it holds a lock or simply proceeds; the read-through happens inside the request path, which is where the stampede math lives.
3. The origin query executes
The origin query runs: a feed assembly spanning multiple tables and maybe an upstream call — 50-300ms of real work. The request that drew the short straw now carries that full origin latency; this is the fill cost — one request per cold key per TTL period, on average. At steady state that is tiny; all at once (cold cache), it is every request.
4. The key is populated
The handler stores the result: SET feed:home:u42 <json> EX 300 — a 10-30KB payload, ~0.5-1ms. Subsequent requests hit the fast path: one Redis GET, ~0.3-0.8ms, no origin. The cache is warm again for this key... for the next 300 seconds.
5. Concurrent misses collide — the stampede
Here is the failure mode. The key expired at T+300s. Five hundred requests arrive in the same second, all GET the key, all get nil, and all run the 300ms origin query. Redis is now serving 500 writes of the same key; the origin is serving 500 identical queries; the DB pool is saturated (see the connection pool trace); and the 500 requests each ate 300ms of origin latency. The cache's hit rate looks catastrophic even though it did its job — the stampede is a synchronization problem: 500 independent processes all decided to fill at the same moment.
6. Stampede protection engages
The standard fixes, in order of increasing sophistication:
- Jittered TTL:
EX 300 ± rand(0-60). Cheap, kills the synchronized-expiry burst. This alone eliminates most stampedes. - Locking/single-flight: only one in-flight fill per key — a Redis
SETNX fill:feed:u42 EX 5, or in-process single-flight (Go'ssingleflight, Java'scomputeIfAbsent-with-lock). Losers either wait for the winner's value or serve stale. The winner fills; the losers take the cached value milliseconds later. - Refresh-ahead: a background job refreshes keys at 80% of TTL. The origin is hit by exactly one scheduled worker, never by user requests — the request path can't stampede because it never misses.
7. Warm-up pre-populates hot keys
The remaining cold-start problem — restart, deploy, or a known traffic spike (launch day, flash sale) — is solved by warming before traffic arrives. A warm-up job reads the known hot key set (from request logs or a counter) and runs the fill path: 10,000 keys × 100ms each = ~17 minutes of sequential work, or ~2 minutes with 8 parallel workers, all before users arrive. Warm-up is a batch job in the strictest sense (see the batch job trace): same chunking, same checkpointing, same resume-on-crash discipline.
The one sentence
A cache key's life is a cycle — miss, fill, hit, expire — and the only moment that matters is the miss: it must be paid by exactly one request (single-flight), spread over time (jitter), or never paid by a user request at all (refresh-ahead and warm-up).