Distributed Caching Consistency Models

A cache is a second copy of data with a different freshness, which makes every cache a miniature distributed system: the writer (database), the reader (application), and the cache node all hold opinions about what the value is. The interesting engineering is not the cache — it is the consistency protocol between the cache and its source of truth. This article covers the three standard models — cache-aside, write-through, write-behind — and the failure modes that actually bite: invalidation races, stampedes, and coherence.

Cache-aside: the default

In cache-aside (lazy loading), the application owns the cache:

• Read: check cache. On miss, read the database, populate the cache, return.
• Write: write the database, then delete the cache entry — never update it in place.

Deleting instead of updating is deliberate. An in-place update races with concurrent readers: a reader can load the stale value from the database, then overwrite your fresh cache write with it. A delete avoids the race — the next reader misses and reloads from the database.

The invalidation race

Deletes have their own race, and it is the classic cache bug:

T1: cache miss → read DB (old value v1)
T2: write DB (v2) → delete cache entry
T1: cache.set(key, v1)   # stale value lands in cache

T2's invalidation happened before T1's population, so the cache now holds v1 with no future invalidation scheduled — permanently stale until something removes it. The standard mitigation is a TTL backstop: the TTL is not a performance knob, it is your failure bound — the maximum time a stale value can serve after an invalidation race. Single-flight population and cache version stamps reduce the race window but do not close it.

Cache stampedes

When a hot key expires, every concurrent request misses at once. If 10,000 requests hit a key with no cached value, 10,000 database queries fire — the database melts, requests time out, and the cache refills even slower. The machine you built to save the database can destroy it.

func Get(key string) (Value, error) {
    if v, ok := cache.Get(key); ok {
        return v, nil
    }
    return singleflight.Do(key, func() (Value, error) {
        return db.Load(key) // one loader per key, others wait
    })
}

Mitigations: request coalescing (single-flight), probabilistic early expiration (expire entries at TTL × random factor so reloads spread out), and never-expiring hot keys refreshed in the background.

Write-through

Write-through updates the cache and the database synchronously in the write path; reads always hit the cache. Read latency is low and consistent, but every write pays for both stores, and the write path now contains two systems that can disagree. If the database write succeeds and the cache write fails — or vice versa — the two diverge, and you need a repair path (retry, or degrade to cache-aside). Write-through is not more consistent than cache-aside; it moves the consistency problem into the write path, where it is synchronous and visible.

Write-behind: trading durability for latency

Write-behind (write-back) acknowledges the write from the cache and flushes to the database asynchronously. It is the fastest model for writes and the most dangerous:

• A cache crash loses every not-yet-flushed write.
• Flushes are at-least-once, so the database must accept duplicates or the flush must be idempotent.
• Flush ordering matters: if two writes touch related rows, flushing them out of order corrupts state.
• A slow database turns the flush queue into a memory bomb — backpressure is mandatory.

Real write-behind systems exist (Cassandra's memtable-to-SSTable path, SSD FTLs, in-memory stores with async persistence), but they are write-behind to disk, where durability is bounded and the target is local. Write-behind to a remote database under network latency is where "eventually consistent" stops being an abstraction and becomes a data-loss budget.

Invalidation storms and coherence

Coherence is the property that all copies of a key converge and reads observe writes in a defined order. A single cache node gets coherence for free. The moment you have multiple cache copies — per instance, per region, per availability zone — each copy holds independent state, and coherence requires invalidating all of them.

The failure mode is the invalidation storm: one hot record changes (say, a shared configuration or a user with millions of followers), the invalidation fans out to every cache copy, and every copy misses and refetches simultaneously — a stampede with the fan-out of a broadcast. And any copy that misses the invalidation message serves stale data until its TTL expires.

write DB(v2)
  └─ publish invalidation(key) to bus
       ├─ cache A: delete key → miss → refetch
       ├─ cache B: delete key → miss → refetch
       └─ cache C: message lost → serves v1 until TTL

Mitigations: versioned keys (user:123:v2 — stale readers get an old key that still exists, no refetch), bus-based invalidation with deduplication, TTL backstops on every entry, and read-your-writes routing so a writer's own requests bypass the cache.

Choosing a model

Evaluate each model by its failure path, not its happy path. Cache-aside fails by serving stale data briefly (bounded by TTL). Write-through fails by coupling your write path to two systems. Write-behind fails by losing acknowledged writes. The right model is the one whose failure mode your product can survive — and whichever you choose, the TTL is the contract that says how much staleness you ship.