A distributed cache is a partition problem wearing a speed problem's clothes. The machine is simple — get/set with TTLs — but where a key lives, what happens when the node holding it fails, and what happens when the key isn't there are where the interview happens.
Requirements and capacity
KV cache fronting a DB: sub-ms reads at 500k reads/s, 100k writes/s, working set 5TB, average value 50KB → 25GB/s of read bandwidth. 5TB / 64GB per node ≈ 100 nodes with ~50% headroom for replication and rebalancing. At 100k ops/s per node, throughput is fine; the binding constraints are bandwidth and node-failure behavior.
Partitioning: consistent hashing
Hash the key into a ring; each node owns virtual-node ranges. A node join/leave moves only ~1/N of keys — at 100 nodes, ~1% — versus range sharding where a split migrates a contiguous chunk and the two neighbors absorb the storm. The cost: routing is a client-side table (O(log vnodes) lookup), and two clients with slightly different views can double-write a key — version every value and read-repair.
Replication and consistency
Two or three replicas, primary-replica layout: writes to the primary, reads from either, stale reads bounded by replica lag. Quorum (R+W > N) buys linearizability at the cost of more round trips — decide per key class, not globally. The cache is not the system of record: cache-aside (app fills on miss) is the default because it lets the app control what gets cached and when.
Data model
key → (value, ttl, version). TTL is logical expiry (the source of truth changed); LRU/LFU is capacity-driven. Redis mixes them: volatile-lru evicts among TTL'd keys — the TTL expresses freshness, the eviction policy expresses memory pressure.
The miss path is the design
A cache is a filter; its value is the miss rate it prevents. On miss: single-flight — one in-flight DB fetch per key, everyone else waits on that promise, not their own query. Without it, 10k concurrent misses on a cold key is 10k DB queries — the thundering herd. Jittered TTLs (±5–10%) stop synchronized mass expiries from stampeding the DB at midnight. Both are cheap and interviewers expect them.
Hot keys
One key at 500k QPS exceeds any single node. Options: replicate the hot key (same value on N nodes, client hashes key+replica) — best for read-mostly; local L1 cache at the app — best for skewed access; or accept the skew and size the node for it.
Bottlenecks
Single-node throughput (~100k ops/s per Redis instance — shard by key hash beyond that); bandwidth for large values (a 1MB value at 10k QPS is 10GB/s — the network, not the node, fails first); rebalance storms on membership change; and stampedes on expiry. State the failure order: node dies → 1% of keys rehash → replicas absorb reads → rebalance drains the new replica — latency during that window is the real cost.