Phil Karlton famously said there are only two hard things in computer science: cache invalidation and naming things. Cache invalidation is harder because a wrong name is annoying, but a wrong cache is a bug that only manifests under concurrency. The challenge isn't storing data in a cache — it's knowing when that data is stale.
The Cache Consistency Problem
The fundamental issue: your cache and database are two separate systems with no shared transaction. A write to the database doesn't automatically update the cache.
Scenario: Cache-Database inconsistency
Time 1: App reads user #42 → cache miss → reads from DB (name="Alice") → caches it
Time 2: Another app updates user #42 in DB (name="Bob")
Time 3: App reads user #42 → cache hit → returns "Alice" (stale!)The window between Time 2 and Time 3 is the inconsistency window. The goal is to minimize it, not eliminate it (elimination requires distributed transactions, which are expensive).
Cache-Aside (Lazy Loading)
The most common pattern: the application manages the cache explicitly.
import redis
import json
class UserCache:
def __init__(self, db, redis_client):
self.db = db
self.redis = redis_client
self.ttl = 300 # 5 minutes
def get_user(self, user_id):
# Try cache first
cached = self.redis.get(f"user:{user_id}")
if cached:
return json.loads(cached)
# Cache miss: read from DB
user = self.db.query("SELECT * FROM users WHERE id = %s", (user_id,))
# Populate cache
self.redis.setex(
f"user:{user_id}",
self.ttl,
json.dumps(user)
)
return user
def update_user(self, user_id, data):
# Write to DB first
self.db.execute("UPDATE users SET ... WHERE id = %s", (user_id,))
# Invalidate cache (don't update — invalidate)
self.redis.delete(f"user:{user_id}")
# Next read will repopulate cache with fresh dataThe key decision: invalidate or update on write?
# Option 1: Invalidate (delete)
# Pro: Simple, no race conditions
# Con: Cache miss on next read (cold start)
self.redis.delete(f"user:{user_id}")
# Option 2: Update (write-through)
# Pro: No cache miss after write
# Con: Race condition — concurrent writes can set stale data
self.redis.setex(f"user:{user_id}", self.ttl, json.dumps(new_user))
# Race: Write A completes, Write B starts, Write B sets old value from cacheWrite-Through and Write-Behind
Write-Through
The application writes to both cache and database synchronously.
def write_through(user_id, data):
# Write to both atomically (as much as possible)
pipe = redis.pipeline()
pipe.setex(f"user:{user_id}", TTL, json.dumps(data))
pipe.execute()
db.execute("UPDATE users SET ... WHERE id = %s", (user_id,))
# If DB write fails, cache has wrong data
# Need compensating logic or distributed transactionWrite-Behind (Write-Back)
The application writes to cache immediately and asynchronously persists to the database.
import threading
import queue
class WriteBehindCache:
def __init__(self):
self.write_queue = queue.Queue()
self.worker = threading.Thread(target=self._flush_worker, daemon=True)
self.worker.start()
def set(self, key, value):
# Write to cache immediately
self.redis.setex(key, TTL, json.dumps(value))
# Queue DB write
self.write_queue.put((key, value))
def _flush_worker(self):
while True:
key, value = self.write_queue.get()
try:
db.execute("UPDATE users SET ... WHERE id = %s", (extract_id(key),))
except Exception as e:
# Retry or log
self.write_queue.put((key, value))
self.write_queue.task_done()tradeoff / Consistency vs Performance
Cache-aside is the default choice for most applications. Write-through suits high-read workloads where cache misses are expensive. Write-behind suits write-heavy workloads where you can tolerate some data loss.
Cache-Aside is simple and correct but has cache misses. Write-Through eliminates misses but adds write latency. Write-Behind is fastest but risks data loss on crash.
Cache Stampede
When a popular cache key expires, many concurrent requests may try to rebuild it simultaneously, overwhelming the database.
# Cache stampede scenario
def get_popular_product(product_id):
# 1000 concurrent requests all see cache miss
# All 1000 hit the database
# Database is overwhelmed
cached = redis.get(f"product:{product_id}")
if cached:
return json.loads(cached)
# All 1000 requests execute this simultaneously
product = db.query("SELECT * FROM products WHERE id = %s", (product_id,))
redis.setex(f"product:{product_id}", 300, json.dumps(product))
return productSolutions:
import functools
import time
# 1. Locking: Only one process rebuilds the cache
def get_with_lock(key, rebuild_fn, ttl=300):
value = redis.get(key)
if value:
return json.loads(value)
lock_key = f"lock:{key}"
if redis.set(lock_key, "1", nx=True, ex=10): # Acquire lock
try:
value = rebuild_fn()
redis.setex(key, ttl, json.dumps(value))
finally:
redis.delete(lock_key)
return value
else:
# Another process is rebuilding; wait and retry
time.sleep(0.1)
return get_with_lock(key, rebuild_fn, ttl)
# 2. Early expiration: Refresh before it expires
def get_early_expire(key, rebuild_fn, ttl=300, early_expire=0.1):
data = redis.get(key)
if data:
result = json.loads(data)
# Check if we should refresh in background
if result.get("_expires_at", 0) < time.time() + ttl * early_expire:
# Trigger async refresh
threading.Thread(target=refresh_cache, args=(key, rebuild_fn, ttl)).start()
return result.get("value")
return rebuild_fn()
# 3. Probabilistic early refresh
def get_probabilistic(key, rebuild_fn, ttl=300):
data = redis.get(key)
if data:
result = json.loads(data)
# Probabilistically refresh based on remaining TTL
remaining = result.get("_expires_at", 0) - time.time()
if random.random() < 1.0 / (remaining + 1):
threading.Thread(target=refresh_cache, args=(key, rebuild_fn, ttl)).start()
return result.get("value")
return rebuild_fn()Cache warming
Don't wait for cold starts. Pre-populate the cache with predicted hot data.
# Cache warming on deployment
def warm_cache():
# Get top 1000 most-queried products
hot_products = db.query("""
SELECT id, name, price, ...
FROM products
WHERE last_accessed > now() - interval '1 hour'
ORDER BY access_count DESC
LIMIT 1000
""")
pipe = redis.pipeline()
for product in hot_products:
pipe.setex(
f"product:{product['id']}",
300,
json.dumps(product)
)
pipe.execute()
print(f"Warmed {len(hot_products)} products in cache")
# Run on startup
warm_cache()Multi-Level Caching
class MultiLevelCache:
def __init__(self):
self.l1 = {} # In-process dictionary (fastest)
self.l2 = Redis() # Distributed cache (fast)
self.l3 = Database() # Source of truth (slow)
def get(self, key):
# L1: In-process (nanoseconds)
if key in self.l1:
return self.l1[key]
# L2: Redis (milliseconds)
value = self.l2.get(key)
if value:
self.l1[key] = json.loads(value) # Populate L1
return json.loads(value)
# L3: Database (tens of milliseconds)
value = self.l3.query(key)
self.l2.setex(key, 300, json.dumps(value)) # Populate L2
self.l1[key] = value # Populate L1
return value
def invalidate(self, key):
self.l1.pop(key, None)
self.l2.delete(key)Synthesis
Cache invalidation is hard because it requires coordinating two systems without shared transactions. Cache-aside with invalidation is the simplest correct pattern. Write-through and write-behind optimize for specific access patterns but add complexity. Cache stampede protection prevents thundering herds when popular keys expire. The real insight: perfect consistency is expensive, and most applications can tolerate brief inconsistency windows in exchange for the performance benefits of caching.