Rate Limiter Design: Token Bucket, Sliding Window, and Distributed Limits
The inner mechanics of rate limiting: we implement token bucket, fixed window, sliding window log, and sliding window counter on a single machine, then add a second machine and watch the classic failure — a client at the edge of its limit can slip through the gap between two limiter nodes. We cover where the limit state lives, how Redis changes the arithmetic, and the counter-intuitive trade-off that local counters and approximate global limits are the shape most real APIs ship with.
Topics covered:
- Token bucket: burst capacity and refill rate
- Fixed window and the boundary burst
- Sliding window log vs counter memory cost
- Distributed limits and the Redis race
- Returning 429 correctly: headers and retry-after
Related articles
Rate Limiters at Scale
Token bucket vs fixed window vs sliding log, why Redis INCR is the distributed workhorse, and the exactness-vs-latency tradeoff of global limits.
Health Checks and Observability
Active vs passive health checks, circuit breaker interplay, and the four concrete reasons a dead node still gets production traffic.
Capacity Planning: The Math
QPS times latency equals concurrency, utilization targets, and headroom: the arithmetic behind fleet sizing that survives peak traffic.
More in System Design
Load Balancers, Explained
Layer 4 vs layer 7, health checks, stickiness, and the failure modes behind the most trusted box in your architecture diagram.
WatchSearch System Design: Inverted Indexes, Tokenization, and Ranking
How a search system actually works — inverted indexes, tokenization, relevance ranking, and the pipeline between a keystroke and ranked results.
DetailsNotification System Design: Fanout, Retries, and Delivery Channels
Designing a notification system — provider abstraction, fanout, retry policies, and how push, email, and SMS channels fail differently.
DetailsChat System Design: WebSockets, Presence, and Message Ordering
Designing a chat system — WebSocket connections, message ordering, presence, and what happens to undelivered messages when a client goes offline.
DetailsURL Shortener System Design: Encoding, Storage, and Redirects
Designing a URL shortener — base62 encoding, ID generation, redirect caching, and how the write and read paths differ in scale.
DetailsMessage Queue Design: Topics, Consumer Groups, and Delivery Guarantees
How message queues actually work — topics, partitions, consumer groups, exactly-once semantics, and what guarantees the broker really gives.
DetailsDatabase Replication Topologies Visualized: Leader, Multi-Leader, and Quorum
How database replication topologies actually work — single leader, multi leader, quorum, and what happens to reads and writes when a node fails.
DetailsCaching Strategies: Cache Aside, Write Through, and Cache Invalidation
How caching strategies actually work in production systems — cache aside, read through, write through, write back, and where stale reads come from.
DetailsLoad Balancer Algorithms: Round Robin, Least Connections, and Hashing
How load balancer algorithms actually distribute requests — round robin, least connections, IP hashing, and the trade-offs each one makes.
DetailsDepth, delivered weekly
One technical dispatch a week — articles and episode notes before they go public.
One technical dispatch per week. No noise.