The machine here is a write path and a read path with a 100:1 ratio — everything else is detail. State the ratio first, then let every component fall out of it.
Requirements and capacity
Shorten: 10M new URLs/day (~116 writes/s average, ~500/s peak). Clicks: 1B/day → ~11.6k reads/s average, 5x at peak. Storage: 10M/day × 365 × 5 years ≈ 18 billion rows; at ~100 bytes per row (key, url, owner, timestamps) that's ~1.8TB — one large database, or sharded by key hash. State the numbers once; they justify everything after: the read path must be a cache lookup plus a 3xx, and the DB must be nearly idle.
Data model
shortened_urls: key (PK, 7 base62 chars → 62^7 ≈ 3.5 trillion keys), url, owner_id, created_at, expires_at. The key is the primary key because every read is a point lookup on exactly it. An index on url_hash enables dedup ("already shortened — return the existing key") at the cost of a second index write per shorten — decide explicitly whether that trade is worth it. Append-only clicks table for analytics, partitioned by day.
Components
- API service:
POST /shorten(validate URL, generate key),GET /:key(redirect). Stateless, behind a load balancer. - Key generation: base62-encoded counter (collision-free by construction, but a single-writer bottleneck) vs random + retry on unique violation (no hot spot; ~1-in-a-billion collisions at 50M keys). The retry is one DB round trip on a rare path.
- Cache: Redis, write-through on shorten. At 11.6k reads/s a single instance (100k+ ops/s) is idle; hit rate is 95%+ because short links don't rot — the URL behind the key never changes.
- CDN: edge caches the redirect for the top N% of links, cutting origin reads to a fraction of QPS.
The read path, concretely
GET /xK9f2Qp → CDN lookup (miss) → Redis GET (~0.5ms, hit) → on miss, single-row DB lookup (~1–5ms) → 308 with Location. Two round trips for the user; the expensive part only happens on cache miss.
Tradeoffs
- 301 vs 308: 301 is cached by browsers and proxies — great for share links, fatal for click counting, because the browser never re-asks. 308 preserves the method and is the API-safe choice.
- Dedup vs not: dedup saves storage but adds a second index and a lookup on the write path.
- TTL on links: expiring links bound table growth but break old bookmarks; most products keep keys forever and archive.
Bottlenecks
Cold-key misses (newly created links aren't cached — write-through must be immediate); the counter generator if you pick it; dedup index write amplification. The database is never the bottleneck; the cache and the edge are.