Chat has three separable hard parts: transport (the live connection), persistence (history), and fanout (getting a message to N receivers). Say that out loud early — it shows you know where the complexity lives.
Requirements and capacity
100M DAU, 1B messages/day (~11.6k writes/s average, ~3x at peak), p99 delivery <100ms when both parties are online, history retained forever, group chats up to 100k members. Storage: 1B/day × ~200B ≈ 200GB/day → ~73TB/year, or ~1PB over a decade — partition by chat_id and accept that the hot chats are the big ones.
Transport
WebSocket: one long-lived TCP connection per client, held by a gateway. The gateway is memory-bound, not CPU-bound: ~20KB per connection × 1M concurrent connections ≈ 20GB of RAM across the fleet — the connection is the scarce resource. Heartbeats: every 30s per connection → 33k/s at 1M connections; presence is derived from heartbeat recency. SSE and polling are the fallbacks for degraded networks, but polling inverts the latency math — the interviewer will check you know why (websocket-vs-sse-vs-polling).
Data model
messages(chat_id, message_id, sender_id, text, ts, prev_msg_id) — Cassandra-style: partition key chat_id, and message_id is a time-ordered UUID so history reads are sequential within a partition, not random seeks. memberships(chat_id, user_id, joined_at). Per-user inbox: a Redis sorted set of (ts, message_id) refs — the fanout target.
Write path
Message → ingest service → persist to history → enqueue to fanout. The queue decouples the user's publish latency from the fanout cost: the sender sees "delivered" while workers push to N inboxes. Fanout = one inbox write per member; delivery is at-least-once and clients dedup by message_id, because ordering is by message_id (original send time), not server arrival — a retried message keeps its place.
The fanout decision
Fanout-on-write means group size is the multiplier: a 100k-member group costs 100k inbox writes per message — the classic fanout-trace math. Hybrid: fanout-on-write for groups under a threshold, read-time assembly (fetch recent posts of the group and merge) for the megagroups. The threshold comes from measurement: writes-per-message vs merge-cost-per-read × read-rate.
Tradeoffs
History partitioning vs range queries; global ordering vs per-chat ordering (per-chat is enough — nobody needs a global sequence); exactly-once vs at-least-once + dedup (the client is the dedup point).
Bottlenecks
Hot-group fanout; gateway memory and reconnection storms (a gateway dying drops 50k connections that all reconnect at once); history pagination on scroll; and the presence heartbeat floor that every design must pay.