A partition is the event every consensus protocol is designed for. What makes it interesting is the heal: a split cluster doesn't just resume — the minority must surrender, catch up, and rejoin under the majority's authority. Trace a 5-node Raft cluster cut in half.
trace stepSystem
A switch fails. Nodes 1–3 (site A) and 4–5 (site B) can no longer reach each other. Neither side knows it's partitioned at first — detection takes one missed heartbeat window: heartbeats every 100ms, election timeouts at 150–300ms. Both sides run normally for a few hundred milliseconds, the last shared reality being term T, leader = node 1.
trace stepSystem
Node 1's heartbeats stop reaching 4 and 5; 4 and 5's replies stop reaching 1. Everyone's election timers start. The important fact: nobody can tell "the leader is dead" from "the network is cut" — the mechanics are identical from the nodes' point of view.
trace stepSystem
Nodes 4 and 5 time out and enter elections. They can vote for each other — but 2 votes is not a majority of 5. No candidate reaches quorum. The minority side becomes read-only at best (Raft blocks even reads without a leader lease), and its pending client writes fail with timeouts. This is quorum loss, and it is correct behavior: two nodes must never decide anything a 5-node cluster has to agree on.
trace stepSystem
Nodes 1–3 run their own election. Node 2's timer fires first, it wins 2–3 votes (its own + node 1's or 3's), and becomes leader of term T+1. The majority side now has a live leader, a quorum, and the most complete log. Writes resume — this is the "at least one side keeps serving" half of the CAP bargain.
trace stepSystem
Client traffic re-routes to the majority side, which serves and replicates normally — every write commits to at least 2 of 3 nodes, so durability within the majority is intact. The minority's log is now stale: it is missing every write since the partition. Crucially, the stale side's leader-era commits are safe: Raft's election rules ensured the new leader's log covered everything committed before the cut.
trace stepSystem
The switch is fixed. The minority nodes' heartbeats to the majority suddenly succeed. Their stale election timers never fire again — the first heartbeat from the new leader resets them, carrying a higher term that forces them back to follower status. Node 1, the old leader, also steps down when it sees the higher term.
trace stepSystem
The rejoined nodes discover they are behind: their log ends at index 200, the leader is at 500. The leader streams missing entries (or a snapshot if the gap is large — snapshots kick in when the log gap exceeds a threshold like 10,000 entries). Nodes 4 and 5 apply the backlog — at typical apply rates of thousands of entries per second, a 10-minute partition heals in seconds. Once their
matchIndex catches up, they rejoin the quorum: the cluster is 5 again, serving with full durability.on first heartbeat after partition:
if leaderTerm > myTerm: step down to follower
accept AppendEntries from nextIndex
stream gap → snapshot if gap > threshold
apply until matchIndex == leader.commitIndex
resume normal replicationThe complete arc — cut, lose quorum, elect, keep serving, heal, catch up — takes a few hundred milliseconds of outage on the majority side and a few seconds of catch-up on the minority side, and it is why the answer to "what happens when the network fails?" is "exactly this, every time."