The Runtime Theory
SystemInternalsdistributed systems

Raft Log Replication: AppendEntries, Majority Ack, and Commit

A step-by-step walk from a client write to the leader's log, through majority acknowledgement, to the commit index every node applies.

The Runtime Theory Team2 min read07 steps

layer stack

System

HWHardware
KKernel
RTRuntime
APPApplication
SYSSystem
CLIClient
NETNetwork
TLSCrypto
SRVServer

adjacent altitudes in this subsystem are still being traced

trace spine

  1. 01 client write lands on leader
  2. 02 entry appended to leader log
  3. 03 AppendEntries fanout
  4. 04 followers append and ack
  5. 05 majority ack advances commitIndex
  6. 06 apply to state machine
  7. 07 commitIndex propagates

Raft's write path is a single pipeline: client → leader log → majority of logs → commit index → state machine. Walk it entry by entry.

trace stepSystem
A client PUT arrives. If the receiving node is a follower, it answers with redirect to leader (or the client keeps a cached leader hint) and the request travels one extra RTT. Once on the leader, the write is handed to the Raft loop, which serializes it with every other pending write.
trace stepSystem
The leader assigns the entry the next log index (say 42) and appends it to its own log with term = currentTerm. Nothing is durable yet — in etcd the entry is persisted with fsync before the RPC fanout, which costs roughly 0.5–2ms on NVMe. The entry is now "uncommitted": it exists on one node only.
trace stepSystem
The leader sends AppendEntries to every follower in parallel, carrying prevLogIndex=41, prevLogTerm, entries=[42], leaderCommit. Each RPC is one round trip (~0.5–5ms in a datacenter). With pipelining, the leader never waits for the previous ack before sending the next batch — the in-flight window is one RPC per follower.
trace stepSystem
Each follower checks the consistency check: its log at prevLogIndex must exist with term prevLogTerm. If it matches, the follower appends the entry (deleting any conflicting suffix first) and replies success=true. If it doesn't match — the classic case after a leader change — the follower replies success=false with its own lastLogIndex, and the leader walks nextIndex back entry by entry until the logs agree. That backtracking is usually one or two RPCs, not a scan.
trace stepSystem
The leader counts success replies. In a 5-node cluster, entries up to index 42 are committed once nodes 1, 2, and 3 all hold them — 3/5. The leader advances commitIndex = 42. Critically, Raft only commits an entry from its own current term; an entry from a previous term becomes committed implicitly when a current-term entry commits past it (the "commit via overlap" rule).
trace stepSystem
The leader applies log[42] to its state machine — the key/value store, membership change, whatever the entry encodes — and returns the response to the client. Total client-visible latency: one AppendEntries RTT plus fsync, roughly 1–10ms in a healthy cluster.
trace stepSystem
The next heartbeat (≤100ms later) carries leaderCommit=42 to every follower. Each follower that has the entry up to 42 applies it in log order. A read served from a follower can now see the write. Full propagation: one heartbeat interval worst case.
text
on majority_ack(index I):
    if I > commitIndex and log[I].term == currentTerm:
        commitIndex = I
        apply log[commitIndex]
        reply to client

The mechanism costs what it costs: every write is at least one network RTT plus an fsync on the leader, and followers apply asynchronously behind the commit index. Batching many client writes into one AppendEntries amortizes that RTT — etcd batches by time window (~10–20ms of accumulated writes) precisely so throughput scales without multiplying fsyncs.