A database crash is not an error state — it's a designed-for contingency. The write-ahead log exists so that when power dies mid-INSERT, the system can rebuild exact state from the last safe line. Recovery is the machine reading that log backward-to-forward: find the checkpoint, replay to the end, open the doors. The whole thing is a race between durability choices you made weeks ago (checkpoint cadence) and seconds of your time now.
Step 1 — crash and startup
Power loss, kernel panic, kill -9, or a failed pg_ctl restart: shared memory is gone, the buffer pool's dirty pages are whatever made it to disk, and the WAL has the truth. On startup the postmaster reads pg_control — a tiny, constantly-updated file holding the system's state: the current timeline, the checkpoint record's LSN, and the database state flag (in production → in recovery). If pg_control is unreadable/corrupt, recovery can't locate a safe starting point, and you're in backup-restore territory.
Step 2 — the checkpoint as starting line
The checkpoint record is the anchor. Every checkpoint flushes all dirty buffer-pool pages and writes a record: "everything before this LSN is on disk." Recovery can therefore ignore everything before the checkpoint and start replay from its LSN. The tradeoff is baked in: recovery time ≈ WAL volume since the last checkpoint. A checkpoint every 5 minutes with heavy write traffic = minutes of replay; hourly checkpoints on the same workload = an hour+ of replay. This is the knob pair to tune together: max_wal_size (roughly how long between checkpoints) vs acceptable downtime.
Step 3 — redo begins
Recovery reads WAL segments starting at the checkpoint LSN, in order, validating each record's CRC. Corrupt or missing WAL between the checkpoint and the end is fatal (requested WAL segment ... has already been removed = your wal_keep_size/archiving config failed you). Parallel workers (Postgres 16+) can replay independent pages concurrently — the binding constraint is usually reading WAL and writing pages, not CPU.
Step 4 — replaying records
Each WAL record is a physical delta: "page 16385/block 42, bytes 100-200 ← ...". Replay locates the page — in the buffer pool if present, read from disk otherwise — applies the delta, marks it dirty, and does not fsync after each record: replay batches writes and lets the normal checkpoint flush them. If the database crashes during recovery, the next recovery simply starts again from the same checkpoint: redo is idempotent — applying the same record twice is safe because each record's page-LSN check (the page's LSN vs the record's) skips already-applied updates. This is the crucial design property: recovery of a recovery is fine.
Step 5 — torn pages
If the machine lost power during an 8KB page write, the page on disk may be half old, half new — a torn page. Left undetected, replay would apply deltas on top of garbage. Defense: full_page_writes — after each checkpoint, the first write of a page logs the whole page in the WAL, so a torn page is detected (page header LSN newer than the partial data implies — the CRC fails) and rebuilt from the full-page image in the log. That's also why full_page_writes causes WAL bloat right after checkpoints: every first-dirty page is logged whole.
Step 6 — no undo
Here's the elegant part: Postgres doesn't roll back anything. The WAL contains records of uncommitted transactions too (they were logged before commit was decided). What happens to their changes? Nothing — MVCC means their effects are tagged with an XID that is not committed, so they're invisible to every snapshot and will be reclaimed by vacuum. InnoDB works differently: it replays redo, then uses the undo log to actively roll back incomplete transactions' changes to pages. Same guarantee, different cleanup: Postgres lets dead data age out; InnoDB erases it during recovery.
Step 7 — the doors open
Replay reaches the end of WAL (the end-of-recovery record is written, timeline marked consistent), the WAL is trimmed (pg_resetwal territory if corrupt), and the database transitions to normal operation. Recovery time appears as a startup delay — pg_ctl status shows "database system is starting up" while it works.
What it costs
- Base cost: reading pg_control + finding the checkpoint: ~ms.
- Replay: roughly WAL bytes / replay bandwidth (100s of MB/s on SSD — often faster than the crash's write rate).
- The checkpoint trade:
checkpoint_timeout5min +max_wal_size1GB → sub-minute recovery on typical workloads.
$ grep -i "redo\|checkpoint\|recovery" /var/log/postgresql/postgresql.log
LOG: database system was interrupted; last known up at 2026-08-18 14:02:11 UTC
LOG: redo starts at 4/2F100000
LOG: invalid record length at 4/2F1E4A08: wanted 24, got 0
LOG: redo done at 4/2F1E49D0 system usage: CPU: user: 2.31 s, system: 1.28 s
LOG: database system is ready to accept connections"Interrupted at 14:02" to "ready" in ~4 seconds of CPU. Recovery isn't scary when the WAL is healthy — it's a deterministic replay of records you paid to write earlier. The scary version is missing WAL, and that's a backup-and-archiving problem, not a recovery problem.