A transaction touches five pages scattered across a B-tree. Crash mid-write and the tree is neither old nor new — it’s shrapnel. You cannot make five random writes atomic; disks don’t offer that. But you can make one sequential append durable, and that asymmetry is the whole foundation: write what you intend to do into a log, make the log durable, then do it. Postgres WAL, InnoDB redo log, ext4’s journal, Kafka, Raft’s log, RocksDB — one idea, a dozen uniforms. Learn it once with the failure modes attached and half of storage engineering unlocks.
The Protocol
1. append to log: "txn 42: page 7 bytes 96..128 := <new>" (+ more records)
2. append: "txn 42 COMMIT"
3. fsync the log ◀── the transaction is durable HERE
4. report success
5. update the real pages in memory; write them back... eventuallyThe rule that names the technique: no data page reaches disk before the log records describing its changes (write-ahead). Recovery is then mechanical: replay the log forward from the last checkpoint (redo), and for transactions with no COMMIT record, either ignore their records (redo-only designs) or roll them back (undo). Crash at any step and you get either all of txn 42 or none of it.
Notice what got bought. Performance, not just safety: the durable
write path became sequential appends to one file — the access pattern
both disks and SSDs love — while the random-write chaos happens lazily
in memory and trickles back in the background. And group commit
compounds it: ten transactions committing within the same millisecond
share a single fsync (one flush, ten durability guarantees) — the
mechanism behind every “how does this database do 50k commits/sec on
one disk” number, and behind knobs like Postgres’s commit_delay and
Kafka producers’ batching. Latency-vs-throughput at the fsync boundary
is the tunable of durable systems: relaxations like
synchronous_commit = off (ack before flush, bounded loss window on
crash) are honest engineering trades when stated, and quiet data-loss
bugs when unstated — MongoDB’s early unacknowledged-write defaults
taught the industry that lesson publicly.
Where WAL Implementations Get Hurt
The protocol is simple; the storage stack underneath is treacherous, and the classic wounds are all documented incidents:
- fsync must mean it. Consumer drives ack writes from volatile cache; a power cut eats “durable” bytes. Barriers/FUA and honest disks (or capacitor-backed enterprise SSDs) are prerequisites, not options. Worse: Linux fsync error semantics — a failed fsync could mark pages clean, so retrying it “succeeds” over lost data. That’s fsyncgate (2018): Postgres now PANICs on fsync failure, on the theory that crash-and-recover-from-WAL beats believing a liar.
- Torn pages. An 8 KB page write isn’t atomic against 4 KB-sector power loss — half old, half new, checksum garbage. Postgres logs full-page images on first touch after each checkpoint (why WAL volume spikes right after checkpoints); InnoDB routes through its doublewrite buffer. Same threat, two armors.
- Checkpoints exist to bound recovery, and they’re the other
latency villain: flushing all dirty pages at once creates I/O storms
(hence spread/fuzzy checkpointing,
checkpoint_completion_target), while checkpointing rarely means longer crash-recovery replays. The triangle — recovery time, steady-state I/O, WAL retention — has no fourth option.
One more gift for free: the log is a replication feed. Ship WAL to a replica and replay it — physical replication, PITR archives, and (decoded) logical CDC pipelines like Debezium are all “read someone else’s WAL.” The log turns out to be the database’s most reusable interface — which is the observation Kafka generalized into a company, and Raft formalized into consensus: replicate the log, and state machines replaying it agree by construction.
Takeaways
- Random multi-page atomicity is impossible; one durable sequential append is easy — WAL converts the first problem into the second.
- Durability point = log fsync; group commit amortizes it and is where commit throughput comes from.
- The stack lies: verify write caches/barriers, treat fsync failure as fatal (fsyncgate), and armor against torn pages (FPI/doublewrite).
- Checkpoints trade steady-state I/O against recovery time; tune the spread, accept the triangle.
- The WAL is also the replication/CDC/consensus interface — the log is the database, everything else is a cache of it.