fsync and durability guarantees, Part 3: Making commits fast and safe

How PostgreSQL turns the brutal “PANIC on fsync error" rule into a system that is both fast and durable: WAL-first commits, group commit, checkpoints, and full-page writes, and how message queues end up making the very same trade-offs.

Quick recap of Parts 1 and 2

The story so far:

• **Part 1**: write() only reaches the OS page cache; fsync is the syscall that pushes data all the way to durable media; and on a writeback failure the kernel drops the dirty page and reports the error at most once.
• Part 2: PostgreSQL trusted fsync in a way that could silently lose committed data (fsyncgate), and the fix was to PANIC on any fsync error and replay the WAL, never to retry.

That fix raises an obvious question. If a database is willing to crash itself on any fsync error and rebuild from the WAL, then the WAL had better be a complete, durable, and cheap record of every change. This part is about how PostgreSQL makes all three true at once, and how the same ideas show up in message queues.

How Postgres actually achieves durability

Knowing the trap is only half the story. The other half is the positive design that lets PostgreSQL deliver fast, durable transactions despite sitting on top of all this volatility.

The core invariant: WAL-first

PostgreSQL does not make table pages durable at commit time. It makes the write-ahead log durable at commit time.

That single sentence is the answer to “why is PostgreSQL fast even though it’s safe.”

When a transaction commits, the only thing that must hit disk is the WAL record describing what changed. The actual changed table page can stay dirty in memory for minutes. We’ll get it to disk later, at our leisure.

If we crash before the page is written, the WAL has everything we need to reconstruct it on restart.

The layers a write passes through

When a backend modifies a row, the data travels through several caches, and several Postgres processes cooperate to manage them:

Notice that PostgreSQL has its own buffer cache sitting on top of the OS page cache. This “double buffering” looks wasteful at first but is actually deliberate: it gives Postgres full control over eviction order, locking, and concurrency at the granularity of an 8 KB Postgres page.

What COMMIT actually does

When you run BEGIN; UPDATE ... ; COMMIT;

Phase 1: backend modifies a page in shared buffers

The backend finds the target heap page (loading it from disk into shared buffers if necessary), takes the appropriate locks, modifies the tuple in place, and marks the buffer dirty.

At this point:

• The page is changed in Postgres memory.
• The underlying table file on disk is untouched.
• The OS page cache for that file is untouched.
• The only current copy is in shared_buffers.

Phase 2: generate WAL records

Before the modified page can ever be written out, Postgres must generate WAL records describing the change: the tuple update, any index modifications, visibility map changes, etc. These records go into the WAL buffers in shared memory.

Each modified page is tagged with a page LSN (Log Sequence Number), essentially a position in the WAL. The LSN means: “this page reflects changes up through this point in the log.”

Phase 3: COMMIT writes a commit record

When the backend executes COMMIT, Postgres emits a special commit WAL record. But a record in the WAL buffer is still just RAM. It needs to become durable.

Phase 4: flush the WAL

This is the moment that matters for durability. Postgres calls its configured WAL sync method, the **wal_sync_method GUC, which defaults to `fdatasync** on Linux and FreeBSD,fsync_writethrough` on macOS, and **open_datasync** on most other platforms, to push the WAL through the OS page cache, through the block layer, through the device cache, all the way to durable media. Only then is the commit considered durable.

With the default **synchronous_commit=on**, the client doesn't get "committed" back until this flush completes.

Phase 5: return success

At the moment the client sees COMMIT OK:

• WAL is durable through the commit record.
• The heap page may still only exist in shared_buffers.
• The data file on disk is stale.

And that’s fine. Because of the WAL invariant, we can always rebuild the page from the log if we crash.

The whole sequence looks like this:

Why this design is fast

WAL writes are wonderful for performance:

• Append-only. No random seeks.
• Sequential. Even spinning disks love this.
• Batchable. Many records can be flushed in a single sync.
• Small. A WAL record is usually much smaller than the page it describes.

Compare that to forcing every changed table page to disk on commit:

• Scattered across many files.
• Random access patterns.
• Large (typically 8 KB per page).
• Hard to batch.

If a database had to do the second thing on every commit, OLTP performance would be orders of magnitude worse. The WAL is the single most important performance optimization in mainstream relational databases, and it only works because of the WAL-first invariant.

Group commit: one fsync for many transactions

The flush itself, fdatasync on the WAL file, is the expensive part. A spinning disk might manage a few hundred per second; even a fast NVMe caps out in the tens of thousands. If every committing transaction issued its own flush, commit throughput would be hard-capped at the device's flush rate, no matter how many cores or connections you have.

Postgres sidesteps this with group commit, and the whole trick rests on one fact about the kernel. **fdatasync(fd) does not operate on "the bytes this backend wrote." It operates on the file. When a backend asks the kernel to `fdatasync** the current WAL segment, the kernel flushes *every* dirty page of that file down to the storage hardware. It has no idea, and no way to find out, that this page was dirtied by backend A and that one by backend B. If their commit records sit in the same WAL file (and they do: WAL is a single append-only stream), onefdatasync` makes all of them durable in a single I/O. So "Flush the WAL up to LSN X" isn't a Postgres feature, it falls out of how the syscall already works.

How Postgres capitalizes on it: the LSN leader

Postgres layers a small amount of coordination on top of that fact.

1. Shared WAL buffers. Committing backends append their commit records to the same in-memory WAL buffer, each record landing at a monotonically increasing LSN (log sequence number). A backend’s commit is durable the moment the WAL has been flushed past its own commit LSN.
2. Serialize on WALWriteLock. To write and flush the WAL, a backend must hold an internal lightweight lock, WALWriteLock. Before taking it, the backend checks the shared "flushed up to" position. If someone else has already flushed past its LSN, it's done and returns immediately, never touching the disk.
3. The leader flushes for everyone. Whichever backend acquires the lock first becomes the leader. It doesn't flush just its own bytes; it flushes the WAL up to the latest record sitting in the buffer at that moment, sweeping in the commit records of every backend that queued up behind it. One write(), one fdatasync.
4. The free ride. While the leader blocks waiting on the disk, the followers wait on **WALWriteLock (through a special primitive, LWLockAcquireOrWait). When the leader finishes, it updates the shared flushed-LSN and releases the lock. The waiters wake up, recheck that position, and find the WAL has already been flushed past their own commit LSN, so they return `COMMIT OK` to their clients without ever acquiring the lock or issuing any I/O of their own.** (Only a backend whose LSN still isn't covered actually takes the lock, and it becomes the next leader).

So instead of N backends performing N blocking syncs, one backend takes the hit and the other N-1 ride along for free. A bottleneck that would otherwise scale linearly with the commit rate collapses into a single batched sync.

This is group commit. It turns the flush cost from “per transaction” into “per batch,” which is how a Postgres instance commits thousands of transactions per second on hardware that can only do a few hundred flushes per second.

The batching is automatic. Even at the default **commit_delay = 0, any backend that becomes ready to commit while a flush is already in progress rides along in the next batch for free. The optional `commit_delay** (gated bycommit_siblings`, default 5) tells the leader to pause for a handful of microseconds before it flushes, so an even larger group can accumulate behind it: a sliver of latency traded for a bigger batch. It's zero by default because at high client counts the batching forms on its own without any deliberate wait.

“Wait, doesn’t batching break durability?”

This is the natural objection, and it’s worth answering precisely because the answer hinges on a distinction that’s easy to miss.

If a transaction writes its commit record and then waits for someone else’s flush, isn’t it now hostage to that other flush? And aren’t we exposing more transactions to a single point of failure?

The short answer: no, because group commit does not defer durability. It merges fsync calls.

Every committing transaction still obeys the same contract:

Before I return **COMMIT OK** to the client, my commit record must have been flushed all the way to durable media.

Group commit preserves that contract exactly. As the leader mechanism above shows, a follower returns COMMIT OK only after the WAL has been flushed past its own commit LSN; the flush simply happened to be issued by a different backend. Two properties make that safe:

• The wait is on the disk, not on a timer. Followers block on WALWriteLock for microseconds while a sync that was already in flight completes. They aren't parked waiting for a batch to fill up; they're piggybacking on a syscall that was going to run anyway.
• The flush is all-or-nothing for the batch. The OS is flushing a single contiguous byte range of one file: either it lands or it doesn’t. There’s no partial-success state where some transactions in the batch are durable and others aren’t. On success, every backend whose commit LSN is covered returns COMMIT OK . On failure, every one of them sees the error and Postgres PANICs.

Notice what didn’t happen: nobody returned COMMIT OK before their bytes were durable. Each transaction's durability requirement was satisfied before its acknowledgment was sent. The "batching" is purely on the syscall side, client-visible behavior is identical to the no-batching case, just faster.

What about the increased failure surface?

This sounds bad, “if the flush fails, 50 transactions die instead of one!”, but the surface area is identical, just rearranged.

A failing disk doesn’t care whether you call **fdatasync* once or a hundred times. If the underlying media is bad, every flush to that region fails. Group commit changes the granularity of failure batches, not the probability* of failure.

And in Postgres specifically, the granularity argument is moot anyway: when fdatasync on WAL returns an error, the response is PANIC, the entire server crashes and restarts. From PANIC's perspective, the question "did this batch contain 1 transaction or 50?" is irrelevant; the server is going down either way and will recover from the last known-good WAL position.

The cousin that actually does trade durability for speed

It’s worth naming what group commit is not, because it’s adjacent to something that genuinely is a durability tradeoff:

synchronous_commit = off

With that setting, the backend returns COMMIT OK before the WAL flush completes. A crash within the next ~200 ms (the default **wal_writer_delay) silently loses committed transactions. That's the actual unsafe version. Group commit is not that. Group commit gives you full `synchronous_commit=on** durability and amortizes the syscall cost as a free side effect of howfdatasync` works.

Checkpoints: bounding crash recovery time

If we only ever flushed WAL and never flushed dirty data pages, two problems would grow without limit:

• The WAL would get huge, because we couldn’t safely delete any of it, it’s still needed for recovery.
• Crash recovery would get slower and slower, because there’s more to replay.

Checkpoints solve both. Periodically, the checkpointer process:

1. Identifies all dirty pages in shared_buffers.
2. Writes them out to the appropriate data files.
3. Calls fsync on those files to make them durable on disk.
4. Records a checkpoint marker in the WAL.

After a checkpoint completes, all WAL older than the checkpoint marker can be safely discarded, there’s nothing in it that isn’t already reflected in the on-disk data files. Crash recovery only ever needs to replay WAL from the most recent successful checkpoint forward.

Checkpoints trade ongoing I/O for bounded recovery time. They’re a background durability mechanism, not part of the commit hot path.

Background writer vs. Checkpointer

These two are often confused, they’re not the same thing:

• Background writer: A smoother. It writes some dirty buffers gradually so that when a backend needs to evict a buffer (to load a new page into the pool), it usually finds clean ones available. Not a durability mechanism.
• Checkpointer: The durability organizer for data files. Writes dirty pages and fsync relation files in batches at checkpoint boundaries. This is where almost all data-file fsync happen.

A subtle gotcha: Uncommitted pages can reach disk

A page containing changes from a transaction that has not committed yet can still be written to disk.

Why is that safe? Because Postgres doesn’t rely on “only committed bytes reach disk.” It relies on:

• MVCC visibility rules,
• Transaction status records (committed vs. aborted), and
• WAL replay semantics.

If a page with an uncommitted tuple is on disk and then the transaction aborts (or crashes before commit), the physical tuple is there, but the visibility rules treat it as invisible, and the vacuum process will eventually reclaim it.

In database-theory terms, this is a steal + no-force design:

• Steal: dirty pages from uncommitted transactions may be written before the transaction commits.
• No-force: at commit time, we are not forced to write all the transaction’s dirty pages.

Both choices are wins for performance. Both are only safe because the WAL is the source of truth.

Torn pages and full-page writes

Here’s a hazard we haven’t covered. Postgres pages are 8 KB. Most storage devices guarantee atomic writes of 512 bytes or 4 KB, not 8 KB. If power dies in the middle of writing an 8 KB page, the on-disk copy might be half-old and half-new. Useless.

Postgres handles this with full-page writes. On the first modification of a page after each checkpoint, Postgres writes the entire 8 KB page image into the WAL, not just the delta. If recovery encounters a torn page, it can restore the full image from the WAL and then replay the incremental changes on top.

This is why your WAL is bigger than the raw size of changes might suggest, especially right after checkpoints. It’s also why **full_page_writes=off** is dangerous on most consumer hardware.

Crash scenarios: does the design actually work?

Let’s walk through the cases.

Case 1: commit succeeded, data page never reached disk

1. Backend modifies page in shared_buffers.
2. WAL records written.
3. Commit record flushed durably.   ← client sees COMMIT OK
4. Crash happens before heap page is written to disk.

On restart:

• Replay WAL from last checkpoint.
• Find the commit record; transaction is committed.
• Apply the logged changes to the heap page.
• The transaction’s effects are restored.

Case 2: data page reached disk, commit record did not

1. Backend modifies page; page dirtied; later written to disk.
2. Crash happens before commit record is flushed.

On restart:

• Replay WAL: No commit record for this transaction is found.
• Transaction status records mark it as aborted (or unknown).
• The physical tuple changes on disk are not treated as visible.
• MVCC + future vacuum clean them up.

The physical bytes on disk are not the whole truth. Commit status is.

Case 3: torn page during a crash

1. Power dies while an 8 KB page is being written.
2. On-disk copy is half-old and half-new, corrupt.

On restart:

• Recovery finds the page-image WAL record written at the first modification after the last checkpoint.
• Replaces the torn page with the logged image.
• Replays subsequent changes on top.

In every case, the WAL is what keeps the database honest.

How fsync is used asymmetrically in Postgres

Now we can connect this all back to the OS layer:

• WAL files are fsync’ed (or fdatasync, depending on **wal_sync_method**) very frequently, on every commit, in the worst case. This is in the latency-critical path of every transaction.
• Data files are fsync’ed only at checkpoints (plus some edge cases like file creation and truncation). They're not on the commit hot path.

This asymmetry is the entire performance trick. WAL sync is per-commit-ish and data-file sync is checkpoint-ish.

When people complain “my commits are slow,” the bottleneck is almost always one of:

• WAL flush latency
• storage device flush behavior
• wal_sync_method choice for the OS
• Group commit not amortizing as well as it could

It’s rarely about table-file writes.

fdatasync vs fsync: The cheaper sync, and why preallocation matters

Postgres’s group commit, and every batching scheme we’re about to see in message queues, treats “the flush” as one expensive operation. But there are actually two flush syscalls, and picking the right one is worth up to roughly 2× on commit throughput. This is the single most common low-level tuning detail across WAL-based systems, so it’s worth pulling apart on its own.

What each one actually guarantees

• **fsync(fd)* flushes the file's data blocks and* all of its inode metadata: the size, the block map, and the timestamps (mtime, ctime, atime).
• **fdatasync(fd) flushes the data blocks and **only the metadata a reader needs to get that data back. Per POSIX and the Linux man page, that explicitly excludes the timestamps, but it still includes the file size when a write has extended the file, because you cannot correctly read bytes the inode doesn't yet know exist.

So **fdatasync** is not "skip all metadata." It's "skip the metadata a reader doesn't need," which in the steady state means the timestamps. That sounds like a rounding error, a few bytes of inode, until you see what those bytes cost on a journaled filesystem.

Why skipping metadata is worth a whole disk round trip

On a journaled filesystem (ext4, XFS, the realistic production options), a metadata change is not a free in-place edit. The change is first recorded in the filesystem’s own journal, and committing that journal entry durably needs its own sync barrier: a separate flush command pushed all the way to the device. So the real cost breakdown is:

• fdatasync, nothing reader-visible dirty besides data ≈ one barrier: push the data blocks, flush the device cache, done.
• fsync, or any sync after the file changed size ≈ two barriers: the data blocks plus a filesystem-journal commit for the metadata.

The extra cost is almost never the metadata bytes (the inode delta is tiny). It is the extra round trip to the device. A sync barrier is a full-pipeline flush, the kind of thing that costs a millisecond on a spinning disk and eats a meaningful slice of an SSD’s IOPS budget. Pre-WAL OLTP benchmarks on ext4 typically show fdatasync running 1.2×–2× faster than fsync for the same workload; the "roughly halving" figure is the upper end of that range.

The catch, and why WAL files are preallocated

fdatasync only avoids the second barrier if the file's size didn't change. An append that extends the file updates **st_size*, which a reader genuinely needs, so even fdatasync is forced to commit that metadata, dragging the journal barrier right back in. A naive "append a record, fdatasync" loop therefore pays the two-barrier price on every single commit*, and gets none of the benefit.

That one fact is why mature WAL implementations preallocate their segment files:

• PostgreSQL writes WAL in fixed 16 MB segments, fills new segments with zeros up front, and recycles old segments by renaming them rather than creating fresh files.
• Kafka uses preallocated, fixed-size log segments (log.preallocate=true);
• BookKeeper's journal (the storage layer under Pulsar) behaves the same way.

Once a segment is already at full size, every subsequent commit writes into already-allocated blocks of an already-sized file. No inode field a reader cares about changes, no journal commit is triggered, and **fdatasync stays on its single-barrier fast path for the entire life of the segment. Seen this way, preallocation isn't really about saving allocation latency, it is specifically what makes `fdatasync`* viable as a per-commit* sync method at all.

Message queues: the same primitive, a different trade-off

PostgreSQL is not the only system that needs to think about fsync carefully. Any system that promises "if I acknowledged this, you can crash and we won't lose it" faces the same problem.

Message queue and streaming systems, Kafka, RabbitMQ, NATS JetStream, Apache Pulsar, ActiveMQ, all sell some flavor of that promise. And they all have to wrestle with the same trade-offs.

Why write() alone isn't enough

If a broker calls write() to append a message to a log file and immediately acks the producer:

• The data is in the OS page cache, not on disk.
• A power loss in the next few seconds loses the message.
• The producer thinks the message is safe; downstream consumers never see it.

This is the same failure mode that motivates fsync for any durable system.

Why per-message fsync is too expensive

A queue that does one fsync per message caps its throughput at the device's flush rate, which is often 10–1000× below the device's raw write bandwidth. Unacceptable for high-throughput systems.

How real systems handle the trade-off

1. Group commit / batching: Exactly the same trick as PostgreSQL. Accept N messages or wait T milliseconds, then fsync once and ack the entire batch. Latency goes up by a few milliseconds, but throughput goes up by orders of magnitude.
2. Replication as the durability primitive. Kafka, by default, does not fsync on every message. Instead, a message is "committed" when it has been replicated to N in-sync replicas, all in RAM/page cache, none of them having necessarily synced. The bet is that simultaneous power loss across N independent machines is much rarer than single-machine failure. Kafka exposes flush.messages and flush.ms for operators who want both replication and per-N fsyncs as a safety net.
3. Tiered guarantees. RabbitMQ’s classic queues batch persistence over hundreds of milliseconds and may ack publishes before fsync. Its quorum queues are stricter: a publisher confirm means the message has been written and fsync'd on a quorum of nodes. Different durability SKUs for different use cases.
4. Append-only logs. Almost every queue is fundamentally an append-only log. Segments are immutable once full. Only the current tail needs syncing. Old segments are durable simply because nothing changes them.
5. Direct I/O. Some systems (BookKeeper, ScyllaDB) bypass the OS page cache entirely with O_DIRECT. They manage their own buffering and submit I/O explicitly. Sidesteps the page-cache error-handling problems from fsyncgate, at the cost of giving up the OS's read caching.

The shared lesson

Whether you’re a relational database, a message queue, an event store, or a file-format library, the rules are the same:

• write() is a promise to copy bytes into RAM, nothing more.
• fsync is the only standard primitive that pushes data all the way through to durable media.
• fsync is expensive. The cost can usually be amortized by batching many logical operations into one physical sync.
• fsync failures mean data loss has already happened. Recover from your durable log. Don't retry and hope.

Closing the series

We started at the bottom, in Part 1, with a write() that only reaches RAM and fsync that is the sole, expensive bridge to durable media, and we watched a correctly-behaving kernel quietly drop a failed page and report the error at most once. In Part 2 we saw that collide with PostgreSQL's architecture in fsyncgate, where a retried fsync returning success was enough to lose a committed transaction, and why the only safe answer turned out to be "PANIC and replay the WAL." And in this part we saw the positive design that makes that answer affordable: WAL-first commits, group commit, checkpoints, full-page writes, and the fdatasync-plus-preallocation trick, the same toolkit that message queues from Kafka to Pulsar reach for when they make the same promise.

If there is one habit to carry away from all of this, it is a healthy distrust of the word “success.” A successful write() means "copied into RAM." A successful fsync means "no new error since you last asked," not "your data is safe forever." Durability is never a single return code; it is a property of a system that knows where its authoritative log lives, flushes it deliberately, and treats any failure to do so as a reason to stop rather than to retry.

Build as if every fsync will eventually fail, because on a long enough timeline one will, and the systems that survive it are the ones that decided in advance what to do when it does.

Further reading

• PostgreSQL documentation. The chapters on *Reliability and the Write-Ahead Log and [WAL Configuration](https://www.postgresql.org/docs/current/wal-configuration.html)* are excellent and surprisingly approachable.
• PostgreSQL source. [**src/backend/access/transam/xlog.c](https://github.com/postgres/postgres/blob/master/src/backend/access/transam/xlog.c) (the WAL machinery, including the group-commit "leader" logic around WALWriteLock) and `[src/backend/storage/sync/sync.c`](https://github.com/postgres/postgres/blob/master/src/backend/storage/sync/sync.c)** (the post-fsyncgate sync handling).
• The message-queue side. The Kafka design and durability docs (acks, flush.messages, replication); the Apache BookKeeper configuration behind Pulsar's journal (journalSyncData, journalAdaptiveGroupWrites) and the NATS JetStream storage internals, all of which lean on batching and replication rather than a per-message fsync.