“Lock-Free” Wasn’t the Optimisation. Batching Was.

Buried in ZeroMQ’s TCP code is a single line that turns off one of the oldest optimisations in the TCP stack, on purpose, with a comment explaining exactly why. In a different part of the codebase, there’s a queue that people usually describe as “lock-free,” but what actually makes it fast has almost nothing to do with avoiding locks. Both of these are really the same decision, made twice, at two different layers of the stack.

The misconception

People tend to treat “lock-free” as a synonym for “fast.” The assumption is that if you’re not blocking on a mutex, you’ve eliminated the cost of coordinating between threads. That’s not quite right; lock-free algorithms aren’t literally free of locking. The locking just moves down to the hardware level, handled by the CPU’s cache-coherency protocol every time you run an atomic instruction like compare-and-swap (CAS).

That distinction matters more than it looks like it should, because it changes what you optimise. If you think the win is “no locks,” you stop the moment you’ve swapped a mutex for a CAS loop. If you understand the win is really “cheaper, hardware-level coordination instead of OS-level coordination,” you notice you’re still paying that hardware cost on every single operation, and you start asking whether you can pay it less often.

The actual mechanism

Start further back, with the problem ZeroMQ was actually solving. In a naive networked application, the thread that calls send() is the same thread that talks to the kernel's socket buffer. That means the thread's progress is now tied to how fast the network, or the receiver, can keep up. Application logic and network I/O share a thread, so a slow peer or a full socket buffer stalls your application code.

ZeroMQ’s fix is to never let application code touch the network directly. When you callzmq_send(), it doesn't send anything. It queues the message onto an internal pipe and returns immediately. A separate background I/O thread owns the actual socket and does the sending asynchronously, on its own schedule. The application thread is untied from the network entirely. It hands off a message and moves on.

That handoff, from application thread to I/O thread, now sits on the critical path of every message your application sends. So the pipe connecting the two threads has to be extremely fast. The obvious implementation is a queue guarded by a mutex, but that reintroduces exactly the blocking you just tried to eliminate: now the application thread can stall waiting for the I/O thread to release a lock, instead of stalling waiting for the network.

So the pipe is built lock-free instead, using CAS to publish and consume items without ever taking an OS-level lock. Here’s the part that’s easy to miss though: a CAS operation per message is still too expensive once your message rate gets high enough, because every CAS forces a synchronization event across CPU cores. Doing that once per message caps your throughput far below what the hardware could otherwise sustain.

The actual fix is batching. The writer thread accumulates a run of messages into a small buffer that only it can see (no synchronisation needed there, since nothing else can touch it yet), then does a single CAS to publish the entire batch to the reader thread at once. The hardware synchronisation cost gets paid once per batch instead of once per message. The batch size grows automatically under load too: more backlog means more messages get folded into each CAS, which is exactly when you need the amortisation most.

One more constraint makes this tractable. Each pipe has exactly one writer thread and one reader thread, always. Building a lock-free structure that safely handles many writers and many readers at once is a much harder problem, full of subtle correctness bugs, than one with a single writer and single reader. ZeroMQ sidesteps that problem entirely. When a socket needs to fan out to multiple peers, it doesn’t build one cleverer shared structure. It just creates more single-writer, single-reader pipes, one per peer.

Now follow a message the rest of the way. It’s been batched and handed to the I/O thread, which writes it to a TCP socket. TCP has its own, much older batching mechanism: Nagle’s algorithm, which holds small writes back and coalesces them before sending, trading latency for fewer packets. ZeroMQ turns this off, explicitly, socket by socket, using TCP_NODELAY. The reasoning is spelled out directly in the source comment: the library is already doing data batching at its own level, so Nagle's algorithm wouldn't add any throughput. It would only add latency, because now two independent, uncoordinated batching mechanisms are stacked on top of each other, each deciding on its own schedule when to actually let bytes go.

That’s the real design principle here, stated as plainly as source code ever states anything. Batching should happen at exactly one layer, the layer that actually has visibility into whether there’s a backlog worth batching, and every other layer downstream should stay out of the way.

Why it matters

This exact failure mode shows up constantly outside of ZeroMQ, usually as a mystery. The instinct to reach for another buffering knob is natural. The actual fix is almost always to figure out which single layer should own the decision, and stop letting others quietly second-guess it.

The pattern generalises well past networking. An application-level write buffer, a database client’s batching layer, an OS-level page cache, and a storage device’s own write buffer can all coexist in one write path, each with its own idea of when to flush. None of them is individually wrong. Stacked without coordination, they produce exactly the kind of latency that’s miserable to debug, because each layer looks correct in isolation and the interaction only shows up as an aggregate number nobody can trace back to a cause.

Key takeaways

• Lock-free doesn’t mean synchronisation-free. CAS moves the coordination cost to CPU-level cache coherency; it doesn’t eliminate it.
• The performance win in a lock-free pipe usually isn’t the CAS itself. It’s reducing how often you have to pay for one, by batching messages behind a single CAS instead of doing one CAS per message.
• Decoupling an application thread from network I/O (queue and return, instead of a blocking send()) is what frees application logic from network speed. A fast internal pipe is what makes that decoupling viable on every message, not just occasionally.
• Restricting a lock-free structure to exactly one writer and one reader sidesteps a much harder correctness problem. Fan-out gets solved with more pipes, not a cleverer shared structure.
• Disabling Nagle’s algorithm isn’t a general “always do this for low latency” tip here. It’s a specific consequence of already owning batching at a higher layer. Turning it off makes sense because something else is already doing that job.
• When multiple layers in a system can each independently decide to buffer or delay, pick exactly one layer with enough visibility to make that call correctly, and explicitly disable the others rather than letting them interact by accident.