Reducing a C++ kdb+ Market Data Feeder’s Latency from 140 µs to 56 µs

Notes from a personal project, with measurements at each step.

I’d been curious about kdb+ and low-latency C++ patterns for a while, and the best way I know to actually learn something is to build with it. So I started a side project: a C++ market data feeder, built from scratch in my own time, consuming quotes from a synthetic pricing engine over a TCP-based ASCII protocol, parsing them, and publishing the records into a kdb+ tickerplant.

The first working version measured around 140 µs of internal processing latency, from TCP receive to the moment the parsed quote was ready to write into kdb+. After some focused work, that came down to about 56 µs.

This post walks through what I changed, what I measured, and what surprised me. None of it is exotic — there are no kernel-bypass NICs or FPGAs involved. The improvements came from three or four reasonably routine decisions, made in the order the measurements suggested.

Architecture

The pipeline has three threads:

1. An asio I/O thread reading the TCP socket and parsing the ASCII format
2. A processor thread consuming parsed quotes from a lock-free queue
3. A writer thread that submits quotes to kdb+

Between the processor and writer, there was originally an asio::post indirection — the writer ran on its own io_context so the processor could submit a task and return immediately rather than block on the kdb+ write.

TCP → [asio I/O thread] → lock-free queue → [processor thread] → async task → [writer thread] → kdb+

Three threads, two inter-thread queues, two wakeups per message. The architecture decouples I/O from parsing from kdb+ writes, which is useful if any stage might block. The cost is that every hop adds synchronization, potential cache movement, and scheduler latency.

The asio TCP client

Since this was greenfield, I had the chance to set up the wire layer carefully from the start. A few decisions made a measurable difference:

Single io_context with a single dedicated I/O thread. For a small number of long-lived outbound connections, additional asio threads add more synchronization overhead than they save. One thread keeps the model simple and gives me an obvious target if I later want to pin it to a core.

**TCP_NODELAY on every connection.** Nagle's algorithm batches small writes for throughput. For an interactive feed it adds tens to hundreds of microseconds of delay. Disabling it is standard practice for this kind of system.

Tuned SO_RCVBUF. The Linux default receive buffer is fine for streaming workloads but smaller than I wanted for bursty traffic. A larger buffer absorbs spikes without forcing the kernel to drop or stall.

Receive timestamping at the asio handler. The moment the read completion handler fires, I capture a steady_clock time and carry it through the pipeline. Downstream measurements are relative to this timestamp, which makes per-stage attribution straightforward later.

Pre-allocated read buffers, no heap allocation on the hot path. Each connection owns its own receive buffer, sized once at construction. Partial messages accumulate in place and complete messages dispatch without copying.

**async_read_some rather than async_read.** I read whatever the kernel has available and let the length-prefixed framing in the parser figure out message boundaries. Waiting for a fixed number of bytes would have meant unnecessary syscalls when partial data arrived.

Per-connection isolated state. Each connection has its own parser, buffer, and callbacks. No shared mutable state means scaling to additional upstream pricers is a matter of adding connections, not adding locks.

Routine lifecycle handling. Heartbeats run on an asio high-resolution timer in the same I/O thread. Reconnect is automatic with a configurable delay. Shutdown stops the io_context cleanly via executor_work_guard.

With this in place, the wire layer contributes about 1 µs from TCP receive to the asio handler firing. That left everything above it as the next thing to look at.

Adding measurement

I didn’t reach for a profiler. Profilers are useful for finding hot functions but tend to perturb the system at the microsecond scale. Instead I added a rolling counter on the hot path — a running sum and sample count, logged out every 100 messages as an average. Cheap and accurate.

I also cross-checked against kdb+. Three timestamps are recorded per quote:

• Source-side: when the upstream system generated the quote
• TCP receive: when our socket got the packet
• Ready-to-publish: when our writer was about to push it to kdb+

The delta between the second and third is what I was optimizing. The first is upstream-determined.

Cross-checking the in-process measurement against the timestamps in kdb+ was useful throughout the work. When the two agreed, I could move on. When they disagreed, the gap was always worth investigating.

Where the time was going

I started by guessing the bottleneck was the kdb+ record building. The code does about 15 SetValue calls per quote (one per column), and I assumed each was doing real work — interning symbols, allocating K-objects, that kind of thing. I had budgeted around 160 µs for the whole block.

I added a measurement point right after the last SetValue and the number came back 10 µs higher than the previous one. Fifteen SetValue calls cost 10 µs total, not 160. Under a microsecond each. The kdb+ wrapper turned out to be well-engineered — pre-allocated records, type-tagged field assignment, no allocation in the hot path.

So the 140 µs was sitting somewhere else.

Adding more measurement points decomposed the pipeline:

Stage Cost TCP receive → asio handler fires ~1 µs Parse the ASCII format ~8 µs Push to lock-free queue (3.4 KB copy) ~2 µs Wakeup of the processor thread (condvar) 30–60 µs Processor pops the queue ~2 µs Submit task to writer thread (heap allocation + asio post) ~5 µs Writer thread wakeup + scheduling 10–30 µs All the SetValue work in the writer ~10 µs kdb+ submit (sync IPC to tickerplant) ~15 µs

Two costs stood out, neither of them visible in the code itself: the two thread wakeups. A condition variable wait plus futex syscall plus scheduler latency was costing roughly 50 µs per wakeup, and there were two of them per message.

Removing the second hop

The writer thread existed to decouple the kdb+ submit from the processor. But the kdb+ submit takes about 30 µs end-to-end, and the asio hop protecting it was costing more than that.

I removed the indirection. The processor thread now calls the writer’s logic directly. It means the processor blocks during the kdb+ submit, but the submit is faster than the hop was anyway.

Result: 91 µs average.

The takeaway: decoupling abstractions have a cost. If the thing being decoupled is already fast, the decoupling layer can become the slower piece. Worth checking.

Replacing the condvar with spin-backoff

With the second hop gone, the remaining wakeup cost was the processor thread itself, which was sleeping on a cv.wait when the queue was empty.

cv.wait is the standard primitive for this. Under typical Linux conditions, waking from it costs 30–60 µs — the futex syscall plus scheduler latency to put the thread back on a core. At the message rates I was testing, the consumer was sleeping between bursts and paying that cost on the first message of every burst.

I replaced the condvar with a spin-with-backoff pattern. The consumer tries to pop from the queue; if nothing’s there, it spins for a few iterations using _mm_pause (which is gentle on SMT siblings and reduces speculative pressure), with the spin count growing exponentially up to a cap. Once the spin budget runs out, it falls back to a 1 µs sleep_for. When a message arrives, it's detected on the next iteration — no syscall, no scheduler involvement.

I also removed the notify_all() from the producer side, since nothing was waiting on the condvar anymore.

Result: 60 µs average. The variance also dropped noticeably:

Before spin-backoff:  p50=50 µs,  p90=149 µs,  p95=187 µs,  p99=300 µs
After:                p50=40 µs,  p90=65 µs,   p95=85 µs,   p99=130 µs

The tail compressed because the worst-case wakeup was no longer in the picture.

The current steady-state measurement is around 56 µs average, with a minimum near 2.5 µs when everything cooperates.

Things I expected to matter but didn’t

Three intuitions that turned out to be wrong:

• CSV parsing. I was sure atof and tokenization would be expensive. They cost about 8 µs total. Real, but not where the optimization budget belonged.
• The 3.4 KB struct copy between threads. I expected cache-line bouncing to be expensive. It cost about 2 µs.
• The SetValue calls. I had budgeted 160 µs and they cost 10 µs.

The pattern across all three: I overestimated the cost of computation and underestimated the cost of synchronization. In a multi-threaded system, the question is rarely whether code is efficient. It’s where the code crosses a thread, cache, or kernel boundary.

What’s left

I stopped at 56 µs because the marginal cost of further work outweighed what the experiment was teaching me. Some remaining options:

• Move a secondary kdb+ write off the hot thread. A second write to a different table currently runs on the same processor thread and blocks the next message. Moving it to a separate writer would shave 30–60 µs per message.
• Pin threads to cores. The processor and asio threads currently float across cores. Pinning them to two cores on the same NUMA node would warm caches and reduce thread migrations. Probably worth 10–25 µs of average improvement.
• Switch from ASCII to a binary protocol. Bigger change, requires upstream coordination. ASCII parsing caps the system at roughly 50k msg/sec; binary would push that 5–10× higher.
• Kernel-bypass networking. Worth 5–20 µs of TCP-stack latency, but requires specific NICs and significant additional work.

None of these are necessary for what I set out to learn, but the options are there if I revisit the project.

Some general observations

A few things from this work that I’d carry forward:

Build the wire layer carefully from the start. TCP_NODELAY, a single dedicated I/O thread, pre-allocated buffers, and immediate receive timestamping aren't optimizations so much as routine setup for any low-latency TCP client. Done once at the beginning, they don't need revisiting.

Add measurement points, then add more. Decomposing the pipeline takes a few lines of code per measurement point and tells you things that guessing won’t. I was wrong about the bottleneck on my first decomposition and corrected it on the second.

Synchronization is usually the slower thing. In multi-threaded latency-critical code, the cost tends to live at boundaries between threads, between cores, and between user space and the kernel. Computation tends to be cheap by comparison.

Cross-check application measurements against downstream timestamps. What the application thinks it did and what the database recorded aren’t always the same. The disagreements are usually informative.

Decide when to stop. Latency optimization is unbounded — there’s always something more you could do. Stopping when further work no longer has business value is part of the work.

Closing

The system went from ~140 µs to ~56 µs, a factor of about 2.5×. Most of the improvement came from removing one thread hop and one condition variable. The rest was disciplined measurement, on top of a wire layer that was straightforward to set up correctly the first time.

Happy to discuss the work with anyone doing similar things in low-latency C++.