Endee vs Vespa: A Comprehensive Vector Database Benchmark

Vector databases are increasingly at the heart of production AI systems — powering semantic search, recommendation engines, and retrieval-augmented generation (RAG) pipelines. Choosing the right one matters enormously: the wrong choice can mean 3x lower throughput, unpredictable latency spikes under load, or inflated infrastructure costs. This post presents a head-to-head benchmark of Endee and Vespa across dense semantic search workloads, covering throughput (QPS), tail latency (p99), retrieval quality (recall) across quantizations and concurrency levels.

1. Goal of the Analysis

This benchmark was designed to answer a specific set of engineering questions:

• Throughput (QPS): Which system handles more queries per second with equivalent hardware?
• Tail Latency (p99): Which system provides tighter, more predictable worst-case response times?
• Recall: Does quantization degrade retrieval quality, and by how much?
• Quantization: How do different precision levels perform across both systems?

2. Experimental Conditions

2.1 Benchmarking Tools

Dense vector search was benchmarked using VectorDBBench by Zilliz.

Note: Vespa’s support in VectorDBBench originally covered only float32 and binary precision. We extended the VectorDBBench codebase to add support for additional Vespa quantization modes (int8, bfloat16). The extended branch is available at: [git_repo](http://github.com/endee-io/VectorDBBench/tree/vespa_precision_addition).

2.2 Server & Client Configuration

Both server and client ran with the following configuration:

• CPU: 4 vCPUs
• RAM: 16 GB
• OS: Debian GNU/Linux 13 (trixie)
• Storage: NVMe SSD (100 GB)
• Network: Server and client co-located

2.3 Software Versions

• Endee: Binary built from commit [b6788f2](https://github.com/endee-io/endee/commit/b6788f2) — run as compiled binary
• Vespa: Docker image vespaengine/vespa:latest — version 8.669.29
• VectorDBBench: git repo

2.4 Dataset

We use the Cohere 1M dataset: 1,000,000 vectors of 768 dimensions with cosine distance.

2.5 HNSW Index Parameters

Both Endee and Vespa use HNSW-based indexing. The following parameters were used consistently:

• m (Endee) / max-links-per-node (Vespa): 16
• ef_construction (Endee) / neighbors-to-explore-at-insert (Vespa): 128
• ef_search (Endee) / target-hits, hnsw.exploreAdditionalHits (Vespa): 128

Vespa uses different parameter nomenclature (target-hits, hnsw.exploreAdditionalHits) but these were tuned to produce equivalent search behaviour to Endee's ef_search=128.

2.6 Methodology

• All tests were repeated three times and the best result was recorded to minimize variance from transient system load and let the system warm up.
• Dense benchmarking used VectorDBBench with concurrency and TopK varied independently.

3. Comparative Analysis

3.1 Baseline: Comparing All Available Quantizations

We first show a set of comparisons between the different precisions (quantizations) available in Endee and Vespa. Endee’s quantizations are designed to have low storage and compute overhead while capturing the most information, leading to high QPS and recall. Here we keep TopK=30 and client concurrency to 5.

Recall with each precision

QPS with each precision

P99 Latency with each precision

At the baseline configuration (TopK=30, Concurrency=5, 1M Cohere 768D vectors), Endee int8 achieves the highest QPS of 1593 at a recall of 94.66%, while Endee int16 delivers 1368 QPS with 97.33% recall; while Vespa’s best was bfloat16 at 860 QPS and 95.01% recall. Vespa’s p99 latency ranges between 5.8ms to 6.8ms while Endee’s ranges from 2.9ms to 4.0ms — a 50% reduction.

3.2 Varying TopK

Next, we compare different metrics with varying TopK values. We pick the best precision for each vector DB based on QPS, recall, and latency — based on our empirical observations above. We choose int16 for Endee and bfloat16 for Vespa.

The following tests vary TopK from 10 to 1000 at fixed client concurrency = 5.

Recall vs TopK

Endee maintains a 1.9–4.3% higher recall than Vespa across different TopK values.

QPS vs TopK

Endee maintains a 25–63% higher QPS than Vespa across different TopK values.

3.3 Varying Concurrency

Next, we compare different metrics with varying client concurrency. Here again, we pick the same precisions as above to ease comparisons. Concurrency was varied from 2 to 24 to evaluate parallel query scaling behaviour.

QPS vs Concurrency

As client load increases, Endee maintains a smooth throughput saturation curve, saturating at 16 concurrency. Vespa, by contrast, shows an abrupt drop in QPS at concurrency=6. Moreover, Endee’s QPS is up to 2.6x higher than Vespa’s.

Note: To understand the reason for Vespa’s abrupt drop, we drilled deeper and found that it is caused by Vespa’s TCP connections entering a TIME_WAIT state and exhausting the available connections.

Conclusion

Endee consistently outperforms Vespa across the metrics that matter most in production: throughput, tail latency, and retrieval quality.

At baseline, Endee int8 delivers nearly 2x Vespa’s QPS while cutting p99 latency by ~50%. Across varying TopK and concurrency, Endee scales predictably, exhibiting a smooth throughput saturation curve.

For teams running RAG pipelines, semantic search, or recommendation systems at scale, the choice has real infrastructure implications: higher QPS means fewer nodes to serve the same load, and lower p99 latency means a better end-user experience without over-provisioning.

Endee delivers more queries, at higher recall, with lower and more predictable latency — on identical hardware.