Bringing SPLADE to vLLM: My First Open Source Contribution Story

Image by author — Generated with DALL.E 3

This story was written with the assistance of an AI writing program.

For the past year I’ve been living inside vLLM — benchmarking, deploying, tuning, and generally abusing it in every possible way for production LLM workloads. At some point I started thinking:

“I’ve used vLLM so much… it would be nice if my name was on the contributors list.”

This post is the story of how that happened: my first vLLM contribution got merged 🎉 — adding official support for the [naver/splade-v3](https://huggingface.co/naver/splade-v3) sparse retrieval model via a new BertSpladeSparseEmbeddingModel.

Along the way I had to:

• Understand SPLADE properly (not just use it as a black box)
• Design how to fit a sparse lexical model into vLLM’s embedding API
• Make the implementation compatible with Hugging Face SparseEncoder and Text Embeddings Inference (TEI)
• Survive code review from two vLLM maintainers

If you’re curious about sparse retrieval, SPLADE, or want to contribute a new model to vLLM, I hope this write-up helps.

After a year of working with vLLM, I finally submitted — and merged — my first contribution. Here’s the PR: https://github.com/vllm-project/vllm/pull/26339

My first contribution to vLLM

Why SPLADE and why now?

I’ve been recently working on embedding models and retrieval systems. In that process, I naturally hit the usual trio:

• Classic BM25 / bag-of-words (BOW)
• Modern dense retrievers
• And more recently, learned sparse retrievers like SPLADE

While experimenting, I realized:

• Hugging Face TEI already supports sparse models and SPLADE via --pooling splade.
• Sentence-Transformers has SparseEncoder support for SPLADE.
• But vLLM didn’t support sparse SPLADE models at all — especially the widely used **naver/splade-v3**.

Since vLLM is often my go-to for serving models in production, this gap was painful. So I decided my first vLLM contribution would be:

“Add official SPLADE support (starting with naver/splade-v3) to vLLM’s embedding API.”

A quick refresher: dense vs sparse vs SPLADE

SPLADE is a learned sparse retriever built on top of BERT-style models. To understand why it’s interesting, we need a bit of IR context.

1. BOW / BM25 — still strong, but…

Traditional IR systems use bag-of-words models (like BM25). They’re powerful and still strong baselines, but they suffer from the classic:

Vocabulary mismatch — relevant documents may not literally contain the same terms as the query.

BOW models are great because they:

• Support exact term matching
• Use inverted indexes (very efficient)
• Are interpretable (you can see which terms matter)

…but they have no semantic understanding.

2. Dense retrieval — semantic, but not lexical

Dense retrievers map queries and documents into a dense vector space, then use approximate nearest neighbor (ANN) search.

They’re great for:

• Capturing semantic similarity
• Handling paraphrases / synonyms naturally

But they:

• Lose explicit term-level matching
• Don’t directly reuse inverted indexes
• Are harder to interpret

That’s why in practice, dense retrieval is often combined with BM25.

3. Learned sparse models — best of both worlds

Recent sparse models (including SPLADE) aim to:

• Produce sparse vectors over the vocabulary (like BOW)
• But learn those weights using a transformer backbone

The goal is:

• Keep BOW strengths: exact matches, inverted index efficiency, interpretability
• Add neural strengths: latent expansion, better handling of vocabulary mismatch

SPLADE does exactly this.

How SPLADE works (high-level)

SPLADE uses a BERT-style Masked Language Model (MLM) head to turn an input sequence into a sparse vector over the vocabulary.

Let’s simplify the core idea:

1. Tokenize your input sequence: [t = (t_1, t_2, …, t_N)]
2. Pass it through BERT and get hidden states: [(h_1, h_2, …, h_N)]
3. For each input token ( i ) and each vocab token ( j ), predict a term importance logit ( w_ij ) using an MLM-style linear head: *w_ij = transform(h_i) E_j + b_j** where

• ( E_j ): embedding of vocab token ( j )
• transform(.): linear → GeLU → LayerNorm

1. Apply activation to enforce non-negativity and sparsity-like behavior. SPLADE uses: logits_ij = log(1 + ReLU(w_ij))
2. Pool across sequence positions to get a single sparse vector over the vocab:

• SPLADE v1 used sum pooling
• SPLADE v2 introduced max pooling: w_j = max over i in t of logits_ij Empirically, max pooling significantly improved performance, and SPLADE-max became the default.

The end result is a vector of size ≈ |V| ≈ 30k, with most entries zero — a sparse lexical representation, but learned via BERT.

Training uses a ranking loss with in-batch negatives: for each query ( q_i ), positive doc, and negatives (including in-batch negatives), SPLADE maximizes the probability that is ranked above the others.

SPLADE already lives in TEI & Sentence-Transformers

Before touching vLLM, I looked at how SPLADE is exposed in other ecosystems.

1. Text Embeddings Inference (TEI)

TEI can serve SPLADE with a simple Docker command:

model=naver/splade-v3
volume=$PWD/data

docker run --gpus all -p 8080:80 -v $volume:/data --pull always \
  ghcr.io/huggingface/text-embeddings-inference:1.8 \
  --model-id $model --pooling splade

Then you can call:

curl 127.0.0.1:8080/embed_sparse \
  -X POST \
  -d '{"inputs":"I like you."}' \
  -H 'Content-Type: application/json'

and get a sparse list of {index, value} pairs.

2. Sentence-Transformers SparseEncoder

SparseEncoder from Sentence-Transformers also supports SPLADE models:

from sentence_transformers import SparseEncoder
import torch

model = SparseEncoder(
    "naver/splade-v3",
    model_kwargs={"torch_dtype": torch.bfloat16},
)
queries = ["who are you?"]
q_emb = model.encode_query(queries)
print(len(q_emb[0].nonzero()))

This makes SPLADE very convenient in HF-based pipelines — but vLLM was still missing from the picture.

Design goal: SPLADE as a first-class vLLM embedding model

My goal for the PR was:

• Add official support for naver/splade-v3
• Implement SPLADE as a proper vLLM embedding model, not a hack
• Keep it OpenAI-compatible via /v1/embeddings
• Make it numerically consistent with HF SparseEncoder and TEI

At a high level, the implementation needed to:

1. Extend the BERT embedding family with a variant that:

• Uses the MLM head
• Applies SPLADE pooling to produce a sparse vector over vocab

3. Register this model in vLLM’s model registry

4. Support HF override to map architectures to BertSpladeSparseEmbeddingModel

5. Plug into vLLM’s embedding API and pooling tasks with no regressions.

Implementation: wiring SPLADE into vLLM

The core changes were in:

• vllm/model_executor/models/bert.py
• vllm/model_executor/models/registry.py
• Plus some test infra.

1. Model registration

First, I added a new model mapping so vLLM knows how to instantiate SPLADE models:

"BertSpladeSparseEmbeddingModel": ("bert", "BertSpladeSparseEmbeddingModel")

In tests/models/registry.py I registered a concrete example:

"BertSpladeSparseEmbeddingModel": _HfExamplesInfo(
    "naver/splade-v3", is_available_online=False
),

The idea is:

• If the HF config has architectures=["BertSpladeSparseEmbeddingModel"],
• Then vLLM routes it to the BERT family and uses the BertSpladeSparseEmbeddingModel class.

To make that work in practice, I rely on **--hf-overrides** when launching vLLM:

--hf-overrides '{"architectures":["BertSpladeSparseEmbeddingModel"]}'

This allows existing SPLADE checkpoints to work without re-uploading modified configs.

2. Implementing the MLM head: BertMLMHead

SPLADE needs access to the MLM logits, so I implemented an MLM head in bert.py:

class BertMLMHead(nn.Module):
    def __init__(
        self,
        hidden_size: int,
        vocab_size: int,
        layer_norm_eps: float = 1e-12,
    ):
        super().__init__()
        self.dense = nn.Linear(hidden_size, hidden_size)
        self.activation = nn.GELU()
        self.layer_norm = nn.LayerNorm(hidden_size, eps=layer_norm_eps)
        self.decoder = nn.Linear(hidden_size, vocab_size, bias=True)

    def tie_weights_with_embeddings(self, embeddings_weight: torch.Tensor):
            self.decoder.weight = embeddings_weight

    def forward(self, hidden_states: torch.Tensor) -> torch.Tensor:
        x = self.dense(hidden_states)
        x = self.activation(x)
        x = self.layer_norm(x)
        logits = self.decoder(x)
        return logits

Key points:

• It follows the usual BERT MLM head design
• tie_weights_with_embeddings ensures weight tying with the token embeddings
• Output shape: [batch, seq_len, vocab_size]

3. SPLADE pooling: SPLADESparsePooler

Next, I implemented the SPLADE pooling logic:

class SPLADESparsePooler(Pooler):
    """
    SPLADE sparse pooling:
    logits = mlm_head(hidden_states)
      -> log1p(relu(logits))
      -> (max | sum over sequence length)
      -> [V]
    Padding is masked with an attention mask,
    [CLS]/[SEP] is removed, and then pooled.
    """
    # (Implementation details omitted here for brevity)

Core behavior:

1. Run hidden states through BertMLMHead → get logits [B, L, V]
2. Apply log1p(ReLU(logits))
3. Mask out padding and special tokens ([CLS], [SEP])
4. Pool over sequence length using max (or sum if configured)
5. Produce a single sparse vector of size vocab per input: [B, V]

This pooler is then plugged into the standard vLLM embedding pipeline so it’s fully compatible with:

• /v1/embeddings
• /pooling endpoints
• PoolingTask.embed

4. The new model class: BertSpladeSparseEmbeddingModel

Finally, I defined the model itself:

@default_pooling_type("CLS")
class BertSpladeSparseEmbeddingModel(BertEmbeddingModel):
    """
    BertEmbeddingModel + SPLADE sparse embedding.
    - Make logits by self.mlm_head
    - Pool via SPLADESparsePooler(mlm_head,...)
    """
    # (Init wires up BertMLMHead + SPLADESparsePooler)

It reuses most of BertEmbeddingModel logic, but:

• Adds an MLM head for logits
• Uses SPLADESparsePooler instead of dense pooling

Importantly, this was done with no regression for existing dense BERT embedding models.

Serving naver/splade-v3 with vLLM

With the code in place, here’s how I ran SPLADE via vLLM in Docker.

1. Launching vLLM with SPLADE

#!/bin/bash
GPU_ID=0
PORT=8004
MODEL_PATH="/workspace/model_repository"
SERVED_MODEL_NAME="splade-v3"

docker run --runtime nvidia --gpus "device=$GPU_ID" \
  -v models/naver/splade-v3:$MODEL_PATH \
  -p $PORT:8000 \
  --ipc=host \
  vllm/vllm-openai:v0.11.1rc6 \
  --model $MODEL_PATH \
  --trust-remote-code \
  --served-model-name $SERVED_MODEL_NAME \
  --hf-overrides '{"architectures":["BertSpladeSparseEmbeddingModel"]}'

Logs confirmed:

INFO Supported_tasks: ['embed', 'encode']
INFO Starting vLLM API server on http://0.0.0.0:8000

✅ The model initialized successfully with:

• torch.compile graph caching
• KV cache disabled (since this is an embedding-only model)
• /v1/embeddings ready for use

2. Embedding request and sparse preview

I tested the embedding endpoint with a simple Python script:

import requests, json

URL = "http://localhost:8004/v1/embeddings"
payload = {
    "model": "splade-v3",
    "input": "who are you?",
    "task": "embed",
    "normalize": False,
}
resp = requests.post(URL, json=payload)
obj = resp.json()
print(obj.keys())

The response shape looked like:

{
  "id": "embd-c1899570dd224953adf527b49be8120e",
  "object": "list",
  "created": 1759815423,
  "model": "splade-v3",
  "data": {
    "embeddings": [
      /* dense array of size ~30k, mostly zeros */
    ]
  },
  "usage": {
    "prompt_tokens": 9,
    "total_tokens": 9,
    "completion_tokens": 0,
    "prompt_tokens_details": null
  }
}

✨ To interpret this as a sparse vector, I used a small helper:

def extract_vector(r):
    if "data" in r:
        # OpenAI-compatible: embeddings under data.embeddings[0]
        if isinstance(r["data"], dict) and "embeddings" in r["data"]:
            return r["data"]["embeddings"][0]
        # alternative shape: data=[{"embedding": [...]}]
        if isinstance(r["data"], list) and "embedding" in r["data"][0]:
            return r["data"][0]["embedding"]
    if "embeddings" in r:
        first = r["embeddings"][0]
        return first["embedding"] if isinstance(first, dict) and "embedding" in first else first
    raise ValueError(f"Unknown response format: keys={list(r.keys())}")

vec = extract_vector(obj)
sparse = {i: float(v) for i, v in enumerate(vec) if v != 0.0}
preview_items = list(sparse.items())[:10]
print("nonzero count:", len(sparse))
print("preview (first 30):", list(sparse.items())[:30])

Observed output (for "who are you?"):

dict_keys(['id', 'object', 'created', 'model', 'data', 'usage'])
nonzero count: 46
preview (first 30): [
  (1037, 0.2741), (2017, 2.2852), (2024, 1.4453),
  (2040, 2.3203), (2057, 0.2632), (2111, 0.1442),
  (2115, 0.9668), (2529, 0.3230), (2554, 0.2603),
  (2619, 0.0225),
  ...
]

The important part: 46 non-zero entries and the top indices/values matched other frameworks.

Cross-framework consistency: vLLM vs SparseEncoder vs TEI

To be confident in the implementation, I checked SPLADE outputs across three engines:

1. vLLM /v1/embeddings (with my new model)
2. HuggingFace SparseEncoder
3. TEI /embed_sparse

1️⃣ vLLM vs SparseEncoder

Using the SparseEncoder snippet above:

from sentence_transformers import SparseEncoder
import torch

model = SparseEncoder("naver/splade-v3",
                      model_kwargs={"torch_dtype": torch.bfloat16})
queries = ["who are you?"]
q_emb = model.encode_query(queries)
print("nnz:", len(q_emb[0].nonzero()))

Result:

num_queries: 1
nnz of first: 46
preview: [
  (1037, 0.2734), (2017, 2.2812), (2024, 1.4453),
  (2040, 2.3281), (2057, 0.2676),
  ...
]

Comparing vLLM and SparseEncoder:

• Non-zero count (nnz): 46 vs 46
• Top indices: 1037, 2017, 2024, 2040, 2057, … — identical
• Values matched within 1e-4 float tolerance

✅ vLLM SPLADE pooling and vocab alignment are correct.

2️⃣ vLLM vs TEI

I also ran TEI with SPLADE:

docker run --rm --gpus "device=1" -p 8080:80 \
  -v models/naver/splade-v3:/app/models/splade-v3:ro \
  ghcr.io/huggingface/text-embeddings-inference:cuda-1.8 \
  --model-id /app/models/splade-v3 --pooling splade

Tested via curl:

curl localhost:8080/embed_sparse \
  -X POST \
  -H "Content-Type: application/json" \
  -d '{"inputs":"who are you?"}'

Response (simplified):

[
  [
    {"index":1037,"value":0.2771},
    {"index":2017,"value":2.2871},
    {"index":2024,"value":1.4482},
    {"index":2040,"value":2.3242},
    {"index":2057,"value":0.2666},
    {"index":2111,"value":0.1477},
    {"index":2115,"value":0.9683},
    {"index":2529,"value":0.3269},
    {"index":2554,"value":0.2659},
    {"index":2619,"value":0.0260},
    ...
  ]
]

Again:

• Same non-zero count
• Same top indices
• Values very close (minor float differences)

✅ TEI and vLLM are functionally equivalent for SPLADE embeddings.

3️⃣ Summary table

All three produce identical sparse activation patterns and magnitudes (within normal float tolerance), which was the main correctness criterion for the PR.

Notes and design considerations

A few important points from the PR:

• ✅ No regressions to existing BertEmbeddingModel or dense workflows
• ✅ Sparse embedding is fully integrated with PoolingTask.embed
• ✅ Works with FlashAttention and torch.compile graph caching
• ✅ TEI & SparseEncoder parity ensures vLLM can be dropped into hybrid retrieval systems that already use SPLADE elsewhere

The PR also included:

• A clearly documented purpose: “Add SPLADE support for naver/splade-v3”
• A test plan covering vLLM vs HF vs TEI
• Registry and pooling code updates

The contribution process: 5 days, 2 reviewers, a lot of polish

From first commit to merge, the whole process took about five days.

Some observations from going through vLLM’s review process for the first time:

1. Two code reviewers had to approve the PR before it could be merged.
2. The review was very thorough, especially around:

• Code style and consistency
• Avoiding unnecessary complexity
• Performance considerations (extra ops, shape handling, etc.)

3. Several iterations focused on:

• Removing unused or redundant bits
• Simplifying logic in the pooler
• Making the interfaces consistent with the rest of vLLM’s embedding stack

It was a nice reminder that in high-performance libraries, even “just an embedding model” goes through serious scrutiny.

But honestly, that made the merge feel even better.

Takeaways & future ideas

This contribution was relatively small in terms of lines of code, but it meant a lot to me personally:

• I’d been using vLLM heavily for about a year; now my name is in the contributors list.
• I learned how vLLM structures its model registry, poolers, and embedding tasks.
• I got to bridge the gap between sparse IR research (SPLADE) and high-performance inference infra (vLLM).

Some potential next steps I’m thinking about:

• Adding more SPLADE variants (e.g., efficient query/document models)
• Making hybrid dense + sparse retrieval easier to run end-to-end with vLLM
• Benchmarking sparse vs dense vs hybrid using vLLM backends on real-world workloads

If you’re using vLLM and you feel a missing piece — a model, a feature, a small utility — it might be a great candidate for your first contribution. Mine started with:

“Why doesn’t vLLM support SPLADE yet?”

…and ended up as a merged PR adding official support for naver/splade-v3.

If you have questions about integrating SPLADE in vLLM, or about the contribution process itself, feel free to reach out or leave a comment. Don’t forget to hit the like button and subscribe for more content! 😊