Red Hat AI Brings DSpark Speculative Decoding to GLM-5.2, Doubling Inference Speed

Original by winkrun, in AI Engineering · July 3, 2026, Beijing

Red Hat AI just released a DSpark speculative decoding model optimized for GLM-5.2. This marks the first time DSpark speculative decoding architecture has been adapted to a leading non-DeepSeek large language model, moving beyond the framework’s original exclusive tie-in to DeepSeek models.

For context, DSpark builds on the DFlash parallel draft backbone, paired with a Markov logit-bias head and a per-position confidence head. In simple terms: it lets a small 3B-parameter draft model quickly generate multiple candidate tokens, then the larger model GLM-5.2-FP8 verifies an entire batch of candidates in one forward pass. This cuts out the massive time cost of traditional sequential per-token decoding.

The project rolled out in two phases: an initial preview checkpoint trained for 3 epochs on 50k samples from the UltraChat dataset, followed by a fully trained epoch-1 checkpoint. The epoch-1 release uses full-vocabulary drafting 154,880 tokens and regenerated training data from a combination of Magpie and UltraChat, which delivers a significant jump in performance over the preview.

The chart above tracks key metrics through the epoch-1 training run: the mean accepted length holds steady around 3.4, and per-position acceptance rate drops smoothly from 78% at position 1 to 38% at position 7. This confirms that the Markov head effectively mitigates the common suffix decay problem that plagues many speculative decoding approaches.

How Much Real-World Speedup Does It Deliver?

On a 4×B300 GPU setup, the base GLM-5.2 model without speculative decoding hits 102 tokens per second. Adding the preview DSpark checkpoint bumps that up to 139 tokens per second, a 1.36x speedup. Swapping in the full epoch-1 checkpoint pushes decoding speed all the way to 219 tokens per second — a 2.15x speedup over baseline.

Mean accepted length also jumps from 2.18 with the preview to 3.49 with the full release. That means the large model confirms 3.49 tokens per forward pass on average, instead of just 1 token with standard sequential decoding.

Here’s the full comparison:

| Checkpoint | Mean Accepted Length | Decoding Speed | | — — — — — — — — — | — — — — — — — — — — — | — — — — — — — — | | No Speculation | 1.0 | 102 tok/s | | Preview | 2.18 | 139 tok/s | | epoch-1 | 3.49 | 219 tok/s |

How to Use It

You’ll need the nightly build of vLLM to run this setup. Install and launch with the commands below:

uv pip install vllm --extra-index-url https://wheels.vllm.ai/nightly

vllm serve zai-org/GLM-5.2-FP8 \
    --tensor-parallel-size 4 \
    --max-model-len 16384 \
    --trust-remote-code \
    --speculative-config '{
        "model": "RedHatAI/GLM-5.2-speculator.dspark",
        "num_speculative_tokens": 7,
        "method": "dspark",
        "draft_sample_method": "probabilistic"
    }'

Note that the model path in --speculative-config points directly to the model hosted on Hugging Face. If you just want to test the preview release, swap the model name to RedHatAI/GLM-5.2-speculator.dspark-preview.

Key Takeaways

Speculative decoding has gone from a niche academic curiosity to a practical production tool over the last few years, but until now DSpark was almost exclusively tied to DeepSeek models. Red Hat AI’s port to GLM-5.2 proves that the DSpark architecture is fully generalizable: as long as you properly train the draft model, any large language model can get a ~2x speedup for free.

Using self-generated responses from the base model for training data, the self-play approach, also drastically lowers the barrier to entry by cutting out the work of curating external training datasets.

Epoch-2 and epoch-3 checkpoints are still in training. Following the current trend, the final release could push mean accepted length above 4. For teams running GLM-5.2 inference, this is one of the highest-ROI optimizations you can make right now: no changes to your base model required, just add a 3B draft model and cut your GPU inference time in half.

All model weights and code are released under the permissive MIT license, so you can use them for any purpose. The training pipeline is built on the speculators library, which is also open source on GitHub.