open-source

DeepSeek open-sources DeepInfer, the inference engine behind V4.1 Flash, under Apache 2.0

DeepSeek published DeepInfer on October 20. It's the offline engine it uses to serve DeepSeek-V4.1-Flash, and it runs only on NVIDIA SM100 (Blackwell) GPUs and Huawei Ascend 950 NPUs.

Written by Mara at Sig · Oct 21, 2026

The repo ships the engine with DeepSeek's kernels (FlashMLA, DeepGEMM, DeepEP and others), several ways to split work across chips, draft-token decoding, structured outputs (regex, JSON schema or EBNF grammar) and two settings I haven't seen documented this plainly before: batch-invariant inference and deterministic sampling. The only supported model is DeepSeek-V4.1-Flash, and the default checkpoint is V4.1-Flash-1017.

It is not a full serving stack. DeepSeek says the KV cache system, prefill/decode disaggregation and load balancing across replicas are tied to its internal infrastructure and aren't in this release.

DeepSeek also released deepseek-recipe (MIT), which converts between common LLM API formats and model tokens the same way the official DeepSeek API does. Its deployment doc says one cluster serves 155 trillion tokens a day and earns $38 per GPU-hour at off-peak prices. Those are DeepSeek's numbers.

Why it matters

If you self-host V4.1 Flash on Blackwell, you now get DeepSeek's own kernels and a documented way to get identical outputs across runs, which makes evals and regression tests repeatable. On any other hardware it's reading material for now.

Try this

On an SM100 machine, set DEEPINFER_BATCH_INVARIANT=1 and DEEPINFER_ENABLE_MTP_VARLEN=0, pass a fixed seq_seed to SamplingParams, run examples/offline_inference.py twice and diff the outputs. Per the README they should match.

Sources

Back to all news Get Sig Weekly every Monday