DeepSeek open-sources DeepInfer, its production inference engine
The Apache 2.0 framework is the same serving stack DeepSeek uses to run its own models at scale, built to target both NVIDIA and Huawei Ascend hardware.
DeepSeek open-sourced DeepInfer on 20 October, the inference framework it uses to run its own large-scale production services. It’s released under the Apache 2.0 licence, and the project’s README credits vLLM, SGLang and TensorRT-LLM as the work it builds on.
Right now DeepInfer supports DeepSeek-V4.1-Flash. It runs on NVIDIA SM100 GPUs or Huawei Ascend 950 NPUs, and its feature list includes optimised kernels, several parallelism strategies for attention and mixture-of-experts layers, speculative decoding via what DeepSeek calls DSpark verification, asynchronous batch scheduling, and structured output in regex, JSON or EBNF. Building it from source needs Linux, GCC 13 or newer, Python 3.12, and either CUDA 13 or CANN 9.2 depending on the hardware.
Why it matters: if you’re self-hosting DeepSeek’s models, you now get the same engine DeepSeek runs in production rather than relying on third-party support in a general-purpose serving framework. Ascend support also matters if your inference hardware isn’t NVIDIA.
The caveat: it only serves V4.1-Flash so far, not DeepSeek’s full model line, and there are no tagged releases yet, so treat it as early-stage.