DeepSeek has published DeepInfer under the Apache 2.0 licence for offline execution of DeepSeek-V4.1-Flash. The engine includes scheduling, parallel computation and structured outputs. Developers building a complete online service still need separate cache and traffic-management infrastructure. The release makes the computation code available while defining the additional work required to serve application requests.
DeepSeek, the language-model developer, published DeepInfer on 20 October under the Apache 2.0 licence. The open-source engine supports DeepSeek-V4.1-Flash. It performs offline inference, the computation that turns model inputs into generated outputs. A complete online service requires additional components for handling users’ requests. The release provides the engine code separately from that surrounding service infrastructure.[1], [2]
The engine combines data and context parallelism for attention calculations with expert parallelism for mixture-of-experts models. Expert load balancing forms part of that path. Asynchronous processor scheduling overlaps scheduling work with graphics-processor computation. DSpark, its draft-and-verification mechanism, uses confidence estimates when deciding which proposed output tokens to verify.[1]
DeepInfer can constrain outputs with schemas, regular expressions and grammar rules. Its documented hardware paths include NVIDIA SM100 and Ascend 950. The release excludes a key-value cache system, service-level separation of initial prompt processing from later generation, and load balancing across service replicas. An online service needs those layers around the engine. Installation also requires Linux, Python 3.12 and the tools associated with the selected hardware.[1]
