Price terms as published

Each line is the vendor’s own wording, split from the price capture checked on 2026-10-25. Nothing here is rounded, converted or re-worded — the full sentences stay in the review below. Lines that carry only a price continue the plan named on the line above, in the vendor’s own order.

2 price terms for DeepInfer
#Plan / termChecked
1Free (Apache-2.0 open source)2026-10-25
2requires NVIDIA SM100 GPU or Huawei Ascend 950 NPU2026-10-25

One-liner: DeepInfer is the open-source inference framework that powers DeepSeek’s large-scale production services — released Apache-2.0 on 2026-10-20 with offline inference for DeepSeek-V4.1-Flash, built primarily on vLLM and optimized with DeepSeek’s own kernel libraries (FlashMLA, DeepGEMM, DeepEP, TileKernels, DeepSelect).

Everything below was checked against the official repository (GitHub API + raw README, fetched 2026-10-25) and independent coverage.

Verified facts (GitHub API + raw README + third-party, 2026-10-25)

Attribute Value Source
GitHub stars 21,914 (created 2026-10-16; released 2026-10-20) deepseek-ai/DeepInfer via GitHub API, 2026-10-25
License Apache-2.0 GitHub API
Supported model DeepSeek-V4.1-Flash (default checkpoint deepseek-ai/DeepSeek-V4.1-Flash-1017) README
Supported hardware NVIDIA SM100 GPUs; Huawei Ascend 950 NPUs README
Toolchain Linux; GCC ≥ 13; Python 3.12; CUDA 13 (NVIDIA) or CANN 9.2 (Ascend) README
Parallelism Attention DP / balanced CP; MoE EP + EPLB README feature list
Acceleration DSpark + confidence-scheduled verification; SWA bounded replay; async zero-overhead batch scheduler README feature list
Determinism Batch-invariant inference + deterministic sampling (identical outputs for identical inputs, NVIDIA only) README feature list
Structured output regex, JSON schema, or EBNF grammar constraints README feature list
Kernels FlashMLA, DeepGEMM, DeepEP, TileKernels, DeepSelect, custom TileLang kernels README feature list
Not included KV cache system, VE/Prefill/Decode disaggregation (most engine functionality implemented), cross-replica load balancing README “Required for a complete inference service”
Heritage Built primarily on vLLM; inspired by SGLang and TensorRT-LLM README acknowledgments
Benchmarks Prefill on 4 SM100 GPUs; decode on 64 SM100 GPUs, each within one scale-up domain README performance section

This is an official-terms entry: we have not built or benchmarked DeepInfer, so there is no score and no hands-on claims.

Getting it running

DeepInfer is a build-from-source project for Linux, not a pip install. The README’s flow: create an empty Python 3.12 environment, export DEEPINFER_ACCELERATOR=nvidia (or ascend), run ./build.sh, then python examples/offline_inference.py — the default checkpoint is downloaded into the Hugging Face cache on first run, or you point DEEPINFER_HF_MODEL_DIR at a local copy. Prerequisites are exacting: GCC ≥ 13, CUDA 13 for NVIDIA (CANN 9.2 for Ascend), and the SM100/Ascend 950 hardware class. Optimizations are toggled with env flags documented in the README (e.g., DEEPINFER_ENABLE_MTP_VARLEN=1 for DSpark, DEEPINFER_BATCH_INVARIANT=1 for deterministic sampling on NVIDIA). A complete serving deployment additionally needs a KV cache system, disaggregation plumbing, and load balancing, which the release explicitly leaves out.

What it costs

The framework is free under Apache-2.0 — no license fee, no per-token charge. The costs are operational and hardware: you need NVIDIA SM100 (Blackwell Ultra-class) GPUs or Huawei Ascend 950 NPUs with CUDA 13/CANN 9.2, and the missing serving components (KV cache system, disaggregation, load balancing) mean production deployment is an engineering project, not a turnkey service. For teams without that hardware, DeepSeek’s hosted API remains the simpler path; DeepInfer is aimed at organizations that want to serve DeepSeek-V4.1-Flash at scale themselves.

How it differs from alternatives

  • vLLM / SGLang / TensorRT-LLM are the general-purpose serving engines DeepInfer builds on and competes with. DeepInfer adds DeepSeek’s production-specific optimizations: MoE expert-parallel with EPLB load balancing, DSpark draft-token verification with confidence scheduling, SWA bounded replay to cut decoder prefill, and batch-invariant/deterministic sampling — features tuned to DeepSeek-V4.1-Flash’s architecture rather than a broad model zoo.
  • Ollama / llama.cpp are local-first and consumer-friendly but not built for 64-GPU scale-up serving with disaggregation; DeepInfer targets datacenter-scale throughput on specific accelerators.
  • Hosted DeepSeek API requires no hardware but means sending data to DeepSeek and paying per token; DeepInfer is the self-hosted counterpart with all the hardware and ops costs that implies.
  • Other open inference projects on this site (e.g., Colibri’s pure-C MoE inference) target lightweight local CPU/GPU runs; DeepInfer sits at the opposite end — production-scale, GPU-heavy, and tied to one model family today.

Where to stay honest

  • One model, two accelerator families today: DeepInfer supports DeepSeek-V4.1-Flash on SM100 and Ascend 950 only — no general model support, no consumer GPUs (SM100 is the Blackwell Ultra datacenter class).
  • A release, not a full serving stack: the README explicitly lists the KV cache system, disaggregation, and load balancing as not included; teams should not expect a drop-in vLLM replacement for every workload yet.
  • Exacting build requirements (GCC ≥ 13, CUDA 13/CANN 9.2, Python 3.12) and benchmarks measured on 4–64 SM100 GPUs — headline numbers assume that hardware.
  • We did not build or run it; this is an official-terms directory entry verified against the repo and coverage as of 2026-10-25.

Who it’s for

  • Organizations with NVIDIA SM100 or Huawei Ascend 950 capacity that want to serve DeepSeek-V4.1-Flash at production scale with DeepSeek’s own engine
  • Infra teams evaluating self-hosted alternatives to the hosted DeepSeek API
  • Performance engineers interested in MoE EP/EPLB, DSpark verification, and SWA replay techniques
  • Not for: consumer/local inference (use Colibri, Ollama, or llama.cpp), general multi-model serving (use vLLM/SGLang), or anyone without the listed hardware

Sources