minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

[Cookbook] Add NVFP4 options for DeepSeek-V4 Flash Official (0731) and Pro Official (0813) (#37351)

Browse Source 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
zijiexia
2026-09-01 08:03:39 +00:00
committed by GitHub
co-authored by Claude Fable 5
parent 03b33cbe5d
commit 379e33d87e
3 changed files with 189 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/cookbook/autoregressive/DeepSeek/DeepSeek-V4.mdx
+9 -1
View File
@@ -311,7 +311,10 @@ sglang serve \
The [`nvidia/DeepSeek-V4-Pro-NVFP4`](https://huggingface.co/nvidia/DeepSeek-V4-Pro-NVFP4) and
[`nvidia/DeepSeek-V4-Flash-NVFP4`](https://huggingface.co/nvidia/DeepSeek-V4-Flash-NVFP4) checkpoints
quantize MoE experts to **NVFP4** while keeping attention and dense layers in
**FP8**. It requires `--moe-runner-backend flashinfer_trtllm_routed` which will be automatically selected if not provided.
**FP8**. The official releases have matching NVFP4 checkpoints at
[`nvidia/DeepSeek-V4-Flash-0731-NVFP4`](https://huggingface.co/nvidia/DeepSeek-V4-Flash-0731-NVFP4) and
[`nvidia/DeepSeek-V4-Pro-0813-NVFP4`](https://huggingface.co/nvidia/DeepSeek-V4-Pro-0813-NVFP4).
All of them require `--moe-runner-backend flashinfer_trtllm_routed` which will be automatically selected if not provided.
```bash Command
sglang serve \
@@ -331,6 +334,11 @@ Requires Blackwell (SM100+). The MTP layer in this checkpoint stays
MXFP4-packed and is routed through the `Mxfp4FlashinferTrtllmMoEMethod` path
automatically.
The official (0731 / 0813) NVFP4 checkpoints preserve the bundled DSpark draft
head, so their low-latency recipes use `--speculative-algorithm DSPARK` instead
of the EAGLE/MTP shape flags — same as the corresponding original-precision
official checkpoints.
<a id="hopper-note" />
**Hopper (H100 / H200) note**