minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 13 Packages Projects Releases Wiki Activity

[DeepSeek V4] Add W4A4 MegaMoE server flag (#35918)

Browse Source
This commit is contained in:
Baizhou Zhang
2026-08-21 18:44:18 -07:00
committed by GitHub
parent 0be2a209ac
commit 3b5909de0e
15 changed files with 147 additions and 87 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/cookbook/autoregressive/DeepSeek/DeepSeek-V4.mdx
+4 -5
View File
@@ -332,14 +332,13 @@ HiCache and MegaMoE are **not** supported on RTX PRO 6000.
MegaMoE fuses expert dispatch + GEMM into a single kernel for higher throughput
on MoE layers. To enable it, use the **MegaMoE** chip in the Playground
below — the playground will swap `--moe-a2a-backend deepep` for
`--moe-a2a-backend megamoe` and add the relevant env vars automatically.
`--moe-a2a-backend megamoe` and add the relevant launch settings automatically.
Two variants are exposed:
- **W4A8** — default MegaMoE kernel (FP4 weights, FP8 activations).
- **W4A4** — adds `SGLANG_OPT_DEEPGEMM_MEGA_MOE_USE_FP4_ACTS=1` and
`SGLANG_OPT_DEEPGEMM_MEGA_MOE_USE_MXF4_KIND=1` to run the custom W4A4
kernel (FP4 activations). Higher throughput with negligible accuracy drop
(~89.5 GPQA on Pro).
- **W4A4** — adds `--enable-w4a4-mxfp4-megamoe` to run the custom W4A4 kernel
(FP4 activations). The flag configures the required DeepGEMM settings.
Higher throughput with negligible accuracy drop (~89.5 GPQA on Pro).
Notes:
- The W4A8 / W4A4 variants above are **Blackwell-only** (B200 / B300 / GB200 / GB300). On **Hopper (SM90, H100 / H200)** use the all-FP8 MegaMoE path described below instead.