minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

[docs] Fix V4 Pro balanced recipe (#26451)

Browse Source 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Baizhou Zhang
2026-05-26 23:50:26 -07:00
committed by GitHub
co-authored by Claude Opus 4.7
parent 9060509214
commit d6032c04b6
2 changed files with 6 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs_new/cookbook/autoregressive/DeepSeek/DeepSeek-V4.mdx
+2 -2
View File
@@ -145,8 +145,8 @@ Two variants are exposed:
(~89.5 GPQA on Pro). (~89.5 GPQA on Pro).
Notes: Notes:
- MegaMoE is **not** supported on Hopper (H100 / H200) nor on the `low-latency` / `cp` settings. When running MegaMoE, don't set `--moe-runner-backend` manually. - MegaMoE is **not** supported on Hopper (H100 / H200) nor on the `low-latency` / `balanced` / `cp` settings — it is only wired into the `max-throughput` recipe on Blackwell. When running MegaMoE, don't set `--moe-runner-backend` manually.
- Adjust `SGLANG_OPT_DEEPGEMM_MEGA_MOE_NUM_MAX_TOKENS_PER_RANK` based on your workload and memory usage. Setting higher number of tokens for MegaMoE requires more HBM space. (recommended: 4096 for balanced, 8320 for max-throughput). - Adjust `SGLANG_OPT_DEEPGEMM_MEGA_MOE_NUM_MAX_TOKENS_PER_RANK` based on your workload and memory usage. Setting higher number of tokens for MegaMoE requires more HBM space. (recommended: 8320 for max-throughput).
**GB300 PD-Disagg cross-pod MNNVL** **GB300 PD-Disagg cross-pod MNNVL**
docs_new/src/snippets/autoregressive/deepseek-v4-deployment.jsx
+4 -10
View File
@@ -119,11 +119,11 @@ export const DeepSeekV4Deployment = () => {
const MARLIN_EFFHW = new Set(["h200-fp4", "h100"]); const MARLIN_EFFHW = new Set(["h200-fp4", "h100"]);
const MARLIN_LABEL = { "h200-fp4": "H200 (FP4)", h100: "H100 (FP4)" }; const MARLIN_LABEL = { "h200-fp4": "H200 (FP4)", h100: "H100 (FP4)" };
// MegaMoE is only wired into the deepep-replacing recipes on Blackwell // MegaMoE is only wired into the max-throughput recipe on Blackwell.
// (balanced / max-throughput). Disabled on Hopper (H100 / H200, both FP4 // Disabled on Hopper (H100 / H200, both FP4 and FP8), on
// and FP8), on low-latency / cp recipes, and on PD-Disagg (the cookbook's // low-latency / balanced / cp recipes, and on PD-Disagg (the cookbook's
// PD command builder doesn't emit the megamoe backend / env vars yet). // PD command builder doesn't emit the megamoe backend / env vars yet).
const MEGAMOE_UNSUPPORTED_RECIPES = new Set(["low-latency", "cp", "pd-disagg"]); const MEGAMOE_UNSUPPORTED_RECIPES = new Set(["low-latency", "balanced", "cp", "pd-disagg"]);
const MEGAMOE_UNSUPPORTED_HARDWARE = new Set(["h100", "h200"]); const MEGAMOE_UNSUPPORTED_HARDWARE = new Set(["h100", "h200"]);
const isMegamoeUnsupported = (vals) => const isMegamoeUnsupported = (vals) =>
MEGAMOE_UNSUPPORTED_HARDWARE.has(vals.hardware) || MEGAMOE_UNSUPPORTED_HARDWARE.has(vals.hardware) ||
@@ -864,12 +864,6 @@ export const DeepSeekV4Deployment = () => {
if (megamoe !== "disabled" && recipe === "max-throughput") { if (megamoe !== "disabled" && recipe === "max-throughput") {
megamoeEnv.push("SGLANG_OPT_DEEPGEMM_MEGA_MOE_NUM_MAX_TOKENS_PER_RANK=8320"); megamoeEnv.push("SGLANG_OPT_DEEPGEMM_MEGA_MOE_NUM_MAX_TOKENS_PER_RANK=8320");
} }
// Blackwell balanced always runs with MTP (1/1/2) — when MegaMoE is layered on
// top, cap the per-rank dispatch buffer at 4096 to keep MoE memory in budget.
// (megamoe is gated to Blackwell by MEGAMOE_UNSUPPORTED_HARDWARE.)
if (megamoe !== "disabled" && recipe === "balanced") {
megamoeEnv.push("SGLANG_OPT_DEEPGEMM_MEGA_MOE_NUM_MAX_TOKENS_PER_RANK=4096");
}
if (megamoe === "w4a4") { if (megamoe === "w4a4") {
megamoeEnv.push("SGLANG_OPT_DEEPGEMM_MEGA_MOE_USE_FP4_ACTS=1"); megamoeEnv.push("SGLANG_OPT_DEEPGEMM_MEGA_MOE_USE_FP4_ACTS=1");
megamoeEnv.push("SGLANG_OPT_DEEPGEMM_MEGA_MOE_USE_MXF4_KIND=1"); megamoeEnv.push("SGLANG_OPT_DEEPGEMM_MEGA_MOE_USE_MXF4_KIND=1");