diff --git a/docs/cookbook/diffusion/MiniMax/MiniMax-H3.mdx b/docs/cookbook/diffusion/MiniMax/MiniMax-H3.mdx index c752a83d2f..c99f34c573 100644 --- a/docs/cookbook/diffusion/MiniMax/MiniMax-H3.mdx +++ b/docs/cookbook/diffusion/MiniMax/MiniMax-H3.mdx @@ -1690,9 +1690,12 @@ pays 2 to 3 s of allocator growth and kernel setup. "Steady s/NFE" is the mean of forwards 2 to 8. The OpenVDN reference rows ran the released inference stack (`8nfe_tuned_fp8.yaml`, `infer_ulysses.py`) on the same host and the same clip length, with its Ulysses layout (`parallel.softmax_ranks`) -swept at every GPU count rather than assumed. Every row below comes from one -campaign, one arm at a time on an otherwise idle node; a repeated arm moves by -about 3% across nodes, so treat smaller differences as noise: +swept at every GPU count rather than assumed. The SGLang rows were re-measured +in one campaign at `f1a2ee763d`, one arm at a time on an otherwise idle node, +each with the [8×B200 recipe](#7-vdn-h3-hybrid-attention-8-step-distill) and +only `--num-gpus` and `--quantization` changed; the OpenVDN reference rows come +from an earlier campaign on the same host type. A repeated arm moves by about +3% across nodes, so treat smaller differences as noise: | Config | Steady s/NFE | Denoise (8 forwards) | Decode | Peak/GPU | | --- | ---: | ---: | ---: | ---: | @@ -1702,18 +1705,18 @@ about 3% across nodes, so treat smaller differences as noise: | OpenVDN reference, FP8, 2× B200 Ulysses (standard / 1+1 branch-parallel) | 4.87 / 5.04 | 39.0 / 40.3 s | — | — | | OpenVDN reference, FP8, 4× B200 Ulysses (3+1 branch-parallel) | 2.59 | 20.7 s | — | — | | OpenVDN reference, FP8, 8× B200 Ulysses (5+3 / 4+4 / standard) | 1.40 / 1.47 / 1.52 | 11.2 / 11.7 / 12.2 s | — | — | -| SGLang, `--quantization bf16`, 1× B200 | 7.45 | 58.8 s | 8.8 s | 156,298 MB | -| SGLang, `--quantization fp8` (→ online `mxfp8` on SM100+), 1× B200 | 6.03 | 47.6 s | 8.7 s | 128,418 MB | -| SGLang default (online `mxfp8`), 2× B200 Ulysses2, `--performance-mode speed`, served-shape warmup | **3.30** | **25.9 s** | 4.7 s | 106,738 MB | -| SGLang, `--quantization bf16`, 4× B200 Ulysses4, `--performance-mode speed`, served-shape warmup | 2.06 | 16.2 s | 2.3 s | 108,434 MB | -| SGLang default (online `mxfp8`), 4× B200 Ulysses4, `--performance-mode speed`, served-shape warmup | **1.67** | **13.1 s** | 2.4 s | 85,244 MB | -| SGLang default (online `mxfp8`), 8× B200 Ulysses8, `--performance-mode speed`, served-shape warmup | **0.88** | **6.9 s** | 1.5 s | 79,972 MB | -| SGLang, per-channel fp8 weight scales (the online `fp8` path of other models; on SM100+ `--quantization fp8` resolves to `mxfp8`, so this arm selects the per-channel path explicitly), 8× B200 Ulysses8 | 0.98 | 7.7 s | 1.5 s | 79,504 MB | +| SGLang, `--quantization bf16`, 1× B200, `--performance-mode speed`, served-shape warmup | 7.27 | 58.2 s | 7.0 s | 156,298 MB | +| SGLang, `--quantization fp8` (→ online `mxfp8` on SM100+), 1× B200, `--performance-mode speed`, served-shape warmup | 5.93 | 47.4 s | 7.0 s | 128,420 MB | +| SGLang default (online `mxfp8`), 2× B200 Ulysses2, `--performance-mode speed`, served-shape warmup | **3.25** | **26.0 s** | 3.7 s | 102,664 MB | +| SGLang, `--quantization bf16`, 4× B200 Ulysses4, `--performance-mode speed`, served-shape warmup | 2.05 | 16.4 s | 2.1 s | 104,358 MB | +| SGLang default (online `mxfp8`), 4× B200 Ulysses4, `--performance-mode speed`, served-shape warmup | **1.67** | **13.3 s** | 2.1 s | 81,168 MB | +| SGLang default (online `mxfp8`), 8× B200 Ulysses8, `--performance-mode speed`, served-shape warmup | **0.85** | **6.8 s** | 1.4 s | 76,918 MB | +| SGLang, per-channel fp8 weight scales (the online `fp8` path of other models; on SM100+ `--quantization fp8` resolves to `mxfp8` and no flag selects the per-channel path, so this arm forces it), 8× B200 Ulysses8 | 0.91 | 7.3 s | 1.4 s | 76,450 MB | -On 8× B200 the SGLang Ulysses8 path runs the paper workload at 0.88 s/NFE -(7.0 GPU-seconds per NFE) against the published 1.40 s/NFE of OpenVDN's 5+3 +On 8× B200 the SGLang Ulysses8 path runs the paper workload at 0.85 s/NFE +(6.8 GPU-seconds per NFE) against the published 1.40 s/NFE of OpenVDN's 5+3 branch-parallel layout; the whole request (text encoding, 8 forwards, joint -decode) completes in about 8.7 s after warmup. At this GPU count the step is +decode) completes in about 8.2 s after warmup. At this GPU count the step is launch- and copy-bound in the linear branch rather than FLOP-bound, and the transformer runs online `mxfp8` by default (e4m3 with one E8M0 scale per 32 elements into cuBLASLt's block-scaled GEMM, with the activation quant fused @@ -1722,11 +1725,11 @@ into the adaLN and SwiGLU producers). Against the published 8× B200 headline (1.40 s/NFE, 11.2 GPU-seconds per NFE), the SGLang 4× B200 run spends 6.7 GPU-seconds per NFE, with half the GPUs and half the all-to-all fan-out. At equal GPU count -SGLang is 1.48× faster at two cards (3.30 vs 4.87 s/NFE), 1.55× at four -(1.67 vs 2.59) and 1.60× at eight (0.88 vs 1.40), while on one card the two -stacks are within 4% of each other (6.03 vs 6.25 s/NFE quantized, 7.45 vs 7.66 +SGLang is 1.50× faster at two cards (3.25 vs 4.87 s/NFE), 1.56× at four +(1.67 vs 2.59) and 1.66× at eight (0.85 vs 1.40), while on one card the two +stacks are within 5% of each other (5.93 vs 6.25 s/NFE quantized, 7.27 vs 7.66 in BF16). The advantage is therefore parallel scaling rather than per-GPU -kernel work: SGLang holds 86-91% parallel efficiency from 2 to 8 cards against +kernel work: SGLang holds 88-91% parallel efficiency from 2 to 8 cards against the reference stack's 56-64%, and sweeping the reference stack's own layout does not close it (its best split is 8% better than standard Ulysses at eight cards, 1.40 against 1.52 s/NFE). diff --git a/python/sglang/multimodal_gen/.agents/skills/sglang-diffusion-benchmark-profile/scripts/bench_diffusion_denoise.py b/python/sglang/multimodal_gen/.agents/skills/sglang-diffusion-benchmark-profile/scripts/bench_diffusion_denoise.py index 765eccafbe..4ed6217de5 100755 --- a/python/sglang/multimodal_gen/.agents/skills/sglang-diffusion-benchmark-profile/scripts/bench_diffusion_denoise.py +++ b/python/sglang/multimodal_gen/.agents/skills/sglang-diffusion-benchmark-profile/scripts/bench_diffusion_denoise.py @@ -441,7 +441,7 @@ MODELS = { "aspect_ratio": "16:9", "duration_seconds": 10.0, }, - "num_inference_steps": 9, + "num_inference_steps": 8, }, "extra_args": [ "--num-gpus=4", @@ -451,7 +451,7 @@ MODELS = { ], "force_eager": True, }, - # OpenVDN paper workload: 1344x768, 14.375 s (latent_t 102), t2va, 9 grid points = 8 NFE + # OpenVDN paper workload: 1344x768, 14.375 s (latent_t 102), t2va, 8 steps = 8 NFE "vdn-h3": { "path": "OpenVDN/vdn-minimax-h3", "prompt": ( @@ -467,7 +467,7 @@ MODELS = { "aspect_ratio": "16:9", "duration_seconds": 14.375, }, - "num_inference_steps": 9, + "num_inference_steps": 8, }, "extra_args": [ "--num-gpus=8",