[docs] Tell Qwen3.8-27B DFLASH2 users to build from main (#35753)

This commit is contained in:
Jimmy Shong
2026-08-20 23:34:44 +00:00
committed by GitHub
parent 779e593bd1
commit 1a138e13b9
6 changed files with 122 additions and 25 deletions
@@ -20,6 +20,9 @@ For all methods and hardware platforms, see the [official SGLang installation gu
pip install --upgrade pip
pip install uv
uv pip install sglang
# For the DFLASH2 cells only — DFlash2 selector support is newer than the
# latest release, so install from source (Method 2) instead of the line above.
```
Then run the **Python** output of the command panel below in that environment.
@@ -30,6 +33,10 @@ Then run the **Python** output of the command panel below in that environment.
```bash Command
docker pull lmsysorg/sglang:qwen38-27b
# For the DFLASH2 cells only — that tag predates DFlash2 selector support,
# so pull a nightly built from main instead:
# docker pull lmsysorg/sglang:dev-nightly-0820
```
For how to launch the image, see [Install → Method 3: Using Docker](../../../docs/get-started/install#method-3-using-docker). Substitute the inner `sglang serve ...` with what the command generator below produces.
@@ -38,6 +45,17 @@ For how to launch the image, see [Install → Method 3: Using Docker](../../../d
</Tabs>
<Warning>
**DFLASH2 needs a build that tracks main.** DFlash2 landed in
[#35371](https://github.com/sgl-project/sglang/pull/35371), and DFlash2 + NVFP4
— the quantized `lm_head` path — in
[#35496](https://github.com/sgl-project/sglang/pull/35496). Both are newer than
the pinned `lmsysorg/sglang:qwen38-27b` tag and than the latest PyPI release; a
build without #35496 fails on the NVFP4 cells at boot with `requires a dense
FP16/BF16/FP32 target lm_head`. Every other recipe on this page — no
speculation, MTP, DSpark — runs on the pinned tag as written.
</Warning>
</Accordion>
Pick your card + checkpoint precision to generate the launch command. The model runs single-GPU on every supported card — H200, RTX PRO 6000, RTX 5090 and DGX Spark — and ships one operating point.
@@ -56,19 +74,34 @@ import { Qwen38MambaRatioCalculator } from "/src/snippets/_qwen38_mamba_ratio_ca
<Deployment config={config} />
<Warning>
**Picking DFLASH2 requires SGLang built from main.** DFlash2 landed in
[#35371](https://github.com/sgl-project/sglang/pull/35371), and DFlash2 +
NVFP4 — the quantized `lm_head` path — in
[#35496](https://github.com/sgl-project/sglang/pull/35496). Both are newer
than the `lmsysorg/sglang:qwen38-27b` tag pinned in
[Install SGLang](#install) and than the latest PyPI release; on NVFP4 a build
without #35496 fails at boot with `requires a dense FP16/BF16/FP32 target
lm_head`. Every other Speculative Decoding pick — None, EAGLE, DSPARK — runs
on the pinned tag as written.
</Warning>
<Note>
The RTX 5090 and RTX PRO 6000 cells above — including every Speculative
Decoding / Serving Strategy / SSM dtype combination — were validated at
ISL 8192 / OSL 1024, concurrency 1 — for DFLASH2, to that full standard on
NVFP4, and to boot-and-serve on the RTX PRO 6000 BF16/FP8 cells. On the
remaining platforms the DFLASH2 pick is offered but not yet exercised, and
the composed command carries a `# DFLASH2 on this platform: final
verification in progress` hint line while it is selected. The DGX Spark cells cover that same combination set minus
DFLASH2, but to a weaker standard: each was confirmed to **boot and
serve** at ISL 8192 / OSL 1024, concurrency 1, with no throughput or
acceptance-length numbers taken. The remaining platforms' recipes carry their
original validation, which covers the default overlay picks (plus MTP on
GB300); non-default overlay picks there are valid but unmeasured.
ISL 8192 / OSL 1024, concurrency 1; for DFLASH2, to that full standard on
NVFP4 and to boot-and-serve on the RTX PRO 6000 BF16/FP8 cells. The DGX Spark
cells cover the same combination set minus DFLASH2, but to a weaker standard:
each was confirmed to **boot and serve** at ISL 8192 / OSL 1024,
concurrency 1, with no throughput or acceptance-length numbers taken. The
H200 and GB300 recipes carry their original validation, which covers the
default overlay picks (plus MTP on GB300); non-default overlay picks there
are valid but unmeasured.
DFLASH2 has not been exercised on H200, DGX Spark or GB300: selecting it
there switches the cell's badge to **Final Verification In Progress** until
that round lands. The other picks on those platforms keep the validation
described above.
</Note>
### Mamba ratio calculator
@@ -234,8 +267,8 @@ checkpoint's calibration scales automatically.
is NVFP4-packed; the BF16 and FP8 checkpoints keep a dense head).
Validation: NVFP4 measured end-to-end on RTX PRO 6000 and RTX 5090; the
RTX PRO 6000 BF16/FP8 cells boot and serve; on H200, DGX Spark and GB300
the pick is offered with a final-verification-in-progress hint line
above the command. The
those cells carry the **Final Verification In Progress**
badge. The
RTX PRO 6000 recipe needs no changes. On the 32GB RTX 5090 prefer
`--mamba-ssm-dtype bfloat16` at `--mem-fraction-static 0.90`: measured
strictly better than float32 for this draft (6.1 vs 8.3 ms TPOT, accept
+14
View File
@@ -169,6 +169,20 @@ for (const path of walk(CONFIGS)) {
}
}
};
// A cell may report its badge per selection (`verificationStatus` as a
// function of sel), so it has to survive the same space the predicates do —
// it renders on every pick, and an unrecognized return silently downgrades
// the badge to "Not Verified" rather than erroring in the browser.
const VERIFY_STATES = ["verified", "in-progress", "unverified"];
for (const [i, cell] of (config.cells || []).entries()) {
if (typeof cell.verificationStatus !== "function") continue;
probe((sel) => {
const out = cell.verificationStatus(sel);
if (out !== undefined && !VERIFY_STATES.includes(out)) {
throw new Error(`returned ${JSON.stringify(out)}, expected one of [${VERIFY_STATES}]`);
}
}, `cells[${i}].verificationStatus`);
}
for (const dim of [...(config.matchDims || []), ...(config.overlayDims || [])]) {
if (typeof dim.showWhen === "function") probe(dim.showWhen, `${dim.id}.showWhen`);
for (const opt of (dim.options || [])) {
+13 -4
View File
@@ -38,7 +38,10 @@
// `verified` is the boolean badge baseline.
// `verificationStatus` overrides it with a third state —
// "verified" | "in-progress" | "unverified" — for a recipe
// whose verification round is open rather than absent.
// whose verification round is open rather than absent. It
// may also be a function of the selection, for a cell whose
// verification depends on an overlay pick (e.g. one
// speculative option still being validated).
// modelNames HF slug lookup, `hw|variant|quant`, `variant|quant`,
// `hw|quant`, `quant`, `hw`, then `default`
// placeholders {{KEY}} → {target: 'command'|'curl', label, default?}
@@ -443,8 +446,14 @@ export const Deployment = ({ config, benchmarks }) => {
typeof v === "string"
? (VERIFY_LABEL[v] ? v : "unverified")
: (v ? "verified" : "unverified");
const cellVerifyStatus = (c) =>
c ? verifyStatusOf(c.verificationStatus ?? c.verified) : "unverified";
const cellVerifyStatus = (c, sel) => {
if (!c) return "unverified";
const v =
typeof c.verificationStatus === "function"
? c.verificationStatus(sel)
: c.verificationStatus;
return verifyStatusOf(v ?? c.verified);
};
// Two kinds of selector row:
// match dims participate in cell lookup (cell.match[dim] === sel[dim])
@@ -1240,7 +1249,7 @@ export const Deployment = ({ config, benchmarks }) => {
// ==== 5. Derived values ====
const s = makeStyles(isDark);
const cell = findCell(config.cells, sel);
const verifyStatus = cellVerifyStatus(cell);
const verifyStatus = cellVerifyStatus(cell, sel);
// Pin the calculator-computed ratio into the rendered command (before the
// host/port tail); cells themselves stay ratio-free.
const cellWithRatio = (() => {
+35 -8
View File
@@ -122,17 +122,12 @@ export const config = {
// selector projects through the target lm_head — including
// quantized heads — so it runs on the NVFP4 checkpoint too.
// Validated on the SM120 pair (NVFP4 measured end to end; the
// RTX PRO 6000 BF16/FP8 cells boot-and-serve). On the other
// platforms the pick is offered with the in-progress hint below
// — the recipe composes from validated cells but has not been
// exercised there yet.
// RTX PRO 6000 BF16/FP8 cells boot-and-serve). The platforms where
// it has not been exercised carry verificationStatus "in-progress"
// on their cells.
disabled: (sel) => sel.hw === "rtx5090" && sel.quant !== "nvfp4",
disableReason:
"On the 32GB RTX 5090 the DFlash2 draft model only fits on top of the NVFP4 weights",
hints: (sel) =>
["rtx6000", "rtx5090"].includes(sel.hw)
? []
: ["DFLASH2 on this platform: final verification in progress"],
// 5090: mem-fraction re-pins like DSPARK's, and fp32 additionally
// re-pins the ratio — the balanced L=9216 value leaves the state
// pool one slot short at every serviceable mem-fraction (see the
@@ -405,6 +400,10 @@ export const config = {
// runnable, but not a recipe this page ships.
match: { hw: "h200", variant: "default", quant: "fp8", nodes: "single" },
verified: true,
// DFLASH2 has not been exercised on this platform; every other overlay
// pick keeps this cell's original validation.
verificationStatus: (sel) =>
sel.spec === "dflash" ? "in-progress" : "verified",
env: [],
flags: [
"--trust-remote-code",
@@ -424,6 +423,10 @@ export const config = {
// H200, BF16 reference checkpoint (~54GB of weights).
match: { hw: "h200", variant: "default", quant: "bf16", nodes: "single" },
verified: true,
// DFLASH2 has not been exercised on this platform; every other overlay
// pick keeps this cell's original validation.
verificationStatus: (sel) =>
sel.spec === "dflash" ? "in-progress" : "verified",
env: [],
flags: [
"--trust-remote-code",
@@ -545,6 +548,10 @@ export const config = {
// exercises the 4-bit `lm_head` this checkpoint quantizes, with no shape
// error.
verified: true,
// DFLASH2 has not been exercised on this platform; every other overlay
// pick keeps this cell's original validation.
verificationStatus: (sel) =>
sel.spec === "dflash" ? "in-progress" : "verified",
env: [],
flags: [
"--trust-remote-code",
@@ -563,6 +570,10 @@ export const config = {
match: { hw: "dgx-spark", variant: "default", quant: "fp8", nodes: "single" },
// All 12 overlay combinations served on GB10.
verified: true,
// DFLASH2 has not been exercised on this platform; every other overlay
// pick keeps this cell's original validation.
verificationStatus: (sel) =>
sel.spec === "dflash" ? "in-progress" : "verified",
env: [],
flags: [
"--trust-remote-code",
@@ -582,6 +593,10 @@ export const config = {
// All 12 overlay combinations served on GB10. Heaviest checkpoint, so
// it holds the sweep's tightest cell: DSPARK + float32 + extra_buffer.
verified: true,
// DFLASH2 has not been exercised on this platform; every other overlay
// pick keeps this cell's original validation.
verificationStatus: (sel) =>
sel.spec === "dflash" ? "in-progress" : "verified",
env: [],
flags: [
"--trust-remote-code",
@@ -603,6 +618,10 @@ export const config = {
{
match: { hw: "gb300", variant: "default", quant: "nvfp4", nodes: "single" },
verified: true,
// DFLASH2 has not been exercised on this platform; every other overlay
// pick keeps this cell's original validation.
verificationStatus: (sel) =>
sel.spec === "dflash" ? "in-progress" : "verified",
env: [],
flags: [
"--trust-remote-code",
@@ -619,6 +638,10 @@ export const config = {
{
match: { hw: "gb300", variant: "default", quant: "fp8", nodes: "single" },
verified: true,
// DFLASH2 has not been exercised on this platform; every other overlay
// pick keeps this cell's original validation.
verificationStatus: (sel) =>
sel.spec === "dflash" ? "in-progress" : "verified",
env: [],
flags: [
"--trust-remote-code",
@@ -635,6 +658,10 @@ export const config = {
{
match: { hw: "gb300", variant: "default", quant: "bf16", nodes: "single" },
verified: true,
// DFLASH2 has not been exercised on this platform; every other overlay
// pick keeps this cell's original validation.
verificationStatus: (sel) =>
sel.spec === "dflash" ? "in-progress" : "verified",
env: [],
flags: [
"--trust-remote-code",