[Docs] Add Qwen3.8 cookbook (#34587)

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
zijiexia
2026-08-12 08:07:46 -07:00
committed by GitHub
co-authored by Claude Opus 5
parent 197832bcf5
commit 8e7c07fae7
9 changed files with 1305 additions and 6 deletions
@@ -164,7 +164,7 @@ schemas (full reference in the `_playground.jsx` header):
| `moe` | Backend select (incl. MegaMoE) + EP knob; picking the MegaMoE backend reveals a Quantization sub-select (W4A8/W4A4) | Model is MoE and supports multiple `--moe-*-backend` choices. For Blackwell MoE kernel-fusion, give the `megamoe` backend option a `requiresHw` (and optional `excludesStrategy`) gate, then add a sibling `megamoeQuant` block (`{stripEnv, options}`): W4A8 = `NUM_MAX` only, W4A4 adds the FP4-activations env vars; both strip the DeepEP dispatch env. |
| `parsers` | Multi-toggle | Model has reasoning / tool-call parsers. |
| `speculative` | Single-select chip group | Model has spec-decoding presets you want to expose. |
| `pdDisagg` | Mode + transfer backend (+ optional per-backend env via `envWhen` hw-gate) + IB device + optional `router{port, command}` | Model supports prefill/decode disaggregation. When a PD role is active and `router` is set, the playground shows the router (SGLang Model Gateway) launch command as a separate companion block and retargets the cURL modal to `router.port` (clients hit the router, not the role servers). |
| `pdDisagg` | Mode + transfer backend (+ optional per-backend env via `envWhen` hw-gate) + IB device + optional `router{port, command}` | Model supports prefill/decode disaggregation. A `modes[]` entry may carry `flags` / `env` that only that role needs (`prefill`'s `--load-balance-method`, `decode`'s `--disaggregation-decode-polling-interval`, ...); they are emitted only while that role is selected, and a flag whose head the base cell already sets is replaced rather than duplicated. Put role-specific settings here, NOT in the cells — a cell carries one recipe, and the role is a Playground overlay. In `router.command` the ports MUST be `{{PREFILL_PORT}}` / `{{DECODE_PORT}}` / `{{ROUTER_PORT}}`: the engine substitutes them from its own `PD_PORTS` (prefill 30000, decode **30100**), so a literal port silently points the router at something the generated decode command never binds. When a PD role is active and `router` is set, the playground shows the router (SGLang Model Gateway) launch command as a companion block and retargets the cURL modal to `router.port`. |
| `hicache` | Enable + storage + write policy | Model is large enough that hierarchical KV cache matters. |
| `hisparse` | Enable + host-ratio select; whole card gated on the live PD-Disagg mode being `decode` | DSA-style model (DeepSeek-V3.2 / V4, GLM-5) that supports decode-side hierarchical sparse attention. |
| `flagSelects` | A config-declared **list** of single-selects, each `{ id, title, stripPrefixes, options }` (option = `{ id, label, flags?, hide?, disable?, disableReason? }`); a flagless option is the "none"/accuracy-safe choice | A titled single-select that picks one value of a flag family the other axes don't model — e.g. KV-cache dtype (`--kv-cache-dtype`), mamba scheduler strategy (`--mamba-scheduler-strategy`). Generic: no engine change to add another. |
@@ -277,14 +277,18 @@ sgl-eval run gsm8k \\
// `auto` is a sentinel (emits no --disaggregation-ib-device flag).
ibDevices: [{ id: "auto", label: "Auto" }, "mlx5_0", "mlx5_7"],
// Router fronting prefill + decode; substitute <prefill-host>/<decode-host>.
// ALWAYS use {{PREFILL_PORT}} / {{DECODE_PORT}} / {{ROUTER_PORT}} — the
// engine fills them from its own PD_PORTS (prefill serves on 30000, decode
// on 30100). A literal port here will not track the engine and sends the
// router at a port the generated decode command does not listen on.
router: {
port: 8000,
command:
`python3 -m sglang_router.launch_router \\
--pd-disaggregation \\
--prefill http://<prefill-host>:30000 \\
--decode http://<decode-host>:30001 \\
--host 0.0.0.0 --port 8000 \\
--prefill http://<prefill-host>:{{PREFILL_PORT}} \\
--decode http://<decode-host>:{{DECODE_PORT}} \\
--host 0.0.0.0 --port {{ROUTER_PORT}} \\
--disable-circuit-breaker \\
--health-check-interval-secs 999999`,
},