docs: add cookbook-migrate-model skill from the Qwen3.5 pilot (#27845)

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
zijiexia
2026-06-12 18:31:52 -07:00
committed by GitHub
co-authored by Claude Fable 5
parent 5633ca8599
commit 29ac249be1
9 changed files with 579 additions and 22 deletions
+60 -9
View File
@@ -48,6 +48,19 @@ than restating.
(`{id,label,vram,vendor}`), **not** added to the engine catalog.
- `placeholders` declares every `{{KEY}}` used in `curl` or any cell.
- `modelNames` covers every cell (by `hw|variant|quant` triple or `variant|quant` pair).
- `strategies` count matches the page's operating points — 1 recipe → a single `balanced`;
2 → `low-latency` + `high-throughput`; 3 → the full trio. Tiers apply per
(hw × variant × quant) combination: a single-recipe combination must park under its
semantically honest tier (clear slant → that tier, e.g. a workstation card under
`low-latency`; no slant → `balanced`, e.g. a CPU platform) — **flag a no-slant recipe
parked under low-latency/high-throughput**. Mixed unions
like [low-latency, balanced, high-throughput] with per-selection greying are fine. Also
flag model-specific ids (e.g. `mtp`), and flag an INVERTED speculative mapping — the
deterministic default is MTP/spec-decoding ON → `low-latency`, OFF → `high-throughput`
(at saturation the draft+verify overhead outweighs the speedup); the reverse needs an
explicit maintainer-confirmed justification in the PR. The MDX strategy bullets describe serving semantics
in the DSv4 style (single-user chat / typical multi-user / batch jobs), not internal
toggles.
- `dockerImages` covers the hw ids that have cells (else users hit the `:dev` fallback).
- `multiNodeHints` present ONLY for hw whose fabric needs manual NIC env (e.g. `gb200`
NVL72) — NOT every `multi-N` hw (standard-IB DeepEP / Marlin multi-node don't need it).
@@ -55,9 +68,13 @@ than restating.
template's `model` field is a free-form input prefilled from this value; if the config
omits the `github` block, the engine falls back to `deepseek-ai/deepseek-v4` and the
page's submissions get mislabeled.
- `playgroundFeatures` axes are pruned to what the model supports — no empty/stub axes
(the `moe` axis's MegaMoE backend option + `megamoeQuant` block only on Blackwell MoE,
gated by `requiresHw`; `hisparse` only DSA-style; `pdDisagg.router` only with a PD topology).
- `playgroundFeatures` is opt-OUT: the **general axes ship on every cookbook by default**
(`attention` TP/CP/DP-Attn, `moe` backend+EP for MoE models, `parsers`, `speculative`,
`pdDisagg`, `hicache`) — flag a missing general axis unless the model genuinely cannot
use it. Model-specific axes only where applicable (MegaMoE backend + `megamoeQuant`
only on Blackwell MoE, gated by `requiresHw`; `hisparse` only DSA-style). Knobs that are
meaningless for a subset of variants/hw are `disable`d with a reason, not silently live
(e.g. MoE knobs greyed on dense variants). No empty/stub axes.
- **No leftover `__TOKEN__`** — the config was stamped from the template and every
placeholder is filled (`grep -rn '__[A-Z_]*__'` on the new config/benchmarks/MDX returns
nothing).
@@ -71,7 +88,18 @@ than restating.
- NO `--nnodes` / `--node-rank` / `--dist-init-addr` literals in multi-node cells
(the renderer injects them from `match.nodes`).
- NO literal `--host` / `--port` — use `{{HOST_IP}}` / `{{PORT}}`.
- Flag order: `--model-path` first, then parallelism, then MoE, then tuning, `--host`/`--port`
- NO `--reasoning-parser` / `--tool-call-parser` in any cell — parsers are a
Playground-only feature added on top of the base command (DSv4 convention);
flag any cell that bakes them in.
- Accuracy-degrading flags in cells — runtime quant below the checkpoint
(e.g. MegaMoE W4A4 — DSv4 gates it behind the Playground's `megamoeQuant`)
and lossy `--kv-cache-dtype` (e.g. `fp8_e4m3` over a higher-precision-KV
checkpoint): **flag for explicit maintainer confirmation**. Output quality
should be exactly what the quant chip declares, so absent a recorded
sign-off in the PR (e.g. carried verbatim from a measured legacy recipe's
default command), request the flag move to Playground/tips.
- Flag order: `--model-path` first (an optional `--trust-remote-code` may precede it —
the DSv4 cells do), then parallelism, then MoE, then tuning, `--host`/`--port`
last (the playground's insert anchors assume this).
- TP/memory sanity: `model_weight_GB / (tp × gpu_mem)` fits with ~2030% headroom
(BF16 ≈ params×2 GB, FP8 ≈ ×1, FP4 ≈ ×0.5; MoE uses **total** weight, not active params).
@@ -88,6 +116,13 @@ than restating.
from the `sglang serve` deploy command.
- `sglang_version` is a real build the author ran (a release, or `dev`/nightly) — not a
guessed/placeholder value (no leftover `0.0.0`).
- **Consistent accuracy harness across entries**: every value under one `accuracyLabels`
column must be produced by the SAME harness — flag a page that, say, measures one
platform's GSM8K with `few_shot_gsm8k --num-questions 200` and another's with
`run_eval --eval-name gsm8k --num-examples 1319` and shows both as one "GSM8K %"
(the scores aren't comparable). Either standardize on one harness (matching
`benchmarkCommands.accuracy`) or require an explicit per-entry note. Common when folding
a second contributor's measurements (e.g. an AMD/ROCm PR) into the page.
### 5. Doc ↔ config parity (the #1 finding)
- Any `sglang serve` command shown in MDX prose (config tips, benchmark section) must
@@ -133,6 +168,16 @@ than restating.
or `../`-relative links. `docs.sglang.io` is canonical.
- No Google-Drive image links (don't render). Shell placeholders are `export VAR=<value>`,
not `${VAR}` (a bash no-op).
- **Parser ids must exist in the code registries** on the PR's target branch: every
`--reasoning-parser X` / `--tool-call-parser Y` named in prose or in
`playgroundFeatures.parsers` flags is a registered key in
`python/sglang/srt/parser/reasoning_parser.py` (DetectorMap) /
`python/sglang/srt/function_call/function_call_parser.py` (ToolCallParserEnum) —
prose naming a near-miss id (e.g. the reasoning id where the tool id differs) is a
factual bug. `--…-parser auto` is acceptable ONLY if the template-detection rules
(`python/sglang/srt/managers/template_detection.py`) actually resolve THIS model's
chat template to the right parser — no rule match means auto silently disables the
parser; when in doubt require explicit ids (the DSv4 page pins explicit ids).
### 9b. MDX authoring (Mintlify) — detail in `cookbook-add-model/references/mintlify-authoring.md`
- **Forbidden syntax**: no Docusaurus admonitions (`:::`), `@site`/`@theme`, GitHub alert
@@ -140,17 +185,23 @@ than restating.
or unknown components. `<CardGroup>`/`<Card>` only on category `intro.mdx`, not model pages.
- Code fences are **labeled** (e.g. `python Example` / `bash Command` / `text Output` after
the opening fence); a fenced block nested inside another uses four backticks outside.
- Every runnable invocation block is followed by `**Output Example:**` + a `text Output`
fenced block (real output, or `Pending update...` only with user acknowledgement).
- §3 commands and outputs are **collapsible** (DeepSeek-V4 pattern): every runnable
example wrapped in an `<Accordion>`, its real output in a following
`<Accordion title="Example Output">` (`Pending update...` only with user
acknowledgement). Flag bare/inline example blocks and `**Output Example:**` headings.
- Reasoning-parser example matches the parser's **output shape**: separate-field
(`reasoning_content` + `content`) vs inline `<think>` tags parsed out of `content`.
- No hardcoded sampling params (`temperature` / `top_p`) in sample code (SGLang uses
`generation_config.json` defaults); listing them in §1 informationally is fine.
### 10. Quantization rules
- FP4 is Blackwell-only (B200/B300/GB300) — never AMD; AMD FP4 chips must be `disabled`.
- BF16 / FP8 work on NVIDIA and AMD. FP8 configs adding `--kv-cache-dtype fp8_e4m3` should
note the accuracy trade-off.
- **NVFP4** checkpoints are Blackwell-only (B200/B300/GB300) — never AMD. An AMD FP4 cell
is legitimate ONLY when the vendor published an **MXFP4** checkpoint for it (e.g.
`amd/Qwen3.5-397B-A17B-MXFP4` on MI355X) — verify the HF repo resolves; otherwise the
AMD FP4 chip must be absent/`disabled`.
- BF16 / FP8 work on NVIDIA and AMD. `--kv-cache-dtype fp8_e4m3` in a cell is an
accuracy-degrading flag — see §3 (needs explicit maintainer sign-off; default
home is Playground/tips).
### 11. Scope
- Changes match the PR title. Flag global changes hiding behind a platform-specific title