docs: add cookbook-migrate-model skill from the Qwen3.5 pilot (#27845)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
zijiexia
co-authored by
Claude Fable 5
Claude Fable 5
parent
5633ca8599
commit
29ac249be1
@@ -32,7 +32,7 @@ the full contract):
|
||||
| `hardware` | `{id,label,vram,vendor}[]` | Optional. GPUs the shared `HARDWARE_CATALOG` doesn't carry (workstation / desktop / future chips, e.g. RTX PRO 6000). The engine merges these into the catalog, so a model-specific GPU is config data — **no engine-catalog edit**. Also add the id to `supportedHardware`. |
|
||||
| `variants` | `{id, label, subtitle?}[]` | 2nd-dim option list. Use `default` / single-element if the model has no variant axis. |
|
||||
| `quantizations` | `{id, label}[]` | 3rd-dim option list. |
|
||||
| `strategies` | `{id, label}[]` | 4th-dim option list. Common ids: `low-latency`, `balanced`, `high-throughput`. |
|
||||
| `strategies` | `{id, label}[]` | 4th-dim option list. Canonical ids: `low-latency` / `balanced` / `high-throughput` (never model-specific ids like `mtp`). **The count follows the page's operating points**: one recipe → a single `balanced`; two → `low-latency` + `high-throughput`; three → the full trio (the ideal). Tiers apply per (hw × variant × quant) combination — a single-recipe combination parks under its semantically honest tier (clear slant → that tier, e.g. DSv4's RTX 6000 → `low-latency`; no slant → `balanced`, e.g. Qwen3.5's Xeon); the page's list is the union and the engine greys unused chips per selection. Never invent a recipe just to fill chips. When two recipes differ by MTP / speculative decoding, the assignment is deterministic: spec ON → `low-latency`, spec OFF → `high-throughput` (at saturation the draft+verify overhead outweighs the speedup — same reason DSv4's high-throughput recipes disable MTP). The recurring markers in the other direction: dp-attention ON (MLA-attention models) and EP / DP+EP ON (MoE models) → `high-throughput`. |
|
||||
| `nodesOptions` | `{id, label}[]` | 5th-dim option list. The `id` MUST be `single` or `multi-N` — the engine parses N from the id for `--nnodes`. |
|
||||
| `cells` | `{match, verified?, env, flags}[]` | One per supported (hw × variant × quant × strategy × nodes) combination. See §2.2. |
|
||||
| `modelNames` | `{[key]: string}` | HF slug lookup. Keys are either `hw\|variant\|quant` (most specific) or `variant\|quant` (fallback). |
|
||||
@@ -91,6 +91,16 @@ Each cell describes one verified (or auto-estimated) launch recipe.
|
||||
flags, then tuning knobs, with `--host` / `--port` last. The playground
|
||||
engine assumes this ordering when inserting overrides (its anchors target
|
||||
`--model-path` / `--tp` / etc., and inserts before the `--host` tail).
|
||||
- Accuracy-degrading flags don't belong in cells by default: a cell's
|
||||
output quality should be exactly what its quantization chip declares.
|
||||
Runtime quant below the checkpoint's precision (e.g. MegaMoE **W4A4** —
|
||||
DSv4 gates it behind the Playground's `megamoeQuant` opt-in) and lossy
|
||||
KV-cache dtypes (`--kv-cache-dtype fp8_e4m3` over a higher-precision-KV
|
||||
checkpoint) default to Playground opt-ins or §2-tips material. If the
|
||||
model's recipe genuinely needs one in a cell, **flag it to the user and
|
||||
get explicit confirmation** — never ship it silently. (Migrations are the
|
||||
sanctioned exception: a flag baked into the legacy recipe's default
|
||||
command keeps verbatim — see the migrate skill.)
|
||||
|
||||
**Cells are denormalized on purpose** — common flags repeat across cells.
|
||||
This makes each cell self-contained and easy to verify. When sweeping a
|
||||
@@ -103,9 +113,14 @@ automatically.
|
||||
|
||||
## 2.3 Configure `playgroundFeatures` (optional)
|
||||
|
||||
The Playground widget is opt-in per axis. Add only the axes that make sense
|
||||
for this model. Recognised axis keys and their schemas (full reference in
|
||||
the `_playground.jsx` header):
|
||||
The Playground is **opt-out, not opt-in**: every cookbook ships the general
|
||||
axes by default — `attention` (TP/CP/DP-Attn), `moe` (backend + EP, for MoE
|
||||
models), `parsers`, `speculative`, `pdDisagg`, `hicache` — then adds
|
||||
model-specific axes (e.g. MegaMoE for DeepSeek-V4) and deletes ONLY the axes
|
||||
this model genuinely cannot use (e.g. `hisparse` on non-DSA models, `moe` on a
|
||||
pure-dense model). Knobs that don't apply to a subset of variants/hw get
|
||||
`disable` + `disableReason`, not removal. Recognised axis keys and their
|
||||
schemas (full reference in the `_playground.jsx` header):
|
||||
|
||||
| Axis key | Widget | Use when |
|
||||
|---|---|---|
|
||||
|
||||
@@ -7,6 +7,10 @@ cookbook is data-only and never needs this. The current 7 built-in axes
|
||||
(MegaMoE is not its own axis — it lives inside `moe` as a backend option +
|
||||
a `megamoeQuant` sub-select.) Only add a new axis if a real cookbook needs it
|
||||
and the feature does not fit any existing axis. Touches `_playground.jsx` only.
|
||||
One routine trigger: the `cookbook-migrate-model` skill requires every legacy
|
||||
control to survive as a dimension or a Playground axis — a legacy feature no
|
||||
built-in axis covers (e.g. Nemotron3's "KV Cache DType" radio) lands here
|
||||
first, as its own engine PR ahead of the migration PR.
|
||||
|
||||
For the per-model config/cells/MDX reference see [authoring-reference.md](authoring-reference.md).
|
||||
|
||||
|
||||
@@ -61,9 +61,12 @@ table is a live reference.
|
||||
- **Tool-call follow-up on thinking models**: the final assistant turn may put text in
|
||||
`reasoning_content` instead of (or with) `content` — print both so the output isn't a
|
||||
misleading `None`.
|
||||
- **Every runnable block** is immediately followed by `**Output Example:**` + a
|
||||
` ```text Output ` block with **real** server output (verbatim, not paraphrased).
|
||||
`Pending update...` is acceptable only with the user's explicit acknowledgement.
|
||||
- **§3 commands and outputs are collapsible (required)**: every runnable example
|
||||
lives in an `<Accordion title="… (Python)">` and its **real** server output
|
||||
(verbatim, not paraphrased) in an immediately following
|
||||
`<Accordion title="Example Output">` — match the DeepSeek-V4 §3 pattern. No
|
||||
inline `**Output Example:**` headings / bare blocks. `Pending update...` is
|
||||
acceptable only with the user's explicit acknowledgement.
|
||||
- **Do not hardcode sampling params** (`temperature`, `top_p`) in sample code — SGLang
|
||||
uses `generation_config.json` defaults. Listing "Recommended generation" in §1 is fine.
|
||||
- Format raw API objects (`ChatCompletionMessage(...)`) into readable Reasoning /
|
||||
|
||||
Reference in New Issue
Block a user