diff --git a/.claude/skills/cookbook-add-model/references/engine-axis.md b/.claude/skills/cookbook-add-model/references/engine-axis.md index 92747e189..9caea4582 100644 --- a/.claude/skills/cookbook-add-model/references/engine-axis.md +++ b/.claude/skills/cookbook-add-model/references/engine-axis.md @@ -4,13 +4,36 @@ Loaded on demand by the `cookbook-add-model` skill. **Rare** — adding a model cookbook is data-only and never needs this. The current 7 built-in axes (`attention`, `moe`, `parsers`, `speculative`, `pdDisagg`, `hicache`, `hisparse`) already cover the SGLang feature surface most cookbooks need. -(MegaMoE is not its own axis — it lives inside `moe` as a backend option + -a `megamoeQuant` sub-select.) Only add a new axis if a real cookbook needs it -and the feature does not fit any existing axis. Touches `_playground.jsx` only. -One routine trigger: the `cookbook-migrate-model` skill requires every legacy -control to survive as a dimension or a Playground axis — a legacy feature no -built-in axis covers (e.g. Nemotron3's "KV Cache DType" radio) lands here -first, as its own engine PR ahead of the migration PR. + +**Model-specific features are config DATA, not engine code.** The axis +handlers read options / flags / env / gating straight from +`config.playgroundFeatures`, so a model-specific feature is added as data on +an existing axis with NO engine edit — MegaMoE W4A4 is not its own axis, it's +config data on `moe` (a `megamoe` backend option + a `megamoeQuant` +sub-select). Reach for this file ONLY when a feature's *shape* — its +title + the flag family it strips + its option/state model — is something no +existing axis can express. + +**When you do extend, build a GENERIC primitive, never a model-named handler.** +The right unit is a reusable shape (e.g. "a titled single-select that strips a +configurable flag family and splices the picked option's flags" — exactly the +`speculative` handler's shape, minus its hardcoded title + `--speculative-*` +strip list). Parameterize title, strip-prefixes, and options from config so +the next model of that shape is pure config. Do NOT add a `kvcache` / +`` handler that hardcodes one model's flag — that's the +model-specific-code-in-the-engine anti-pattern this architecture exists to +avoid. (Precedent: Nemotron3's "KV Cache DType" and Qwen3's mamba-cache select +are the SAME single-select shape → one generic primitive serves both, then +both are config.) + +**A new axis is backward-compatible — zero churn on existing configs.** The +runtime is opt-in per key: the apply/render loop does `const fc = +pgFeatures[axisId]; if (!fc) continue;`, so any config that doesn't declare +the key never sees the axis. And a model-specific axis does NOT join the +opt-out "general axes ship on every cookbook" set (that set is an authoring +convention for NEW configs, not a runtime default) — so review-pr won't flag +existing pages for lacking it, and you never touch a merged config. Only the +models that expose the control declare it. Touches `_playground.jsx` only. For the per-model config/cells/MDX reference see [authoring-reference.md](authoring-reference.md). @@ -20,12 +43,15 @@ For the per-model config/cells/MDX reference see [authoring-reference.md](author Before touching the engine, confirm: -- The feature is a STABLE part of the SGLang CLI surface (will appear in - multiple cookbooks, not one-off). -- The feature cannot be expressed as a new option inside an existing axis - (e.g. a new MoE backend belongs in `moe.backend.options`, not a new - axis). -- The feature has a clean strip-prefix → emit-flag pattern. +- The feature cannot be expressed as data on an existing axis (a new MoE + backend belongs in `moe.backend.options`; a new parser in `parsers.items`; + a new spec preset in `speculative.options`). This is the common case — + most "new features" are new options, not new shapes. +- The *shape* is genuinely new (state model + strip pattern), AND you are + adding it as a GENERIC config-parameterized axis (title / strip-prefixes / + options all from config), not a one-model handler. If you'd hardcode a + specific flag like `--kv-cache-dtype`, stop — generalize the shape instead. +- The shape has a clean strip-prefix → emit-flag pattern. If unsure, add it as data first (in one cookbook's config under an existing axis) before promoting it to a built-in axis. diff --git a/.claude/skills/cookbook-migrate-model/SKILL.md b/.claude/skills/cookbook-migrate-model/SKILL.md index 882123760..e7ff742d4 100644 --- a/.claude/skills/cookbook-migrate-model/SKILL.md +++ b/.claude/skills/cookbook-migrate-model/SKILL.md @@ -63,10 +63,12 @@ your dispatch prompt, or ask for it. including combos that look memory-infeasible; keep them verbatim and list them in the PR body for the re-verification track. 4. **Engines are read-only.** `_deployment.jsx` / `_playground.jsx` must not - change in a migration PR. If the model needs an engine capability (a new - axis, accuracy labels, …), that is a separate prior PR. The every-feature - rule (step 2) triggers this routinely: a legacy control no built-in axis - covers means an engine-axis PR lands first. + change in a migration PR. Model-specific features are config DATA consumed + by generic axis handlers (MegaMoE precedent), so they need NO engine + change — only a genuinely new control *shape* does, and then as a one-time + generic primitive (never a model-named handler) on a separate prior PR + (engine-axis.md). KV Cache DType / mamba-cache select trigger this once for + the whole round; afterwards both are config. 5. **`github.cookbookModel` must be set** (`/`, e.g. `qwen/qwen3.5`) and the block never pruned — without it Submit ↗ mislabels as deepseek-v4. The issue template itself needs NO edits (free-form input). @@ -113,10 +115,14 @@ ON — EXCEPT parsers: `--reasoning-parser`/`--tool-call-parser` are NEVER baked into cells, they are Playground-only (DSv4 convention). **Every legacy control survives as an interactive control** — a dimension or a Playground axis, never a tips-only -mention. A feature none of the built-in axes covers (Nemotron3's "KV Cache -DType" radio is the precedent) still lands in the Playground: add the axis -via a separate PRIOR engine PR (engine-axis.md), keeping the migration PR -itself data-only (hard rule 4). The strategy count follows the page's +mention — and a model-specific control is **config data, not engine code** +(MegaMoE W4A4 is all DSv4 config on the existing `moe` axis). It's pure +config whenever it fits an existing axis's data schema. Only a genuinely new +*shape* (Nemotron3's "KV Cache DType" — a titled single-select stripping a +flag family no axis manages) needs the engine, and then as a ONE-TIME +generic config-parameterized primitive (never a model-named handler) on a +separate PRIOR engine PR, keeping the migration PR data-only (hard rule 4, +engine-axis.md). The strategy count follows the page's operating points: **one recipe → a single `balanced`; two → `low-latency` + `high-throughput`; three → the full trio (the ideal)**. The tiers apply per (hw × variant × quant) combination — a single-recipe combination on a diff --git a/.claude/skills/cookbook-migrate-model/references/dimension-mapping.md b/.claude/skills/cookbook-migrate-model/references/dimension-mapping.md index 27b31758d..0bbb23aa7 100644 --- a/.claude/skills/cookbook-migrate-model/references/dimension-mapping.md +++ b/.claude/skills/cookbook-migrate-model/references/dimension-mapping.md @@ -13,9 +13,9 @@ this file is about the *mapping decisions*. | model-size / model-name radio | `variants` | One variant per deployable checkpoint family; single `{id:"default"}` when there's no variant axis (then `modelNames` keys drop the variant half). | | quantization radio | `quantizations` | Real precision ids (`bf16`/`fp8`/`fp4`/`int4`/…). One `fp4` id even when checkpoints differ per vendor — route via `hw\|variant\|quant` triple keys in `modelNames` (NVFP4 on Blackwell vs AMD MXFP4 is the precedent); per-hw greying falls out of which cells exist. | | toggle that **couples** with other parts of the command (changes TP/mem/EP), OR one the legacy page labels with **operating-point words** | `strategies` | The Playground applies pure flag diffs — it cannot do coupled changes. Example: Qwen3.5's MTP toggle bumps TP on three H100 combos → strategies `low-latency` (MTP on) / `high-throughput` (MTP off). **Naming counts like coupling**: GLM-5.1's / Kimi-K2.6's `dpattention` adds only `--dp N --enable-dp-attention` (uncoupled), but its options are subtitled "Low Latency" / "High Throughput" — the page's own named operating-point split → strategies; a flag-only spec toggle riding alongside it stays a Playground axis and bakes per its legacy default. GPU-count radios (GLM-4.7, MiniMax-M2.5/2.7) → budget-tier strategies with the legacy SUPPORT matrix preserved by which cells exist. Strategy count follows the page's operating points: 1 → `balanced`, 2 → `low-latency`+`high-throughput`, 3 → the full trio (§4). | -| toggle that only adds/removes its own flags | Playground axis (+ bake, EXCEPT parsers and accuracy-degrading flags) | **Parsers (`--reasoning-parser` / `--tool-call-parser`) are NEVER baked into cells** — Deployment commands ship without them regardless of the legacy default or the measured command; the `parsers` axis adds them on top (DSv4 convention; cells mirror the legacy generator's parsers-OFF output). Accuracy-degrading toggles are never baked either — §2 caveats (axis-only, accuracy-safe cells). Other flag-only toggles: legacy default ON → bake into cells AND declare the axis so users can strip (red strikethrough); default OFF → keep cells clean, axis preset only. MTP/EAGLE presets → `speculative` axis; dp-attention → a strategy when the legacy page labels it as the operating-point split or when coupled (see the row above), else `attention.dpAttn`. **No fitting built-in axis ≠ drop the feature** — EVERY legacy control survives as an interactive control (a dimension or a Playground axis), never a tips-only mention. When none of the built-in axes fits (Nemotron3's "KV Cache DType" radio is the precedent), add the axis via the engine-axis flow: a separate PRIOR engine PR, then the migration config declares it (hard rule 4). | +| toggle that only adds/removes its own flags | Playground axis (+ bake, EXCEPT parsers and accuracy-degrading flags) | **Parsers (`--reasoning-parser` / `--tool-call-parser`) are NEVER baked into cells** — Deployment commands ship without them regardless of the legacy default or the measured command; the `parsers` axis adds them on top (DSv4 convention; cells mirror the legacy generator's parsers-OFF output). Accuracy-degrading toggles are never baked either — §2 caveats (axis-only, accuracy-safe cells). Other flag-only toggles: legacy default ON → bake into cells AND declare the axis so users can strip (red strikethrough); default OFF → keep cells clean, axis preset only. MTP/EAGLE presets → `speculative` axis; dp-attention → a strategy when the legacy page labels it as the operating-point split or when coupled (see the row above), else `attention.dpAttn`. **EVERY legacy control survives as an interactive control** (a dimension or a Playground axis), never a tips-only mention — but a model-specific control is **config DATA, not engine code**: the axis handler reads options/flags/env/gating straight from `config.playgroundFeatures` (MegaMoE W4A4 is entirely DSv4 config data on the existing `moe` axis — no per-model engine edit). A control that fits an existing axis's data schema is therefore pure config, full stop. Only when the *shape* is genuinely new — a titled single-select that strips a flag family no axis manages, e.g. Nemotron3's "KV Cache DType" (`--kv-cache-dtype`) — does the engine need that shape, and then you add it ONCE as a **generic config-parameterized primitive** (title + strip-prefixes + options all from config), **never** a model-named `kvcache` handler; afterwards this feature and every future one of its shape are pure config. Such a primitive is backward-compatible — runtime is opt-in per key (`if (!fc) continue`), and a model-specific axis is NOT added to the opt-out general set — so it churns zero existing configs (engine-axis.md). | | per-combo hidden option (e.g. spec hidden on Xeon) | absent cells | Don't create cells for combos the legacy widget couldn't produce; the engine greys them automatically. `# Error:` pseudo-commands → no cell + explanation in §2 tips and/or a chip `disable`/`disableReason`. | -| coupled secondary knob (e.g. mamba cache V1/V2) | cells + Playground axis | Bake the correct value per cell following the legacy coupling (Qwen3.5: MTP ⇒ `--mamba-scheduler-strategy extra_buffer` on NVIDIA; AMD/Xeon ⇒ V1/no flag) and document the coupling in §2 tips — AND surface the knob as a Playground axis like every other legacy feature (row above; add the axis when none fits). Baking alone is NOT enough — the every-feature rule supersedes the pilot's cells+prose-only treatment of Qwen3.5's mamba knob (retrofit pending); Qwen3.6 / Qwen3-Coder-Next carry the same knob, so their migrations need the axis in place first. | +| coupled secondary knob (e.g. mamba cache V1/V2) | cells + Playground axis | Bake the correct value per cell following the legacy coupling (Qwen3.5: MTP ⇒ `--mamba-scheduler-strategy extra_buffer` on NVIDIA; AMD/Xeon ⇒ V1/no flag) and document the coupling in §2 tips — AND surface the knob as a Playground axis like every other legacy feature (row above; add the axis when none fits). Baking alone is NOT enough — the every-feature rule supersedes the pilot's cells+prose-only treatment of Qwen3.5's mamba knob (retrofit pending). The mamba knob is the same single-select shape as KV Cache DType, so it rides the SAME generic primitive (row above) — once that lands, Qwen3.6 / Qwen3-Coder-Next declare it purely in config. | ## 2. Command rewrite table (the ONLY allowed normalizations) @@ -44,13 +44,14 @@ Caveats discovered in the pilot: asking: - offered as a legacy **selectable option/toggle** → never select it; cells mirror the accuracy-safe side (even if the legacy default was the - lossy side). The option itself **must survive as a Playground axis** — the user's choice may not degrade to a - tips mention. Use the existing axis when one fits (DSv4 gates W4A4 - behind `megamoeQuant`); when the playground lacks one — e.g. - Nemotron3-Ultra's "KV Cache DType" radio (None default / fp8_e4m3 / - bf16) has no kv-cache axis today — **add the axis** via the engine-axis - flow: a separate PRIOR engine PR (hard rule 4, like accuracyLabels - #27842), then the migration config declares it; + lossy side). The option itself **must survive as a Playground control** + — the user's choice may not degrade to a tips mention. Express it as + config data on the fitting axis (DSv4 gates W4A4 behind `megamoeQuant`); + when no axis models its shape — e.g. Nemotron3-Ultra's "KV Cache DType" + radio (None default / fp8_e4m3 / bf16) is a titled single-select the + playground has no shape for today — that shape is added ONCE as a + generic config-parameterized primitive (row above / engine-axis.md), + then this config and every future one declare it as data; - baked into the recipe's **unconditional/default command** → keep it verbatim. The legacy measurements ran with it, and fp8 KV halves KV memory — stripping could OOM the recipe. Expect this pattern: legacy diff --git a/docs_new/src/snippets/_playground.jsx b/docs_new/src/snippets/_playground.jsx index 561096f39..baa8406f3 100644 --- a/docs_new/src/snippets/_playground.jsx +++ b/docs_new/src/snippets/_playground.jsx @@ -13,6 +13,8 @@ // pdDisagg — role + transfer backend + IB device + optional router // hicache — enable + backend + write policy // hisparse — enable + host ratio (decode-only) +// flagSelects — generic: a config-declared LIST of single-selects, each with +// its own title + strip-prefixes + options (no per-feature code) // // Adding an axis = one entry in AXIS_HANDLERS below; nothing else switches on // an axis id. Each handler implements initState / revertHidden / apply / @@ -915,6 +917,107 @@ export const Playground = ({ config }) => { }, }, + // ---- Axis: Flag Selects (generic, config-declared) ---------------------- + // A LIST of single-selects, each declared entirely in config: + // { id, title, stripPrefixes: [...], options: [{ id, label, flags? }] } + // Same shape as `speculative` minus its hardcoded title + strip list: pick + // an option → strip the family, splice the option's flags. A flagless + // option is the "none" / accuracy-safe choice (matches a base carrying none + // of the family). Model-specific controls (KV-cache dtype, mamba scheduler + // strategy, …) live here as DATA — no per-feature engine code. Supports + // multiple selects per page. State: { [selectId]: optionId | null } + // (null = inherit base). + flagSelects: { + initState: (fc) => { + const out = {}; + for (const spec of (fc || [])) out[spec.id] = null; + return out; + }, + + // Per select: match base's family flags (first token ∈ stripPrefixes) + // against each option's flags. A flagless option matches an empty family. + deriveFromBase: (cell, fc) => { + const flags = (cell && cell.flags) || []; + const out = {}; + for (const spec of (fc || [])) { + const prefixes = spec.stripPrefixes || []; + const fam = flags.filter((f) => prefixes.includes(f.split(/[\s=]/)[0])); + let hit = null; + for (const opt of (spec.options || [])) { + const of = opt.flags || []; + if (of.length === fam.length && of.every((x) => fam.includes(x))) { + hit = opt.id; break; + } + } + out[spec.id] = hit; + } + return out; + }, + + revertHidden: (value, fc, base, h) => { + let changed = false; + const next = { ...value }; + for (const spec of (fc || [])) { + const cur = next[spec.id]; + if (cur !== null && cur !== undefined + && h.isHidden(spec.options, cur, base)) { + next[spec.id] = null; changed = true; + } + } + return changed ? next : value; + }, + + apply: ({ flags, env, value, fc, sel, h, derived }) => { + const evalBase = { + ...(sel || {}), + dpAttnOn: h.hasFlag(flags, "--enable-dp-attention"), + pdMode: h.findFlagArg(flags, "--disaggregation-mode") || "off", + }; + for (const spec of (fc || [])) { + const v = value ? value[spec.id] : null; + if (v === null || v === undefined) continue; // inherit base + const d = derived ? derived[spec.id] : null; + if (v === d) continue; // already == base + const opt = (spec.options || []).find((o) => o.id === v); + if (!opt) continue; + if (h.evaluateChip(opt, evalBase).disabled) continue; + flags = h.stripFlagsByFirstToken(flags, spec.stripPrefixes || []); + if (opt.flags && opt.flags.length) { + flags = h.insertBeforeTail(flags, opt.flags); + } + } + return { flags, env }; + }, + + render: ({ axisId, value, setValue, fc, base, s, h, renderChip, derived }) => { + const cards = []; + for (const spec of (fc || [])) { + const opts = (spec.options || []) + .map((o) => h.evaluateChip(o, base)) + .filter((c) => !c.hidden); + if (!opts.length) continue; + const explicit = value ? value[spec.id] : null; + const display = (explicit !== null && explicit !== undefined) + ? explicit : (derived ? derived[spec.id] : null); + cards.push( +
+
+ {spec.title} + {opts.map((c) => ( + + {renderChip(c.label, display, c.value, + () => setValue({ ...value, [spec.id]: c.value }), + { disabled: c.disabled, disabledReason: c.disableReason })} + + ))} +
+
+ ); + } + return cards.length ? cards : null; + }, + }, + }; // ==========================================================================