feat(cookbook): generic config-declared flagSelects playground axis (#28128)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
29128f31fd
commit
d988d5d681
@@ -4,13 +4,36 @@ Loaded on demand by the `cookbook-add-model` skill. **Rare** — adding a model
|
||||
cookbook is data-only and never needs this. The current 7 built-in axes
|
||||
(`attention`, `moe`, `parsers`, `speculative`, `pdDisagg`, `hicache`,
|
||||
`hisparse`) already cover the SGLang feature surface most cookbooks need.
|
||||
(MegaMoE is not its own axis — it lives inside `moe` as a backend option +
|
||||
a `megamoeQuant` sub-select.) Only add a new axis if a real cookbook needs it
|
||||
and the feature does not fit any existing axis. Touches `_playground.jsx` only.
|
||||
One routine trigger: the `cookbook-migrate-model` skill requires every legacy
|
||||
control to survive as a dimension or a Playground axis — a legacy feature no
|
||||
built-in axis covers (e.g. Nemotron3's "KV Cache DType" radio) lands here
|
||||
first, as its own engine PR ahead of the migration PR.
|
||||
|
||||
**Model-specific features are config DATA, not engine code.** The axis
|
||||
handlers read options / flags / env / gating straight from
|
||||
`config.playgroundFeatures`, so a model-specific feature is added as data on
|
||||
an existing axis with NO engine edit — MegaMoE W4A4 is not its own axis, it's
|
||||
config data on `moe` (a `megamoe` backend option + a `megamoeQuant`
|
||||
sub-select). Reach for this file ONLY when a feature's *shape* — its
|
||||
title + the flag family it strips + its option/state model — is something no
|
||||
existing axis can express.
|
||||
|
||||
**When you do extend, build a GENERIC primitive, never a model-named handler.**
|
||||
The right unit is a reusable shape (e.g. "a titled single-select that strips a
|
||||
configurable flag family and splices the picked option's flags" — exactly the
|
||||
`speculative` handler's shape, minus its hardcoded title + `--speculative-*`
|
||||
strip list). Parameterize title, strip-prefixes, and options from config so
|
||||
the next model of that shape is pure config. Do NOT add a `kvcache` /
|
||||
`<model-feature>` handler that hardcodes one model's flag — that's the
|
||||
model-specific-code-in-the-engine anti-pattern this architecture exists to
|
||||
avoid. (Precedent: Nemotron3's "KV Cache DType" and Qwen3's mamba-cache select
|
||||
are the SAME single-select shape → one generic primitive serves both, then
|
||||
both are config.)
|
||||
|
||||
**A new axis is backward-compatible — zero churn on existing configs.** The
|
||||
runtime is opt-in per key: the apply/render loop does `const fc =
|
||||
pgFeatures[axisId]; if (!fc) continue;`, so any config that doesn't declare
|
||||
the key never sees the axis. And a model-specific axis does NOT join the
|
||||
opt-out "general axes ship on every cookbook" set (that set is an authoring
|
||||
convention for NEW configs, not a runtime default) — so review-pr won't flag
|
||||
existing pages for lacking it, and you never touch a merged config. Only the
|
||||
models that expose the control declare it. Touches `_playground.jsx` only.
|
||||
|
||||
For the per-model config/cells/MDX reference see [authoring-reference.md](authoring-reference.md).
|
||||
|
||||
@@ -20,12 +43,15 @@ For the per-model config/cells/MDX reference see [authoring-reference.md](author
|
||||
|
||||
Before touching the engine, confirm:
|
||||
|
||||
- The feature is a STABLE part of the SGLang CLI surface (will appear in
|
||||
multiple cookbooks, not one-off).
|
||||
- The feature cannot be expressed as a new option inside an existing axis
|
||||
(e.g. a new MoE backend belongs in `moe.backend.options`, not a new
|
||||
axis).
|
||||
- The feature has a clean strip-prefix → emit-flag pattern.
|
||||
- The feature cannot be expressed as data on an existing axis (a new MoE
|
||||
backend belongs in `moe.backend.options`; a new parser in `parsers.items`;
|
||||
a new spec preset in `speculative.options`). This is the common case —
|
||||
most "new features" are new options, not new shapes.
|
||||
- The *shape* is genuinely new (state model + strip pattern), AND you are
|
||||
adding it as a GENERIC config-parameterized axis (title / strip-prefixes /
|
||||
options all from config), not a one-model handler. If you'd hardcode a
|
||||
specific flag like `--kv-cache-dtype`, stop — generalize the shape instead.
|
||||
- The shape has a clean strip-prefix → emit-flag pattern.
|
||||
|
||||
If unsure, add it as data first (in one cookbook's config under an
|
||||
existing axis) before promoting it to a built-in axis.
|
||||
|
||||
@@ -63,10 +63,12 @@ your dispatch prompt, or ask for it.
|
||||
including combos that look memory-infeasible; keep them verbatim and list
|
||||
them in the PR body for the re-verification track.
|
||||
4. **Engines are read-only.** `_deployment.jsx` / `_playground.jsx` must not
|
||||
change in a migration PR. If the model needs an engine capability (a new
|
||||
axis, accuracy labels, …), that is a separate prior PR. The every-feature
|
||||
rule (step 2) triggers this routinely: a legacy control no built-in axis
|
||||
covers means an engine-axis PR lands first.
|
||||
change in a migration PR. Model-specific features are config DATA consumed
|
||||
by generic axis handlers (MegaMoE precedent), so they need NO engine
|
||||
change — only a genuinely new control *shape* does, and then as a one-time
|
||||
generic primitive (never a model-named handler) on a separate prior PR
|
||||
(engine-axis.md). KV Cache DType / mamba-cache select trigger this once for
|
||||
the whole round; afterwards both are config.
|
||||
5. **`github.cookbookModel` must be set** (`<hf-org>/<page-slug>`, e.g.
|
||||
`qwen/qwen3.5`) and the block never pruned — without it Submit ↗ mislabels
|
||||
as deepseek-v4. The issue template itself needs NO edits (free-form input).
|
||||
@@ -113,10 +115,14 @@ ON — EXCEPT parsers:
|
||||
`--reasoning-parser`/`--tool-call-parser` are NEVER baked into cells, they are
|
||||
Playground-only (DSv4 convention). **Every legacy control survives as an
|
||||
interactive control** — a dimension or a Playground axis, never a tips-only
|
||||
mention. A feature none of the built-in axes covers (Nemotron3's "KV Cache
|
||||
DType" radio is the precedent) still lands in the Playground: add the axis
|
||||
via a separate PRIOR engine PR (engine-axis.md), keeping the migration PR
|
||||
itself data-only (hard rule 4). The strategy count follows the page's
|
||||
mention — and a model-specific control is **config data, not engine code**
|
||||
(MegaMoE W4A4 is all DSv4 config on the existing `moe` axis). It's pure
|
||||
config whenever it fits an existing axis's data schema. Only a genuinely new
|
||||
*shape* (Nemotron3's "KV Cache DType" — a titled single-select stripping a
|
||||
flag family no axis manages) needs the engine, and then as a ONE-TIME
|
||||
generic config-parameterized primitive (never a model-named handler) on a
|
||||
separate PRIOR engine PR, keeping the migration PR data-only (hard rule 4,
|
||||
engine-axis.md). The strategy count follows the page's
|
||||
operating points: **one recipe → a single `balanced`; two → `low-latency` +
|
||||
`high-throughput`; three → the full trio (the ideal)**. The tiers apply per
|
||||
(hw × variant × quant) combination — a single-recipe combination on a
|
||||
|
||||
@@ -13,9 +13,9 @@ this file is about the *mapping decisions*.
|
||||
| model-size / model-name radio | `variants` | One variant per deployable checkpoint family; single `{id:"default"}` when there's no variant axis (then `modelNames` keys drop the variant half). |
|
||||
| quantization radio | `quantizations` | Real precision ids (`bf16`/`fp8`/`fp4`/`int4`/…). One `fp4` id even when checkpoints differ per vendor — route via `hw\|variant\|quant` triple keys in `modelNames` (NVFP4 on Blackwell vs AMD MXFP4 is the precedent); per-hw greying falls out of which cells exist. |
|
||||
| toggle that **couples** with other parts of the command (changes TP/mem/EP), OR one the legacy page labels with **operating-point words** | `strategies` | The Playground applies pure flag diffs — it cannot do coupled changes. Example: Qwen3.5's MTP toggle bumps TP on three H100 combos → strategies `low-latency` (MTP on) / `high-throughput` (MTP off). **Naming counts like coupling**: GLM-5.1's / Kimi-K2.6's `dpattention` adds only `--dp N --enable-dp-attention` (uncoupled), but its options are subtitled "Low Latency" / "High Throughput" — the page's own named operating-point split → strategies; a flag-only spec toggle riding alongside it stays a Playground axis and bakes per its legacy default. GPU-count radios (GLM-4.7, MiniMax-M2.5/2.7) → budget-tier strategies with the legacy SUPPORT matrix preserved by which cells exist. Strategy count follows the page's operating points: 1 → `balanced`, 2 → `low-latency`+`high-throughput`, 3 → the full trio (§4). |
|
||||
| toggle that only adds/removes its own flags | Playground axis (+ bake, EXCEPT parsers and accuracy-degrading flags) | **Parsers (`--reasoning-parser` / `--tool-call-parser`) are NEVER baked into cells** — Deployment commands ship without them regardless of the legacy default or the measured command; the `parsers` axis adds them on top (DSv4 convention; cells mirror the legacy generator's parsers-OFF output). Accuracy-degrading toggles are never baked either — §2 caveats (axis-only, accuracy-safe cells). Other flag-only toggles: legacy default ON → bake into cells AND declare the axis so users can strip (red strikethrough); default OFF → keep cells clean, axis preset only. MTP/EAGLE presets → `speculative` axis; dp-attention → a strategy when the legacy page labels it as the operating-point split or when coupled (see the row above), else `attention.dpAttn`. **No fitting built-in axis ≠ drop the feature** — EVERY legacy control survives as an interactive control (a dimension or a Playground axis), never a tips-only mention. When none of the built-in axes fits (Nemotron3's "KV Cache DType" radio is the precedent), add the axis via the engine-axis flow: a separate PRIOR engine PR, then the migration config declares it (hard rule 4). |
|
||||
| toggle that only adds/removes its own flags | Playground axis (+ bake, EXCEPT parsers and accuracy-degrading flags) | **Parsers (`--reasoning-parser` / `--tool-call-parser`) are NEVER baked into cells** — Deployment commands ship without them regardless of the legacy default or the measured command; the `parsers` axis adds them on top (DSv4 convention; cells mirror the legacy generator's parsers-OFF output). Accuracy-degrading toggles are never baked either — §2 caveats (axis-only, accuracy-safe cells). Other flag-only toggles: legacy default ON → bake into cells AND declare the axis so users can strip (red strikethrough); default OFF → keep cells clean, axis preset only. MTP/EAGLE presets → `speculative` axis; dp-attention → a strategy when the legacy page labels it as the operating-point split or when coupled (see the row above), else `attention.dpAttn`. **EVERY legacy control survives as an interactive control** (a dimension or a Playground axis), never a tips-only mention — but a model-specific control is **config DATA, not engine code**: the axis handler reads options/flags/env/gating straight from `config.playgroundFeatures` (MegaMoE W4A4 is entirely DSv4 config data on the existing `moe` axis — no per-model engine edit). A control that fits an existing axis's data schema is therefore pure config, full stop. Only when the *shape* is genuinely new — a titled single-select that strips a flag family no axis manages, e.g. Nemotron3's "KV Cache DType" (`--kv-cache-dtype`) — does the engine need that shape, and then you add it ONCE as a **generic config-parameterized primitive** (title + strip-prefixes + options all from config), **never** a model-named `kvcache` handler; afterwards this feature and every future one of its shape are pure config. Such a primitive is backward-compatible — runtime is opt-in per key (`if (!fc) continue`), and a model-specific axis is NOT added to the opt-out general set — so it churns zero existing configs (engine-axis.md). |
|
||||
| per-combo hidden option (e.g. spec hidden on Xeon) | absent cells | Don't create cells for combos the legacy widget couldn't produce; the engine greys them automatically. `# Error:` pseudo-commands → no cell + explanation in §2 tips and/or a chip `disable`/`disableReason`. |
|
||||
| coupled secondary knob (e.g. mamba cache V1/V2) | cells + Playground axis | Bake the correct value per cell following the legacy coupling (Qwen3.5: MTP ⇒ `--mamba-scheduler-strategy extra_buffer` on NVIDIA; AMD/Xeon ⇒ V1/no flag) and document the coupling in §2 tips — AND surface the knob as a Playground axis like every other legacy feature (row above; add the axis when none fits). Baking alone is NOT enough — the every-feature rule supersedes the pilot's cells+prose-only treatment of Qwen3.5's mamba knob (retrofit pending); Qwen3.6 / Qwen3-Coder-Next carry the same knob, so their migrations need the axis in place first. |
|
||||
| coupled secondary knob (e.g. mamba cache V1/V2) | cells + Playground axis | Bake the correct value per cell following the legacy coupling (Qwen3.5: MTP ⇒ `--mamba-scheduler-strategy extra_buffer` on NVIDIA; AMD/Xeon ⇒ V1/no flag) and document the coupling in §2 tips — AND surface the knob as a Playground axis like every other legacy feature (row above; add the axis when none fits). Baking alone is NOT enough — the every-feature rule supersedes the pilot's cells+prose-only treatment of Qwen3.5's mamba knob (retrofit pending). The mamba knob is the same single-select shape as KV Cache DType, so it rides the SAME generic primitive (row above) — once that lands, Qwen3.6 / Qwen3-Coder-Next declare it purely in config. |
|
||||
|
||||
## 2. Command rewrite table (the ONLY allowed normalizations)
|
||||
|
||||
@@ -44,13 +44,14 @@ Caveats discovered in the pilot:
|
||||
asking:
|
||||
- offered as a legacy **selectable option/toggle** → never select it;
|
||||
cells mirror the accuracy-safe side (even if the legacy default was the
|
||||
lossy side). The option itself **must survive as a Playground axis** — the user's choice may not degrade to a
|
||||
tips mention. Use the existing axis when one fits (DSv4 gates W4A4
|
||||
behind `megamoeQuant`); when the playground lacks one — e.g.
|
||||
Nemotron3-Ultra's "KV Cache DType" radio (None default / fp8_e4m3 /
|
||||
bf16) has no kv-cache axis today — **add the axis** via the engine-axis
|
||||
flow: a separate PRIOR engine PR (hard rule 4, like accuracyLabels
|
||||
#27842), then the migration config declares it;
|
||||
lossy side). The option itself **must survive as a Playground control**
|
||||
— the user's choice may not degrade to a tips mention. Express it as
|
||||
config data on the fitting axis (DSv4 gates W4A4 behind `megamoeQuant`);
|
||||
when no axis models its shape — e.g. Nemotron3-Ultra's "KV Cache DType"
|
||||
radio (None default / fp8_e4m3 / bf16) is a titled single-select the
|
||||
playground has no shape for today — that shape is added ONCE as a
|
||||
generic config-parameterized primitive (row above / engine-axis.md),
|
||||
then this config and every future one declare it as data;
|
||||
- baked into the recipe's **unconditional/default command** → keep it
|
||||
verbatim. The legacy measurements ran with it, and fp8 KV halves KV
|
||||
memory — stripping could OOM the recipe. Expect this pattern: legacy
|
||||
|
||||
Reference in New Issue
Block a user