feat(cookbook): generic config-declared flagSelects playground axis (#28128)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
zijiexia
co-authored by
Claude Opus 4.8
Claude Opus 4.8
parent
29128f31fd
commit
d988d5d681
@@ -4,13 +4,36 @@ Loaded on demand by the `cookbook-add-model` skill. **Rare** — adding a model
|
||||
cookbook is data-only and never needs this. The current 7 built-in axes
|
||||
(`attention`, `moe`, `parsers`, `speculative`, `pdDisagg`, `hicache`,
|
||||
`hisparse`) already cover the SGLang feature surface most cookbooks need.
|
||||
(MegaMoE is not its own axis — it lives inside `moe` as a backend option +
|
||||
a `megamoeQuant` sub-select.) Only add a new axis if a real cookbook needs it
|
||||
and the feature does not fit any existing axis. Touches `_playground.jsx` only.
|
||||
One routine trigger: the `cookbook-migrate-model` skill requires every legacy
|
||||
control to survive as a dimension or a Playground axis — a legacy feature no
|
||||
built-in axis covers (e.g. Nemotron3's "KV Cache DType" radio) lands here
|
||||
first, as its own engine PR ahead of the migration PR.
|
||||
|
||||
**Model-specific features are config DATA, not engine code.** The axis
|
||||
handlers read options / flags / env / gating straight from
|
||||
`config.playgroundFeatures`, so a model-specific feature is added as data on
|
||||
an existing axis with NO engine edit — MegaMoE W4A4 is not its own axis, it's
|
||||
config data on `moe` (a `megamoe` backend option + a `megamoeQuant`
|
||||
sub-select). Reach for this file ONLY when a feature's *shape* — its
|
||||
title + the flag family it strips + its option/state model — is something no
|
||||
existing axis can express.
|
||||
|
||||
**When you do extend, build a GENERIC primitive, never a model-named handler.**
|
||||
The right unit is a reusable shape (e.g. "a titled single-select that strips a
|
||||
configurable flag family and splices the picked option's flags" — exactly the
|
||||
`speculative` handler's shape, minus its hardcoded title + `--speculative-*`
|
||||
strip list). Parameterize title, strip-prefixes, and options from config so
|
||||
the next model of that shape is pure config. Do NOT add a `kvcache` /
|
||||
`<model-feature>` handler that hardcodes one model's flag — that's the
|
||||
model-specific-code-in-the-engine anti-pattern this architecture exists to
|
||||
avoid. (Precedent: Nemotron3's "KV Cache DType" and Qwen3's mamba-cache select
|
||||
are the SAME single-select shape → one generic primitive serves both, then
|
||||
both are config.)
|
||||
|
||||
**A new axis is backward-compatible — zero churn on existing configs.** The
|
||||
runtime is opt-in per key: the apply/render loop does `const fc =
|
||||
pgFeatures[axisId]; if (!fc) continue;`, so any config that doesn't declare
|
||||
the key never sees the axis. And a model-specific axis does NOT join the
|
||||
opt-out "general axes ship on every cookbook" set (that set is an authoring
|
||||
convention for NEW configs, not a runtime default) — so review-pr won't flag
|
||||
existing pages for lacking it, and you never touch a merged config. Only the
|
||||
models that expose the control declare it. Touches `_playground.jsx` only.
|
||||
|
||||
For the per-model config/cells/MDX reference see [authoring-reference.md](authoring-reference.md).
|
||||
|
||||
@@ -20,12 +43,15 @@ For the per-model config/cells/MDX reference see [authoring-reference.md](author
|
||||
|
||||
Before touching the engine, confirm:
|
||||
|
||||
- The feature is a STABLE part of the SGLang CLI surface (will appear in
|
||||
multiple cookbooks, not one-off).
|
||||
- The feature cannot be expressed as a new option inside an existing axis
|
||||
(e.g. a new MoE backend belongs in `moe.backend.options`, not a new
|
||||
axis).
|
||||
- The feature has a clean strip-prefix → emit-flag pattern.
|
||||
- The feature cannot be expressed as data on an existing axis (a new MoE
|
||||
backend belongs in `moe.backend.options`; a new parser in `parsers.items`;
|
||||
a new spec preset in `speculative.options`). This is the common case —
|
||||
most "new features" are new options, not new shapes.
|
||||
- The *shape* is genuinely new (state model + strip pattern), AND you are
|
||||
adding it as a GENERIC config-parameterized axis (title / strip-prefixes /
|
||||
options all from config), not a one-model handler. If you'd hardcode a
|
||||
specific flag like `--kv-cache-dtype`, stop — generalize the shape instead.
|
||||
- The shape has a clean strip-prefix → emit-flag pattern.
|
||||
|
||||
If unsure, add it as data first (in one cookbook's config under an
|
||||
existing axis) before promoting it to a built-in axis.
|
||||
|
||||
@@ -63,10 +63,12 @@ your dispatch prompt, or ask for it.
|
||||
including combos that look memory-infeasible; keep them verbatim and list
|
||||
them in the PR body for the re-verification track.
|
||||
4. **Engines are read-only.** `_deployment.jsx` / `_playground.jsx` must not
|
||||
change in a migration PR. If the model needs an engine capability (a new
|
||||
axis, accuracy labels, …), that is a separate prior PR. The every-feature
|
||||
rule (step 2) triggers this routinely: a legacy control no built-in axis
|
||||
covers means an engine-axis PR lands first.
|
||||
change in a migration PR. Model-specific features are config DATA consumed
|
||||
by generic axis handlers (MegaMoE precedent), so they need NO engine
|
||||
change — only a genuinely new control *shape* does, and then as a one-time
|
||||
generic primitive (never a model-named handler) on a separate prior PR
|
||||
(engine-axis.md). KV Cache DType / mamba-cache select trigger this once for
|
||||
the whole round; afterwards both are config.
|
||||
5. **`github.cookbookModel` must be set** (`<hf-org>/<page-slug>`, e.g.
|
||||
`qwen/qwen3.5`) and the block never pruned — without it Submit ↗ mislabels
|
||||
as deepseek-v4. The issue template itself needs NO edits (free-form input).
|
||||
@@ -113,10 +115,14 @@ ON — EXCEPT parsers:
|
||||
`--reasoning-parser`/`--tool-call-parser` are NEVER baked into cells, they are
|
||||
Playground-only (DSv4 convention). **Every legacy control survives as an
|
||||
interactive control** — a dimension or a Playground axis, never a tips-only
|
||||
mention. A feature none of the built-in axes covers (Nemotron3's "KV Cache
|
||||
DType" radio is the precedent) still lands in the Playground: add the axis
|
||||
via a separate PRIOR engine PR (engine-axis.md), keeping the migration PR
|
||||
itself data-only (hard rule 4). The strategy count follows the page's
|
||||
mention — and a model-specific control is **config data, not engine code**
|
||||
(MegaMoE W4A4 is all DSv4 config on the existing `moe` axis). It's pure
|
||||
config whenever it fits an existing axis's data schema. Only a genuinely new
|
||||
*shape* (Nemotron3's "KV Cache DType" — a titled single-select stripping a
|
||||
flag family no axis manages) needs the engine, and then as a ONE-TIME
|
||||
generic config-parameterized primitive (never a model-named handler) on a
|
||||
separate PRIOR engine PR, keeping the migration PR data-only (hard rule 4,
|
||||
engine-axis.md). The strategy count follows the page's
|
||||
operating points: **one recipe → a single `balanced`; two → `low-latency` +
|
||||
`high-throughput`; three → the full trio (the ideal)**. The tiers apply per
|
||||
(hw × variant × quant) combination — a single-recipe combination on a
|
||||
|
||||
@@ -13,9 +13,9 @@ this file is about the *mapping decisions*.
|
||||
| model-size / model-name radio | `variants` | One variant per deployable checkpoint family; single `{id:"default"}` when there's no variant axis (then `modelNames` keys drop the variant half). |
|
||||
| quantization radio | `quantizations` | Real precision ids (`bf16`/`fp8`/`fp4`/`int4`/…). One `fp4` id even when checkpoints differ per vendor — route via `hw\|variant\|quant` triple keys in `modelNames` (NVFP4 on Blackwell vs AMD MXFP4 is the precedent); per-hw greying falls out of which cells exist. |
|
||||
| toggle that **couples** with other parts of the command (changes TP/mem/EP), OR one the legacy page labels with **operating-point words** | `strategies` | The Playground applies pure flag diffs — it cannot do coupled changes. Example: Qwen3.5's MTP toggle bumps TP on three H100 combos → strategies `low-latency` (MTP on) / `high-throughput` (MTP off). **Naming counts like coupling**: GLM-5.1's / Kimi-K2.6's `dpattention` adds only `--dp N --enable-dp-attention` (uncoupled), but its options are subtitled "Low Latency" / "High Throughput" — the page's own named operating-point split → strategies; a flag-only spec toggle riding alongside it stays a Playground axis and bakes per its legacy default. GPU-count radios (GLM-4.7, MiniMax-M2.5/2.7) → budget-tier strategies with the legacy SUPPORT matrix preserved by which cells exist. Strategy count follows the page's operating points: 1 → `balanced`, 2 → `low-latency`+`high-throughput`, 3 → the full trio (§4). |
|
||||
| toggle that only adds/removes its own flags | Playground axis (+ bake, EXCEPT parsers and accuracy-degrading flags) | **Parsers (`--reasoning-parser` / `--tool-call-parser`) are NEVER baked into cells** — Deployment commands ship without them regardless of the legacy default or the measured command; the `parsers` axis adds them on top (DSv4 convention; cells mirror the legacy generator's parsers-OFF output). Accuracy-degrading toggles are never baked either — §2 caveats (axis-only, accuracy-safe cells). Other flag-only toggles: legacy default ON → bake into cells AND declare the axis so users can strip (red strikethrough); default OFF → keep cells clean, axis preset only. MTP/EAGLE presets → `speculative` axis; dp-attention → a strategy when the legacy page labels it as the operating-point split or when coupled (see the row above), else `attention.dpAttn`. **No fitting built-in axis ≠ drop the feature** — EVERY legacy control survives as an interactive control (a dimension or a Playground axis), never a tips-only mention. When none of the built-in axes fits (Nemotron3's "KV Cache DType" radio is the precedent), add the axis via the engine-axis flow: a separate PRIOR engine PR, then the migration config declares it (hard rule 4). |
|
||||
| toggle that only adds/removes its own flags | Playground axis (+ bake, EXCEPT parsers and accuracy-degrading flags) | **Parsers (`--reasoning-parser` / `--tool-call-parser`) are NEVER baked into cells** — Deployment commands ship without them regardless of the legacy default or the measured command; the `parsers` axis adds them on top (DSv4 convention; cells mirror the legacy generator's parsers-OFF output). Accuracy-degrading toggles are never baked either — §2 caveats (axis-only, accuracy-safe cells). Other flag-only toggles: legacy default ON → bake into cells AND declare the axis so users can strip (red strikethrough); default OFF → keep cells clean, axis preset only. MTP/EAGLE presets → `speculative` axis; dp-attention → a strategy when the legacy page labels it as the operating-point split or when coupled (see the row above), else `attention.dpAttn`. **EVERY legacy control survives as an interactive control** (a dimension or a Playground axis), never a tips-only mention — but a model-specific control is **config DATA, not engine code**: the axis handler reads options/flags/env/gating straight from `config.playgroundFeatures` (MegaMoE W4A4 is entirely DSv4 config data on the existing `moe` axis — no per-model engine edit). A control that fits an existing axis's data schema is therefore pure config, full stop. Only when the *shape* is genuinely new — a titled single-select that strips a flag family no axis manages, e.g. Nemotron3's "KV Cache DType" (`--kv-cache-dtype`) — does the engine need that shape, and then you add it ONCE as a **generic config-parameterized primitive** (title + strip-prefixes + options all from config), **never** a model-named `kvcache` handler; afterwards this feature and every future one of its shape are pure config. Such a primitive is backward-compatible — runtime is opt-in per key (`if (!fc) continue`), and a model-specific axis is NOT added to the opt-out general set — so it churns zero existing configs (engine-axis.md). |
|
||||
| per-combo hidden option (e.g. spec hidden on Xeon) | absent cells | Don't create cells for combos the legacy widget couldn't produce; the engine greys them automatically. `# Error:` pseudo-commands → no cell + explanation in §2 tips and/or a chip `disable`/`disableReason`. |
|
||||
| coupled secondary knob (e.g. mamba cache V1/V2) | cells + Playground axis | Bake the correct value per cell following the legacy coupling (Qwen3.5: MTP ⇒ `--mamba-scheduler-strategy extra_buffer` on NVIDIA; AMD/Xeon ⇒ V1/no flag) and document the coupling in §2 tips — AND surface the knob as a Playground axis like every other legacy feature (row above; add the axis when none fits). Baking alone is NOT enough — the every-feature rule supersedes the pilot's cells+prose-only treatment of Qwen3.5's mamba knob (retrofit pending); Qwen3.6 / Qwen3-Coder-Next carry the same knob, so their migrations need the axis in place first. |
|
||||
| coupled secondary knob (e.g. mamba cache V1/V2) | cells + Playground axis | Bake the correct value per cell following the legacy coupling (Qwen3.5: MTP ⇒ `--mamba-scheduler-strategy extra_buffer` on NVIDIA; AMD/Xeon ⇒ V1/no flag) and document the coupling in §2 tips — AND surface the knob as a Playground axis like every other legacy feature (row above; add the axis when none fits). Baking alone is NOT enough — the every-feature rule supersedes the pilot's cells+prose-only treatment of Qwen3.5's mamba knob (retrofit pending). The mamba knob is the same single-select shape as KV Cache DType, so it rides the SAME generic primitive (row above) — once that lands, Qwen3.6 / Qwen3-Coder-Next declare it purely in config. |
|
||||
|
||||
## 2. Command rewrite table (the ONLY allowed normalizations)
|
||||
|
||||
@@ -44,13 +44,14 @@ Caveats discovered in the pilot:
|
||||
asking:
|
||||
- offered as a legacy **selectable option/toggle** → never select it;
|
||||
cells mirror the accuracy-safe side (even if the legacy default was the
|
||||
lossy side). The option itself **must survive as a Playground axis** — the user's choice may not degrade to a
|
||||
tips mention. Use the existing axis when one fits (DSv4 gates W4A4
|
||||
behind `megamoeQuant`); when the playground lacks one — e.g.
|
||||
Nemotron3-Ultra's "KV Cache DType" radio (None default / fp8_e4m3 /
|
||||
bf16) has no kv-cache axis today — **add the axis** via the engine-axis
|
||||
flow: a separate PRIOR engine PR (hard rule 4, like accuracyLabels
|
||||
#27842), then the migration config declares it;
|
||||
lossy side). The option itself **must survive as a Playground control**
|
||||
— the user's choice may not degrade to a tips mention. Express it as
|
||||
config data on the fitting axis (DSv4 gates W4A4 behind `megamoeQuant`);
|
||||
when no axis models its shape — e.g. Nemotron3-Ultra's "KV Cache DType"
|
||||
radio (None default / fp8_e4m3 / bf16) is a titled single-select the
|
||||
playground has no shape for today — that shape is added ONCE as a
|
||||
generic config-parameterized primitive (row above / engine-axis.md),
|
||||
then this config and every future one declare it as data;
|
||||
- baked into the recipe's **unconditional/default command** → keep it
|
||||
verbatim. The legacy measurements ran with it, and fp8 KV halves KV
|
||||
memory — stripping could OOM the recipe. Expect this pattern: legacy
|
||||
|
||||
@@ -13,6 +13,8 @@
|
||||
// pdDisagg — role + transfer backend + IB device + optional router
|
||||
// hicache — enable + backend + write policy
|
||||
// hisparse — enable + host ratio (decode-only)
|
||||
// flagSelects — generic: a config-declared LIST of single-selects, each with
|
||||
// its own title + strip-prefixes + options (no per-feature code)
|
||||
//
|
||||
// Adding an axis = one entry in AXIS_HANDLERS below; nothing else switches on
|
||||
// an axis id. Each handler implements initState / revertHidden / apply /
|
||||
@@ -915,6 +917,107 @@ export const Playground = ({ config }) => {
|
||||
},
|
||||
},
|
||||
|
||||
// ---- Axis: Flag Selects (generic, config-declared) ----------------------
|
||||
// A LIST of single-selects, each declared entirely in config:
|
||||
// { id, title, stripPrefixes: [...], options: [{ id, label, flags? }] }
|
||||
// Same shape as `speculative` minus its hardcoded title + strip list: pick
|
||||
// an option → strip the family, splice the option's flags. A flagless
|
||||
// option is the "none" / accuracy-safe choice (matches a base carrying none
|
||||
// of the family). Model-specific controls (KV-cache dtype, mamba scheduler
|
||||
// strategy, …) live here as DATA — no per-feature engine code. Supports
|
||||
// multiple selects per page. State: { [selectId]: optionId | null }
|
||||
// (null = inherit base).
|
||||
flagSelects: {
|
||||
initState: (fc) => {
|
||||
const out = {};
|
||||
for (const spec of (fc || [])) out[spec.id] = null;
|
||||
return out;
|
||||
},
|
||||
|
||||
// Per select: match base's family flags (first token ∈ stripPrefixes)
|
||||
// against each option's flags. A flagless option matches an empty family.
|
||||
deriveFromBase: (cell, fc) => {
|
||||
const flags = (cell && cell.flags) || [];
|
||||
const out = {};
|
||||
for (const spec of (fc || [])) {
|
||||
const prefixes = spec.stripPrefixes || [];
|
||||
const fam = flags.filter((f) => prefixes.includes(f.split(/[\s=]/)[0]));
|
||||
let hit = null;
|
||||
for (const opt of (spec.options || [])) {
|
||||
const of = opt.flags || [];
|
||||
if (of.length === fam.length && of.every((x) => fam.includes(x))) {
|
||||
hit = opt.id; break;
|
||||
}
|
||||
}
|
||||
out[spec.id] = hit;
|
||||
}
|
||||
return out;
|
||||
},
|
||||
|
||||
revertHidden: (value, fc, base, h) => {
|
||||
let changed = false;
|
||||
const next = { ...value };
|
||||
for (const spec of (fc || [])) {
|
||||
const cur = next[spec.id];
|
||||
if (cur !== null && cur !== undefined
|
||||
&& h.isHidden(spec.options, cur, base)) {
|
||||
next[spec.id] = null; changed = true;
|
||||
}
|
||||
}
|
||||
return changed ? next : value;
|
||||
},
|
||||
|
||||
apply: ({ flags, env, value, fc, sel, h, derived }) => {
|
||||
const evalBase = {
|
||||
...(sel || {}),
|
||||
dpAttnOn: h.hasFlag(flags, "--enable-dp-attention"),
|
||||
pdMode: h.findFlagArg(flags, "--disaggregation-mode") || "off",
|
||||
};
|
||||
for (const spec of (fc || [])) {
|
||||
const v = value ? value[spec.id] : null;
|
||||
if (v === null || v === undefined) continue; // inherit base
|
||||
const d = derived ? derived[spec.id] : null;
|
||||
if (v === d) continue; // already == base
|
||||
const opt = (spec.options || []).find((o) => o.id === v);
|
||||
if (!opt) continue;
|
||||
if (h.evaluateChip(opt, evalBase).disabled) continue;
|
||||
flags = h.stripFlagsByFirstToken(flags, spec.stripPrefixes || []);
|
||||
if (opt.flags && opt.flags.length) {
|
||||
flags = h.insertBeforeTail(flags, opt.flags);
|
||||
}
|
||||
}
|
||||
return { flags, env };
|
||||
},
|
||||
|
||||
render: ({ axisId, value, setValue, fc, base, s, h, renderChip, derived }) => {
|
||||
const cards = [];
|
||||
for (const spec of (fc || [])) {
|
||||
const opts = (spec.options || [])
|
||||
.map((o) => h.evaluateChip(o, base))
|
||||
.filter((c) => !c.hidden);
|
||||
if (!opts.length) continue;
|
||||
const explicit = value ? value[spec.id] : null;
|
||||
const display = (explicit !== null && explicit !== undefined)
|
||||
? explicit : (derived ? derived[spec.id] : null);
|
||||
cards.push(
|
||||
<div key={`${axisId}-${spec.id}`} style={s.card}>
|
||||
<div style={s.compactRow}>
|
||||
<span style={s.axisTitle}>{spec.title}</span>
|
||||
{opts.map((c) => (
|
||||
<span key={c.value} style={s.field}>
|
||||
{renderChip(c.label, display, c.value,
|
||||
() => setValue({ ...value, [spec.id]: c.value }),
|
||||
{ disabled: c.disabled, disabledReason: c.disableReason })}
|
||||
</span>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
return cards.length ? cards : null;
|
||||
},
|
||||
},
|
||||
|
||||
};
|
||||
|
||||
// ==========================================================================
|
||||
|
||||
Reference in New Issue
Block a user