Cookbook renovation (#26885)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
zijiexia
co-authored by
Claude Opus 4.8
Claude Opus 4.8
parent
6365d6faee
commit
d1777d1f6d
@@ -0,0 +1,220 @@
|
||||
# Cookbook config reference (fields · cells · playground · MDX)
|
||||
|
||||
Loaded on demand by the `cookbook-add-model` skill. This is the field-by-field
|
||||
contract for when the clone needs more than a rename. The two engine files are
|
||||
the canonical specs — read their headers first:
|
||||
|
||||
- [`_deployment.jsx`](../../../../docs_new/src/snippets/_deployment.jsx) — the 5-dim matrix widget; lists every config field.
|
||||
- [`_playground.jsx`](../../../../docs_new/src/snippets/_playground.jsx) — the diff-based override widget; lists the `playgroundFeatures` axes + the `AXIS_HANDLERS` interface.
|
||||
|
||||
Engine extension (adding a new playground axis) lives in [engine-axis.md](engine-axis.md).
|
||||
|
||||
---
|
||||
|
||||
## 2.1 Create the config file
|
||||
|
||||
**Path**: `docs_new/src/snippets/configs/<vendor>/<model>.jsx`. The vendor folder is
|
||||
the HuggingFace org (`deepseek-ai`, `Qwen`, `moonshotai`, ...); the file
|
||||
name is a short hyphenated model id (`deepseek-v4`, `qwen3.5`, ...).
|
||||
|
||||
**Shape**: must be a single `export const config = { ... }` literal. Do not
|
||||
use function calls, spreads, fragment refs, or IIFE — Mintlify re-evaluates
|
||||
this export at hydration time with module-level identifiers out of scope,
|
||||
and any non-literal value crashes with `ReferenceError`.
|
||||
|
||||
**Required fields** (engine reads these — see the `_deployment.jsx` header for
|
||||
the full contract):
|
||||
|
||||
| Field | Type | Purpose |
|
||||
|---|---|---|
|
||||
| `modelName` | string | Display label only. Not used for HF slug — see `modelNames`. |
|
||||
| `supportedHardware` | `string[]` | Which hw ids appear in the catalog. Subset of `HARDWARE_CATALOG` (in `_deployment.jsx`) ∪ `config.hardware`. Listing an id makes its button appear; if no cell uses it, the engine greys it out automatically. |
|
||||
| `hardware` | `{id,label,vram,vendor}[]` | Optional. GPUs the shared `HARDWARE_CATALOG` doesn't carry (workstation / desktop / future chips, e.g. RTX PRO 6000). The engine merges these into the catalog, so a model-specific GPU is config data — **no engine-catalog edit**. Also add the id to `supportedHardware`. |
|
||||
| `variants` | `{id, label, subtitle?}[]` | 2nd-dim option list. Use `default` / single-element if the model has no variant axis. |
|
||||
| `quantizations` | `{id, label}[]` | 3rd-dim option list. |
|
||||
| `strategies` | `{id, label}[]` | 4th-dim option list. Common ids: `low-latency`, `balanced`, `high-throughput`. |
|
||||
| `nodesOptions` | `{id, label}[]` | 5th-dim option list. The `id` MUST be `single` or `multi-N` — the engine parses N from the id for `--nnodes`. |
|
||||
| `cells` | `{match, verified?, env, flags}[]` | One per supported (hw × variant × quant × strategy × nodes) combination. See §2.2. |
|
||||
| `modelNames` | `{[key]: string}` | HF slug lookup. Keys are either `hw\|variant\|quant` (most specific) or `variant\|quant` (fallback). |
|
||||
| `placeholders` | `{[key]: {target, label, default?}}` | `{{KEY}}` interpolation map for command + curl. `target` is `'command'` or `'curl'`. Editable through the Env modal. |
|
||||
| `curl` | string | cURL template. Uses `{{MODEL_NAME}}` + placeholder keys. |
|
||||
|
||||
**Optional fields**:
|
||||
|
||||
| Field | Type | Purpose |
|
||||
|---|---|---|
|
||||
| `multiNodeHints` | `{[hwId]: string[]}` | Lines prepended as `# ...` comments to multi-node commands (env-var hints). Per-hw, and only for hw whose **cluster fabric needs manual NIC config** (e.g. `gb200` NVL72/MNNVL → NVSHMEM/Gloo hints). NOT every multi-N hw needs an entry — standard-IB DeepEP (h200) auto-detects the HCA, and Marlin multi-node (h100) uses no DeepEP/NVSHMEM at all. |
|
||||
| `dockerImages` | `{[hwId]: string}` | Per-hw image name for `docker run` framing. **Ask the user which sglang build the recipes ran on; don't guess a supporting release.** Falls back to `lmsysorg/sglang:dev` if missing — also the sensible default when unsure. |
|
||||
| `playgroundFeatures` | `{[axisId]: {...}}` | Opts into the Playground widget. See §2.3. |
|
||||
| `benchmarkCommands` | `{speed: string, accuracy: {[accKey]: string \| {[variant]: string}}, numPromptsByConc?: {[c]: number}}` | Powers the benchmark card's **"⚡ Reproduce"** modal. `speed` is ONE `bench_serving` template; the engine fills `{{DATASET}}`/`{{ISL}}`/`{{OSL}}` from each cell's `speed[].workload`, the chip-picked `{{MAX_CONCURRENCY}}`, and `{{NUM_PROMPTS}}` (resolved `workload.num_prompts ?? numPromptsByConc[c] ?? max(c*2, 200)`). `accuracy` maps an accuracy field (e.g. `gsm8k_pct`) to a per-eval template — a string, OR a `{flash, pro, …}` object keyed by variant when the command differs per variant (e.g. GPQA/AIME `--max-tokens`). The modal renders a chip per eval (one command area, like Speed). Both also use `{{MODEL_NAME}}` + `{{CURL_HOST}}`/`{{CURL_PORT}}` like `curl`. Optional; the button only appears when this AND `benchmarks` are present. |
|
||||
| `defaultAccuracy` | `{[variant]: {[accKey]: number}}` | Model-level accuracy applied to **every** cell of a variant (e.g. GPQA Diamond / AIME25 — hardware-independent). Merged UNDER each cell's measured `accuracy` (a per-cell value wins), so you set a variant's score once instead of copying it onto every benchmark entry. Keys must match `ACCURACY_LABELS` + `benchmarkCommands.accuracy`. |
|
||||
| `github` | `{owner?, repo?, issueTemplate?, cookbookModel?}` | Overrides for the "Submit verified cell" CTA in the playground. Defaults: `sgl-project/sglang` + `3-playground-verified-cell.yml` + `"deepseek-ai/deepseek-v4"`. Set `cookbookModel` to the value that matches the `model` dropdown in your issue template so it's pre-selected when the issue opens. |
|
||||
|
||||
## 2.2 Author the 5-dim matrix (`cells[]`)
|
||||
|
||||
Each cell describes one verified (or auto-estimated) launch recipe.
|
||||
|
||||
```js
|
||||
{
|
||||
match: { hw: "b200", variant: "flash", quant: "fp4",
|
||||
strategy: "low-latency", nodes: "single" },
|
||||
verified: true, // green "Verified" badge; absence = yellow
|
||||
env: [
|
||||
"SGLANG_DEEPEP_NUM_MAX_DISPATCH_TOKENS_PER_RANK=1024",
|
||||
],
|
||||
flags: [
|
||||
"--trust-remote-code",
|
||||
"--model-path {{MODEL_NAME}}", // {{MODEL_NAME}} resolves from modelNames
|
||||
"--tp 4",
|
||||
"--moe-runner-backend flashinfer_mxfp4",
|
||||
"--host {{HOST_IP}}",
|
||||
"--port {{PORT}}",
|
||||
],
|
||||
},
|
||||
```
|
||||
|
||||
**Rules**:
|
||||
|
||||
- `match` MUST contain exactly the 5 keys: `hw`, `variant`, `quant`,
|
||||
`strategy`, `nodes`. The engine looks up cells by tuple equality.
|
||||
- `env` and `flags` are FLAT literals. The engine does NOT expand
|
||||
fragments, aliases, or templates — it consumes them verbatim
|
||||
(only `{{PLACEHOLDER}}` substitutions happen at render time).
|
||||
- DO NOT include `--nnodes` / `--node-rank` / `--dist-init-addr` in
|
||||
`cell.flags` for multi-node cells. The renderer injects them
|
||||
automatically from `match.nodes` (`multi-N` → N nodes).
|
||||
- DO NOT include `--host` / `--port` literally — use `{{HOST_IP}}` /
|
||||
`{{PORT}}` placeholders so users can override through the Env modal.
|
||||
- Order flags as: `--model-path` first (after any `--trust-remote-code`),
|
||||
then parallelism (`--tp`, `--dp`, `--enable-dp-attention`), then MoE
|
||||
flags, then tuning knobs, with `--host` / `--port` last. The playground
|
||||
engine assumes this ordering when inserting overrides (its anchors target
|
||||
`--model-path` / `--tp` / etc., and inserts before the `--host` tail).
|
||||
|
||||
**Cells are denormalized on purpose** — common flags repeat across cells.
|
||||
This makes each cell self-contained and easy to verify. When sweeping a
|
||||
common change, edit every cell.
|
||||
|
||||
**Avoid premature cells**: only add a cell for a (hw × variant × quant ×
|
||||
strategy × nodes) combination if you have a recipe that has been tested or
|
||||
at least sanity-checked. The engine greys out un-listed combinations
|
||||
automatically.
|
||||
|
||||
## 2.3 Configure `playgroundFeatures` (optional)
|
||||
|
||||
The Playground widget is opt-in per axis. Add only the axes that make sense
|
||||
for this model. Recognised axis keys and their schemas (full reference in
|
||||
the `_playground.jsx` header):
|
||||
|
||||
| Axis key | Widget | Use when |
|
||||
|---|---|---|
|
||||
| `attention` | TP / CP / DP-Attention sub-knobs (DP-Attention is a combined knob: its value is the DP degree AND toggles `--enable-dp-attention`) | Model exposes parallelism knobs in its cells (§2.2) and you want users to override them. |
|
||||
| `moe` | Backend select + EP knob | Model is MoE and supports multiple `--moe-*-backend` choices. |
|
||||
| `parsers` | Multi-toggle | Model has reasoning / tool-call parsers. |
|
||||
| `speculative` | Single-select chip group | Model has spec-decoding presets you want to expose. |
|
||||
| `pdDisagg` | Mode + transfer backend (+ optional per-backend env via `envWhen` hw-gate) + IB device + optional `router{port, command}` | Model supports prefill/decode disaggregation. When a PD role is active and `router` is set, the playground shows the router (SGLang Model Gateway) launch command as a separate companion block and retargets the cURL modal to `router.port` (clients hit the router, not the role servers). |
|
||||
| `hicache` | Enable + storage + write policy | Model is large enough that hierarchical KV cache matters. |
|
||||
| `hisparse` | Enable + host-ratio select; whole card gated on the live PD-Disagg mode being `decode` | DSA-style model (DeepSeek-V3.2 / V4, GLM-5) that supports decode-side hierarchical sparse attention. |
|
||||
| `megamoe` | Single-select with hw/strategy gating | Blackwell-only kernel fusion variant. |
|
||||
|
||||
**Per-chip constraints**: any chip entry in any axis can be wrapped with
|
||||
`hide` / `disable` constraint objects:
|
||||
|
||||
```js
|
||||
{ value: 16, disable: { nodes: ["single"] },
|
||||
disableReason: "TP=16 requires 16 ranks — switch the Deploy panel's Nodes to Multi-Nodes first." }
|
||||
```
|
||||
|
||||
- `hide` — chip omitted entirely (use for hard impossibilities).
|
||||
- `disable` — chip greyed out with tooltip (soft warning).
|
||||
- Constraints are AND across keys, OR within each key's array.
|
||||
- Bare `disabled: true` / `disable: true` is a static always-disabled form
|
||||
(used for "Coming soon" chips).
|
||||
|
||||
## 2.4 Create the MDX page
|
||||
|
||||
Path: `docs_new/cookbook/<category>/<Vendor>/<Model>.mdx`. Import both widgets and
|
||||
the per-model config, render them inside the relevant sections:
|
||||
|
||||
```mdx
|
||||
## Deployment
|
||||
|
||||
import { Deployment } from "/src/snippets/_deployment.jsx";
|
||||
import { config } from "/src/snippets/configs/<vendor>/<model>.jsx";
|
||||
import { benchmarks } from "/src/snippets/configs/<vendor>/<model>-benchmarks.jsx";
|
||||
|
||||
{/* Install is a PREREQUISITE — keep it compact + collapsed at the top of the
|
||||
Deploy section (NOT a numbered section). Tabs mirror the widget's
|
||||
Python/Docker toggle. */}
|
||||
<a id="install" />
|
||||
<Accordion title="Install SGLang">
|
||||
<Tabs>
|
||||
<Tab title="Python (pip / uv)">…pip / uv install…</Tab>
|
||||
<Tab title="Docker">…docker pull + a `docker run … sglang serve` example…</Tab>
|
||||
</Tabs>
|
||||
</Accordion>
|
||||
|
||||
<Deployment config={config} benchmarks={benchmarks} />
|
||||
|
||||
[model-specific tuning notes, caveats, links]
|
||||
|
||||
## Playground
|
||||
|
||||
import { Playground } from "/src/snippets/_playground.jsx";
|
||||
|
||||
<Playground config={config} />
|
||||
```
|
||||
|
||||
**Heading slugs matter** — the two widgets cross-link by scrolling to each
|
||||
other's section id (Mintlify auto-slugs headings: lowercase, spaces →
|
||||
hyphens, punctuation dropped). The engines look up:
|
||||
|
||||
- the **Deploy** panel by id `deployment` (falls back to `deploy`) — used
|
||||
by the Playground's "↑ Switch base" button and by deep-link scroll-on-
|
||||
load. Title the section `## Deployment` (or `## Deploy`).
|
||||
- the **Playground** by id `playground` — used by `_deployment.jsx`'s
|
||||
"Open the Playground →" link. Title the section `## Playground`.
|
||||
|
||||
Avoid numbered headings like `## 3. Model Deployment` (slug
|
||||
`3-model-deployment`) for these two sections — the cross-links would break.
|
||||
The Playground reads the Deploy selection live via the URL hash + the
|
||||
`sglang-deploy-sel` custom event, so the two can live in different parts
|
||||
of the page.
|
||||
|
||||
The `benchmarks` prop is **optional**. It points at a sibling
|
||||
`<model>-benchmarks.jsx` file (one entry per cell, keyed by the same
|
||||
`match` tuple) that renders an accuracy + speed sub-card under the command
|
||||
box; omit the import and the prop if the cookbook has no measured numbers
|
||||
yet. See the `_deployment.jsx` header and `deepseek-v4-benchmarks.jsx` for
|
||||
the full speed/accuracy schema.
|
||||
|
||||
To let users *reproduce* those numbers, add a `benchmarkCommands` block to
|
||||
the config (§2.1, next to `curl`). When present alongside `benchmarks`, the
|
||||
benchmark card grows a **"⚡ Reproduce"** button that opens a modal listing
|
||||
the runnable commands for the current cell — one `bench_serving` command for
|
||||
Speed (with concurrency chips that rewrite `--max-concurrency`) plus an
|
||||
Accuracy command with a chip per eval. No separate benchmark section needed.
|
||||
|
||||
---
|
||||
|
||||
## Pitfalls (authoring)
|
||||
|
||||
**Stale URL hash hydration** — If a user shares a link from an old cell
|
||||
catalog and the hash names an impossible combination, `_deployment.jsx`'s
|
||||
`validateSelection` snaps to the nearest real cell. The Playground reads
|
||||
the hash too — make sure cookbook removals don't leave dangling shared
|
||||
links pointing at hardware/quant combos that no longer exist.
|
||||
|
||||
**Mintlify constraints** — Module-level statements are stripped. The config
|
||||
MUST be a single `export const config = { ... }` literal — no function calls,
|
||||
spreads, fragment refs, or IIFE (Mintlify re-evaluates the export at hydration
|
||||
with module-level identifiers out of scope; any non-literal crashes with
|
||||
`ReferenceError`). In MDX, capitalized JSX tags get rebound — use the built-in
|
||||
Mintlify components (`<Accordion>`, `<Tabs>`, `<Card>`, ...) as documented.
|
||||
Avoid `!(x in y)` anywhere (Mintlify's AST walker crashes on it) — use
|
||||
`obj.key === undefined`.
|
||||
|
||||
**Per-cell denormalization** — Cells repeat common flags on purpose. Do
|
||||
not factor them into a shared `commonFlags` array — Mintlify will fail
|
||||
to inline the reference. If you need to sweep a flag across cells, do it
|
||||
with a global find-replace in the config file.
|
||||
@@ -0,0 +1,242 @@
|
||||
# Engine extension: add a new playground feature axis
|
||||
|
||||
Loaded on demand by the `cookbook-add-model` skill. **Rare** — adding a model
|
||||
cookbook is data-only and never needs this. The current 8 built-in axes
|
||||
(`attention`, `moe`, `parsers`, `speculative`, `pdDisagg`, `hicache`,
|
||||
`hisparse`, `megamoe`) already cover the SGLang feature surface most cookbooks
|
||||
need. Only add a new axis if a real cookbook needs it and the feature does not
|
||||
fit any existing axis. Touches `_playground.jsx` only.
|
||||
|
||||
For the per-model config/cells/MDX reference see [authoring-reference.md](authoring-reference.md).
|
||||
|
||||
---
|
||||
|
||||
## 3.1 Decide
|
||||
|
||||
Before touching the engine, confirm:
|
||||
|
||||
- The feature is a STABLE part of the SGLang CLI surface (will appear in
|
||||
multiple cookbooks, not one-off).
|
||||
- The feature cannot be expressed as a new option inside an existing axis
|
||||
(e.g. a new MoE backend belongs in `moe.backend.options`, not a new
|
||||
axis).
|
||||
- The feature has a clean strip-prefix → emit-flag pattern.
|
||||
|
||||
If unsure, add it as data first (in one cookbook's config under an
|
||||
existing axis) before promoting it to a built-in axis.
|
||||
|
||||
## 3.2 Pick the axis id and state shape
|
||||
|
||||
The axis id is the key in both `config.playgroundFeatures` and the
|
||||
internal `deltas` object. Use camelCase, descriptive but short:
|
||||
`mambaCache`, `attentionBackend`, `kvCacheDtype`.
|
||||
|
||||
The state shape is whatever `initState` returns. Common shapes:
|
||||
|
||||
- Single-select: a string sentinel (e.g. `"disabled"` / `"current"` / an
|
||||
option id).
|
||||
- Multi-toggle: `{[itemId]: bool}`.
|
||||
- Sub-knobs: `{[knobId]: value | null}`.
|
||||
- Compound (axis with its own internal sub-state, like PD-Disagg's
|
||||
`{mode, ibDevice}`): a plain object.
|
||||
|
||||
Pick ONE "inherit base" sentinel and document it in the handler comment.
|
||||
|
||||
## 3.3 Implement the handler
|
||||
|
||||
Add one entry to `AXIS_HANDLERS` in `_playground.jsx`.
|
||||
The handler owns everything: state init, apply (strip+insert), hidden-revert,
|
||||
AND the JSX render. Engine main loop iterates `AXIS_HANDLERS` and calls each
|
||||
method by name — adding a new axis is genuinely a one-place change.
|
||||
|
||||
Template:
|
||||
|
||||
```js
|
||||
// ---- Axis: <Title> ----------------------------------------------------
|
||||
// <one-paragraph description of what this axis controls and why it
|
||||
// exists. Mention the SGLang feature it wraps and the strip/insert
|
||||
// policy.>
|
||||
<axisId>: {
|
||||
initState: (fc) => /* initial state value */,
|
||||
|
||||
// Called when base cell changes. Return new value if the picked option
|
||||
// is now hidden by a constraint; otherwise return value unchanged.
|
||||
// Disabled picks are intentionally NOT auto-reverted (soft warning).
|
||||
revertHidden: (value, fc, base, h) => {
|
||||
// ... return value or a new value
|
||||
return value;
|
||||
},
|
||||
|
||||
// Pure function. Receives the current (flags, env) and returns the next
|
||||
// (flags, env). Do NOT mutate inputs. The `value` argument is whatever
|
||||
// initState returned. The `fc` argument is config.playgroundFeatures[axisId].
|
||||
// The `sel` argument is the current base cell selection. The `h`
|
||||
// argument is the helpers bundle (strip/insert primitives + anchors).
|
||||
apply: ({ flags, env, value, fc, sel, h, derived }) => {
|
||||
if (/* value is the inherit-base sentinel */) return { flags, env };
|
||||
flags = h.stripFlagsByFirstToken(flags, [/* prefixes this axis owns */]);
|
||||
if (/* an option is picked */) {
|
||||
flags = h.insertAfter(flags, h.ANCHOR_NEAR_<X>, [/* new flags */]);
|
||||
// or: flags = h.insertBeforeTail(flags, [/* new flags */]);
|
||||
// if the axis mutates env:
|
||||
// env = h.stripEnvByPrefix(env, fc.stripEnv || []);
|
||||
// env = [...env, /* additional env vars */];
|
||||
}
|
||||
return { flags, env };
|
||||
},
|
||||
|
||||
// Optional: read the base cell's flag array back into the same shape
|
||||
// initState/apply use. Render shows this as the default selection
|
||||
// (dropdown option or checked chip) when the state slot is the inherit
|
||||
// sentinel — so the user sees the cell's actual --tp / MoE backend /
|
||||
// spec preset instead of an opaque "Auto." When derive returns a real
|
||||
// value, the inherit-sentinel option is hidden from the control. Apply
|
||||
// also receives the derived
|
||||
// value (as `derived`) and may use it as a no-op shortcut when the
|
||||
// user's pick matches base. Skip when your axis owns flags that never
|
||||
// appear in base cells (PD-Disagg / HiCache / MegaMoE).
|
||||
// deriveFromBase: (cell, fc, h) => ({ ... }) | null,
|
||||
|
||||
// Optional: hints for the renderer. Currently only pdDisagg uses this
|
||||
// to report its role banner. Omit if not needed.
|
||||
// getRenderHints: (value, fc) => ({ pdMode: ... }) | null,
|
||||
|
||||
// Returns the axis card JSX. The outer div MUST have key={axisId} so
|
||||
// React can track it in the engine's map loop. Return null for
|
||||
// axis-level gating (e.g. MegaMoE on Hopper). Lay out as a single
|
||||
// compact horizontal row: title on the left, fields after.
|
||||
render: ({ axisId, value, setValue, fc, base, s, h, renderChip, renderSelect, derived }) => {
|
||||
if (/* axis-level gating fails */) return null;
|
||||
return (
|
||||
<div key={axisId} style={s.card}>
|
||||
<div style={s.compactRow}>
|
||||
<span style={s.axisTitle}>Axis Title</span>
|
||||
{/* For multi-option fields, use renderSelect(...) — the default.
|
||||
For on/off toggles or single-select chip groups, use
|
||||
renderChip instead (see "Control choice" in the conventions
|
||||
below). Read state from `value`; write via `setValue(next)`
|
||||
(replaces the whole axis slot). */}
|
||||
<span style={s.field}>
|
||||
<span style={s.fieldLabel}>Field</span>
|
||||
{renderSelect(value.slot, fc.entries, (v) =>
|
||||
setValue({ ...value, slot: v }), base)}
|
||||
</span>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
},
|
||||
},
|
||||
```
|
||||
|
||||
**Important conventions**:
|
||||
|
||||
- Insert the entry in the position you want it rendered. `AXIS_HANDLERS`
|
||||
is iterated in insertion order for both render and apply.
|
||||
- Use `h.ANCHOR_NEAR_*` constants for insertion. Add a new anchor to the
|
||||
helpers bundle if your axis needs to land somewhere new in the flag
|
||||
block.
|
||||
- Use lowercase HTML JSX tags only. Capitalized tags get rebound by
|
||||
Mintlify.
|
||||
- Inside `render`, read state via `value` (the slice for this axis).
|
||||
Write state via `setValue(next)` (replaces the whole slice). For
|
||||
compound axes, do `setValue({ ...value, [k]: nextK })`.
|
||||
- Layout: one `s.compactRow` per axis card, `s.axisTitle` for the
|
||||
leading label, one `s.field` per (label + input) pair.
|
||||
- Control choice — `renderSelect` vs `renderChip`:
|
||||
- `renderSelect(current, entries, onPick, base, labelFor?, opts?)` is
|
||||
the **default** compact control (a `<select>` dropdown). It filters
|
||||
hidden chips and disables greyed-out ones internally — no per-chip
|
||||
`evaluateChip` loop needed in the render body. Most axes use it
|
||||
(attention, moe, pdDisagg, hisparse, hicache, megamoe). Pass
|
||||
`{ hideValues: [<sentinel>] }` when your `deriveFromBase` resolved to
|
||||
a real value, so the inherit-sentinel ("Auto" / "Inherited" /
|
||||
"current") doesn't clutter the dropdown.
|
||||
- `renderChip(label, current, value, onPick, { disabled?, disabledReason? })`
|
||||
renders a **button** instead of a dropdown row. Use it for a chip
|
||||
group when you want the options laid out as buttons. It serves two
|
||||
shapes:
|
||||
- **Multi-toggle** (Parsers) — one independent on/off chip per item;
|
||||
`current` is that item's effective bool, `value` is `true`, so the
|
||||
chip is "checked" when the item is on.
|
||||
- **Single-select** (Speculative) — a radio-style group; pass the
|
||||
group's effective value as `current` and each option's id as
|
||||
`value`, so exactly one chip is checked (`current === value`).
|
||||
Chip groups own their visibility/disable filtering: loop
|
||||
`h.evaluateChip(opt, base)` in the render body, skip `c.hidden`,
|
||||
filter the inherit-sentinel yourself when `deriveFromBase` resolved
|
||||
to a real value, and forward `c.disabled` / `c.disableReason` into
|
||||
`renderChip`'s opts (this is what surfaces a disabled chip's tooltip,
|
||||
e.g. a "Coming soon" entry).
|
||||
- Selected chips use the same terracotta (`#D45D44`) as the Deploy
|
||||
panel's selected button, so both widgets read as one
|
||||
visual system. Don't introduce a per-axis accent color.
|
||||
- Default-from-base: if your axis can be read out of base cells'
|
||||
flags, implement `deriveFromBase` and have your render show the
|
||||
derived value when state is the sentinel (e.g.
|
||||
`const eff = value.tp !== null ? value.tp : (derived && derived.tp)`).
|
||||
This is what makes a fresh playground load show the user's actual
|
||||
recipe instead of "auto." Flag-parsing helpers on `h`:
|
||||
`parseIntFlag`, `hasFlag`, `findFlagArg`.
|
||||
- **Avoid the `in` operator wrapped in unary** (`!(x in y)`). Mintlify's
|
||||
AST walker crashes on it (`TypeError: this[e] is not a function`). Use
|
||||
`obj.key === undefined` or `obj.id !== undefined` instead. Bare
|
||||
`if (key in obj)` (no surrounding `!`) is fine.
|
||||
|
||||
## 3.4 Document the per-cookbook schema
|
||||
|
||||
Edit the file header in `_playground.jsx` to add your new axis to the
|
||||
"Recognised keys" list, with a one-line description of its schema.
|
||||
Optionally add a paragraph below explaining its strip/insert policy.
|
||||
|
||||
Update the §2.3 axis table in [authoring-reference.md](authoring-reference.md) to list the new axis.
|
||||
|
||||
## 3.5 Migrate cookbooks that need it
|
||||
|
||||
For each cookbook that should expose this axis, add a
|
||||
`playgroundFeatures.<axisId>` entry to its config. Verify the chip group
|
||||
renders, options apply correctly, and the diff matches expectations.
|
||||
|
||||
---
|
||||
|
||||
## Pitfalls (engine work)
|
||||
|
||||
**Insertion anchor misses** — `insertAfter` falls back to right-after
|
||||
`--model-path` if none of its anchor prefixes are present. If your axis
|
||||
emits flags that should land somewhere specific, include the most likely
|
||||
anchor prefixes in your call. Order doesn't matter (set semantics).
|
||||
|
||||
**Conditional strips** — Some axes strip ONLY when overridden
|
||||
(`attention.tp`, `moe.backend`, `speculative`, `megamoe`). Others strip
|
||||
UNCONDITIONALLY whenever declared (`parsers`, `pdDisagg`, `hicache`). The
|
||||
header comment in `AXIS_HANDLERS` documents which policy each axis uses;
|
||||
follow the same pattern when adding a new axis. If unsure, prefer
|
||||
conditional strip — it preserves base behavior when the user does not
|
||||
opt in.
|
||||
|
||||
**Closure of `AXIS_HANDLERS`** — Inside a handler method, you can
|
||||
reference `AXIS_HANDLERS.<otherAxis>` for cross-handler calls (megamoe
|
||||
does this for `_gateOpen`). This works because `AXIS_HANDLERS` is in
|
||||
lexical scope. Do NOT use this for general logic — it tightly couples
|
||||
handlers. Reserve it for one handler's helpers shared between its own
|
||||
`render` and `revertHidden`.
|
||||
|
||||
---
|
||||
|
||||
## Review checklist for a new-axis PR
|
||||
|
||||
- [ ] `AXIS_HANDLERS` is the ONLY place that mentions the new axis id
|
||||
(apart from per-cookbook config). No `if (axisId === '<new>')`
|
||||
branches anywhere in the engine.
|
||||
- [ ] `initState` is deterministic and idempotent (does not depend on
|
||||
the base cell).
|
||||
- [ ] `apply` is pure — does not mutate inputs.
|
||||
- [ ] `revertHidden` returns the same reference when nothing changed
|
||||
(avoids unnecessary re-renders).
|
||||
- [ ] `render` returns `null` when axis-level gating fails (whole card
|
||||
hidden) — does not render an empty placeholder.
|
||||
- [ ] `render` sets `key={axisId}` on its outer element.
|
||||
- [ ] No `!(x in y)` patterns introduced (Mintlify AST walker crashes).
|
||||
- [ ] File header lists the new axis in "Recognised keys".
|
||||
- [ ] The §2.3 table in `authoring-reference.md` lists the new axis.
|
||||
- [ ] One existing cookbook config is updated to consume the new axis,
|
||||
and visual verification shows the diff is correct.
|
||||
@@ -0,0 +1,107 @@
|
||||
# MDX authoring rules (Mintlify) + invocation-example patterns
|
||||
|
||||
Loaded on demand by the `cookbook-add-model` skill (Phase 5, writing the page prose).
|
||||
These are model-agnostic Mintlify hygiene rules — the most common review findings.
|
||||
The cookbook is **Mintlify**, not Docusaurus.
|
||||
|
||||
## Mintlify syntax
|
||||
|
||||
**Allowed components**: `<Card>`, `<CardGroup>`, `<Note>`, `<Tip>`, `<Warning>`,
|
||||
`<Info>`, `<Accordion>`, `<AccordionGroup>`, `<Steps>`, `<Step>`, `<Tabs>`, `<Tab>`,
|
||||
`<CodeGroup>`, `<Frame>`, `<Icon>`.
|
||||
|
||||
**Forbidden** (flag every occurrence):
|
||||
- Docusaurus admonitions (`:::note` / `:::warning` / …) — use `<Note>` / `<Warning>`.
|
||||
- `@site/...` / `@theme/...` imports — use absolute `/src/snippets/...`.
|
||||
- GitHub alert blocks (`> [!NOTE]`, `> [!WARNING]`).
|
||||
- **Markdown pipe tables** on new pages — use JSX `<table>` (see below).
|
||||
- Inline `<details>` / `<summary>` — use `<Accordion>`.
|
||||
- Unknown / non-Mintlify components.
|
||||
- `<CardGroup>` / `<Card>` on individual model pages — those are for category `intro.mdx` only.
|
||||
|
||||
**Code fences**: always labeled — ` ```python Example `, ` ```bash Command `,
|
||||
` ```shell Command `, ` ```text Output `. When nesting a fenced block inside another,
|
||||
the **outer** fence uses four backticks.
|
||||
|
||||
**Internal links**: root-relative, no extension (`/cookbook/<category>/<Vendor>/<Model>`);
|
||||
`docs.sglang.io` is canonical. Flag `.md`/`.mdx` extensions and `../`-relative page links
|
||||
in body prose. (Existing cookbook pages do use `../../../docs/...` for cross-links into
|
||||
the non-cookbook docs tree — that's the established exception; don't introduce new ones.)
|
||||
|
||||
## JSX tables (required for all tables on new pages)
|
||||
|
||||
```jsx
|
||||
<table style={{width: "100%", borderCollapse: "collapse", tableLayout: "fixed"}}>
|
||||
<thead>
|
||||
<tr style={{borderBottom: "2px solid #d55816"}}>
|
||||
<th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700}}>Col</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr><td style={{padding: "9px 12px"}}>cell</td></tr>
|
||||
</tbody>
|
||||
</table>
|
||||
```
|
||||
|
||||
Alternate column background colors (`rgba(255,255,255,0.02)` / `0.05`) for readability;
|
||||
adjust `<colgroup>` widths for 3- or 5-column tables. The DeepSeek-V4 page §1 variants
|
||||
table is a live reference.
|
||||
|
||||
## Invocation-example patterns (§3 Advanced Usage)
|
||||
|
||||
- **Reasoning-parser output shape must match the example**:
|
||||
- *Separate-field* parsers (most qwen/glm, `kimi_k2`, `deepseek-v4`): thinking lands
|
||||
in `message.reasoning_content`, answer in `message.content` — print both.
|
||||
- *Inline-tag* parsers (e.g. `minimax-append-think`): thinking is wrapped in
|
||||
`<think>...</think>` **inside** `message.content` — the client parses the tags; for
|
||||
streaming, buffer and split on the markers.
|
||||
Pick the pattern from the model card / SGLang docs for that specific parser.
|
||||
- **Hybrid reasoning models**: show both thinking-on (default) and thinking-off
|
||||
(`extra_body={"chat_template_kwargs": {"thinking": False}}` or `enable_thinking: False`).
|
||||
- **Tool-call follow-up on thinking models**: the final assistant turn may put text in
|
||||
`reasoning_content` instead of (or with) `content` — print both so the output isn't a
|
||||
misleading `None`.
|
||||
- **Every runnable block** is immediately followed by `**Output Example:**` + a
|
||||
` ```text Output ` block with **real** server output (verbatim, not paraphrased).
|
||||
`Pending update...` is acceptable only with the user's explicit acknowledgement.
|
||||
- **Do not hardcode sampling params** (`temperature`, `top_p`) in sample code — SGLang
|
||||
uses `generation_config.json` defaults. Listing "Recommended generation" in §1 is fine.
|
||||
- Format raw API objects (`ChatCompletionMessage(...)`) into readable Reasoning /
|
||||
Content / Tool Calls blocks.
|
||||
|
||||
## Frontmatter
|
||||
|
||||
- **Top-level `description:`** is the canonical field — it sets the page's SEO meta
|
||||
description (`og:`/`twitter:description` fall back to it) AND renders as the visible
|
||||
**subtitle** under the title, filling the header band before the first heading. Give every
|
||||
page a one-line top-level `description` (a lede / value prop) — without it, a page that
|
||||
opens straight into `## Deployment` shows an empty gap under the title (the title and
|
||||
`## Deployment` are the same size, so they read as two bare headings). Do **not** put the
|
||||
description inside a `metatags` block — `metatags` is for other/custom tags, and
|
||||
`metatags.description` is redundant with (and non-canonical vs) the top-level field.
|
||||
- **Write it for SEO** (it doubles as the search-result snippet): front-load the exact
|
||||
model name + intent — e.g. `Deploy <Model> with SGLang — …` — aim for ~150–160 chars, and
|
||||
pack secondary keywords (variants + sizes, `Mixture-of-Experts` / architecture, target
|
||||
GPUs). Phrase it as a value prop, not a generic "`<Model>` is a … model" intro.
|
||||
- Frontmatter MUST be the first thing in the file — no comment or blank line before the
|
||||
opening `---`.
|
||||
|
||||
## Commands & ports
|
||||
|
||||
- **Deploy/launch** commands use `sglang serve --model-path …` — never
|
||||
`python -m sglang.launch_server` / `python3 -m sglang.launch_server` (deprecated).
|
||||
- **Benchmark workload** commands use `python3 -m sglang.bench_serving …` (never bare
|
||||
`python -m`); built-in accuracy scripts use `python3 benchmark/...`.
|
||||
- Port **30000** everywhere on a page — launch, curl, client `base_url`, and bench must
|
||||
agree. Keep one canonical deploy command (the Deploy widget) and don't re-paste launch
|
||||
commands across sections; the documented command must match the widget's output for the
|
||||
same selection (doc ↔ config parity).
|
||||
|
||||
## Factual hygiene
|
||||
|
||||
- License must match the actual HuggingFace license (don't copy from another model).
|
||||
- HF URLs resolve to a real model; Docker images from `lmsysorg/sglang`.
|
||||
- No Google-Drive image links (they don't render); host images in the repo.
|
||||
- Shell placeholders are `export VAR=<value>`, not `export VAR=${VAR}` (a bash no-op).
|
||||
- `tag: NEW` is sparing — at most one per `<category>/<Vendor>/` dir (the newest); strip
|
||||
it from siblings when adding a new NEW page.
|
||||
@@ -0,0 +1,76 @@
|
||||
# Vendor card logo (new brand only)
|
||||
|
||||
A new vendor/brand in the cookbook landing grid needs a card logo at
|
||||
`docs_new/cards/logos/<org-slug>.png`. **Ask the user for the brand's logo, then generate
|
||||
the conforming PNG** — never invent, copy, or hallucinate one, and never ship a
|
||||
non-conforming file. Reference: PR #27400 (added `tencent.png` + `poolside.png`).
|
||||
|
||||
If the org already has a card/logo, do nothing here — only update the `<Card href>`.
|
||||
|
||||
## Spec (match the existing logos exactly)
|
||||
|
||||
| Property | Value |
|
||||
|---|---|
|
||||
| Path | `docs_new/cards/logos/<org-slug>.png` — lowercase, matches the `img=` in the `<Card>` |
|
||||
| Canvas | **940 × 525** px |
|
||||
| Mode | **RGBA**, fully **transparent** background |
|
||||
| Content | **Icon-only** — the brand glyph/mark (the "swirl"), **no wordmark text** |
|
||||
| Placement | centered; glyph ≈ 0.33 × width and ≈ 0.50 × height of the canvas |
|
||||
|
||||
Why icon-only + transparent: cards render on both light and dark backgrounds, so a baked-in
|
||||
(usually black) wordmark vanishes on dark. `deepseek.png` / `ernie.png` are 940×525 RGBA
|
||||
exemplars — eyeball your output against them.
|
||||
|
||||
## 1. Get the source
|
||||
|
||||
Ask the user for the brand logo (SVG preferred → crisp + already transparent; else a high-res
|
||||
transparent PNG, or a link to the official press/brand asset). Prefer an **icon-only** source;
|
||||
if they only have a full lockup, ask them to crop the glyph, or crop it yourself.
|
||||
|
||||
If the user **pasted** an image inline, it may not be on disk — recover the base64 `image`
|
||||
block from the session transcript (`~/.claude/projects/<slug>/*.jsonl`) and decode it to a file.
|
||||
|
||||
## 2. Generate (Pillow)
|
||||
|
||||
There's no system Pillow — use a venv:
|
||||
|
||||
```bash
|
||||
python3 -m venv /tmp/logo-venv && /tmp/logo-venv/bin/pip install -q Pillow
|
||||
```
|
||||
|
||||
```python
|
||||
from PIL import Image
|
||||
src = Image.open("SOURCE").convert("RGBA") # icon-only, already transparent
|
||||
W, H = 940, 525
|
||||
target_h = round(H * 0.50) # glyph ≈ half the canvas height
|
||||
scale = target_h / src.height
|
||||
glyph = src.resize((round(src.width * scale), target_h), Image.LANCZOS)
|
||||
canvas = Image.new("RGBA", (W, H), (0, 0, 0, 0)) # transparent
|
||||
canvas.paste(glyph, ((W - glyph.width) // 2, (H - glyph.height) // 2), glyph)
|
||||
canvas.save("docs_new/cards/logos/<org-slug>.png")
|
||||
```
|
||||
|
||||
Notes:
|
||||
- **Wordmark present?** Crop to the glyph first (or ask the user for an icon-only asset). Don't
|
||||
ship text in the logo.
|
||||
- **Solid background?** Don't auto-strip it (risky) — ask the user for a transparent source.
|
||||
- **SVG source?** Rasterize at high res first (`cairosvg` / `rsvg-convert`), then run the above.
|
||||
- If the glyph is much wider than tall, cap by width instead (≈ 0.33 × W) so it doesn't overflow.
|
||||
|
||||
## 3. Verify
|
||||
|
||||
```bash
|
||||
sips -g pixelWidth -g pixelHeight -g hasAlpha docs_new/cards/logos/<org-slug>.png
|
||||
# → pixelWidth: 940 pixelHeight: 525 hasAlpha: yes
|
||||
```
|
||||
|
||||
## 4. Wire + track + validate
|
||||
|
||||
```bash
|
||||
# Card in the landing grid (keep card order aligned with the docs.json nav order):
|
||||
# <Card title="<NavGroup>" mode="card"
|
||||
# href="/cookbook/<category>/<Vendor>/<Model>"
|
||||
# img="/cards/logos/<org-slug>.png" />
|
||||
git add -f docs_new/cards/logos/<org-slug>.png # root .gitignore ignores *.png repo-wide
|
||||
cd docs_new && mint validate && mint broken-links # confirms the card href + img resolve
|
||||
```
|
||||
Reference in New Issue
Block a user