docs(cookbook): re-benchmark DeepSeek-V4 on sglang 0.5.15 (#31363)

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Zijie Xia <zijie.xia@radixark.ai>

.claude/skills/cookbook-add-model/references/authoring-reference.md

@@ -46,10 +46,10 @@ the full contract):
 | `multiNodeHints` | `{[hwId]: string[]}` | Lines prepended as `# ...` comments to multi-node commands (env-var hints). Per-hw, and only for hw whose **cluster fabric needs manual NIC config** (e.g. `gb200` NVL72/MNNVL → NVSHMEM/Gloo hints). NOT every multi-N hw needs an entry — standard-IB DeepEP (h200) auto-detects the HCA, and Marlin multi-node (h100) uses no DeepEP/NVSHMEM at all. |
 | `dockerImages` | `{[key]: string}` | Image for `docker run` framing, keyed by `hw\|quant` (most specific) then `hw`. Use a `hw\|quant` key only when one quant on a shared GPU needs a different image (e.g. an NVFP4 dev build on b300/gb300 while FP8/BF16 stay on the release image); otherwise key by plain `hw`. **Ask the user which sglang build the recipes ran on; don't guess a supporting release.** Falls back to `lmsysorg/sglang:dev` if missing — also the sensible default when unsure. |
 | `playgroundFeatures` | `{[axisId]: {...}}` | Opts into the Playground widget. See §2.3. |
-| `benchmarkCommands` | `{speed: string, accuracy: {[accKey]: string \| {[variant]: string}}, numPromptsByConc?: {[c]: number}}` | Powers the benchmark card's **"⚡ Reproduce"** modal. `speed` is ONE `bench_serving` template; the engine fills `{{DATASET}}`/`{{ISL}}`/`{{OSL}}` from each cell's `speed[].workload`, the chip-picked `{{MAX_CONCURRENCY}}`, and `{{NUM_PROMPTS}}` (resolved `workload.num_prompts ?? numPromptsByConc[c] ?? max(c*2, 200)`). `accuracy` maps an accuracy field (e.g. `gsm8k_pct`) to a per-eval template — a string, OR a `{flash, pro, …}` object keyed by variant when the command differs per variant (e.g. GPQA/AIME `--max-tokens`). The modal renders a chip per eval (one command area, like Speed). Both also use `{{MODEL_NAME}}` + `{{CURL_HOST}}`/`{{CURL_PORT}}` like `curl`. Optional; the button only appears when this AND `benchmarks` are present. |
+| `benchmarkCommands` | `{speed: string, accuracy: {[accKey]: string \| {[variant]: string}}, numPromptsByConc?: {[c]: number}}` | Powers the benchmark card's **"⚡ Reproduce"** modal. `speed` is ONE `bench_serving` template; the engine fills `{{DATASET}}`/`{{ISL}}`/`{{OSL}}` from each cell's `speed[].workload`, the chip-picked `{{MAX_CONCURRENCY}}`, and `{{NUM_PROMPTS}}` (resolved `workload.num_prompts ?? numPromptsByConc[c] ?? max(c*2, 200)`). `accuracy` maps an accuracy field (e.g. `gsm8k_pct`) to a per-eval template — a string, OR a `{flash, pro, …}` object keyed by variant when the command differs per variant (e.g. GPQA/AIME `--max-tokens`). The modal renders a chip per eval (one command area, like Speed). Both also use `{{MODEL_NAME}}` + `{{CURL_HOST}}`/`{{CURL_PORT}}` like `curl`. `speed` should carry `--flush-cache` (bench_serving's `random` prompts are deterministic — warm reruns hit the radix cache and inflate throughput; measure cache-cold). Optional; the button only appears when this AND `benchmarks` are present. |
 | `defaultAccuracy` | `{[variant]: {[accKey]: number}}` | Model-level accuracy applied to **every** cell of a variant (e.g. GPQA Diamond / AIME25 — hardware-independent). Merged UNDER each cell's measured `accuracy` (a per-cell value wins), so you set a variant's score once instead of copying it onto every benchmark entry. Keys must match `accuracyLabels` (below) + `benchmarkCommands.accuracy`. |
 | `accuracyLabels` | `[key, label, unit][]` | The eval set rendered in the benchmark card and the "⚡ Reproduce" modal — **the engine ships no default**, every config declares its own (e.g. DSv4: GPQA/AIME25/GSM8K; Qwen3.5: GSM8K/MMMU). Required whenever the benchmarks carry accuracy data; without it the accuracy rows silently don't render. Every key used in `benchmarks[].accuracy`, `defaultAccuracy`, and `benchmarkCommands.accuracy` must appear here. |
-| `latencyPercentile` | `"Mean" \| "P50"` | Optional, **temporary**; the percentile the benchmark TTFT/TPOT values are. **Default `"P50"`** — the card renders `TTFT (<pct>)` / `TPOT (<pct>)`. Set `"Mean"` only for legacy data recorded as Mean (being re-measured to P50). `tokens_per_sec_per_gpu` is stored as **total (in+out)/GPU** = `output tok/s/GPU × (isl+osl)/osl`, shown by the card as-is. |
+| `latencyPercentile` | `"Mean" \| "P50"` | Optional, **temporary**; the percentile the benchmark TTFT/TPOT values are. **Default `"P50"`** — the card renders `TTFT (<pct>)` / `TPOT (<pct>)`. Set `"Mean"` only for legacy data recorded as Mean (being re-measured to P50). A benchmarks entry may carry its own `latencyPercentile` to override the page value per cell (entry → config → `"P50"`). `tokens_per_sec_per_gpu` is stored as **total (in+out)/GPU** = `output tok/s/GPU × (isl+osl)/osl`, shown by the card as-is. |
 | `github` | `{owner?, repo?, issueTemplate?, cookbookModel?}` | Overrides for the "Submit verified cell" CTA in the playground. Defaults: `sgl-project/sglang` + `3-playground-verified-cell.yml` + `"deepseek-ai/deepseek-v4"`. Set `cookbookModel` to the model's HF id (`<hf-org>/<model-slug>`); it prefills the issue template's free-form `model` input when the issue opens. **Don't prune this block** — without it the engine falls back to `deepseek-ai/deepseek-v4` and submissions from your page get mislabeled. |
 
 ## 2.2 Author the 5-dim matrix (`cells[]`)

.claude/skills/cookbook-add-model/templates/benchmarks.jsx.tmpl

@@ -7,7 +7,8 @@
 // "pending" until an entry has a non-null speed metric or accuracy. Speed shape:
 //   speed: [{ workload: {dataset, isl, osl, max_concurrency}, ttft_ms, tpot_ms,
 //             tokens_per_sec_per_gpu }, ...]
-//   - ttft_ms/tpot_ms are P50 (median); set config.latencyPercentile ("P50" default, or "Mean").
+//   - ttft_ms/tpot_ms are P50 (median); set config.latencyPercentile ("P50" default, or "Mean");
+//     an entry-level latencyPercentile overrides the page value per cell.
 //   - tokens_per_sec_per_gpu = total (in+out) tok/s/GPU
 //     (= output tok/s ÷ GPUs × (isl+osl)/osl). interactivity is derived = 1000/TPOT (tokens/s/user).
 // Per-cell `accuracy: { <key>: <pct> }` overrides the config's defaultAccuracy.

.claude/skills/cookbook-add-model/templates/config.jsx.tmpl

@@ -97,7 +97,8 @@ export const config = {
   --model {{MODEL_NAME}} \\
   --dataset-name {{DATASET}} \\
   --random-input-len {{ISL}} --random-output-len {{OSL}} \\
-  --num-prompts {{NUM_PROMPTS}} --max-concurrency {{MAX_CONCURRENCY}}`,
+  --num-prompts {{NUM_PROMPTS}} --max-concurrency {{MAX_CONCURRENCY}} \\
+  --flush-cache`,
     // One entry per accuracy field. A value is a string, OR a {[variant]: string}
     // object when the command differs per variant. Keys must match ACCURACY_LABELS
     // in _deployment.jsx + the per-cell/defaultAccuracy keys.

.claude/skills/cookbook-migrate-model/SKILL.md

@@ -177,7 +177,8 @@ One entry per measured block only (cells without entries already render
 total (in+out) tok/s/GPU = `output tok/s ÷ (tp × nnodes) ×
 (isl+osl)/osl` — stored directly (the card shows it as-is). TTFT/TPOT
 take the P50 (median) rows; set `config.latencyPercentile` (default `"P50"`; use
-`"Mean"` only for legacy Mean-recorded data — temporary, being migrated to P50).
+`"Mean"` only for legacy Mean-recorded data — temporary, being migrated to P50; an
+entry-level `latencyPercentile` overrides the page value per cell).
 Put the workload's
 `num_prompts` into `workload`. **`config.accuracyLabels` is required whenever
 the benchmarks carry accuracy data** — the engine ships no default eval set

.claude/skills/cookbook-review-pr/SKILL.md

@@ -115,12 +115,17 @@ than restating.
 - A benchmark's quantization must match a variant actually listed — `(BF16)` on a model
   that only released FP8/FP4 is a factual bug.
 - `benchmarkCommands.speed` is `python3 -m sglang.bench_serving` (the workload), separate
-  from the `sglang serve` deploy command.
+  from the `sglang serve` deploy command, and should carry `--flush-cache`: bench_serving's
+  `random` prompts are deterministic, so a warm rerun hits the radix cache and inflates
+  throughput — speed numbers are measured cache-cold.
 - `sglang_version` is a real build the author ran (a release, or `dev`/nightly) — not a
   guessed/placeholder value (no leftover `0.0.0`).
 - **Latency percentile**: `config.latencyPercentile` (default `"P50"`, or `"Mean"`) matches the
-  percentile the TTFT/TPOT values actually are — the card renders `TTFT (<pct>)`. (`"Mean"` is
-  temporary — legacy data is being re-measured to P50.)
+  percentile the TTFT/TPOT values actually are — the card renders `TTFT (<pct>)`. A benchmarks
+  entry may carry its own `latencyPercentile` to override the page value per cell
+  (entry → config → `"P50"`): on a P50 page, kept legacy Mean cells must set it — a
+  `sglang_version` tag alone doesn't convey the percentile. (`"Mean"` is temporary — legacy
+  data is being re-measured to P50.)
 - **Throughput convention**: `tokens_per_sec_per_gpu` is stored as **total (in+out)/GPU**
   = `output tok/s/GPU × (isl+osl)/osl`, shown by the card as-is. Flag output-only values.
 - **Consistent accuracy harness across entries**: every value under one `accuracyLabels`