docs(cookbook): total (input+output) throughput per GPU + percentile latency labels (#30214)

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 00:39:48 -07:00
parent 5f98f62a8a
commit 7c9bb316cf
13 changed files with 168 additions and 132 deletions
@@ -6,7 +6,10 @@
 // One entry per cell `match` tuple (same 5 keys as config cells). The card stays
 // "pending" until an entry has a non-null speed metric or accuracy. Speed shape:
 //   speed: [{ workload: {dataset, isl, osl, max_concurrency}, ttft_ms, tpot_ms,
-//             tokens_per_sec_per_gpu }, ...]   // interactivity is derived (1000/TPOT)
+//             tokens_per_sec_per_gpu }, ...]
+//   - ttft_ms/tpot_ms are P50 (median); set config.latencyPercentile ("P50" default, or "Mean").
+//   - tokens_per_sec_per_gpu = total (in+out) tok/s/GPU
+//     (= output tok/s ÷ GPUs × (isl+osl)/osl). interactivity is derived = 1000/TPOT (tokens/s/user).
 // Per-cell `accuracy: { <key>: <pct> }` overrides the config's defaultAccuracy.

 export const benchmarks = [