docs(cookbook): re-benchmark DeepSeek-V4 on sglang 0.5.15 (#31363)

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Zijie Xia <zijie.xia@radixark.ai>
This commit is contained in:
Douglas Yang
2026-07-21 22:55:36 +00:00
committed by GitHub
co-authored by Claude Opus 4.8 Zijie Xia
parent 9057db9417
commit 4a55fdba0b
8 changed files with 231 additions and 71 deletions
+8 -3
View File
@@ -115,12 +115,17 @@ than restating.
- A benchmark's quantization must match a variant actually listed — `(BF16)` on a model
that only released FP8/FP4 is a factual bug.
- `benchmarkCommands.speed` is `python3 -m sglang.bench_serving` (the workload), separate
from the `sglang serve` deploy command.
from the `sglang serve` deploy command, and should carry `--flush-cache`: bench_serving's
`random` prompts are deterministic, so a warm rerun hits the radix cache and inflates
throughput — speed numbers are measured cache-cold.
- `sglang_version` is a real build the author ran (a release, or `dev`/nightly) — not a
guessed/placeholder value (no leftover `0.0.0`).
- **Latency percentile**: `config.latencyPercentile` (default `"P50"`, or `"Mean"`) matches the
percentile the TTFT/TPOT values actually are — the card renders `TTFT (<pct>)`. (`"Mean"` is
temporary — legacy data is being re-measured to P50.)
percentile the TTFT/TPOT values actually are — the card renders `TTFT (<pct>)`. A benchmarks
entry may carry its own `latencyPercentile` to override the page value per cell
(entry → config → `"P50"`): on a P50 page, kept legacy Mean cells must set it — a
`sglang_version` tag alone doesn't convey the percentile. (`"Mean"` is temporary — legacy
data is being re-measured to P50.)
- **Throughput convention**: `tokens_per_sec_per_gpu` is stored as **total (in+out)/GPU**
= `output tok/s/GPU × (isl+osl)/osl`, shown by the card as-is. Flag output-only values.
- **Consistent accuracy harness across entries**: every value under one `accuracyLabels`