# feat(bench): add SPEED-Bench dataset support to bench_serving (#24149)

Co-authored-by: zijiexia <37504505+zijiexia@users.noreply.github.com>
Co-authored-by: Khoa Pham <khoa.pham@radixark.ai>

docs_new/docs/developer_guide/bench_serving.mdx

@@ -65,6 +65,7 @@ Select with `--dataset-name`:
 - `image`: generates images and wraps them in chat messages; supports custom resolutions, multiple formats, and different content types
 - `generated-shared-prefix`: synthetic dataset with shared long system prompts and short questions
 - `mmmu`: samples from MMMU (Math split) and includes images
+- `speed-bench`: [SPEED-Bench](https://huggingface.co/datasets/nvidia/SPEED-Bench) (**SPEculative Evaluation Dataset**) — a unified benchmark for evaluating [Speculative Decoding (SD)](https://arxiv.org/abs/2604.09557) algorithms. Uses the Throughput split, which provides fixed-length input sequences (1K–32K tokens) grouped into three output-entropy categories (`low_entropy`, `mixed`, `high_entropy`). Requires a pre-downloaded JSONL file passed via `--dataset-path`.
 
 Common dataset flags:
 
@@ -90,6 +91,12 @@ Image dataset flags (for `image`):
 - `--image-format`: Image format (jpeg or png)
 - `--image-content`: Image content type (random or blank)
 
+SPEED-Bench flags (for `speed-bench`):
+
+- `--dataset-path PATH`: path to the pre-downloaded SPEED-Bench Throughput JSONL (e.g., `throughput_1k.jsonl`). Use the [SPEED-Bench measurement framework](https://github.com/NVIDIA/Model-Optimizer/tree/main/examples/specdec_bench) to generate it.
+- `--speed-bench-category`: filter to one entropy category: `low_entropy`, `mixed`, or `high_entropy` (default: all)
+- `--speed-bench-output-len`: fixed number of output tokens per request (default: 512)
+
 ### Examples
 
 1. To benchmark image dataset with 3 images per request, 500 prompts, 512 input length, and 512 output length, you can run:
@@ -125,6 +132,23 @@ python3 -m sglang.bench_serving \
     --random-range-ratio 0.5
 ```
 
+3. To benchmark speculative decoding throughput using SPEED-Bench (mixed-entropy category, 1K ISL), you can run:
+
+```bash Command
+python -m sglang.launch_server --model-path meta-llama/Llama-3.1-8B-Instruct \
+    --speculative-algorithm EAGLE --speculative-draft-model-path <draft-model-path>
+```
+
+```bash Command
+python3 -m sglang.bench_serving \
+    --backend sglang \
+    --dataset-name speed-bench \
+    --dataset-path /path/to/throughput_1k.jsonl \
+    --speed-bench-category mixed \
+    --speed-bench-output-len 512 \
+    --num-prompts 512
+```
+
 ### Choosing model and tokenizer
 
 - `--model` is required unless the backend exposes `GET /v1/models`, in which case the first model ID is auto-selected.