minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 6 Packages Projects Releases Wiki Activity

[Docs] GLM-5.3/5.3-Flash cookbooks: enable reasoning/tool-call parsers by default via auto (#40497)

Browse Source 
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
This commit is contained in:
Brayden Zhong
2026-09-22 13:54:08 +08:00
committed by GitHub
co-authored by Xinyuan Tong
parent 04c0913434
commit a0781f2714
4 changed files with 83 additions and 74 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/cookbook/autoregressive/GLM/GLM-5.3-Flash.mdx
+5 -5
View File
@@ -10,10 +10,10 @@ tag: NEW
<Accordion title="Install SGLang">
Use an SGLang build that includes GLM-5.3-Flash support.
Use an SGLang build that includes GLM-5.3-Flash support (v0.5.20 or later).
```bash Command
docker pull lmsysorg/sglang:glm-5.3-flash
docker pull lmsysorg/sglang:latest
```
The deployment panel can render a complete `docker run` command for the selected hardware and options. See [Install SGLang with Docker](/docs/get-started/install#method-3-using-docker) for host setup.
@@ -104,7 +104,7 @@ The **Speculative** card in the Playground changes the algorithm without leaving
- **EAGLE / MTP 5-1-6** is exactly what Low Latency serves, so a Low Latency base starts on this chip. Pick it from a High Throughput base to keep that recipe's other settings and add the MTP head.
- **Off (greedy)** strips the whole `--speculative-*` family, which is what High Throughput already starts from.
- **DFlash2** swaps the in-checkpoint MTP head for the trained block-diffusion draft in [`incoai/GLM-5.3-Flash-DFlash2`](https://huggingface.co/incoai/GLM-5.3-Flash-DFlash2). The draft proposes a whole block per step and the target verifies it in one forward pass, so output quality stays the target's. Its block size comes from the draft checkpoint, and the draft runs on `fa4` rather than the target's DSA backends. It needs a build that carries the GLM-5.3-Flash hidden-state capture from [PR #36708](https://github.com/sgl-project/sglang/pull/36708), which is merged into the [PR #36507](https://github.com/sgl-project/sglang/pull/36507) support branch (`xinyuan/glm-5.3-flash-support`) rather than into `main`, so the image pinned above is not enough on its own — pull that branch at its current head, or add #36708's commit on top of an older checkout. The draft repository is also access-gated: request access on its model page, then download it alongside the target before serving. This combination is not yet measured on the cookbook hardware, so treat it as a starting point.
- **DFlash2** swaps the in-checkpoint MTP head for the trained block-diffusion draft in [`incoai/GLM-5.3-Flash-DFlash2`](https://huggingface.co/incoai/GLM-5.3-Flash-DFlash2). The draft proposes a whole block per step and the target verifies it in one forward pass, so output quality stays the target's. Its block size comes from the draft checkpoint, and the draft runs on `fa4` rather than the target's DSA backends. The hidden-state capture it needs ([PR #36708](https://github.com/sgl-project/sglang/pull/36708)) shipped with the GLM-5.3-Flash support in v0.5.20, so the image pinned above is enough. The draft repository is access-gated: request access on its model page, then download it alongside the target before serving. This combination is not yet measured on the cookbook hardware, so treat it as a starting point.
Neither algorithm runs with DP-Attention; the card disables the affected chips and names the reason.
@@ -134,13 +134,13 @@ The default multimodal feature transport is automatic, and on a single CUDA node
### 3.1 Reasoning
Thinking is enabled by the checkpoint's generation configuration, and generated commands enable `--reasoning-parser glm45` by default. The OpenAI-compatible API then places thinking in `message.reasoning_content` and the final answer in `message.content`. You can disable **Reasoning Parser** in the Playground when an integration needs the raw response format.
Thinking is enabled by the checkpoint's generation configuration, and generated commands enable `--reasoning-parser auto` (which resolves to `glm45` for GLM-5.3-Flash) by default. The OpenAI-compatible API then places thinking in `message.reasoning_content` and the final answer in `message.content`. You can disable **Reasoning Parser** in the Playground when an integration needs the raw response format.
To disable thinking for a request, pass `chat_template_kwargs: {"thinking": false}` in the request body.
### 3.2 Tool calling
Generated commands enable `--tool-call-parser glm47` by default, so structured calls are returned in `message.tool_calls`. You can disable **Tool Call Parser** in the Playground when tool calling is not needed. On follow-up turns, read both `reasoning_content` and `content` because a thinking model can use either field around tool execution.
Generated commands enable `--tool-call-parser auto` (which resolves to `glm47` for GLM-5.3-Flash) by default, so structured calls are returned in `message.tool_calls`. You can disable **Tool Call Parser** in the Playground when tool calling is not needed. On follow-up turns, read both `reasoning_content` and `content` because a thinking model can use either field around tool execution.
### 3.3 Multimodal serving
docs/cookbook/autoregressive/GLM/GLM-5.3.mdx
+4 -4
View File
@@ -103,7 +103,7 @@ import { Playground } from "/src/snippets/_playground.jsx";
- **DeepSeek Sparse Attention (DSA).** GLM-5.3 uses the `glm_moe_dsa` architecture; SGLang auto-selects the DSA attention backends (`flashmla_sparse` prefill, `fa3` decode, `sgl-kernel` indexer topk). No attention-backend flag is needed on the supported hardware. SGLang also auto-selects the KV-cache dtype for DSA models — `fp8_e4m3` on Blackwell (B200/GB300/B300, which then routes DSA through the TensorRT-LLM backend) and `bf16` on Hopper (H200) — so no `--kv-cache-dtype` flag is required. On Hopper, pairing `--kv-cache-dtype fp8_e4m3` with `--dsa-prefill-backend flashmla_sparse_q8 --dsa-decode-backend flashmla_kv` selects the native FP8 sparse prefill kernel (computes directly on the fp8 KV cache with no fp8→bf16 dequantization round-trip; GLM-5.3's 64 query heads match the kernel's native tile) — see the [DeepSeek-V3.2 page](../DeepSeek/DeepSeek-V3_2) for kernel details; the optional `SGLANG_ENABLE_DSA_Q8KV8_*` performance env vars are documented in `python/sglang/srt/environ.py`.
- **MTP / speculative decoding.** The checkpoint ships one nextn layer. Enable EAGLE MTP for lower latency (`--speculative-algorithm EAGLE --speculative-num-steps 5 --speculative-eagle-topk 1 --speculative-num-draft-tokens 6` for low-latency; `1-1-2` for balanced). The config's `index_share_for_mtp_iteration` reuses the DSA indexer's topk across draft steps (effective only at `--speculative-eagle-topk 1`). Watch the server's reported **accept length** and adjust `--speculative-num-steps` / `--speculative-num-draft-tokens`: lower the draft length when rejected draft tokens create excess verification work.
- **DFlash2 (block-diffusion draft).** The **Speculative** card in the [Playground above](#playground) also offers **DFlash2**, which replaces the in-checkpoint MTP layer with the separately trained block-diffusion drafter [`incoai/GLM-5.3-DFlash2`](https://huggingface.co/incoai/GLM-5.3-DFlash2). It proposes a whole block per step and the target verifies the block in one forward pass, so output quality stays the target's. The block size — 8, i.e. 7 draft tokens per verification step — comes from the draft checkpoint's own `dflash_config`, so no `--speculative-num-draft-tokens` is passed; the draft is a small dense model and runs on `fa4` instead of the target's DSA backends. Two prerequisites: the DFlash2 drafter ([PR #35371](https://github.com/sgl-project/sglang/pull/35371)) merged **after v0.5.18**, so install SGLang from `main` (or use a nightly image) rather than the release this page pins; and DFLASH runs on **CUDA/NPU only** and rejects **DP-Attention**, so turn DP-Attention off in the **Attention** card before selecting it on a high-throughput base. The draft repository is public but licensed CC BY-NC-ND 4.0 for research and evaluation.
- **DFlash2 (block-diffusion draft).** The **Speculative** card in the [Playground above](#playground) also offers **DFlash2**, which replaces the in-checkpoint MTP layer with the separately trained block-diffusion drafter [`incoai/GLM-5.3-DFlash2`](https://huggingface.co/incoai/GLM-5.3-DFlash2). It proposes a whole block per step and the target verifies the block in one forward pass, so output quality stays the target's. The block size — 8, i.e. 7 draft tokens per verification step — comes from the draft checkpoint's own `dflash_config`, so no `--speculative-num-draft-tokens` is passed; the draft is a small dense model and runs on `fa4` instead of the target's DSA backends. Note that DFLASH runs on **CUDA/NPU only** and rejects **DP-Attention**, so turn DP-Attention off in the **Attention** card before selecting it on a high-throughput base. The draft repository is public but licensed CC BY-NC-ND 4.0 for research and evaluation.
- **Memory.** The FP8 weights are large (MoE total, not active params). Start around `--mem-fraction-static 0.8` on H200 (TP8) and tune up; raise it for the 4-GPU GB300 single-node layout (TP4).
- **DP-Attention + DeepEP** for the balanced/high-throughput strategies spreads attention across data-parallel ranks and routes MoE through DeepEP.
- **BF16 weights need more GPUs.** The full-precision build (`zai-org/GLM-5.3-BF16`, ~1.5 TB) does not fit a single 8×H200 / 8×B200 / 4×GB300 node. It fits single-node on **8×B300** (TP8, ~2.1 TB HBM); on the smaller GPUs it needs a **multi-node** layout (e.g. 2×8×H200 or 2×8×B200 at TP16, 2×4×GB300 at TP8). FP8 is the recommended deployment. Use the same DSA / MTP / chunked-prefill guidance as FP8.
@@ -117,7 +117,7 @@ import { Playground } from "/src/snippets/_playground.jsx";
### 3.1 Reasoning
GLM-5.3 is a reasoning model. Enable the `glm45` reasoning parser (toggle **Reasoning Parser** in the **Parsers** card of the [Playground above](#playground)) to separate thinking from the final answer — thinking lands in `message.reasoning_content`, the answer in `message.content`. The chat template defaults `clear_thinking` to `false`; for multi-turn chat, pass `chat_template_kwargs: {"clear_thinking": True}` so previous reasoning is cleared before the next response.
GLM-5.3 is a reasoning model, and generated commands enable `--reasoning-parser auto` (which resolves to `glm45` for GLM-5.3) by default so thinking is separated from the final answer — thinking lands in `message.reasoning_content`, the answer in `message.content`. Without the parser the server returns the thinking and the answer as one `content` string with a stray `</think>` between them, because the chat template opens `<think>` in the generation prompt. You can disable **Reasoning Parser** in the **Parsers** card of the [Playground above](#playground) when an integration needs that raw format. The chat template defaults `clear_thinking` to `false`; for multi-turn chat, pass `chat_template_kwargs: {"clear_thinking": True}` so previous reasoning is cleared before the next response.
**Reasoning effort.** Pass `chat_template_kwargs: {"reasoning_effort": ...}` to select `low`, `high`, or `max`. If you omit it or pass another value, the template uses `max`.
@@ -164,7 +164,7 @@ Here is how you can calculate it:
### 3.2 Tool Calling
Enable the `glm47` tool-call parser (toggle **Tool Call Parser** in the **Parsers** card of the [Playground above](#playground)) to surface structured tool calls via `message.tool_calls`. GLM-5.3 emits the newer `<tool_call>…<arg_key>…<arg_value>…` format, so it needs the **`glm47`** parser — the older `glm45` parser does not parse it (the call would be left as raw text in `content`). On thinking mode the turn also fills `reasoning_content`, so print both fields.
Generated commands enable `--tool-call-parser auto` by default, so structured calls are returned in `message.tool_calls` with `finish_reason: "tool_calls"`. `auto` resolves to **`glm47`** for GLM-5.3: the model emits the newer `<tool_call>…<arg_key>…<arg_value>…` format, which the older `glm45` parser does not parse (the call would be left as raw text in `content`). Running with no tool-call parser fails the same way, and `finish_reason` stays `"stop"`, so an agent loop never sees the call. You can disable **Tool Call Parser** in the **Parsers** card of the [Playground above](#playground) when tool calling is not needed. On thinking mode the turn also fills `reasoning_content`, so print both fields.
<Accordion title="Tool Calling Example (Python)">
@@ -218,7 +218,7 @@ For long-context, prefix-heavy workloads, enable hierarchical KV caching to spil
### 3.4 Claude Code Integration
GLM-5.3's strong reasoning + tool-calling makes it a good backend for [Claude Code](https://code.claude.com/docs/en/overview), Anthropic's agentic CLI. SGLang exposes the Anthropic-compatible `/v1/messages` endpoint on every server, so Claude Code can talk to a GLM-5.3 server with only environment variables — no code change. Launch the server with `--reasoning-parser glm45 --tool-call-parser glm47` (any recipe from the Deployment panel above works), then:
GLM-5.3's strong reasoning + tool-calling makes it a good backend for [Claude Code](https://code.claude.com/docs/en/overview), Anthropic's agentic CLI. SGLang exposes the Anthropic-compatible `/v1/messages` endpoint on every server, so Claude Code can talk to a GLM-5.3 server with only environment variables — no code change. Launch the server with `--reasoning-parser auto --tool-call-parser auto` (any recipe from the Deployment panel above works), then:
```bash Command
export ANTHROPIC_BASE_URL="http://127.0.0.1:30000"
docs/src/snippets/configs/zai-org/glm-5.3-flash.jsx
+38 -56
View File
@@ -119,6 +119,32 @@ export const config = {
},
],
},
// Parser flags live in one overlay dim so every generated command gets
// them without per-cell duplication; the Parsers card toggles derive
// on/off from the composed flags. `auto` needs the GLM-5.3 template
// detection (v0.5.20+).
{
id: "parsers",
title: "Parsers",
default: "auto",
options: [
{
id: "auto",
label: "Auto (glm45 + glm47)",
stripPrefixes: ["--reasoning-parser", "--tool-call-parser"],
flags: [
"--reasoning-parser auto",
"--tool-call-parser auto",
],
},
{
id: "off",
label: "Off",
stripPrefixes: ["--reasoning-parser", "--tool-call-parser"],
flags: [],
},
],
},
],
modelNames: {
@@ -174,15 +200,16 @@ sgl-eval run gsm8k \\
["aime2026_pct", "AIME 2026", "%"],
],
// Support is not in a public sglang release yet, so the nightly images do
// not work; every NVIDIA lane uses the purpose-built CUDA 13 image.
// v0.5.20 (= latest) carries GLM-5.3-Flash support (#36507) and the GLM-5.3
// template parser detection (#38297) that `--*-parser auto` needs; the old
// glm-5.3-flash dev image (2026-09-03) predates #38297 and misdetects.
dockerImages: {
gb300: "lmsysorg/sglang:glm-5.3-flash",
h100: "lmsysorg/sglang:glm-5.3-flash",
h200: "lmsysorg/sglang:glm-5.3-flash",
b200: "lmsysorg/sglang:glm-5.3-flash",
b300: "lmsysorg/sglang:glm-5.3-flash",
gb200: "lmsysorg/sglang:glm-5.3-flash",
gb300: "lmsysorg/sglang:latest",
h100: "lmsysorg/sglang:latest",
h200: "lmsysorg/sglang:latest",
b200: "lmsysorg/sglang:latest",
b300: "lmsysorg/sglang:latest",
gb200: "lmsysorg/sglang:latest",
},
github: {
@@ -259,8 +286,8 @@ sgl-eval run gsm8k \\
parsers: {
items: [
{ id: "reasoning", label: "Reasoning Parser", flag: "--reasoning-parser glm45" },
{ id: "toolCall", label: "Tool Call Parser", flag: "--tool-call-parser glm47" },
{ id: "reasoning", label: "Reasoning Parser", flag: "--reasoning-parser auto" },
{ id: "toolCall", label: "Tool Call Parser", flag: "--tool-call-parser auto" },
],
},
@@ -305,12 +332,7 @@ sgl-eval run gsm8k \\
"--speculative-draft-model-path incoai/GLM-5.3-Flash-DFlash2",
"--speculative-draft-attention-backend fa4",
],
// DFLASH needs this model's hidden-state capture, which landed on the
// GLM-5.3-Flash support branch (PR #36708 into #36507's
// xinyuan/glm-5.3-flash-support), not on main — so it postdates the
// image the Install accordion pins. Drop this note once #36507 merges
// and a published image carries it.
note: "⚠️ Needs the GLM-5.3-Flash hidden-state capture from PR #36708. It is merged into the PR #36507 support branch (xinyuan/glm-5.3-flash-support), not into main, so pull that branch at its current head — or add #36708's commit on top of an older checkout — before serving. The lmsysorg/sglang:glm-5.3-flash image alone is not enough.",
note: "⚠️ The draft checkpoint incoai/GLM-5.3-Flash-DFlash2 is access-gated: request access on its Hugging Face page, then download it alongside the target before serving.",
disable: [
{
when: { dpAttnOn: [true] },
@@ -346,8 +368,6 @@ sgl-eval run gsm8k \\
"--speculative-num-steps 5",
"--speculative-eagle-topk 1",
"--speculative-num-draft-tokens 6",
"--reasoning-parser glm45",
"--tool-call-parser glm47",
"--host {{HOST_IP}}",
"--port {{PORT}}",
],
@@ -371,8 +391,6 @@ sgl-eval run gsm8k \\
"--dsa-decode-backend trtllm",
"--kv-cache-dtype fp8_e4m3",
"--moe-runner-backend flashinfer_trtllm",
"--reasoning-parser glm45",
"--tool-call-parser glm47",
"--host {{HOST_IP}}",
"--port {{PORT}}",
],
@@ -406,8 +424,6 @@ sgl-eval run gsm8k \\
"--speculative-num-steps 5",
"--speculative-eagle-topk 1",
"--speculative-num-draft-tokens 6",
"--reasoning-parser glm45",
"--tool-call-parser glm47",
"--mem-fraction-static 0.85",
"--cuda-graph-max-bs-decode 32",
"--host {{HOST_IP}}",
@@ -434,8 +450,6 @@ sgl-eval run gsm8k \\
"--dsa-decode-backend tilelang",
"--kv-cache-dtype bfloat16",
"--moe-runner-backend flashinfer_cutlass",
"--reasoning-parser glm45",
"--tool-call-parser glm47",
"--mem-fraction-static 0.85",
"--host {{HOST_IP}}",
"--port {{PORT}}",
@@ -461,8 +475,6 @@ sgl-eval run gsm8k \\
"--speculative-num-steps 5",
"--speculative-eagle-topk 1",
"--speculative-num-draft-tokens 6",
"--reasoning-parser glm45",
"--tool-call-parser glm47",
"--mem-fraction-static 0.85",
"--cuda-graph-max-bs-decode 32",
"--host {{HOST_IP}}",
@@ -482,8 +494,6 @@ sgl-eval run gsm8k \\
"--dsa-decode-backend tilelang",
"--kv-cache-dtype bfloat16",
"--moe-runner-backend flashinfer_cutlass",
"--reasoning-parser glm45",
"--tool-call-parser glm47",
"--mem-fraction-static 0.85",
"--host {{HOST_IP}}",
"--port {{PORT}}",
@@ -506,8 +516,6 @@ sgl-eval run gsm8k \\
"--speculative-num-steps 5",
"--speculative-eagle-topk 1",
"--speculative-num-draft-tokens 6",
"--reasoning-parser glm45",
"--tool-call-parser glm47",
"--mem-fraction-static 0.85",
"--cuda-graph-max-bs-decode 32",
"--host {{HOST_IP}}",
@@ -527,8 +535,6 @@ sgl-eval run gsm8k \\
"--dsa-decode-backend tilelang",
"--kv-cache-dtype bfloat16",
"--moe-runner-backend flashinfer_cutlass",
"--reasoning-parser glm45",
"--tool-call-parser glm47",
"--mem-fraction-static 0.85",
"--host {{HOST_IP}}",
"--port {{PORT}}",
@@ -551,8 +557,6 @@ sgl-eval run gsm8k \\
"--speculative-num-steps 5",
"--speculative-eagle-topk 1",
"--speculative-num-draft-tokens 6",
"--reasoning-parser glm45",
"--tool-call-parser glm47",
"--mem-fraction-static 0.85",
"--cuda-graph-max-bs-decode 32",
"--host {{HOST_IP}}",
@@ -572,8 +576,6 @@ sgl-eval run gsm8k \\
"--dsa-decode-backend tilelang",
"--kv-cache-dtype bfloat16",
"--moe-runner-backend flashinfer_cutlass",
"--reasoning-parser glm45",
"--tool-call-parser glm47",
"--mem-fraction-static 0.85",
"--host {{HOST_IP}}",
"--port {{PORT}}",
@@ -603,8 +605,6 @@ sgl-eval run gsm8k \\
"--speculative-num-steps 5",
"--speculative-eagle-topk 1",
"--speculative-num-draft-tokens 6",
"--reasoning-parser glm45",
"--tool-call-parser glm47",
"--host {{HOST_IP}}",
"--port {{PORT}}",
],
@@ -626,8 +626,6 @@ sgl-eval run gsm8k \\
"--dsa-decode-backend tilelang",
"--kv-cache-dtype bfloat16",
"--moe-runner-backend deep_gemm",
"--reasoning-parser glm45",
"--tool-call-parser glm47",
"--host {{HOST_IP}}",
"--port {{PORT}}",
],
@@ -655,8 +653,6 @@ sgl-eval run gsm8k \\
"--speculative-num-steps 5",
"--speculative-eagle-topk 1",
"--speculative-num-draft-tokens 6",
"--reasoning-parser glm45",
"--tool-call-parser glm47",
"--host {{HOST_IP}}",
"--port {{PORT}}",
],
@@ -677,8 +673,6 @@ sgl-eval run gsm8k \\
"--dsa-decode-backend tilelang",
"--kv-cache-dtype bfloat16",
"--moe-runner-backend deep_gemm",
"--reasoning-parser glm45",
"--tool-call-parser glm47",
"--host {{HOST_IP}}",
"--port {{PORT}}",
],
@@ -701,8 +695,6 @@ sgl-eval run gsm8k \\
"--speculative-num-steps 5",
"--speculative-eagle-topk 1",
"--speculative-num-draft-tokens 6",
"--reasoning-parser glm45",
"--tool-call-parser glm47",
"--host {{HOST_IP}}",
"--port {{PORT}}",
],
@@ -722,8 +714,6 @@ sgl-eval run gsm8k \\
"--dsa-decode-backend trtllm",
"--kv-cache-dtype fp8_e4m3",
"--moe-runner-backend flashinfer_trtllm",
"--reasoning-parser glm45",
"--tool-call-parser glm47",
"--host {{HOST_IP}}",
"--port {{PORT}}",
],
@@ -746,8 +736,6 @@ sgl-eval run gsm8k \\
"--speculative-num-steps 5",
"--speculative-eagle-topk 1",
"--speculative-num-draft-tokens 6",
"--reasoning-parser glm45",
"--tool-call-parser glm47",
"--host {{HOST_IP}}",
"--port {{PORT}}",
],
@@ -767,8 +755,6 @@ sgl-eval run gsm8k \\
"--dsa-decode-backend trtllm",
"--kv-cache-dtype fp8_e4m3",
"--moe-runner-backend flashinfer_trtllm",
"--reasoning-parser glm45",
"--tool-call-parser glm47",
"--host {{HOST_IP}}",
"--port {{PORT}}",
],
@@ -789,8 +775,6 @@ sgl-eval run gsm8k \\
"--speculative-num-steps 5",
"--speculative-eagle-topk 1",
"--speculative-num-draft-tokens 6",
"--reasoning-parser glm45",
"--tool-call-parser glm47",
"--host {{HOST_IP}}",
"--port {{PORT}}",
],
@@ -807,8 +791,6 @@ sgl-eval run gsm8k \\
"--dsa-decode-backend trtllm",
"--kv-cache-dtype fp8_e4m3",
"--moe-runner-backend flashinfer_trtllm",
"--reasoning-parser glm45",
"--tool-call-parser glm47",
"--host {{HOST_IP}}",
"--port {{PORT}}",
],
docs/src/snippets/configs/zai-org/glm-5.3.jsx
+36 -9
View File
@@ -96,15 +96,45 @@ sgl-eval run aime25 \\
b200: "lmsysorg/sglang:latest",
gb300: "lmsysorg/sglang:latest",
b300: "lmsysorg/sglang:latest",
mi355x: "lmsysorg/sglang-rocm:v0.5.13.post1-rocm720-mi35x-20260618",
mi325x: "lmsysorg/sglang-rocm:v0.5.13.post1-rocm700-mi30x-20260616",
mi300x: "lmsysorg/sglang-rocm:v0.5.13.post1-rocm700-mi30x-20260616",
// >= v0.5.20 so `--*-parser auto` detects GLM-5.3 (#38297); the rocm700
// line stopped at v0.5.19, so mi30x moves to the rocm720 build.
mi355x: "lmsysorg/sglang-rocm:v0.5.20-rocm720-mi35x-20260920",
mi325x: "lmsysorg/sglang-rocm:v0.5.20-rocm720-mi30x-20260920",
mi300x: "lmsysorg/sglang-rocm:v0.5.20-rocm720-mi30x-20260920",
},
github: {
cookbookModel: "zai-org/glm-5.3",
},
// Parser flags live in one overlay dim so every generated command gets them
// without per-cell duplication; the Parsers card toggles derive on/off from
// the composed flags. `auto` needs the GLM-5.3 template detection (v0.5.20+).
overlayDims: [
{
id: "parsers",
title: "Parsers",
default: "auto",
options: [
{
id: "auto",
label: "Auto (glm45 + glm47)",
stripPrefixes: ["--reasoning-parser", "--tool-call-parser"],
flags: [
"--reasoning-parser auto",
"--tool-call-parser auto",
],
},
{
id: "off",
label: "Off",
stripPrefixes: ["--reasoning-parser", "--tool-call-parser"],
flags: [],
},
],
},
],
playgroundFeatures: {
// ----- Card 1: "Attention Parallelism" -----
@@ -162,8 +192,8 @@ sgl-eval run aime25 \\
// ----- Card 3: "Parsers" -----
parsers: {
items: [
{ id: "reasoning", label: "Reasoning Parser", flag: "--reasoning-parser glm45" },
{ id: "toolCall", label: "Tool Call Parser", flag: "--tool-call-parser glm47" },
{ id: "reasoning", label: "Reasoning Parser", flag: "--reasoning-parser auto" },
{ id: "toolCall", label: "Tool Call Parser", flag: "--tool-call-parser auto" },
],
},
@@ -194,10 +224,7 @@ sgl-eval run aime25 \\
flags: ["--speculative-algorithm DFLASH",
"--speculative-draft-model-path incoai/GLM-5.3-DFlash2",
"--speculative-draft-attention-backend fa4"],
// The DFlash2 drafter (PR #35371) merged after v0.5.18, so neither the
// release wheel nor the lmsysorg/sglang:latest image this page pins
// carries it. Drop this note once a release ships it.
note: "⚠️ Needs a nightly image: the DFlash2 drafter (PR #35371) is not in the release wheel nor the lmsysorg/sglang:latest image this page pins — install SGLang from main or use a lmsysorg/sglang:dev image. The draft is a separate checkpoint, so fetch incoai/GLM-5.3-DFlash2 alongside the target; it is public but licensed CC BY-NC-ND 4.0 for research and evaluation.",
note: "⚠️ The draft is a separate checkpoint: fetch incoai/GLM-5.3-DFlash2 alongside the target. It is public but licensed CC BY-NC-ND 4.0 for research and evaluation.",
disable: [
{ when: { dpAttnOn: [true] },
reason: "DFLASH speculative decoding does not support DP-Attention — the server rejects the combination at startup. Turn DP-Attention off in the Attention card above (the high-throughput recipes enable it)." },