minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

[XPU] Remove redundant xpu graph backend and make xpu graph opt-in by default (#29911)

Browse Source
This commit is contained in:
Cao E
2026-07-03 15:58:56 +08:00
committed by GitHub
parent 67697fb891
commit 9df16b5ba9
17 changed files with 68 additions and 140 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs_new/docs/hardware-platforms/xpu.mdx
+13 -4
View File
@@ -147,9 +147,18 @@ SGLang enables XPU graph capture to reduce per-step kernel-launch overhead.
| Phase | Backend | Mechanism | Default |
|---|---|---|---|
| Decode | `full` | One `torch.xpu.XPUGraph` per batch size, captured on startup | **On** |
| Decode | `full` | One `torch.xpu.XPUGraph` per batch size, captured on startup | **Off** (opt-in) |
| Prefill | `tc_piecewise` | `torch.compile` + XPU graph, one graph segment per token-length bucket | **Off** (opt-in) |
### Enable Decode Graph
Decode graph capture is **opt-in** on XPU. Enable it explicitly:
```bash
python -m sglang.launch_server --model-path <MODEL> --device xpu \
--cuda-graph-backend-decode full
```
### Enable Prefill Graph
Prefill graph capture is **opt-in** on XPU and requires `torch.compile`
@@ -195,10 +204,10 @@ python -m sglang.launch_server --model-path <MODEL> --device xpu \
### Disable XPU Graph
To opt out of one or both phases:
Both phases are disabled by default. To explicitly disable them anyway:
```bash
# Disable decode graph
# Disable decode graph (already off by default; explicit form)
python -m sglang.launch_server --model-path <MODEL> --device xpu \
--cuda-graph-backend-decode=disabled
@@ -236,7 +245,7 @@ python -m sglang.launch_server \
| Argument | XPU allowed values | Default | Description |
|---|---|---|---|
| `--cuda-graph-backend-decode` | `full`, `disabled` | `full` | Backend for the decode phase. Only `full` is supported on XPU. |
| `--cuda-graph-backend-decode` | `full`, `disabled` | `disabled` | Backend for the decode phase. Only `full` is supported on XPU. Set to `full` to enable. |
| `--cuda-graph-backend-prefill` | `tc_piecewise`, `disabled` | `disabled`* | Backend for the prefill phase. Must be set to `tc_piecewise` explicitly to enable. |
| `--cuda-graph-tc-compiler` | `eager`, `inductor` | `eager` | Compiler for `tc_piecewise` prefill subgraphs. `inductor` produces more optimized code but has longer startup. |
| `--cuda-graph-bs-prefill` | list of ints | auto | Explicit token-length buckets to capture for prefill. |