[diffusion] Enable LongCat breakable CUDA graphs (#35724)

This commit is contained in:
Xiaoyu Zhang
2026-08-21 17:59:16 +08:00
committed by GitHub
parent e7a37c8550
commit a5c52a9358
7 changed files with 90 additions and 4 deletions
+12
View File
@@ -207,6 +207,18 @@ For supported native pipelines, set `SGLANG_CACHE_DIT_ENABLED=true` to enable Ca
For supported image pipelines, breakable CUDA graph can be enabled with `--enable-breakable-cuda-graph`, but you must declare every served resolution in `--warmup-resolutions` so warmup captures matching graph signatures.
LongCat-Image is supported by this path. Its DiT always consumes a fixed
512-token prompt body, so different prompt lengths reuse the same graph without
model-specific padding. For a 1024x1024 deployment, use:
```bash Command
sglang serve --model-path meituan-longcat/LongCat-Image \
--enable-breakable-cuda-graph \
--warmup-resolutions 1024x1024 \
--enable-torch-compile false \
--port 30010
```
### Component Residency
Use `--component-residency COMPONENT=MODE` to assign one runtime residency mode to each native pipeline component: