[diffusion] cli: support component attention backend overrides (#24320)

This commit is contained in:
Mick
2026-05-05 08:39:27 +08:00
committed by GitHub
parent 078f84d80d
commit 2f7d99b7f7
10 changed files with 444 additions and 43 deletions
+23
View File
@@ -78,6 +78,7 @@ Use `sglang generate --help` and `sglang serve --help` for the full argument lis
- `--sp-degree {N}`: sequence parallelism size
- `--ulysses-degree {N}` and `--ring-degree {N}`: USP parallelism controls
- `--attention-backend {BACKEND}`: attention backend for native SGLang pipelines
- `--component-attention-backends {MAP}`: per-component attention backend overrides, for example `text_encoder=torch_sdpa,transformer=fa`
- `--attention-backend-config {CONFIG}`: attention backend configuration
### Sampling and output
@@ -195,6 +196,28 @@ sglang serve \
The component key must match the key in the model's `model_index.json`, and the path must be either a Hugging Face repo ID or a complete component directory.
## Component Attention Backend Overrides
Use `--component-attention-backends` when one pipeline component needs a different native attention backend from the global `--attention-backend`.
```bash
sglang generate \
--model-path Lightricks/LTX-2.3 \
--attention-backend fa \
--component-attention-backends text_encoder=torch_sdpa
```
The component key must match a pipeline module key such as `text_encoder`, `text_encoder_2`, `transformer`, `transformer_2`, or `connectors`. Component overrides take precedence over the global `--attention-backend` only while that component is being constructed.
You can also pass dotted CLI entries:
```bash
sglang generate \
--model-path <MODEL_PATH_OR_ID> \
--component-attention-backends.text_encoder torch_sdpa \
--component-attention-backends.transformer fa
```
## Diffusers Backend
Use `--backend diffusers` to force vanilla diffusers pipelines when no native SGLang implementation exists or when a model requires a custom pipeline class.