[diffusion] cli: support component attention backend overrides (#24320)

This commit is contained in:
Mick
2026-05-05 08:39:27 +08:00
committed by GitHub
parent 078f84d80d
commit 2f7d99b7f7
10 changed files with 444 additions and 43 deletions
@@ -42,8 +42,9 @@ For SGLang-native pipelines, the CLI accepts the lowercase names of `AttentionBa
The selection order in `runtime/layers/attention/selector.py` is:
1. `global_force_attn_backend(...)` / `global_force_attn_backend_context_manager(...)`
2. CLI `--attention-backend` (`ServerArgs.attention_backend`)
3. Auto selection (platform capability, dtype, and installed packages)
2. Component override from `--component-attention-backends` while that component is being constructed
3. CLI `--attention-backend` (`ServerArgs.attention_backend`)
4. Auto selection (platform capability, dtype, and installed packages)
## Configuration
@@ -122,6 +123,20 @@ sglang generate \
--attention-backend torch_sdpa
```
### Override one component
Use component overrides when a specific module needs different attention semantics from the main transformer:
```bash
sglang generate \
--model-path <MODEL_PATH_OR_ID> \
--prompt "..." \
--attention-backend fa \
--component-attention-backends text_encoder=torch_sdpa
```
Component keys match pipeline module names from `model_index.json`, such as `text_encoder`, `text_encoder_2`, `transformer`, `transformer_2`, or `connectors`.
### Using Sliding Tile Attention (STA)
```bash