[diffusion] Reject unsafe quality=high BCG replay (#36008)

This commit is contained in:
Xiaoyu Zhang
2026-08-24 08:50:26 +08:00
committed by GitHub
parent f4448e677f
commit 447048dba2
4 changed files with 61 additions and 1 deletions
+1 -1
View File
@@ -97,7 +97,7 @@ Use `sglang generate --help` and `sglang serve --help` for the full argument lis
- `--warmup-mode {off|request|server}`: control startup warmup for `sglang serve`; `off` skips warmup, `request` primes the request path, and `server` runs a full synthetic server warmup before serving traffic
- `--enable-torch-compile {true|false}`: compile native diffusion hot paths. When no warmup mode is configured, this also enables server warmup so first real requests do not pay compile latency.
- `--offload-during-compile {true|false}`: when compile warmup is active, temporarily layerwise-offload DiT weights and move resident non-DiT components off-device so `max-autotune` fits on tighter-memory GPUs; the configured serving residency is restored before real traffic. Skipped under existing layerwise offload, Cache-DiT, or FSDP.
- `--enable-breakable-cuda-graph {true|false}`: capture supported DiT forwards as breakable CUDA graph segments to reduce launch overhead. Requires `--warmup-resolutions` for every served resolution because each resolution is captured separately.
- `--enable-breakable-cuda-graph {true|false}`: capture supported DiT forwards as breakable CUDA graph segments to reduce launch overhead. Requires `--warmup-resolutions` for every served resolution because each resolution is captured separately. A `quality=high` request is rejected when it would mount request-scoped DiT fusions that were not present during lossless graph capture; VAE-only high-quality paths remain compatible.
- `--bcg-text-buckets {N...}`: prompt-length padding buckets for breakable CUDA graph capture/replay reuse.
- `--attention-backend {BACKEND}`: attention backend for native SGLang and diffusers pipelines
- `--component-attention-backends {MAP}`: per-component attention backend overrides, for example `text_encoder=torch_sdpa,transformer=fa`
@@ -48,6 +48,15 @@ The `quality` field in a **video response** body is unrelated. It is Sora-compat
`quality` participates in the dynamic-batch signature, so mixed-quality traffic is batched separately and the transition happens safely at a batch boundary. Mounting is all-or-nothing: if any marked site on a transformer fails its static guards, no site on that transformer is fused.
<Warning>
Do not combine request-gated DiT fusions with `--enable-breakable-cuda-graph`.
BCG warmup captures the lossless module branches before a high-quality request
mounts its DiT fusions, so replay would bypass the requested kernels. SGLang
rejects this combination for models with eligible DiT quality sites. Models
whose high-quality path changes only VAE decode remain allowed because BCG
captures the DiT only.
</Warning>
These fusion families mount under `quality="high"`:
| Fusion | What it folds |