[diffusion] Reject unsafe quality=high BCG replay (#36008)
This commit is contained in:
@@ -97,7 +97,7 @@ Use `sglang generate --help` and `sglang serve --help` for the full argument lis
|
||||
- `--warmup-mode {off|request|server}`: control startup warmup for `sglang serve`; `off` skips warmup, `request` primes the request path, and `server` runs a full synthetic server warmup before serving traffic
|
||||
- `--enable-torch-compile {true|false}`: compile native diffusion hot paths. When no warmup mode is configured, this also enables server warmup so first real requests do not pay compile latency.
|
||||
- `--offload-during-compile {true|false}`: when compile warmup is active, temporarily layerwise-offload DiT weights and move resident non-DiT components off-device so `max-autotune` fits on tighter-memory GPUs; the configured serving residency is restored before real traffic. Skipped under existing layerwise offload, Cache-DiT, or FSDP.
|
||||
- `--enable-breakable-cuda-graph {true|false}`: capture supported DiT forwards as breakable CUDA graph segments to reduce launch overhead. Requires `--warmup-resolutions` for every served resolution because each resolution is captured separately.
|
||||
- `--enable-breakable-cuda-graph {true|false}`: capture supported DiT forwards as breakable CUDA graph segments to reduce launch overhead. Requires `--warmup-resolutions` for every served resolution because each resolution is captured separately. A `quality=high` request is rejected when it would mount request-scoped DiT fusions that were not present during lossless graph capture; VAE-only high-quality paths remain compatible.
|
||||
- `--bcg-text-buckets {N...}`: prompt-length padding buckets for breakable CUDA graph capture/replay reuse.
|
||||
- `--attention-backend {BACKEND}`: attention backend for native SGLang and diffusers pipelines
|
||||
- `--component-attention-backends {MAP}`: per-component attention backend overrides, for example `text_encoder=torch_sdpa,transformer=fa`
|
||||
|
||||
@@ -48,6 +48,15 @@ The `quality` field in a **video response** body is unrelated. It is Sora-compat
|
||||
|
||||
`quality` participates in the dynamic-batch signature, so mixed-quality traffic is batched separately and the transition happens safely at a batch boundary. Mounting is all-or-nothing: if any marked site on a transformer fails its static guards, no site on that transformer is fused.
|
||||
|
||||
<Warning>
|
||||
Do not combine request-gated DiT fusions with `--enable-breakable-cuda-graph`.
|
||||
BCG warmup captures the lossless module branches before a high-quality request
|
||||
mounts its DiT fusions, so replay would bypass the requested kernels. SGLang
|
||||
rejects this combination for models with eligible DiT quality sites. Models
|
||||
whose high-quality path changes only VAE decode remain allowed because BCG
|
||||
captures the DiT only.
|
||||
</Warning>
|
||||
|
||||
These fusion families mount under `quality="high"`:
|
||||
|
||||
| Fusion | What it folds |
|
||||
|
||||
Reference in New Issue
Block a user