[diffusion] feat: unify encoder folding and batch data-parallel encoding (#30211)

2026-07-30 20:15:22 +08:00
parent 1f04eaab6a
commit db3da62333
14 changed files with 505 additions and 60 deletions
@@ -83,6 +83,7 @@ Use `sglang generate --help` and `sglang serve --help` for the full argument lis
 - `--sp-degree {N}`: sequence parallelism size
 - `--ulysses-degree {N}` and `--ring-degree {N}`: USP parallelism controls
 - `--enable-cfg-parallel {true|false}`: enable or explicitly disable CFG parallelism
+- `--encoder-parallel {auto|fold|dp|replicate}`: how the text/image encoders use the GPUs the DiT replica leaves idle during encoding. `auto` (the default for `generate`) TP-folds an encoder wide enough to pay for the per-layer all-reduce and replicates the rest; `fold` forces the shard whenever the dims allow it; `dp` splits a batched encode across ranks (needs `--batching-max-size > 1` to engage, and is the `serve` default); `replicate` encodes redundantly on every rank. `fold` and `replicate` are bitwise-identical to single-GPU encoding. See [Encoder Parallelism](/docs/sglang-diffusion/encoder_parallel).
 - `--warmup-mode {off|request|server}`: control startup warmup for `sglang serve`; `off` skips warmup, `request` primes the request path, and `server` runs a full synthetic server warmup before serving traffic
 - `--enable-torch-compile {true|false}`: compile native diffusion hot paths. When no warmup mode is configured, this also enables server warmup so first real requests do not pay compile latency.
 - `--offload-during-compile {true|false}`: when compile warmup is active, temporarily layerwise-offload DiT weights and move resident non-DiT components off-device so `max-autotune` fits on tighter-memory GPUs; the configured serving residency is restored before real traffic. Skipped under existing layerwise offload, Cache-DiT, or FSDP.