[diffusion] feat: unify encoder folding and batch data-parallel encoding (#30211)

docs_new/docs.json

@@ -401,6 +401,10 @@
       "source": "/diffusion/performance/ring_sp_performance.html",
       "destination": "/docs/sglang-diffusion/ring_sp_performance"
     },
+    {
+      "source": "/diffusion/performance/encoder_parallel.html",
+      "destination": "/docs/sglang-diffusion/encoder_parallel"
+    },
     {
       "source": "/diffusion/quantization.html",
       "destination": "/docs/sglang-diffusion/quantization"
@@ -1332,6 +1336,7 @@
                   "docs/sglang-diffusion/deployment_cookbook",
                   "docs/sglang-diffusion/attention_backends",
                   "docs/sglang-diffusion/ring_sp_performance",
+                  "docs/sglang-diffusion/encoder_parallel",
                   "docs/sglang-diffusion/dynamic_batching",
                   {
                     "group": "Caching Acceleration",

docs_new/docs/sglang-diffusion/api/cli.mdx

@@ -83,6 +83,7 @@ Use `sglang generate --help` and `sglang serve --help` for the full argument lis
 - `--sp-degree {N}`: sequence parallelism size
 - `--ulysses-degree {N}` and `--ring-degree {N}`: USP parallelism controls
 - `--enable-cfg-parallel {true|false}`: enable or explicitly disable CFG parallelism
+- `--encoder-parallel {auto|fold|dp|replicate}`: how the text/image encoders use the GPUs the DiT replica leaves idle during encoding. `auto` (the default for `generate`) TP-folds an encoder wide enough to pay for the per-layer all-reduce and replicates the rest; `fold` forces the shard whenever the dims allow it; `dp` splits a batched encode across ranks (needs `--batching-max-size > 1` to engage, and is the `serve` default); `replicate` encodes redundantly on every rank. `fold` and `replicate` are bitwise-identical to single-GPU encoding. See [Encoder Parallelism](/docs/sglang-diffusion/encoder_parallel).
 - `--warmup-mode {off|request|server}`: control startup warmup for `sglang serve`; `off` skips warmup, `request` primes the request path, and `server` runs a full synthetic server warmup before serving traffic
 - `--enable-torch-compile {true|false}`: compile native diffusion hot paths. When no warmup mode is configured, this also enables server warmup so first real requests do not pay compile latency.
 - `--offload-during-compile {true|false}`: when compile warmup is active, temporarily layerwise-offload DiT weights and move resident non-DiT components off-device so `max-autotune` fits on tighter-memory GPUs; the configured serving residency is restored before real traffic. Skipped under existing layerwise offload, Cache-DiT, or FSDP.

docs_new/docs/sglang-diffusion/encoder_parallel.mdx

@@ -0,0 +1,106 @@
+---
+title: "Encoder Parallelism"
+tag: "preserve"
+metatags:
+    description: "Configure how SGLang Diffusion spreads text and image encoding across GPUs: parallel folding, batch data-parallel encoding, or replication."
+---
+
+While the DiT denoises, the text and image encoders are idle — and while they
+encode, the whole DiT replica is idle. `--encoder-parallel` decides how to use
+those otherwise-unused GPUs for the encoding stage.
+
+```bash
+--encoder-parallel {auto,fold,dp,replicate}
+```
+
+| Mode | What it does | Use when |
+| --- | --- | --- |
+| `auto` | Picks `fold`, `dp`, or `replicate` per encoder from its width and the request's batch width | Default for `generate`; you want the decision made per encoder |
+| `fold` | TP-shards the encoder weights across the idle DiT replica | One wide encoder dominates a single-request encode |
+| `dp` | Each rank encodes its slice of the prompt batch, then the outputs are all-gathered | Default for `serve`; needs `--batching-max-size > 1` to engage |
+| `replicate` | Every rank encodes the whole batch redundantly | You want the encoding stage to match single-GPU numerics exactly |
+
+The two accelerated modes are mutually exclusive per encoder: folding shards the
+weights for the lifetime of the loaded model, so a folded encoder cannot also be
+data-parallel.
+
+## Which Mode Wins
+
+Measured on H100 across T5 (hidden 4096), Qwen3 (2560), and CLIP-L (768) at
+batch 1–8 and replica sizes 2 and 4:
+
+- **Folding** pays when the encoder is wide enough that sharding its GEMMs beats
+  the per-layer all-reduce it adds. T5 gains; Qwen3 (+35%) and CLIP-L (+50%) get
+  slower, so folding is gated at hidden ≥ 4096. Its benefit also saturates as
+  the replica grows, since each rank's slice keeps shrinking.
+- **Data-parallel** pays only when the encode is compute-bound, which needs a
+  wide encoder (hidden ≥ 1024 — CLIP-L is slower at every batch and replica
+  measured) and more than one prompt in a single encode call.
+- **Replication** is the right answer whenever neither condition holds, which is
+  most single-request latency work.
+
+`auto` encodes exactly these rules, so prefer it unless you are pinning a
+configuration you measured yourself.
+
+## Numerics
+
+`fold` and `replicate` are bitwise-identical to single-GPU encoding: folding
+shards a GEMM and reduces it, which is the same arithmetic the unsharded kernel
+performs.
+
+`dp` is **not** bitwise-identical. Each rank runs the full unsharded encoder on
+a smaller batch, so the GEMM tiling and reduction order differ from the batched
+reference — the same floating-point reordering class as choosing a different
+attention backend or parallelism strategy, not a precision loss. The gathered
+result is mathematically equivalent, and per-request results stay deterministic
+for a fixed batch shape, but embeddings will not match a `replicate` run
+bit-for-bit, and long video sampling can amplify the difference into visible
+frame differences. Use `replicate` (or `fold`) when you need bit-exact
+reproducibility against a single-GPU reference, e.g. when refreshing consistency
+baselines.
+
+## Recommended Commands
+
+Throughput serving. `serve` already defaults to `dp`, but a single encode call
+must carry more than one prompt for it to engage, so raise the batching ceiling
+too — an encoder flag deliberately does not change DiT batching for you:
+
+```bash
+sglang serve \
+  --model-path Qwen/Qwen-Image-2512 \
+  --model-type diffusion \
+  --num-gpus 2 \
+  --encoder-parallel dp \
+  --batching-max-size 2
+```
+
+Single-request latency with one wide text encoder:
+
+```bash
+sglang serve \
+  --model-path Wan-AI/Wan2.2-TI2V-5B-Diffusers \
+  --model-type diffusion \
+  --num-gpus 4 \
+  --ulysses-degree 4 \
+  --encoder-parallel fold
+```
+
+Bit-exact reproducibility against a single-GPU reference:
+
+```bash
+sglang serve \
+  --model-path Qwen/Qwen-Image-2512 \
+  --model-type diffusion \
+  --num-gpus 2 \
+  --encoder-parallel replicate
+```
+
+## Interaction With Other Flags
+
+- **Tensor / data parallel**: `dp` requires a replicated encoder, so it is
+  skipped when `--tp-size > 1` or `--dp-size > 1`.
+- **Dynamic batching**: `dp` only pays with a wide batch, so selecting it raises
+  the default batching ceiling. See [Inference Batching](./dynamic_batching).
+- **Sequence parallelism**: independent — SP splits the DiT's latent sequence,
+  encoder parallelism splits the encoding stage. See
+  [Sequence Parallelism](./ring_sp_performance).

docs_new/docs/sglang-diffusion/index.mdx

@@ -44,6 +44,7 @@ sglang serve --model-path Qwen/Qwen-Image --port 30010
 - [Deployment and Performance Modes](/docs/sglang-diffusion/deployment_cookbook): choose `--performance-mode`, offload, FSDP, CFG parallelism, SP, and TP
 - [Attention Backends](/docs/sglang-diffusion/attention_backends): choose the best backend for your model and hardware
 - [Sequence Parallelism](/docs/sglang-diffusion/ring_sp_performance): configure SP, Ulysses, and ring-based splitting for long sequences
+- [Encoder Parallelism](/docs/sglang-diffusion/encoder_parallel): fold, data-parallel, or replicate the text/image encoders across idle GPUs
 - [Inference Batching](/docs/sglang-diffusion/dynamic_batching): batch compatible native diffusion requests during serving
 - [Progressive Resolution Generation](/docs/sglang-diffusion/progressive_resolution): run early denoising steps at lower latent resolution for selected pipelines
 - [Environment Variables](/docs/sglang-diffusion/environment_variables): platform, caching, storage, and debugging configuration

docs_new/docs/sglang-diffusion/performance-optimization.mdx

@@ -51,6 +51,11 @@ These settings should preserve model behavior while changing residency, parallel
       <td style={{padding: "9px 12px"}}>Long image/video sequences need sequence-level parallelism.</td>
       <td style={{padding: "9px 12px"}}><a href="./ring_sp_performance">Sequence Parallelism</a></td>
     </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500}}><code>--encoder-parallel</code></td>
+      <td style={{padding: "9px 12px"}}>Text/image encoding is a visible share of the request and the DiT replica sits idle during it.</td>
+      <td style={{padding: "9px 12px"}}><a href="./encoder_parallel">Encoder Parallelism</a></td>
+    </tr>
     <tr>
       <td style={{padding: "9px 12px", fontWeight: 500}}>Attention backend</td>
       <td style={{padding: "9px 12px"}}>Kernel choice dominates DiT latency or memory.</td>