[diffusion] refactor: validate and document spectrum controls (#33851)

2026-08-06 23:23:11 +08:00
parent 44bde3911a
commit 7195b8e4c7
10 changed files with 143 additions and 9 deletions
@@ -81,7 +81,7 @@ Use `sglang generate --help` and `sglang serve --help` for the full argument lis
 - `--lora-merge-mode {auto|merge|dynamic}`: choose how LoRA is applied. `auto` statically merges regular weights and uses dynamic LoRA for FSDP-sharded weights to avoid full-gather peaks.
 - `--num-gpus {N}`: number of GPUs to use
 - `--performance-mode {manual|auto|speed|memory}` / `--mode`: preset for latency/throughput and memory defaults. `auto` is the default and keeps safe offload defaults, using FSDP only for validated DiT-offload replacement paths; `speed` also enables `--enable-torch-compile` unless the model-specific deployment config opts out or you explicitly disable it. Use `manual` to keep performance-related server args under explicit user control. Explicit offload, FSDP, and parallelism flags take precedence in all modes.
- `--tp-size {N}`: tensor parallelism size, mainly for encoders
+- `--tp-size {N}`: tensor parallelism size. Depending on the pipeline, it can shard the DiT, one or more encoders, or both.
 - `--sp-degree {N}`: sequence parallelism size
 - `--dp-size {N}` (alias `--data-parallel-size`): number of data-parallel replicas. Each replica is a full copy of the engine on `num_gpus / N` GPUs with its own ingress; generation requests round-robin across replicas, realtime sessions stick to the replica holding their state, and control operations (weights, LoRA, memory occupation, shutdown) apply to every replica. Combines with the other parallelism axes (`num_gpus = dp × cfg × tp × sp`); monolithic serving only.
 - `--ulysses-degree {N}` and `--ring-degree {N}`: USP parallelism controls
@@ -183,7 +183,7 @@ sglang generate \
 HTTP server-only arguments are ignored by `sglang generate`.
 </Note>

-For supported pipelines, Cache-DiT can be enabled with `SGLANG_CACHE_DIT_ENABLED=true` or `--cache-dit-config`. See [Cache-DiT](../cache_dit).
+For supported native pipelines, set `SGLANG_CACHE_DIT_ENABLED=true` to enable Cache-DiT. For the diffusers backend, use `--backend diffusers --cache-dit-config ...`. See [Cache-DiT](../cache_dit).

 For supported image pipelines, breakable CUDA graph can be enabled with `--enable-breakable-cuda-graph`, but you must declare every served resolution in `--warmup-resolutions` so warmup captures matching graph signatures.

@@ -14,7 +14,7 @@ Backend selection is performed by the shared attention layers (e.g. `LocalAttent
 When using the diffusers backend, `--attention-backend` is passed through to diffusers'
 `set_attention_backend` (e.g., `flash`, `_flash_3_hub`, `sage`, `xformers`, `native`).

- **CUDA**: prefers FlashAttention (FA3/FA4) when supported; otherwise falls back to PyTorch SDPA.
+- **CUDA**: prefers FlashAttention (FA3/FA4) when supported; otherwise falls back to PyTorch SDPA. On SM100/B200, dense non-causal fp16/bf16 native attention prefers cuDNN SDPA and falls back to FA4 if cuDNN has no compatible kernel.
 - **ROCm**: uses FlashAttention when available; otherwise falls back to PyTorch SDPA.
 - **Intel XPU**: uses XPU Flash Attention backend (fp16/bf16, head sizes 64/96/128/192/256); otherwise falls back to PyTorch SDPA.
 - **MUSA**: uses FlashAttention when available; also supports Sage Attention when installed; otherwise falls back to PyTorch SDPA.
@@ -3,11 +3,11 @@ title: "Caching Acceleration"
 description: "Compare caching acceleration strategies for diffusion models."
 tag: "approx"
 ---
-SGLang provides two complementary caching strategies for Diffusion Transformer (DiT) models. Both reduce denoising cost by skipping redundant computation, but they operate at different levels.
+SGLang provides three complementary caching strategies for Diffusion Transformer (DiT) models. All reduce denoising cost by skipping redundant computation, but they operate at different levels.

 ## Overview

-SGLang supports two complementary caching approaches:
+SGLang supports three complementary caching approaches:

 <table style={{width: "100%", borderCollapse: "collapse", tableLayout: "fixed"}}>
  <colgroup>
@@ -37,6 +37,12 @@ SGLang supports two complementary caching approaches:
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>Skip entire denoising steps based on L1 similarity</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Simple, built-in</td>
    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Spectrum</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Timestep-level</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>Forecast DiT features to skip selected denoising steps</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Experimental, model-validated tuning</td>
+    </tr>
  </tbody>
 </table>

@@ -86,6 +92,14 @@ See [TeaCache](./teacache) for detailed documentation.

 For Flux and Qwen models, TeaCache is automatically disabled when CFG is enabled.

+## Spectrum
+
+Spectrum forecasts DiT features and skips selected denoising steps. It is
+approximate and currently applies only to selected native implementation paths.
+
+See [Spectrum Acceleration](./spectrum) for supported model families,
+constraints, and request controls.
+

 ## References

@@ -119,7 +119,7 @@ Rows are grouped when a family shares the same runtime path or optimization supp
            <td>MiniMax-H3</td>
            <td><div className="sgd-id-list"><code>MiniMaxAI/MiniMax-H3</code></div></td>
            <td>T2VA / FL2VA / Ref2VA, 768p at 24 fps with synchronized audio</td>
-            <td><span className="sgd-chip">Cache-DiT</span><span className="sgd-chip">Online FP8</span></td>
+            <td><span className="sgd-chip">Cache-DiT</span><span className="sgd-chip">Sage</span><span className="sgd-chip">Online FP8</span></td>
          </tr>
          <tr>
            <td>Wan2.1 Fun</td>
@@ -560,7 +560,7 @@ Optimization columns are abbreviated to keep the matrix readable:
      <td style={{padding: "9px 8px", backgroundColor: "rgba(255,255,255,0.02)"}}>768p · 24 fps</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>❌</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>❌</td>
-      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>❌</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>✅</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>❌</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>❌</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>❌</td>
@@ -34,7 +34,7 @@ sglang serve --model-path Qwen/Qwen-Image --port 30010
 - [CLI](/docs/sglang-diffusion/api/cli): run one-off generation jobs or launch a persistent server
 - [OpenAI-Compatible API](/docs/sglang-diffusion/api/openai_api): send image and video requests to the HTTP server
 - [Performance Overview](/docs/sglang-diffusion/performance-optimization): choose speed, memory, parallelism, caching, and quality-tradeoff levers
- [Caching Acceleration](/docs/sglang-diffusion/caching-acceleration): use Cache-DiT or TeaCache to reduce denoising cost
+- [Caching Acceleration](/docs/sglang-diffusion/caching-acceleration): use Cache-DiT, TeaCache, or Spectrum to reduce denoising cost
 - [Quantization](/docs/sglang-diffusion/quantization): load quantized transformer checkpoints
 - [Contributing](/docs/sglang-diffusion/contributing): contribution workflow, adding new models, and CI perf baselines

@@ -0,0 +1,54 @@
+---
+title: "Spectrum Acceleration"
+description: "Approximate request-scoped denoising-step acceleration."
+tag: "approx"
+---
+
+Spectrum forecasts DiT features and skips selected denoising steps. It is an
+approximation: validate visual or video quality and latency on the exact model,
+shape, hardware, and sampling settings you plan to deploy.
+
+## Quick start
+
+```bash
+sglang generate \
+  --model-path Wan-AI/Wan2.1-T2V-1.3B-Diffusers \
+  --prompt "A paper boat floating through a misty mountain lake" \
+  --enable-spectrum \
+  --save-output
+```
+
+## Scope and constraints
+
+- Available only on native FLUX.1, Wan, HunyuanVideo, and SD3 implementation
+  paths. It is not a `--backend diffusers` feature and does not currently
+  cover FLUX.2.
+- The request control is available through `sglang generate` and Python
+  sampling parameters. It is not a `sglang serve` or OpenAI-server request
+  option yet.
+- Spectrum and `--enable-teacache` are mutually exclusive.
+- Start with the defaults. `--debug` adds shadow-prediction validation work, so
+  its latency is not representative of normal Spectrum execution.
+
+## Advanced controls
+
+Use `--enable-spectrum` explicitly. Providing any Spectrum override also
+enables it, but that implicit behavior is intended for scripts rather than new
+commands.
+
+| Flag | Default | Purpose |
+| --- | --- | --- |
+| `--spectrum-window-size` | `2.0` | Initial step-skipping window |
+| `--spectrum-flex-window` | `0.75` | Window growth after a real forward |
+| `--spectrum-warmup-steps` | `5` | Initial exact DiT forwards |
+| `--spectrum-m` | `4` | Chebyshev basis count |
+| `--spectrum-lam` | `0.1` | Ridge regularization |
+| `--spectrum-tau-num-steps` | `50` | Chebyshev time horizon |
+| `--history-size` | `100` | Recent feature-history capacity |
+| `--taylor-order` | `1` | Local predictor order (`1`, `2`, or `3`) |
+| `--w` | `1.0` | Chebyshev/Taylor blend weight |
+
+<Warning>
+These controls trade speed against output fidelity. Change one control at a
+time and retain a lossless baseline for comparison.
+</Warning>