[diffusion] feat: support FastH3 (4-step VSA-distilled MiniMax-H3) with a VSA-H3 attention backend (#37480)

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-09-02 21:39:54 +08:00
parent 99b9109553
commit f586654518
26 changed files with 1778 additions and 14 deletions
@@ -643,7 +643,71 @@ SGLang projects those adapter factors onto the pruned coordinates at load time.
 A structurally modified checkpoint without that metadata still fails closed,
 and packed GGUF weights remain incompatible with LoRA.

-## 6. Sampling and output controls
+## 6. FastH3: 4-step distilled preview
+
+[FastVideo/FastVideo-FastH3-4-step-Preview-v1-VSA-DataFree](https://huggingface.co/FastVideo/FastVideo-FastH3-4-step-Preview-v1-VSA-DataFree)
+is a 4-step DMD2 distillation of MiniMax-H3, trained data-free with Video
+Sparse Attention (VSA) at 0.9 sparsity and 64-token tiles. Only the T2VA
+capability was distilled: requests must use `task: "t2va"`, and `fl2va` /
+`ref2va` requests are rejected. The checkpoint inherits the MiniMax-H3
+Community License.
+
+Pass the repository directly to `--model-path`. The flat native-Diffusers
+upload is materialized into the base-H3 layout through a registered model
+overlay; the only non-symlink step is a one-time re-serialization of the
+roughly 10 GB video VAE on first launch.
+
+```bash 4×B300 VSA-H3
+sglang serve \
+  --model-path FastVideo/FastVideo-FastH3-4-step-Preview-v1-VSA-DataFree \
+  --num-gpus 4 \
+  --attention-backend video_sparse_attn_h3 \
+  --attention-backend-config '{"VSA_sparsity": 0.9}' \
+  --port 30010
+```
+
+Requests use the same asynchronous video endpoint as the base model, with
+`task: "t2va"`, `conditions: []`, and a target such as
+`{"short_edge": 768, "aspect_ratio": "16:9", "duration_seconds": 5.0}`. The
+request default is `num_inference_steps: 5`: five points on the standard
+shift-12/shift-3 sigma grid, i.e. the four distilled DiT evaluations. Any
+other step count is rejected.
+
+`video_sparse_attn_h3` (VSA-H3) is the trained sparse policy: an in-tree
+Triton block-sparse kernel (SM90 / SM100 / SM103) over segment-pure prefix
+tiles and (4, 4, 4) video tiles, driven by the checkpoint's trained
+`to_gate_compress` compression branch. Only the DiT runs sparse; the token
+refiner, text encoder, and VAEs keep their dense defaults. Ulysses sequence
+parallelism is supported. See
+[Attention Backends](/docs/sglang-diffusion/attention_backends) for
+`VSA_sparsity`, `vsa_mode`, `vsa_dense_first_n_steps`, and
+`vsa_dense_layers`. Every dense backend that runs on base H3 (`fa`,
+`torch_sdpa`, ...) also runs on the FastH3 weights without VSA flags, and
+`sglang generate` takes the same flags as `sglang serve`.
+
+Measured latencies for the 4× B300 recipe are in
+[FastH3 on B300](#fasth3-on-b300).
+
+FastH3 rejects deployment options that do not apply to the distilled preview
+instead of silently ignoring them: `--model-variant`, `quality: "high"`,
+`fl2va` / `ref2va` requests, and, with VSA-H3, `--ring-degree` greater than 1,
+`torch.compile`, and breakable CUDA graph execution.
+
+<Warning>
+The sibling `FastVideo/FastVideo-FastH3-4-step-Preview-v1-LoRA` adapters carry
+full-rank `.diff` / `.diff_b` deltas and `set_weight` gate tensors beyond the
+LoRA contract; `--lora-path` rejects them with an explicit error. Serve the
+merged VSA-DataFree checkpoint above instead.
+</Warning>
+
+<Note>
+Upstream labels this checkpoint a preview. Quality gaps versus base H3 on hard
+motion and fine detail are properties of the released distillation, not of the
+SGLang port. Use base MiniMax-H3 when output quality matters more than
+latency.
+</Note>
+
+## 7. Sampling and output controls

 MiniMax-H3 supports more than one output per prompt. The video API accepts
 `num_outputs_per_prompt` (or OpenAI-compatible `n`) from 1 through 10. Offline
@@ -818,7 +882,7 @@ has completed, but the `quality: "high"` path above remains fail-closed
 to the audited 4×H200 workload.
 </Warning>

-## 7. Feature contracts and advanced recipes
+## 8. Feature contracts and advanced recipes

 The generated command already contains the recommended topology and encoder
 setting. Use the detailed reference below only when applying an optional
@@ -1059,7 +1123,7 @@ fold decision is not node-boundary aware:

 </Tabs>

-## 8. Configuration notes
+## 9. Configuration notes

 - MiniMax-H3 produces the canonical 24 fps output; request duration is expressed through `target.duration_seconds`.
 - `target.duration_seconds` must be between 4 and 15 seconds, inclusive. The command picker defaults to the verified 5-second profile.
@@ -1078,7 +1142,7 @@ fold decision is not node-boundary aware:
 - `speed` keeps model components resident, while `auto` applies the model-aware 120 GiB residency threshold. `memory` prioritizes avoiding OOM and includes the executable VAE decoder in its default layerwise set. A measured recipe with sufficient headroom can opt into `--component-residency vae=resident`; the 2×H100 CI recipe does this because the VAE's 4.8 GiB/GPU cost avoids repeated decoder transfers during tiled decode. DiT residency and prefetch knobs remain scoped to the DiT. Use `speed` only after confirming that the complete target workload fits.
 - Breakable CUDA graph execution is an explicit opt-in, not part of the recommended `speed` preset. It requires `--enable-breakable-cuda-graph`, every served size in `--warmup-resolutions`, and `--bcg-text-buckets` that cover the live H3 condition sequence. The validated 1344×768 Ref2VA recipe uses 5504; other task profiles and reference sets may need a different value. It preserves eager output for matching captured signatures, but graph capture consumes additional GPU memory and may provide little latency benefit when Ulysses attention and collectives dominate, so benchmark it on the target topology before enabling it.

-## 9. Benchmarks
+## 10. Benchmarks

 The picker exposes resident and FSDP profiles on NVIDIA datacenter GPUs. GPU
 counts are properties of the selected recipes, not a claim that every platform
@@ -1189,6 +1253,30 @@ python3 -m sglang.multimodal_gen.benchmarks.bench_serving \
 | Ref2VA | FP8 | fold | 112.0 s | 34.44 s | **27.12 s** | 52,816 MB |
 | Ref2VA | FP8 | replicate | 116.0 s | 33.42 s | **27.12 s** | 93,396 MB |

+### FastH3 on B300
+
+The same 4× B300 host served [FastH3](#6-fasth3-4-step-distilled-preview)
+with the VSA-H3 recipe above (1344×768 at 24 fps with audio, `task: "t2va"`,
+`num_inference_steps: 5`, seed 1000, eager BF16, Ulysses4, `VSA_sparsity` 0.9).
+E2E is the client wall clock of a `/v1/videos` request including decode,
+muxing, and file output, median of three requests after one warm request;
+the stage columns are the server timings of the same request. H3 aligns the
+requested durations to 124, 243, and 362 frames. Client RTF is E2E divided by
+the video duration:
+
+| Requested / aligned | Encoder | Denoise (4 forwards) | Decode | Transport + MP4 | E2E | Client RTF | Peak/GPU |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
+| 5 s / 124 | 0.07 s | 2.18 s | 0.87 s | 0.9 s | **4.1 s** | 0.79 | 95,744 MB |
+| 10 s / 243 | 0.07 s | 4.83 s | 1.71 s | 1.3 s | **8.0 s** | 0.79 | 102,666 MB |
+| 15 s / 362 | 0.07 s | 8.80 s | 2.56 s | 1.8 s | **13.3 s** | 0.88 | 110,774 MB |
+
+All three requests finish faster than playback. Dense FA on the same weights
+and topology takes 3.77 / 9.84 / 18.45 s (`sglang generate`, stage sum): it is
+competitive at 5 s, and VSA-H3 pulls ahead from 10 s on. At 5 s,
+TP2 + Ulysses2 (3.42 s, 62,290 MB), FSDP + Ulysses4 (3.42 s, 50,984 MB), and
+online `--quantization fp8` (2.93 s, 64,204 MB) trade a little latency for
+peak memory.
+
 ### H200 topology comparison

 The same four-card H200 host completed both lossless resident placements with
@@ -79,6 +79,11 @@ For SGLang-native pipelines, the CLI accepts the lowercase names of `AttentionBa
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)", whiteSpace: "nowrap"}}>`VIDEO_SPARSE_ATTN`</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>Requires <code>vsa</code>. Configure <code>sparsity</code> via <code>--attention-backend-config</code>.</td>
    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>`video_sparse_attn_h3`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)", whiteSpace: "nowrap"}}>`VIDEO_SPARSE_ATTN_H3`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>Video Sparse Attention for MiniMax-H3 / FastH3 (VSA-H3). In-tree Triton block-sparse kernel (SM90 / SM100 / SM103); no external package. Configure via <code>--attention-backend-config</code>.</td>
+    </tr>
    <tr>
      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>`vmoba_attn`</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)", whiteSpace: "nowrap"}}>`VMOBA_ATTN`</td>
@@ -238,6 +243,67 @@ SpargeAttention is approximate even when `topk=1`: the recommended upstream
 kernel quantizes attention through SageAttention2. Validate output quality and
 end-to-end latency on the target model and resolution before deployment.

+**Video Sparse Attention for H3 (`video_sparse_attn_h3`)**
+
+<table style={{width: "100%", borderCollapse: "collapse", tableLayout: "fixed"}}>
+  <colgroup>
+    <col style={{width: "24%"}} />
+    <col style={{width: "14%"}} />
+    <col style={{width: "44%"}} />
+    <col style={{width: "18%"}} />
+  </colgroup>
+  <thead>
+    <tr style={{borderBottom: "2px solid #d55816"}}>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Parameter</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>Type</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Description</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>Default</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>`VSA_sparsity`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`float`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>Fraction of video tiles excluded from the top-k selection (0.0 - 1.0). `0.9` is the FastH3 trained policy.</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`0.9`</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>`vsa_mode`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`str`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>`exempt`: non-video keys (text/audio prefix tiles) are always selected. `compete`: they compete with video tiles in the top-k.</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`exempt`</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>`vsa_dense_first_n_steps`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`int`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>Use dense attention for the first N denoising steps.</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`0`</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>`vsa_dense_layers`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`list[int]`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>Layer indices kept dense, e.g. `[0, 1]`.</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`[]`</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>`vsa_tile_size`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`int`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>Kernel tile size. Only `64` (the trained (4, 4, 4) geometry) is accepted.</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`64`</td>
+    </tr>
+  </tbody>
+</table>
+
+VSA-H3 constraints:
+
+- Only the DiT runs sparse; the token refiner, text encoder, and VAEs keep
+  their dense defaults. An explicit `--component-attention-backends
+  text_encoder=fa` is rejected because the H3 text encoder has SDPA-only layers.
+- Uses the checkpoint's trained `to_gate_compress` compression branch. Base
+  MiniMax-H3 weights load zero gates and run pure sparse.
+- Ulysses sequence parallelism is supported; `--ring-degree` greater than 1,
+  `torch.compile`, and breakable CUDA graph execution are rejected.
+
 **V-MoBA (`vmoba_attn`)**

 <table style={{width: "100%", borderCollapse: "collapse", tableLayout: "fixed"}}>
@@ -549,6 +615,16 @@ end-to-end latency on the target model and resolution before deployment.
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>❌</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>CUDA-only. Requires <code>vsa</code>. Configure <code>sparsity</code> via <code>--attention-backend-config</code>.</td>
    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>`video_sparse_attn_h3`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Yes</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>No</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>No</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>No</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>❌</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>❌</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>CUDA-only (SM90 / SM100 / SM103). In-tree Triton kernel, no external dependency. Configure via <code>--attention-backend-config</code>.</td>
+    </tr>
    <tr>
      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}><code>sla_attn</code></td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Yes</td>
@@ -137,6 +137,12 @@ Rows are grouped when a family shares the same runtime path or optimization supp
            <td>T2VA / FL2VA / Ref2VA, 768p at 24 fps with synchronized audio</td>
            <td><span className="sgd-chip">Cache-DiT</span><span className="sgd-chip">Sage</span><span className="sgd-chip">Online FP8</span><span className="sgd-chip">GGUF</span></td>
          </tr>
+          <tr>
+            <td>FastH3</td>
+            <td><div className="sgd-id-list"><code>FastVideo/FastVideo-FastH3-4-step-Preview-v1-VSA-DataFree</code></div></td>
+            <td>T2VA only, 4-step distilled, 768p at 24 fps with synchronized audio</td>
+            <td><span className="sgd-chip">VSA-H3</span></td>
+          </tr>
          <tr>
            <td>Wan2.1 Fun</td>
            <td><div className="sgd-id-list"><code>weizhou03/Wan2.1-Fun-1.3B-InP-Diffusers</code></div></td>
@@ -621,6 +627,21 @@ Optimization columns are abbreviated to keep the matrix readable:
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>❌</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>❌</td>
    </tr>
+<tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>FastH3 4-step (T2VA only)</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}><code>FastVideo/FastVideo-FastH3-4-step-Preview-v1-VSA-DataFree</code></td>
+      <td style={{padding: "9px 8px", backgroundColor: "rgba(255,255,255,0.02)"}}>768p · 24 fps</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>❌</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>❌</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>✅</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>✅</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>❌</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>❌</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>❌</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>❌</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>❌</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>❌</td>
+    </tr>
 <tr>
      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>LTX-2.3 (one/two-stage/TI2V/HQ)</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}><code>Lightricks/LTX-2.3</code></td>
@@ -704,6 +725,11 @@ Optimization columns are abbreviated to keep the matrix readable:
   difference is transformer depth and width, picked up from
   `transformer/config.json` at load time. A single checkpoint serves T2V,
   I2V (`--image-path`), and T2I (`--num-frames 1`).
+6. FastH3's VSA column refers to the dedicated `video_sparse_attn_h3` (VSA-H3)
+   backend. Dense backends that run on base MiniMax-H3 also run on the FastH3
+   weights.
+   FastH3 serves `t2va` only and rejects `--model-variant` and
+   `quality: "high"`.

 </Accordion>

@@ -373,7 +373,7 @@ For a pipeline whose primary DiT is named `transformer`, the shorter

 MiniMax-H3 supports this path while preserving its required FP32 patch,
 timestep, and output projections. See the
-[MiniMax-H3 cookbook](/cookbook/diffusion/MiniMax/MiniMax-H3#7-feature-contracts-and-advanced-recipes)
+[MiniMax-H3 cookbook](/cookbook/diffusion/MiniMax/MiniMax-H3#8-feature-contracts-and-advanced-recipes)
 for its distributed serving recipe.

 ### MXFP4 Online Quantization
@@ -432,7 +432,7 @@ projections take that path.
 <Warning>
 `kitchen_int8` is approximate and is not a consistency ground-truth mode.
 The BF16 path is unchanged when `comfy-kitchen` is not installed. See the
-[MiniMax-H3 cookbook](/cookbook/diffusion/MiniMax/MiniMax-H3#7-feature-contracts-and-advanced-recipes)
+[MiniMax-H3 cookbook](/cookbook/diffusion/MiniMax/MiniMax-H3#8-feature-contracts-and-advanced-recipes)
 for the 24 GB offload recipe, including why `vae` must stay out of
 `--layerwise-offload-components`.
 </Warning>