diff --git a/docs/cookbook/diffusion/MiniMax/MiniMax-H3.mdx b/docs/cookbook/diffusion/MiniMax/MiniMax-H3.mdx index fb9fdfe4f..e159e6ddd 100644 --- a/docs/cookbook/diffusion/MiniMax/MiniMax-H3.mdx +++ b/docs/cookbook/diffusion/MiniMax/MiniMax-H3.mdx @@ -1187,6 +1187,59 @@ benchmark latency as well as visual and audio quality on the target workload. + + +SubBlock sparse attention applies training-free block sparsity to MiniMax-H3's +long, non-causal DiT self-attention. The BF16 path supports CUDA SM90, SM100, +and SM120. Other compute capabilities, including SM103/B300, are rejected. +Ulysses sequence parallelism is supported; Ring parallelism is not. + +The global backend selects the H3 DiT lazily, so keep the Qwen text encoder on +its compatible dense backend with a component override: + +```bash Command +sglang serve \ + --model-path MiniMaxAI/MiniMax-H3 \ + --model-variant fl2va \ + --num-gpus 8 \ + --ulysses-degree 8 \ + --attention-backend subblock_sparse_attn \ + --component-attention-backends text_encoder=fa \ + --attention-backend-config '{"sparsity": 0.75, "skip_first_steps": 10}' \ + --port 30010 +``` + +Use `text_encoder=torch_sdpa` on SM120. The default configuration drops at +most 75% of key blocks after the first 10 denoise forwards; short sequences, +cross-attention, and unsupported shapes continue on the dense path. Tune +`sparsity` and `skip_first_steps` together with visual and audio quality checks. + +On H100/H200 (SM90), `compute_mode=sage_fp8` switches the sparse kernel to +online INT8 Q/K and FP8 P/V compute. Install the optional kernel and add the +compute mode: + +```bash Command +pip install git+https://github.com/thu-ml/SpargeAttn.git --no-build-isolation + +sglang serve \ + --model-path MiniMaxAI/MiniMax-H3 \ + --model-variant fl2va \ + --num-gpus 8 \ + --ulysses-degree 8 \ + --attention-backend subblock_sparse_attn \ + --component-attention-backends text_encoder=fa \ + --attention-backend-config '{"compute_mode": "sage_fp8", "sparsity": 0.75}' \ + --port 30010 +``` + + +Both SubBlock routing and `sage_fp8` are approximate. Use dense BF16 attention +for consistency ground truth, and validate the selected sparsity on the target +resolution, duration, task, and checkpoint. + + + + The picker explicitly writes `--encoder-parallel auto` in every single-node diff --git a/docs/docs/sglang-diffusion/attention_backends.mdx b/docs/docs/sglang-diffusion/attention_backends.mdx index 1f594a604..e9536aaee 100644 --- a/docs/docs/sglang-diffusion/attention_backends.mdx +++ b/docs/docs/sglang-diffusion/attention_backends.mdx @@ -79,6 +79,11 @@ For SGLang-native pipelines, the CLI accepts the lowercase names of `AttentionBa `VIDEO_SPARSE_ATTN` Requires vsa. Configure sparsity via --attention-backend-config. + + `subblock_sparse_attn` + `SUBBLOCK_SPARSE_ATTN` + Training-free SubBlock sparsity for MiniMax-H3 on CUDA SM90, SM100, and SM120. The default compute mode is BF16; compute_mode=sage_fp8 is an approximate SM90-only path that requires SpargeAttn. See the MiniMax-H3 recipe. + `video_sparse_attn_h3` `VIDEO_SPARSE_ATTN_H3` diff --git a/python/sglang/multimodal_gen/README.md b/python/sglang/multimodal_gen/README.md index c6e97c4dd..8f4f434de 100644 --- a/python/sglang/multimodal_gen/README.md +++ b/python/sglang/multimodal_gen/README.md @@ -84,6 +84,8 @@ loaded component: - `resident` keeps the complete component on the accelerator. - `component-offload` stores the complete component on CPU between uses. +- `snapshot-offload` keeps a CPU weight snapshot while the complete component + runs on the accelerator, avoiding a weight copy back to CPU after each use. - `layerwise-offload` streams the component's declared layers from CPU. `COMPONENT` can be an exact `model_index.json` key or one of `all`, `dit`,