[NPU] [Diffusion] Support MiniMax H3 on Ascend NPU's (#33569)

Co-authored-by: ronnie_zheng <zl19940307@163.com>
This commit is contained in:
Артем Савкин
2026-08-25 09:44:14 +03:00
committed by GitHub
co-authored by ronnie_zheng
parent b7f9fca26e
commit 61b67316d8
21 changed files with 1147 additions and 693 deletions
@@ -369,6 +369,70 @@ values. SGLang rejects a request outside that coverage instead of silently
changing conditioning. Cache mode supports the matching unquantized checkpoint
only.
### Serve MiniMax-H3 on Ascend NPUs
For Ascend NPU, follow the
[NPU installation guide](/docs/hardware-platforms/ascend-npus/getting-started/installation)
before starting the server.
The Ascend commands below explicitly enable the Cache-DiT configuration used
for the reported performance measurements. Remove these `SGLANG_CACHE_DIT_*`
variables to use lossless denoising. See the **Ascend NPU topology comparison**
in the Benchmarks section for the measured eight- and four-NPU latency.
The measured latency configuration also passes `--dit-cpu-offload false` to
keep the transformer resident on the NPUs. Omit this flag when lower device
memory usage is more important than avoiding CPU-to-NPU transfer latency.
For an eight-NPU host, the validated topology is TP2 + SP4 with Laser
Attention. Use Ascend Flash Attention by replacing `laser_attn` with `fa`.
```bash 8-NPU
SGLANG_CACHE_DIT_ENABLED=true \
SGLANG_CACHE_DIT_FN=2 \
SGLANG_CACHE_DIT_BN=1 \
SGLANG_CACHE_DIT_WARMUP=4 \
SGLANG_CACHE_DIT_RDT=0.4 \
SGLANG_CACHE_DIT_MC=4 \
SGLANG_CACHE_DIT_TAYLORSEER=true \
SGLANG_CACHE_DIT_TS_ORDER=2 \
HCCL_BUFFSIZE=256 sglang serve \
--model-path MiniMaxAI/MiniMax-H3 \
--model-type diffusion \
--model-variant fl2va \
--dit-cpu-offload false \
--num-gpus 8 \
--tp-size 2 \
--sp-degree 4 \
--attention-backend laser_attn \
--port 30088 \
--component-residency text_encoder=layerwise-offload
```
For a four-NPU host, use TP2 + SP2:
```bash 4-NPU
SGLANG_CACHE_DIT_ENABLED=true \
SGLANG_CACHE_DIT_FN=2 \
SGLANG_CACHE_DIT_BN=1 \
SGLANG_CACHE_DIT_WARMUP=4 \
SGLANG_CACHE_DIT_RDT=0.4 \
SGLANG_CACHE_DIT_MC=4 \
SGLANG_CACHE_DIT_TAYLORSEER=true \
SGLANG_CACHE_DIT_TS_ORDER=2 \
HCCL_BUFFSIZE=256 sglang serve \
--model-path MiniMaxAI/MiniMax-H3 \
--model-type diffusion \
--model-variant fl2va \
--dit-cpu-offload false \
--num-gpus 4 \
--tp-size 2 \
--sp-degree 2 \
--attention-backend laser_attn \
--port 30088 \
--component-residency text_encoder=layerwise-offload
```
## 4. Generate video and audio
MiniMax-H3 uses the asynchronous OpenAI-compatible video endpoint. Choose a
@@ -1087,10 +1151,27 @@ the configurations with collected measurements:
| B200 | 8× Ulysses8 resident | 4× FSDP + Ulysses4 |
| H200 | 4× Ulysses4 resident | 4× FSDP + Ulysses4; 4× TP2 + Ulysses2; 2 nodes × 8× Ulysses8×Ring2 cross-node |
| H100 | 4× TP2 + Ulysses2 resident | 4× TP4 + Ulysses1; 4× FSDP + Ulysses4 |
| Ascend NPU | 8 NPUs, TP2 + SP4, Laser Attention | 4 NPUs, TP2 + SP2, Laser Attention |
| MI300X / MI355X | 8× Ulysses8 resident | 1×, 2×, and 4× scaling runs |
| RTX 5090 | 2× TP2 + layerwise offload | — |
| RTX 4090 24 GB | 1× layerwise offload + `kitchen_int8` | Approximate attention backends are opt-in |
### Ascend NPU topology comparison
Both topologies used Laser Attention and the explicit Cache-DiT configuration
from the Ascend launch commands, with `--dit-cpu-offload false` keeping the DiT
resident. The measured workload was one 5-second T2VA request at 1344×768,
124 frames, 24 fps, and 50 inference steps.
| NPU count | Topology | End-to-end latency |
| ---: | --- | ---: |
| 8 | TP2 + SP4 | **55.07 s** |
| 4 | TP2 + SP2 | **103.57 s** |
These are individual end-to-end measurements for each topology, not averages.
The eight-NPU topology had 46.8% lower end-to-end latency than the four-NPU
topology.
### B300 precision and encoder placement
A 12-configuration sweep on a single 8× B300 host, covering both checkpoint