[NPU] [Diffusion] Support MiniMax H3 on Ascend NPU's (#33569)
Co-authored-by: ronnie_zheng <zl19940307@163.com>
This commit is contained in:
co-authored by
ronnie_zheng
parent
b7f9fca26e
commit
61b67316d8
@@ -369,6 +369,70 @@ values. SGLang rejects a request outside that coverage instead of silently
|
||||
changing conditioning. Cache mode supports the matching unquantized checkpoint
|
||||
only.
|
||||
|
||||
### Serve MiniMax-H3 on Ascend NPUs
|
||||
|
||||
For Ascend NPU, follow the
|
||||
[NPU installation guide](/docs/hardware-platforms/ascend-npus/getting-started/installation)
|
||||
before starting the server.
|
||||
|
||||
The Ascend commands below explicitly enable the Cache-DiT configuration used
|
||||
for the reported performance measurements. Remove these `SGLANG_CACHE_DIT_*`
|
||||
variables to use lossless denoising. See the **Ascend NPU topology comparison**
|
||||
in the Benchmarks section for the measured eight- and four-NPU latency.
|
||||
|
||||
The measured latency configuration also passes `--dit-cpu-offload false` to
|
||||
keep the transformer resident on the NPUs. Omit this flag when lower device
|
||||
memory usage is more important than avoiding CPU-to-NPU transfer latency.
|
||||
|
||||
For an eight-NPU host, the validated topology is TP2 + SP4 with Laser
|
||||
Attention. Use Ascend Flash Attention by replacing `laser_attn` with `fa`.
|
||||
|
||||
```bash 8-NPU
|
||||
SGLANG_CACHE_DIT_ENABLED=true \
|
||||
SGLANG_CACHE_DIT_FN=2 \
|
||||
SGLANG_CACHE_DIT_BN=1 \
|
||||
SGLANG_CACHE_DIT_WARMUP=4 \
|
||||
SGLANG_CACHE_DIT_RDT=0.4 \
|
||||
SGLANG_CACHE_DIT_MC=4 \
|
||||
SGLANG_CACHE_DIT_TAYLORSEER=true \
|
||||
SGLANG_CACHE_DIT_TS_ORDER=2 \
|
||||
HCCL_BUFFSIZE=256 sglang serve \
|
||||
--model-path MiniMaxAI/MiniMax-H3 \
|
||||
--model-type diffusion \
|
||||
--model-variant fl2va \
|
||||
--dit-cpu-offload false \
|
||||
--num-gpus 8 \
|
||||
--tp-size 2 \
|
||||
--sp-degree 4 \
|
||||
--attention-backend laser_attn \
|
||||
--port 30088 \
|
||||
--component-residency text_encoder=layerwise-offload
|
||||
```
|
||||
|
||||
For a four-NPU host, use TP2 + SP2:
|
||||
|
||||
```bash 4-NPU
|
||||
SGLANG_CACHE_DIT_ENABLED=true \
|
||||
SGLANG_CACHE_DIT_FN=2 \
|
||||
SGLANG_CACHE_DIT_BN=1 \
|
||||
SGLANG_CACHE_DIT_WARMUP=4 \
|
||||
SGLANG_CACHE_DIT_RDT=0.4 \
|
||||
SGLANG_CACHE_DIT_MC=4 \
|
||||
SGLANG_CACHE_DIT_TAYLORSEER=true \
|
||||
SGLANG_CACHE_DIT_TS_ORDER=2 \
|
||||
HCCL_BUFFSIZE=256 sglang serve \
|
||||
--model-path MiniMaxAI/MiniMax-H3 \
|
||||
--model-type diffusion \
|
||||
--model-variant fl2va \
|
||||
--dit-cpu-offload false \
|
||||
--num-gpus 4 \
|
||||
--tp-size 2 \
|
||||
--sp-degree 2 \
|
||||
--attention-backend laser_attn \
|
||||
--port 30088 \
|
||||
--component-residency text_encoder=layerwise-offload
|
||||
```
|
||||
|
||||
## 4. Generate video and audio
|
||||
|
||||
MiniMax-H3 uses the asynchronous OpenAI-compatible video endpoint. Choose a
|
||||
@@ -1087,10 +1151,27 @@ the configurations with collected measurements:
|
||||
| B200 | 8× Ulysses8 resident | 4× FSDP + Ulysses4 |
|
||||
| H200 | 4× Ulysses4 resident | 4× FSDP + Ulysses4; 4× TP2 + Ulysses2; 2 nodes × 8× Ulysses8×Ring2 cross-node |
|
||||
| H100 | 4× TP2 + Ulysses2 resident | 4× TP4 + Ulysses1; 4× FSDP + Ulysses4 |
|
||||
| Ascend NPU | 8 NPUs, TP2 + SP4, Laser Attention | 4 NPUs, TP2 + SP2, Laser Attention |
|
||||
| MI300X / MI355X | 8× Ulysses8 resident | 1×, 2×, and 4× scaling runs |
|
||||
| RTX 5090 | 2× TP2 + layerwise offload | — |
|
||||
| RTX 4090 24 GB | 1× layerwise offload + `kitchen_int8` | Approximate attention backends are opt-in |
|
||||
|
||||
### Ascend NPU topology comparison
|
||||
|
||||
Both topologies used Laser Attention and the explicit Cache-DiT configuration
|
||||
from the Ascend launch commands, with `--dit-cpu-offload false` keeping the DiT
|
||||
resident. The measured workload was one 5-second T2VA request at 1344×768,
|
||||
124 frames, 24 fps, and 50 inference steps.
|
||||
|
||||
| NPU count | Topology | End-to-end latency |
|
||||
| ---: | --- | ---: |
|
||||
| 8 | TP2 + SP4 | **55.07 s** |
|
||||
| 4 | TP2 + SP2 | **103.57 s** |
|
||||
|
||||
These are individual end-to-end measurements for each topology, not averages.
|
||||
The eight-NPU topology had 46.8% lower end-to-end latency than the four-NPU
|
||||
topology.
|
||||
|
||||
### B300 precision and encoder placement
|
||||
|
||||
A 12-configuration sweep on a single 8× B300 host, covering both checkpoint
|
||||
|
||||
Reference in New Issue
Block a user