[diffusion] feat: rebuild minimax-h3 adaln outputs on demand (#34650)

Co-authored-by: Mick <mickjagger19@icloud.com>
This commit is contained in:
triple-mu
2026-08-14 15:33:51 +08:00
committed by GitHub
co-authored by Mick
parent 2622e013eb
commit a86edcdc0a
12 changed files with 872 additions and 23 deletions
@@ -174,6 +174,52 @@ server environment.
For MiniMax-H3, `--performance-mode speed` deliberately keeps the DiT eager. The current `torch.compile` path changes the model's numerical output, so it is not enabled implicitly by any recommended lossless preset. An explicit `--enable-torch-compile true` remains available for controlled experiments, but it should not be used to generate consistency ground truth.
### Advanced: precomputed AdaLN cache
The [model card](https://huggingface.co/MiniMaxAI/MiniMax-H3) notes that about
13B H3 parameters are AdaLN branches whose outputs can be precomputed for
inference. The public base checkpoint contains the original branches, not a
ready-to-use cache. SGLang therefore keeps the standard path as the default.
<Warning>
This is an experimental deployment path. It is intentionally disabled unless
you provide an explicitly generated cache; end-to-end numerical and peak-memory
validation remains required before using it in production.
</Warning>
When an inference-only deployment has a fixed sampling schedule, build a cache
from the already materialized transformer directory on CUDA, then pass it to
the usual `sglang serve` command. This does not alter the denoising formula:
the cache stores the BF16 outputs of the original AdaLN linears.
```bash Command
python -m sglang.multimodal_gen.tools.build_minimax_h3_adaln_cache \
--transformer-path "$TRANSFORMER_PATH" \
--model-variant fl2va \
--mode t2va \
--num-inference-steps 50 \
--flow-shift 12 \
--audio-flow-shift 3 \
--output /models/minimax-h3-fl2va-adaln-50step.safetensors
sglang serve \
--model-path MiniMaxAI/MiniMax-H3 \
--model-variant fl2va \
--minimax-h3-adaln-cache-path /models/minimax-h3-fl2va-adaln-50step.safetensors \
--num-gpus 4 \
--tp-size 2 \
--ulysses-degree 2 \
--port 30010
```
`$TRANSFORMER_PATH` is the `FL2VA/transformer` or `Ref2VA/transformer`
directory in the normal SGLang/Hugging Face snapshot; the builder never
downloads a second copy. A cache only covers the scheduler settings used to
create it, including its mode, step count, flow shifts, and condition noise
values. SGLang rejects a request outside that coverage instead of silently
changing conditioning. Cache mode supports the matching unquantized checkpoint
only.
## 4. Generate video and audio
MiniMax-H3 uses the asynchronous OpenAI-compatible video endpoint. Choose a
+1
View File
@@ -77,6 +77,7 @@ Use `sglang generate --help` and `sglang serve --help` for the full argument lis
- `--model-path {MODEL}`: model path or Hugging Face model ID
- `--served-model-name {NAME}`: stable model name exposed by serving APIs. Defaults to `--model-id` when set, otherwise `--model-path`.
- `--model-variant {NAME}`: semantic checkpoint variant to load when one model repository contains multiple weight partitions. The pipeline maps this stable name to the repository layout before loading; for example, MiniMax-H3 accepts `fl2va` and `ref2va`. This is a server/load-time choice, unlike a request's `task`.
- `--minimax-h3-adaln-cache-path {FILE}`: advanced MiniMax-H3-only inference cache. It replaces the checkpoint's AdaLN projection weights with precomputed outputs and only accepts requests whose exact FP32 timestep plan is included in the cache. It requires unquantized weights and the matching model variant.
- `--model-subfolder {PATH}`: advanced direct override for a component subfolder inside the model repository. Prefer `--model-variant` when the pipeline exposes semantic routing. If both are supplied, they must resolve to the same weight partition.
- `--lora-path {PATH}` and `--lora-nickname {NAME}`: load a LoRA adapter
- `--lora-weight-name {FILE}`: select one adapter file from a repository that contains multiple LoRA revisions. The Hub download is filtered to that file plus JSON metadata, so unused weights are not downloaded.