[diffusion] feat: rebuild minimax-h3 adaln outputs on demand (#34650)
Co-authored-by: Mick <mickjagger19@icloud.com>
This commit is contained in:
@@ -174,6 +174,52 @@ server environment.
|
||||
|
||||
For MiniMax-H3, `--performance-mode speed` deliberately keeps the DiT eager. The current `torch.compile` path changes the model's numerical output, so it is not enabled implicitly by any recommended lossless preset. An explicit `--enable-torch-compile true` remains available for controlled experiments, but it should not be used to generate consistency ground truth.
|
||||
|
||||
### Advanced: precomputed AdaLN cache
|
||||
|
||||
The [model card](https://huggingface.co/MiniMaxAI/MiniMax-H3) notes that about
|
||||
13B H3 parameters are AdaLN branches whose outputs can be precomputed for
|
||||
inference. The public base checkpoint contains the original branches, not a
|
||||
ready-to-use cache. SGLang therefore keeps the standard path as the default.
|
||||
|
||||
<Warning>
|
||||
This is an experimental deployment path. It is intentionally disabled unless
|
||||
you provide an explicitly generated cache; end-to-end numerical and peak-memory
|
||||
validation remains required before using it in production.
|
||||
</Warning>
|
||||
|
||||
When an inference-only deployment has a fixed sampling schedule, build a cache
|
||||
from the already materialized transformer directory on CUDA, then pass it to
|
||||
the usual `sglang serve` command. This does not alter the denoising formula:
|
||||
the cache stores the BF16 outputs of the original AdaLN linears.
|
||||
|
||||
```bash Command
|
||||
python -m sglang.multimodal_gen.tools.build_minimax_h3_adaln_cache \
|
||||
--transformer-path "$TRANSFORMER_PATH" \
|
||||
--model-variant fl2va \
|
||||
--mode t2va \
|
||||
--num-inference-steps 50 \
|
||||
--flow-shift 12 \
|
||||
--audio-flow-shift 3 \
|
||||
--output /models/minimax-h3-fl2va-adaln-50step.safetensors
|
||||
|
||||
sglang serve \
|
||||
--model-path MiniMaxAI/MiniMax-H3 \
|
||||
--model-variant fl2va \
|
||||
--minimax-h3-adaln-cache-path /models/minimax-h3-fl2va-adaln-50step.safetensors \
|
||||
--num-gpus 4 \
|
||||
--tp-size 2 \
|
||||
--ulysses-degree 2 \
|
||||
--port 30010
|
||||
```
|
||||
|
||||
`$TRANSFORMER_PATH` is the `FL2VA/transformer` or `Ref2VA/transformer`
|
||||
directory in the normal SGLang/Hugging Face snapshot; the builder never
|
||||
downloads a second copy. A cache only covers the scheduler settings used to
|
||||
create it, including its mode, step count, flow shifts, and condition noise
|
||||
values. SGLang rejects a request outside that coverage instead of silently
|
||||
changing conditioning. Cache mode supports the matching unquantized checkpoint
|
||||
only.
|
||||
|
||||
## 4. Generate video and audio
|
||||
|
||||
MiniMax-H3 uses the asynchronous OpenAI-compatible video endpoint. Choose a
|
||||
|
||||
Reference in New Issue
Block a user