[diffusion] feat: rebuild minimax-h3 adaln outputs on demand (#34650)
Co-authored-by: Mick <mickjagger19@icloud.com>
This commit is contained in:
@@ -174,6 +174,52 @@ server environment.
|
||||
|
||||
For MiniMax-H3, `--performance-mode speed` deliberately keeps the DiT eager. The current `torch.compile` path changes the model's numerical output, so it is not enabled implicitly by any recommended lossless preset. An explicit `--enable-torch-compile true` remains available for controlled experiments, but it should not be used to generate consistency ground truth.
|
||||
|
||||
### Advanced: precomputed AdaLN cache
|
||||
|
||||
The [model card](https://huggingface.co/MiniMaxAI/MiniMax-H3) notes that about
|
||||
13B H3 parameters are AdaLN branches whose outputs can be precomputed for
|
||||
inference. The public base checkpoint contains the original branches, not a
|
||||
ready-to-use cache. SGLang therefore keeps the standard path as the default.
|
||||
|
||||
<Warning>
|
||||
This is an experimental deployment path. It is intentionally disabled unless
|
||||
you provide an explicitly generated cache; end-to-end numerical and peak-memory
|
||||
validation remains required before using it in production.
|
||||
</Warning>
|
||||
|
||||
When an inference-only deployment has a fixed sampling schedule, build a cache
|
||||
from the already materialized transformer directory on CUDA, then pass it to
|
||||
the usual `sglang serve` command. This does not alter the denoising formula:
|
||||
the cache stores the BF16 outputs of the original AdaLN linears.
|
||||
|
||||
```bash Command
|
||||
python -m sglang.multimodal_gen.tools.build_minimax_h3_adaln_cache \
|
||||
--transformer-path "$TRANSFORMER_PATH" \
|
||||
--model-variant fl2va \
|
||||
--mode t2va \
|
||||
--num-inference-steps 50 \
|
||||
--flow-shift 12 \
|
||||
--audio-flow-shift 3 \
|
||||
--output /models/minimax-h3-fl2va-adaln-50step.safetensors
|
||||
|
||||
sglang serve \
|
||||
--model-path MiniMaxAI/MiniMax-H3 \
|
||||
--model-variant fl2va \
|
||||
--minimax-h3-adaln-cache-path /models/minimax-h3-fl2va-adaln-50step.safetensors \
|
||||
--num-gpus 4 \
|
||||
--tp-size 2 \
|
||||
--ulysses-degree 2 \
|
||||
--port 30010
|
||||
```
|
||||
|
||||
`$TRANSFORMER_PATH` is the `FL2VA/transformer` or `Ref2VA/transformer`
|
||||
directory in the normal SGLang/Hugging Face snapshot; the builder never
|
||||
downloads a second copy. A cache only covers the scheduler settings used to
|
||||
create it, including its mode, step count, flow shifts, and condition noise
|
||||
values. SGLang rejects a request outside that coverage instead of silently
|
||||
changing conditioning. Cache mode supports the matching unquantized checkpoint
|
||||
only.
|
||||
|
||||
## 4. Generate video and audio
|
||||
|
||||
MiniMax-H3 uses the asynchronous OpenAI-compatible video endpoint. Choose a
|
||||
|
||||
@@ -77,6 +77,7 @@ Use `sglang generate --help` and `sglang serve --help` for the full argument lis
|
||||
- `--model-path {MODEL}`: model path or Hugging Face model ID
|
||||
- `--served-model-name {NAME}`: stable model name exposed by serving APIs. Defaults to `--model-id` when set, otherwise `--model-path`.
|
||||
- `--model-variant {NAME}`: semantic checkpoint variant to load when one model repository contains multiple weight partitions. The pipeline maps this stable name to the repository layout before loading; for example, MiniMax-H3 accepts `fl2va` and `ref2va`. This is a server/load-time choice, unlike a request's `task`.
|
||||
- `--minimax-h3-adaln-cache-path {FILE}`: advanced MiniMax-H3-only inference cache. It replaces the checkpoint's AdaLN projection weights with precomputed outputs and only accepts requests whose exact FP32 timestep plan is included in the cache. It requires unquantized weights and the matching model variant.
|
||||
- `--model-subfolder {PATH}`: advanced direct override for a component subfolder inside the model repository. Prefer `--model-variant` when the pipeline exposes semantic routing. If both are supplied, they must resolve to the same weight partition.
|
||||
- `--lora-path {PATH}` and `--lora-nickname {NAME}`: load a LoRA adapter
|
||||
- `--lora-weight-name {FILE}`: select one adapter file from a repository that contains multiple LoRA revisions. The Hub download is filtered to that file plus JSON metadata, so unused weights are not downloaded.
|
||||
|
||||
Reference in New Issue
Block a user