[diffusion] quant: support pruned safetensors checkpoints for minimax-h3 (#35418)

This commit is contained in:
Mick
2026-08-20 19:34:14 +08:00
committed by GitHub
parent 97efc0507c
commit 82c6fc2db9
9 changed files with 672 additions and 41 deletions
@@ -248,6 +248,59 @@ server environment.
For MiniMax-H3, `--performance-mode speed` deliberately keeps the DiT eager. The current `torch.compile` path changes the model's numerical output, so it is not enabled implicitly by any recommended lossless preset. An explicit `--enable-torch-compile true` remains available for controlled experiments, but it should not be used to generate consistency ground truth.
### AdaLN-pruned safetensors transformers
[Comfy-Org/MiniMax-H3](https://huggingface.co/Comfy-Org/MiniMax-H3/tree/main/diffusion_models)
publishes smaller DiT-only checkpoints that replace the original AdaLN branches
with an interpolated curve table. Select one file explicitly; the base model
still supplies the text encoder and VAEs.
```bash Command
sglang serve \
--model-path MiniMaxAI/MiniMax-H3 \
--model-variant fl2va \
--transformer-weights-path \
Comfy-Org/MiniMax-H3/diffusion_models/minimax_h3_fl2va_pruned_bf16.safetensors \
--num-gpus 4 \
--tp-size 2 \
--ulysses-degree 2 \
--performance-mode speed \
--port 30010
```
The pruned checkpoint is approximate and is therefore rejected by
`quality="high"`, which remains limited to the audited official BF16 DiT.
The Comfy `pruned_fp8_scaled` FL2VA and Ref2VA files are also supported. Their
per-layer markers are detected automatically; do not add `--quantization`:
```bash Command
sglang serve \
--model-path MiniMaxAI/MiniMax-H3 \
--model-variant fl2va \
--transformer-weights-path \
Comfy-Org/MiniMax-H3/diffusion_models/minimax_h3_fl2va_pruned_fp8_scaled.safetensors \
--num-gpus 4 \
--tp-size 2 \
--ulysses-degree 2 \
--performance-mode speed \
--port 30010
```
SGLang uses its native static-activation FP8 linear path for attention and
`fc1`. The checkpoint marks `fc2` for full-precision matrix multiplication, so
SGLang retains its FP8 storage but materializes and scales one compute-dtype
`fc2` matrix for each call. This preserves the checkpoint's mixed execution
contract and low resident weight memory, but that part is slower than a fully
quantized FP8 GEMM. TP, Ulysses/Ring sequence parallelism, and
component/layerwise offload are supported; FSDP inference is rejected.
The `pruned_int8_convrot` files are detected but remain unsupported. They
require online regular-Hadamard ConvRot, dynamic INT8 activation quantization,
and a matching W8A8 GEMM. SGLang fails before loading them instead of silently
treating their stored INT8 values as ordinary weights. A native ConvRot kernel
path should be added and benchmarked separately before these files are accepted.
### Advanced: precomputed AdaLN cache
The [model card](https://huggingface.co/MiniMaxAI/MiniMax-H3) notes that about