[diffusion] quant: support pruned safetensors checkpoints for minimax-h3 (#35418)
This commit is contained in:
@@ -248,6 +248,59 @@ server environment.
|
||||
|
||||
For MiniMax-H3, `--performance-mode speed` deliberately keeps the DiT eager. The current `torch.compile` path changes the model's numerical output, so it is not enabled implicitly by any recommended lossless preset. An explicit `--enable-torch-compile true` remains available for controlled experiments, but it should not be used to generate consistency ground truth.
|
||||
|
||||
### AdaLN-pruned safetensors transformers
|
||||
|
||||
[Comfy-Org/MiniMax-H3](https://huggingface.co/Comfy-Org/MiniMax-H3/tree/main/diffusion_models)
|
||||
publishes smaller DiT-only checkpoints that replace the original AdaLN branches
|
||||
with an interpolated curve table. Select one file explicitly; the base model
|
||||
still supplies the text encoder and VAEs.
|
||||
|
||||
```bash Command
|
||||
sglang serve \
|
||||
--model-path MiniMaxAI/MiniMax-H3 \
|
||||
--model-variant fl2va \
|
||||
--transformer-weights-path \
|
||||
Comfy-Org/MiniMax-H3/diffusion_models/minimax_h3_fl2va_pruned_bf16.safetensors \
|
||||
--num-gpus 4 \
|
||||
--tp-size 2 \
|
||||
--ulysses-degree 2 \
|
||||
--performance-mode speed \
|
||||
--port 30010
|
||||
```
|
||||
|
||||
The pruned checkpoint is approximate and is therefore rejected by
|
||||
`quality="high"`, which remains limited to the audited official BF16 DiT.
|
||||
|
||||
The Comfy `pruned_fp8_scaled` FL2VA and Ref2VA files are also supported. Their
|
||||
per-layer markers are detected automatically; do not add `--quantization`:
|
||||
|
||||
```bash Command
|
||||
sglang serve \
|
||||
--model-path MiniMaxAI/MiniMax-H3 \
|
||||
--model-variant fl2va \
|
||||
--transformer-weights-path \
|
||||
Comfy-Org/MiniMax-H3/diffusion_models/minimax_h3_fl2va_pruned_fp8_scaled.safetensors \
|
||||
--num-gpus 4 \
|
||||
--tp-size 2 \
|
||||
--ulysses-degree 2 \
|
||||
--performance-mode speed \
|
||||
--port 30010
|
||||
```
|
||||
|
||||
SGLang uses its native static-activation FP8 linear path for attention and
|
||||
`fc1`. The checkpoint marks `fc2` for full-precision matrix multiplication, so
|
||||
SGLang retains its FP8 storage but materializes and scales one compute-dtype
|
||||
`fc2` matrix for each call. This preserves the checkpoint's mixed execution
|
||||
contract and low resident weight memory, but that part is slower than a fully
|
||||
quantized FP8 GEMM. TP, Ulysses/Ring sequence parallelism, and
|
||||
component/layerwise offload are supported; FSDP inference is rejected.
|
||||
|
||||
The `pruned_int8_convrot` files are detected but remain unsupported. They
|
||||
require online regular-Hadamard ConvRot, dynamic INT8 activation quantization,
|
||||
and a matching W8A8 GEMM. SGLang fails before loading them instead of silently
|
||||
treating their stored INT8 values as ordinary weights. A native ConvRot kernel
|
||||
path should be added and benchmarked separately before these files are accepted.
|
||||
|
||||
### Advanced: precomputed AdaLN cache
|
||||
|
||||
The [model card](https://huggingface.co/MiniMaxAI/MiniMax-H3) notes that about
|
||||
|
||||
@@ -15,7 +15,9 @@ Use these paths:
|
||||
|
||||
- `--model-path`: the base or original model
|
||||
- `--transformer-path`: a quantized transformers-style transformer component directory that already contains its own `config.json`
|
||||
- `--transformer-weights-path`: quantized transformer weights provided as a single safetensors file, a sharded safetensors directory, a local path, or a Hugging Face repo ID
|
||||
- `--transformer-weights-path`: replacement transformer weights in safetensors
|
||||
format (file, directory, or Hub repository/file) or a supported GGUF file
|
||||
(local or Hub)
|
||||
- `--quantization`: apply online quantization to unquantized models at load time (activations are quantized dynamically)
|
||||
- `--quantization-ignored-layers` layer name patterns to keep unquantized (e.g. `attention.to_`)
|
||||
- `--component-paths.text_encoder`: replace a native text encoder with a checkpoint whose `quantization_config` is auto-detected
|
||||
@@ -45,6 +47,12 @@ directory directly as `--model-path`, but that is a compatibility path. If a
|
||||
repo contains multiple candidate checkpoints, pass
|
||||
`--transformer-weights-path` explicitly.
|
||||
|
||||
MiniMax-H3 auto-detects the per-layer metadata in Comfy's
|
||||
`pruned_fp8_scaled` safetensors. Pass one selected FL2VA or Ref2VA file by local
|
||||
path, `owner/repo/path/file.safetensors`, or direct Hugging Face file URL; do
|
||||
not combine it with `--quantization`. MiniMax-H3 GGUF usage is documented in
|
||||
the [MiniMax-H3 cookbook](/cookbook/diffusion/MiniMax/MiniMax-H3#pre-quantized-gguf-transformer).
|
||||
|
||||
## Quant Families
|
||||
|
||||
Here, `quant_family` means a checkpoint and loading family with shared CLI
|
||||
@@ -111,6 +119,22 @@ backend.
|
||||
<td>None</td>
|
||||
<td>Mixed override repos keep the base model separate; full Qwen Image exports can be loaded directly as <code>--model-path</code>; raw exports such as <code>black-forest-labs/FLUX.2-dev-NVFP4</code> still use the weights-path flow</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>gguf</code></td>
|
||||
<td>One selected GGUF DiT file</td>
|
||||
<td><code>--transformer-weights-path</code></td>
|
||||
<td>MiniMax-H3 original or pruned FL2VA / Ref2VA DiTs</td>
|
||||
<td>None</td>
|
||||
<td>CUDA only; auto-detected; supports standard and K-quant GGML types; FSDP and the separate Qwen3-VL text-encoder GGUF files are not supported</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>comfy-fp8</code></td>
|
||||
<td>One selected Comfy safetensors file with per-layer <code>comfy_quant</code> metadata</td>
|
||||
<td><code>--transformer-weights-path</code></td>
|
||||
<td>MiniMax-H3 pruned FL2VA / Ref2VA DiTs</td>
|
||||
<td>None</td>
|
||||
<td>CUDA; auto-detected; TP, sequence parallelism, and component/layerwise offload are supported, while FSDP is not. Checkpoint-marked <code>fc2</code> layers retain FP8 storage and use compute-dtype matmul.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>qvg-kv</code></td>
|
||||
<td>Unquantized model with runtime causal KV-cache compression</td>
|
||||
|
||||
Reference in New Issue
Block a user