[diffusion] feat: support loading serialized convrot w4a4 checkpoints (#36039)

This commit is contained in:
Mick
2026-08-24 19:13:35 +08:00
committed by GitHub
parent 716a6bf10c
commit bfeae4e79a
9 changed files with 407 additions and 1 deletions
@@ -306,6 +306,12 @@ and packed INT4 tensors automatically. Do not add `--quantization`. TP remains
subject to each row-parallel shard preserving the checkpoint's ConvRot group
boundary, and FSDP is rejected.
W4A4 ConvRot files are also detected from their layer metadata. Install a
current `comfy-kitchen`, then pass a full or pruned FL2VA / Ref2VA file such as
`Merserk/MiniMax-H3-INT4-ConvRot/minimax_h3_fl2va_pruned_int4_convrot.safetensors`
to `--transformer-weights-path`. The packed weights stay INT4 and the runtime
honors each layer's activation mode; omit `--quantization`.
### Advanced: precomputed AdaLN cache
The [model card](https://huggingface.co/MiniMaxAI/MiniMax-H3) notes that about
@@ -969,6 +975,16 @@ Install `comfy-kitchen>=0.2.27` and omit `--quantization`. SGLang automatically
loads its W4A8 language linears and tensorwise INT8 embedding; the unmarked
vision tower remains BF16.
W4A4 Qwen3-VL files use the same overlay, for example:
```bash Overlay
--component-paths.text_encoder \
Merserk/MiniMax-H3-INT4-ConvRot/qwen3vl_32b_minimax_h3_int4_convrot.safetensors
```
This checkpoint keeps its unmarked embedding and vision tower in their source
precision; no explicit component quantization option is needed.
The same component option accepts a self-describing Quanto qint8 file without
an additional quantization flag:
@@ -200,6 +200,14 @@ backend.
<td><code>comfy-kitchen&gt;=0.2.27</code></td>
<td>Auto-detected; omit <code>--quantization</code>. Requires SM80+ and validates packed weights, group/channel scales, and optional codebooks before model construction. Mixed encoder files may keep their embedding tensorwise INT8. TP must preserve ConvRot group boundaries; offload is supported and FSDP is not.</td>
</tr>
<tr>
<td><code>comfy-w4a4-convrot</code></td>
<td>Safetensors with serialized <code>convrot_w4a4</code> layer metadata</td>
<td><code>--transformer-weights-path</code> or <code>--component-paths.text_encoder</code></td>
<td>Native DiTs and encoders with matching parameter mappings; MiniMax-H3 FL2VA / Ref2VA DiTs and Qwen3-VL encoder layouts are recognized</td>
<td><code>comfy-kitchen</code></td>
<td>Auto-detected; omit <code>--quantization</code>. Packed INT4 weights use the checkpoint's W4A4 ConvRot kernel and <code>linear_dtype</code>. CUDA requires SM75+; TP must preserve the 64-element quantization and ConvRot group boundaries. Offload is supported and FSDP is not.</td>
</tr>
<tr>
<td><code>quanto-int8</code></td>
<td>One native encoder safetensors file with an embedded Quanto quantization map</td>