[diffusion] feat: support loading serialized convrot w4a4 checkpoints (#36039)
This commit is contained in:
@@ -306,6 +306,12 @@ and packed INT4 tensors automatically. Do not add `--quantization`. TP remains
|
||||
subject to each row-parallel shard preserving the checkpoint's ConvRot group
|
||||
boundary, and FSDP is rejected.
|
||||
|
||||
W4A4 ConvRot files are also detected from their layer metadata. Install a
|
||||
current `comfy-kitchen`, then pass a full or pruned FL2VA / Ref2VA file such as
|
||||
`Merserk/MiniMax-H3-INT4-ConvRot/minimax_h3_fl2va_pruned_int4_convrot.safetensors`
|
||||
to `--transformer-weights-path`. The packed weights stay INT4 and the runtime
|
||||
honors each layer's activation mode; omit `--quantization`.
|
||||
|
||||
### Advanced: precomputed AdaLN cache
|
||||
|
||||
The [model card](https://huggingface.co/MiniMaxAI/MiniMax-H3) notes that about
|
||||
@@ -969,6 +975,16 @@ Install `comfy-kitchen>=0.2.27` and omit `--quantization`. SGLang automatically
|
||||
loads its W4A8 language linears and tensorwise INT8 embedding; the unmarked
|
||||
vision tower remains BF16.
|
||||
|
||||
W4A4 Qwen3-VL files use the same overlay, for example:
|
||||
|
||||
```bash Overlay
|
||||
--component-paths.text_encoder \
|
||||
Merserk/MiniMax-H3-INT4-ConvRot/qwen3vl_32b_minimax_h3_int4_convrot.safetensors
|
||||
```
|
||||
|
||||
This checkpoint keeps its unmarked embedding and vision tower in their source
|
||||
precision; no explicit component quantization option is needed.
|
||||
|
||||
The same component option accepts a self-describing Quanto qint8 file without
|
||||
an additional quantization flag:
|
||||
|
||||
|
||||
@@ -200,6 +200,14 @@ backend.
|
||||
<td><code>comfy-kitchen>=0.2.27</code></td>
|
||||
<td>Auto-detected; omit <code>--quantization</code>. Requires SM80+ and validates packed weights, group/channel scales, and optional codebooks before model construction. Mixed encoder files may keep their embedding tensorwise INT8. TP must preserve ConvRot group boundaries; offload is supported and FSDP is not.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>comfy-w4a4-convrot</code></td>
|
||||
<td>Safetensors with serialized <code>convrot_w4a4</code> layer metadata</td>
|
||||
<td><code>--transformer-weights-path</code> or <code>--component-paths.text_encoder</code></td>
|
||||
<td>Native DiTs and encoders with matching parameter mappings; MiniMax-H3 FL2VA / Ref2VA DiTs and Qwen3-VL encoder layouts are recognized</td>
|
||||
<td><code>comfy-kitchen</code></td>
|
||||
<td>Auto-detected; omit <code>--quantization</code>. Packed INT4 weights use the checkpoint's W4A4 ConvRot kernel and <code>linear_dtype</code>. CUDA requires SM75+; TP must preserve the 64-element quantization and ConvRot group boundaries. Offload is supported and FSDP is not.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>quanto-int8</code></td>
|
||||
<td>One native encoder safetensors file with an embedded Quanto quantization map</td>
|
||||
|
||||
Reference in New Issue
Block a user