[diffusion] feat: support loading mixed w4a8 text encoders (#36037)

This commit is contained in:
Mick
2026-08-24 13:31:27 +08:00
committed by GitHub
parent 344613c159
commit 8df3b9eff9
9 changed files with 260 additions and 34 deletions
@@ -958,6 +958,17 @@ linear layers use FP8 while embeddings, normalization, and the vision tower
remain BF16. This is an approximate serve-time choice and is incompatible with
the strict `quality="high"` deployment contract.
The mixed W4A8 community encoder follows the same component-path contract:
```bash Overlay
--component-paths.text_encoder \
Winnougan/MiniMax-H3-INT4_Convrot_ComfyUI/qwen3vl_32b_minimax_h3-w4a8_convrot.safetensors
```
Install `comfy-kitchen>=0.2.27` and omit `--quantization`. SGLang automatically
loads its W4A8 language linears and tensorwise INT8 embedding; the unmarked
vision tower remains BF16.
</Tab>
<Tab title="Encoder scheduling">
+3 -3
View File
@@ -195,10 +195,10 @@ backend.
<tr>
<td><code>comfy-w4a8-convrot</code></td>
<td>Safetensors with serialized <code>asym_w4a8_int8</code> layer metadata and packed weights</td>
<td><code>--transformer-weights-path</code></td>
<td>MiniMax-H3 FL2VA / Ref2VA DiTs</td>
<td><code>--transformer-weights-path</code> or <code>--component-paths.text_encoder</code></td>
<td>MiniMax-H3 FL2VA / Ref2VA DiTs and its native Qwen3-VL encoder</td>
<td><code>comfy-kitchen&gt;=0.2.27</code></td>
<td>Auto-detected; omit <code>--quantization</code>. Requires SM80+ and validates packed weights, group/channel scales, and optional codebooks before model construction. TP must preserve ConvRot group boundaries; offload is supported and FSDP is not.</td>
<td>Auto-detected; omit <code>--quantization</code>. Requires SM80+ and validates packed weights, group/channel scales, and optional codebooks before model construction. Mixed encoder files may keep their embedding tensorwise INT8. TP must preserve ConvRot group boundaries; offload is supported and FSDP is not.</td>
</tr>
<tr>
<td><code>qvg-kv</code></td>