[diffusion] feat: support loading mixed w4a8 text encoders (#36037)
This commit is contained in:
@@ -958,6 +958,17 @@ linear layers use FP8 while embeddings, normalization, and the vision tower
|
||||
remain BF16. This is an approximate serve-time choice and is incompatible with
|
||||
the strict `quality="high"` deployment contract.
|
||||
|
||||
The mixed W4A8 community encoder follows the same component-path contract:
|
||||
|
||||
```bash Overlay
|
||||
--component-paths.text_encoder \
|
||||
Winnougan/MiniMax-H3-INT4_Convrot_ComfyUI/qwen3vl_32b_minimax_h3-w4a8_convrot.safetensors
|
||||
```
|
||||
|
||||
Install `comfy-kitchen>=0.2.27` and omit `--quantization`. SGLang automatically
|
||||
loads its W4A8 language linears and tensorwise INT8 embedding; the unmarked
|
||||
vision tower remains BF16.
|
||||
|
||||
</Tab>
|
||||
|
||||
<Tab title="Encoder scheduling">
|
||||
|
||||
@@ -195,10 +195,10 @@ backend.
|
||||
<tr>
|
||||
<td><code>comfy-w4a8-convrot</code></td>
|
||||
<td>Safetensors with serialized <code>asym_w4a8_int8</code> layer metadata and packed weights</td>
|
||||
<td><code>--transformer-weights-path</code></td>
|
||||
<td>MiniMax-H3 FL2VA / Ref2VA DiTs</td>
|
||||
<td><code>--transformer-weights-path</code> or <code>--component-paths.text_encoder</code></td>
|
||||
<td>MiniMax-H3 FL2VA / Ref2VA DiTs and its native Qwen3-VL encoder</td>
|
||||
<td><code>comfy-kitchen>=0.2.27</code></td>
|
||||
<td>Auto-detected; omit <code>--quantization</code>. Requires SM80+ and validates packed weights, group/channel scales, and optional codebooks before model construction. TP must preserve ConvRot group boundaries; offload is supported and FSDP is not.</td>
|
||||
<td>Auto-detected; omit <code>--quantization</code>. Requires SM80+ and validates packed weights, group/channel scales, and optional codebooks before model construction. Mixed encoder files may keep their embedding tensorwise INT8. TP must preserve ConvRot group boundaries; offload is supported and FSDP is not.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>qvg-kv</code></td>
|
||||
|
||||
Reference in New Issue
Block a user