[diffusion] feat: support loading Comfy NVFP4-AWQ text encoders (#36046)
This commit is contained in:
@@ -1060,6 +1060,19 @@ W4A4 Qwen3-VL files use the same overlay, for example:
|
||||
This checkpoint keeps its unmarked embedding and vision tower in their source
|
||||
precision; no explicit component quantization option is needed.
|
||||
|
||||
The official Comfy NVFP4-AWQ encoder uses the same flagless overlay:
|
||||
|
||||
```bash Overlay
|
||||
--component-paths.text_encoder \
|
||||
Comfy-Org/MiniMax-H3/text_encoders/qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors
|
||||
```
|
||||
|
||||
SGLang auto-detects its row-wise INT8 embedding, NVFP4 language linears, and
|
||||
AWQ input pre-scales. The weights stay compressed at rest; each active linear
|
||||
is dequantized for a BF16/FP16 matrix multiplication, so this path primarily
|
||||
reduces resident memory rather than encoder latency. Do not add a component or
|
||||
transformer quantization option.
|
||||
|
||||
The same component option accepts a self-describing Quanto qint8 file without
|
||||
an additional quantization flag:
|
||||
|
||||
|
||||
@@ -53,12 +53,12 @@ repo contains multiple candidate checkpoints, pass
|
||||
`--transformer-weights-path` explicitly.
|
||||
|
||||
MiniMax-H3 is a verified example for Comfy safetensors with per-layer metadata,
|
||||
including `pruned_fp8_scaled` and serialized ConvRot INT8. Other Comfy FP8
|
||||
exports are also auto-detected: the presence of an input scale selects static
|
||||
activation scaling, while its absence selects dynamic scaling. Pass one selected
|
||||
FL2VA or Ref2VA DiT file by local path, `owner/repo/path/file.safetensors`, or
|
||||
direct Hugging Face file URL; do not combine it with `--quantization`. Its GGUF
|
||||
usage is documented in
|
||||
including `pruned_fp8_scaled`, serialized ConvRot formats, and the official
|
||||
NVFP4-AWQ Qwen3-VL encoder. Other Comfy FP8 exports are also auto-detected: the
|
||||
presence of an input scale selects static activation scaling, while its absence
|
||||
selects dynamic scaling. Pass one selected DiT or component file by local path,
|
||||
`owner/repo/path/file.safetensors`, or direct Hugging Face file URL; do not
|
||||
combine it with an explicit quantization option. Its GGUF usage is documented in
|
||||
the [MiniMax-H3 cookbook](/cookbook/diffusion/MiniMax/MiniMax-H3#pre-quantized-gguf-transformer).
|
||||
|
||||
## Quantized Component Repositories
|
||||
@@ -212,6 +212,14 @@ backend.
|
||||
<td><code>comfy-kitchen</code></td>
|
||||
<td>Auto-detected; omit <code>--quantization</code>. Each layer dispatches to its serialized W4A4 or INT8 ConvRot kernel. CUDA requires SM75+; TP must preserve each format's quantization and ConvRot group boundaries. Offload is supported and FSDP is not.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>comfy-nvfp4-full-precision</code></td>
|
||||
<td>Safetensors with serialized <code>nvfp4</code> and optional row-wise <code>int8_tensorwise</code> layer metadata</td>
|
||||
<td><code>--component-paths.text_encoder</code></td>
|
||||
<td>MiniMax-H3 native Qwen3-VL encoder</td>
|
||||
<td>None</td>
|
||||
<td>Auto-detected; omit explicit quantization. Preserves packed storage, high-nibble-first weights, swizzled block scales, and AWQ input pre-scales. Each active NVFP4 matrix is dequantized for BF16/FP16 compute, so this is a memory path rather than a native FP4 speed path.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>quanto-int8</code></td>
|
||||
<td>One native encoder safetensors file with an embedded Quanto quantization map</td>
|
||||
|
||||
Reference in New Issue
Block a user