[diffusion] feat: support loading minimax h3 gguf text encoders (#36055)
This commit is contained in:
@@ -1006,6 +1006,21 @@ storage, then dequantizes only the active matrix for BF16/FP16 linear math. Use
|
||||
it as a memory option, not as an INT8 throughput claim. The embedded Quanto map
|
||||
is the selector; adding `--quantization` would describe a different operation.
|
||||
|
||||
Qwen3-VL GGUF files use the same component option:
|
||||
|
||||
```bash Overlay
|
||||
--component-paths.text_encoder \
|
||||
DeepBeepMeep/MiniMax-H3/Qwen3-VL-32B-Instruct/qwen3vl-32B-MiniMax-H3-Q4_K_M.gguf
|
||||
```
|
||||
|
||||
The file selects its own Q4/Q6 layout; do not add `--quantization`. SGLang keeps
|
||||
the aligned vocabulary table and language linears in packed GGML storage. The
|
||||
smaller Comfy vision tower, whose flattened blocks can cross row and TP shard
|
||||
boundaries, is restored one matrix at a time to BF16 during loading.
|
||||
This CUDA-only memory option supports encoder TP and layerwise placement, but
|
||||
not encoder FSDP. Like the Quanto variant, it is approximate and is not a
|
||||
throughput claim.
|
||||
|
||||
</Tab>
|
||||
|
||||
<Tab title="Encoder scheduling">
|
||||
|
||||
@@ -162,11 +162,11 @@ backend.
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>gguf</code></td>
|
||||
<td>One selected GGUF DiT file</td>
|
||||
<td><code>--transformer-weights-path</code></td>
|
||||
<td>MiniMax-H3 original or pruned FL2VA / Ref2VA DiTs</td>
|
||||
<td>One selected GGUF DiT or native encoder file</td>
|
||||
<td><code>--transformer-weights-path</code>, or an explicit encoder file through <code>--component-paths.<component></code></td>
|
||||
<td>MiniMax-H3 original or pruned FL2VA / Ref2VA DiTs and its Qwen3-VL text encoder</td>
|
||||
<td>None</td>
|
||||
<td>CUDA only; auto-detected; supports standard and K-quant GGML types; FSDP and the separate Qwen3-VL text-encoder GGUF files are not supported</td>
|
||||
<td>CUDA only and auto-detected; supports standard and K-quant GGML types. TP requires aligned GGML blocks; component/layerwise offload is supported and FSDP is not.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>comfy-fp8</code></td>
|
||||
@@ -242,11 +242,11 @@ backend.
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>gguf</code></td>
|
||||
<td>A single community <code>.gguf</code> holding the transformer</td>
|
||||
<td><code>--transformer-weights-path</code></td>
|
||||
<td>MiniMax-H3 <code>fl2va</code> (original and pruned AdaLN curve)</td>
|
||||
<td>A single community <code>.gguf</code> holding a transformer or native encoder</td>
|
||||
<td><code>--transformer-weights-path</code> or <code>--component-paths.<component></code></td>
|
||||
<td>MiniMax-H3 <code>fl2va</code> DiT (original and pruned AdaLN curve) and Qwen3-VL text encoder</td>
|
||||
<td>None</td>
|
||||
<td>CUDA only, no FSDP. TP requires GGML-aligned shard boundaries. Shrinks the download and the host memory offload pins (17.5 vs 61.7 GiB for H3) rather than peak VRAM, which offload already bounds. Dequantized per use, so it is not faster. See <a href="#gguf">GGUF</a>.</td>
|
||||
<td>CUDA only, no FSDP. TP requires GGML-aligned shard boundaries. Packed weights reduce checkpoint and resident-weight memory; they are dequantized per use, so this is not a speed claim. See <a href="#gguf">GGUF</a>.</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
Reference in New Issue
Block a user