[diffusion] feat: support loading minimax h3 gguf text encoders (#36055)
This commit is contained in:
@@ -1006,6 +1006,21 @@ storage, then dequantizes only the active matrix for BF16/FP16 linear math. Use
|
||||
it as a memory option, not as an INT8 throughput claim. The embedded Quanto map
|
||||
is the selector; adding `--quantization` would describe a different operation.
|
||||
|
||||
Qwen3-VL GGUF files use the same component option:
|
||||
|
||||
```bash Overlay
|
||||
--component-paths.text_encoder \
|
||||
DeepBeepMeep/MiniMax-H3/Qwen3-VL-32B-Instruct/qwen3vl-32B-MiniMax-H3-Q4_K_M.gguf
|
||||
```
|
||||
|
||||
The file selects its own Q4/Q6 layout; do not add `--quantization`. SGLang keeps
|
||||
the aligned vocabulary table and language linears in packed GGML storage. The
|
||||
smaller Comfy vision tower, whose flattened blocks can cross row and TP shard
|
||||
boundaries, is restored one matrix at a time to BF16 during loading.
|
||||
This CUDA-only memory option supports encoder TP and layerwise placement, but
|
||||
not encoder FSDP. Like the Quanto variant, it is approximate and is not a
|
||||
throughput claim.
|
||||
|
||||
</Tab>
|
||||
|
||||
<Tab title="Encoder scheduling">
|
||||
|
||||
Reference in New Issue
Block a user