[diffusion] feat: support loading self-describing quanto int8 encoders (#36052)
This commit is contained in:
@@ -969,6 +969,19 @@ Install `comfy-kitchen>=0.2.27` and omit `--quantization`. SGLang automatically
|
||||
loads its W4A8 language linears and tensorwise INT8 embedding; the unmarked
|
||||
vision tower remains BF16.
|
||||
|
||||
The same component option accepts a self-describing Quanto qint8 file without
|
||||
an additional quantization flag:
|
||||
|
||||
```bash Overlay
|
||||
--component-paths.text_encoder \
|
||||
DeepBeepMeep/MiniMax-H3/Qwen3-VL-32B-Instruct/Qwen3-VL-32B-Instruct-layer50_quanto_bf16_int8.safetensors
|
||||
```
|
||||
|
||||
This variant keeps the declared language and vision linear weights in qint8
|
||||
storage, then dequantizes only the active matrix for BF16/FP16 linear math. Use
|
||||
it as a memory option, not as an INT8 throughput claim. The embedded Quanto map
|
||||
is the selector; adding `--quantization` would describe a different operation.
|
||||
|
||||
</Tab>
|
||||
|
||||
<Tab title="Encoder scheduling">
|
||||
|
||||
Reference in New Issue
Block a user