[diffusion] feat: load quantized H3 text encoder checkpoints (#34986)

Co-authored-by: Yiqi Yang <yangyiqi8787@gmail.com>
This commit is contained in:
Mick
2026-08-18 09:10:54 +08:00
committed by GitHub
co-authored by Yiqi Yang
parent 0ea262e6e5
commit d55f1c28e2
10 changed files with 351 additions and 5 deletions
@@ -738,6 +738,29 @@ The picker exposes this option only on the B200 and B300 topologies used for
real H3 validation runs.
</Warning>
The Qwen3-VL text encoder can be replaced independently of the DiT. To reduce
its resident memory, point the text-encoder component at the serialized FP8
checkpoint used in validation:
```bash Command
sglang serve \
--model-path MiniMaxAI/MiniMax-H3 \
--model-variant fl2va \
--component-paths.text_encoder Qwen/Qwen3-VL-32B-Instruct-FP8 \
--num-gpus 4 \
--tp-size 2 \
--ulysses-degree 2 \
--performance-mode speed \
--port 30010
```
`--text-encoder-path` is accepted as a shorter alias. No separate quantization
flag is required: SGLang reads the checkpoint's `quantization_config` and
fails closed if the native encoder does not support that format. The language
linear layers use FP8 while embeddings, normalization, and the vision tower
remain BF16. This is an approximate serve-time choice and is incompatible with
the strict `quality="high"` deployment contract.
</Tab>
</Tabs>