[diffusion] feat: load serialized bnb4 components with transformers (#35945)

This commit is contained in:
Mick
2026-08-22 15:33:53 +08:00
committed by GitHub
parent 90354326c7
commit b391ef171f
5 changed files with 281 additions and 84 deletions
@@ -423,6 +423,27 @@ the native encoder explicitly supports. Text-encoder FP8 is approximate, is
not enabled by default, and is rejected by MiniMax-H3's strict
`quality="high"` deployment contract.
## Transformers Component BnB4
Model components that already have a native Transformers loading path can load
serialized BitsAndBytes 4-bit checkpoints with a standard top-level
`quantization_config`. Plain checkpoints keep using an available native SGLang
implementation. For example, replace FLUX's T5 component with the official
Diffusers checkpoint:
```bash Command
sglang serve \
--model-path black-forest-labs/FLUX.1-dev \
--component-paths.text_encoder_2 \
diffusers/FLUX.1-dev-bnb-4bit/text_encoder_2
```
The quantized component must stay resident. SGLang rejects component or
layerwise offload, nonstandard metadata locations, and native-only component
fallbacks for this path instead of silently changing the checkpoint contract.
Diffusion DiT components declared under the Diffusers library use the separate
quantization backends documented above.
## Validated ModelOpt Checkpoints
This section is the canonical support matrix for the thirteen published