minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

[diffusion] feat: admit compatible quantized native encoders (#35962)

Browse Source
This commit is contained in:
Mick
2026-08-22 18:51:01 +08:00
committed by GitHub
parent 382343f860
commit 489e605b35
8 changed files with 66 additions and 105 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/docs/sglang-diffusion/api/cli.mdx
+1 -1
View File
@@ -139,7 +139,7 @@ or a native pipeline's registered module name.
For a native text encoder:
- `--component-paths.text_encoder {MODEL}` replaces the text-encoder checkpoint; `--text-encoder-path {MODEL}` is its shorter alias
- Quantization metadata is auto-detected from that checkpoint. Each native encoder must explicitly support the serialized format; this is not blanket quantization support for every component, and unsupported combinations fail before weight loading.
- Quantization metadata is auto-detected from that checkpoint. The native loader accepts compatible serialized formats without a model-name allowlist and rejects implementations that do not construct the required quantized layers.
The same contract applies to every weighted component: path routing is generic,
while quantized materialization is capability-based. Native auxiliary loaders
docs/docs/sglang-diffusion/quantization.mdx
+1 -22
View File
@@ -70,7 +70,7 @@ paths:
| Component path | Quantized checkpoint behavior |
| --- | --- |
| `transformer`, `transformer_2`, `unconditional_transformer`, `audio_dit`, `video_dit` | Uses the SGLang transformer quantization adapters documented below. |
| `text_encoder*`, `image_encoder*` | Requires the native encoder class to declare support for the detected format. MiniMax-H3 FP8 and model-managed integrations such as Ideogram are supported; unknown combinations fail closed. |
| `text_encoder*`, `image_encoder*` | Passes detected metadata to the registered native encoder. Loading continues only when that implementation constructs compatible quantized layers; unknown or incompatible combinations fail closed. |
| `vae`, `video_vae`, `audio_vae` | A standard top-level Diffusers `quantization_config` is delegated to `AutoModel.from_pretrained`. Native-only VAEs and nested/compression metadata fail closed. |
| Library-managed Transformers or Diffusers components | Delegates to the upstream `from_pretrained` path and inherits its format support and validation behavior. The local PE model uses this path; compatible formats remain model-specific. |
| Native auxiliary components that load raw state dicts | Quantized checkpoints are rejected before model construction until that component has a quantized materialization implementation. This includes connectors, duration heads, bridges, diffusion decoders, sound tokenizers, spatial upsamplers, and vocoders. |
@@ -402,27 +402,6 @@ sglang generate \
Each pattern is matched against the full layer prefix (e.g. `layers.0.attention.to_q`). A layer is skipped and left unquantized if its prefix contains any of the given patterns.
## MiniMax-H3 Text Encoder FP8
MiniMax-H3 can load a serialized FP8 checkpoint for the language linear layers
in its native Qwen3-VL text encoder independently of the DiT. Embeddings,
normalization layers, and the Qwen vision tower remain in BF16.
```bash Command
sglang serve \
--model-path MiniMaxAI/MiniMax-H3 \
--model-variant fl2va \
--component-paths.text_encoder Qwen/Qwen3-VL-32B-Instruct-FP8 \
--num-gpus 4 \
--port 30010
```
`--text-encoder-path` is accepted as a shorter alias. No quantization flag is
needed: SGLang detects the checkpoint metadata and only enables formats that
the native encoder explicitly supports. Text-encoder FP8 is approximate, is
not enabled by default, and is rejected by MiniMax-H3's strict
`quality="high"` deployment contract.
## Transformers Component BnB4
Model components that already have a native Transformers loading path can load