[Diffusion] modelopt diffusion fp8 support for flux1/flux2 and wan2.2 (#22365)

This commit is contained in:
Xiaoyu Zhang
2026-04-10 20:56:57 +08:00
committed by GitHub
parent 8ba9646044
commit 1ff51555f2
10 changed files with 1891 additions and 116 deletions
+37
View File
@@ -43,10 +43,47 @@ backend.
| quant_family | checkpoint form | canonical CLI | supported models | extra dependency | platform / notes |
|------------------|--------------------------------------------------------------------------------------------|------------------------------------------------------|--------------------------------------------------------------|---------------------------------------|-----------------------------------------------------------------------------------------------------------------------|
| `fp8` | Quantized transformer component folder, or safetensors with `quantization_config` metadata | `--transformer-path` or `--transformer-weights-path` | ALL | None | Component-folder and single-file flows are both supported |
| `modelopt-fp8` | Converted ModelOpt FP8 transformer directory or repo with `config.json` | `--transformer-weights-path` | FLUX.2, Wan2.2 | None | Override config is read from the quantized transformer repo; `dit_cpu_offload` and `dit_layerwise_offload` are auto-disabled |
| `nvfp4-modelopt` | NVFP4 safetensors file, sharded directory, or repo providing transformer weights | `--transformer-weights-path` | FLUX.2 | `comfy-kitchen` optional on Blackwell | Blackwell can use a best-performance kit when available; otherwise SGLang falls back to the generic ModelOpt FP4 path |
| `nunchaku-svdq` | Pre-quantized Nunchaku transformer weights, usually named `svdq-{int4\|fp4}_r{rank}-...` | `--transformer-weights-path` | Model-specific support such as Qwen-Image, FLUX, and Z-Image | `nunchaku` | SGLang can infer precision and rank from the filename and supports both `int4` and `nvfp4` |
| `msmodelslim` | Pre-quantized msmodelslim transformer weights | `--model-path` | Wan2.2 family | None | Currently only compatible with the Ascend NPU family and supports both `w8a8` and `w4a4` |
## ModelOpt FP8
### Usage Examples
ModelOpt FP8 checkpoints should be converted into an SGLang-loadable transformer
override first, then loaded with the original base model:
```bash
sglang generate \
--model-path black-forest-labs/FLUX.2-dev \
--transformer-weights-path BBuf/flux2-dev-modelopt-fp8-sglang-transformer \
--prompt "A Logo With Bold Large Text: SGL Diffusion" \
--save-output
```
```bash
sglang generate \
--model-path Wan-AI/Wan2.2-T2V-A14B-Diffusers \
--transformer-weights-path BBuf/wan22-t2v-a14b-modelopt-fp8-sglang-transformer \
--prompt "a fox walking through neon rain" \
--save-output
```
### Notes
- `--transformer-weights-path` is the canonical flag for converted ModelOpt FP8
diffusion checkpoints.
- If the override repo or local directory contains its own `config.json`,
SGLang reads the quantization config from that override instead of relying on
the base model config.
- `dit_cpu_offload` and `dit_layerwise_offload` are automatically disabled for
ModelOpt FP8 checkpoints because the runtime expects the transformed FP8
weights to remain GPU-resident in their column-major layout.
- To build the converted checkpoint yourself from a ModelOpt diffusers export,
use `python -m sglang.multimodal_gen.tools.convert_modelopt_fp8_checkpoint`.
## NVFP4
### Usage Examples