[Diffusion] Add Qwen Image ModelOpt FP8 support (#23155)

Co-authored-by: Mick <mickjagger19@icloud.com>
This commit is contained in:
Xiaoyu Zhang
2026-05-04 00:24:22 +08:00
committed by GitHub
co-authored by Mick
parent 5925572c95
commit f2d1390909
7 changed files with 273 additions and 36 deletions
@@ -109,14 +109,14 @@ backend.
## Validated ModelOpt Checkpoints
This section is the canonical support matrix for the six diffusion ModelOpt
This section is the canonical support matrix for the eight diffusion ModelOpt
checkpoints currently wired up in SGLang docs and B200 CI coverage.
Published checkpoints keep the serialized quantization config as
`quant_method=modelopt`; the FP8 vs NVFP4 split below is a documentation label
derived from `quant_algo`.
Five of the six repos live under `lmsys/*`. The FLUX.2 NVFP4 entry keeps the
Seven of the eight repos live under `lmsys/*`. The FLUX.2 NVFP4 entry keeps the
official `black-forest-labs/FLUX.2-dev-NVFP4` repo.
<table style={{width: "100%", borderCollapse: "collapse", tableLayout: "fixed"}}>
@@ -163,6 +163,22 @@ official `black-forest-labs/FLUX.2-dev-NVFP4` repo.
<td>primary <code>transformer</code> quantized, <code>transformer_2</code> kept BF16</td>
<td>primary-transformer-only path; keep <code>transformer_2</code> on the base checkpoint, and do not describe this as dual-transformer full-model FP8 unless that path is validated separately</td>
</tr>
<tr>
<td><code>FP8</code></td>
<td><code>Qwen/Qwen-Image</code></td>
<td><code>--transformer-path</code></td>
<td><code>lmsys/qwen-image-modelopt-fp8-sglang-transformer</code></td>
<td>single-transformer override, BF16-vs-FP8 image comparison, H100 benchmark, torch-profiler trace</td>
<td>shares the Qwen Image FP8 fallback preset; keep <code>img_in</code>, <code>txt_in</code>, timestep embedder, <code>norm_out.linear</code>, <code>proj_out</code>, <code>img_mod</code>/<code>txt_mod</code>, and <code>img_mlp.net.2</code> in BF16</td>
</tr>
<tr>
<td><code>FP8</code></td>
<td><code>Qwen/Qwen-Image-Edit-2511</code></td>
<td><code>--transformer-path</code></td>
<td><code>lmsys/qwen-image-edit-modelopt-fp8-sglang-transformer</code></td>
<td>TI2I edit smoke, BF16-vs-FP8 image comparison, H100 benchmark</td>
<td>shares <code>QwenImageTransformer2DModel</code> with Qwen Image and uses the same Qwen Image FP8 fallback preset</td>
</tr>
<tr>
<td><code>NVFP4</code></td>
<td><code>black-forest-labs/FLUX.1-dev</code></td>
@@ -190,7 +206,7 @@ official `black-forest-labs/FLUX.2-dev-NVFP4` repo.
</tbody>
</table>
These six checkpoints are also the intended case set for the B200 diffusion CI
These eight checkpoints are also the intended case set for the B200 diffusion CI
job (`multimodal-gen-test-1-b200`).
## ModelOpt FP8
@@ -217,6 +233,23 @@ sglang generate \
--save-output
```
```bash
sglang generate \
--model-path Qwen/Qwen-Image \
--transformer-path lmsys/qwen-image-modelopt-fp8-sglang-transformer \
--prompt "A tiny astronaut reading a book under a glass greenhouse" \
--save-output
```
```bash
sglang generate \
--model-path Qwen/Qwen-Image-Edit-2511 \
--transformer-path lmsys/qwen-image-edit-modelopt-fp8-sglang-transformer \
--image-path /path/to/input.png \
--prompt "Turn the scene into a warm watercolor illustration" \
--save-output
```
### Notes
- `--transformer-path` is the canonical flag for converted ModelOpt FP8