Add HunyuanVideo ModelOpt FP8 diffusion support (#23199)

This commit is contained in:
Xiaoyu Zhang
2026-05-05 19:27:28 +08:00
committed by GitHub
parent 80ccb6b93c
commit 8c703f215e
9 changed files with 300 additions and 48 deletions
@@ -76,7 +76,7 @@ backend.
<td><code>modelopt-fp8</code></td>
<td>Converted ModelOpt FP8 transformer directory or repo with <code>config.json</code></td>
<td><code>--transformer-path</code></td>
<td>FLUX.1, FLUX.2, Wan2.2</td>
<td>FLUX.1, FLUX.2, Wan2.2, HunyuanVideo, Qwen Image, Qwen Image Edit</td>
<td>None</td>
<td>Serialized config stays <code>quant_method=modelopt</code> with <code>quant_algo=FP8</code>; <code>dit_layerwise_offload</code> is supported and <code>dit_cpu_offload</code> stays disabled</td>
</tr>
@@ -109,14 +109,14 @@ backend.
## Validated ModelOpt Checkpoints
This section is the canonical support matrix for the eight diffusion ModelOpt
This section is the canonical support matrix for the nine diffusion ModelOpt
checkpoints currently wired up in SGLang docs and B200 CI coverage.
Published checkpoints keep the serialized quantization config as
`quant_method=modelopt`; the FP8 vs NVFP4 split below is a documentation label
derived from `quant_algo`.
Seven of the eight repos live under `lmsys/*`. The FLUX.2 NVFP4 entry keeps the
Eight of the nine repos live under `lmsys/*`. The FLUX.2 NVFP4 entry keeps the
official `black-forest-labs/FLUX.2-dev-NVFP4` repo.
<table style={{width: "100%", borderCollapse: "collapse", tableLayout: "fixed"}}>
@@ -163,6 +163,14 @@ official `black-forest-labs/FLUX.2-dev-NVFP4` repo.
<td>primary <code>transformer</code> quantized, <code>transformer_2</code> kept BF16</td>
<td>primary-transformer-only path; keep <code>transformer_2</code> on the base checkpoint, and do not describe this as dual-transformer full-model FP8 unless that path is validated separately</td>
</tr>
<tr>
<td><code>FP8</code></td>
<td><code>hunyuanvideo-community/HunyuanVideo</code></td>
<td><code>--transformer-path</code></td>
<td><code>lmsys/hunyuanvideo-modelopt-fp8-sglang-transformer</code></td>
<td>single-transformer override, BF16-vs-FP8 video comparison, H100 benchmark, torch-profiler trace</td>
<td>HunyuanVideo uses different ModelOpt/diffusers and SGLang runtime module names; the converter maps those names before writing FP8 scale tensors and BF16 fallback ignores</td>
</tr>
<tr>
<td><code>FP8</code></td>
<td><code>Qwen/Qwen-Image</code></td>
@@ -176,7 +184,7 @@ official `black-forest-labs/FLUX.2-dev-NVFP4` repo.
<td><code>Qwen/Qwen-Image-Edit-2511</code></td>
<td><code>--transformer-path</code></td>
<td><code>lmsys/qwen-image-edit-modelopt-fp8-sglang-transformer</code></td>
<td>TI2I edit smoke, BF16-vs-FP8 image comparison, H100 benchmark</td>
<td>TI2I edit path, BF16-vs-FP8 image comparison, H100 benchmark</td>
<td>shares <code>QwenImageTransformer2DModel</code> with Qwen Image and uses the same Qwen Image FP8 fallback preset</td>
</tr>
<tr>
@@ -206,7 +214,7 @@ official `black-forest-labs/FLUX.2-dev-NVFP4` repo.
</tbody>
</table>
These eight checkpoints are also the intended case set for the B200 diffusion CI
These nine checkpoints are also the intended case set for the B200 diffusion CI
job (`multimodal-gen-test-1-b200`).
## ModelOpt FP8
@@ -233,6 +241,15 @@ sglang generate \
--save-output
```
```bash
sglang generate \
--model-path hunyuanvideo-community/HunyuanVideo \
--transformer-path lmsys/hunyuanvideo-modelopt-fp8-sglang-transformer \
--height 544 --width 960 --num-frames 17 \
--prompt "A cinematic shot of a red sports car driving through rain at night" \
--save-output
```
```bash
sglang generate \
--model-path Qwen/Qwen-Image \