Add HunyuanVideo ModelOpt FP8 diffusion support (#23199)
This commit is contained in:
@@ -76,7 +76,7 @@ backend.
|
||||
<td><code>modelopt-fp8</code></td>
|
||||
<td>Converted ModelOpt FP8 transformer directory or repo with <code>config.json</code></td>
|
||||
<td><code>--transformer-path</code></td>
|
||||
<td>FLUX.1, FLUX.2, Wan2.2</td>
|
||||
<td>FLUX.1, FLUX.2, Wan2.2, HunyuanVideo, Qwen Image, Qwen Image Edit</td>
|
||||
<td>None</td>
|
||||
<td>Serialized config stays <code>quant_method=modelopt</code> with <code>quant_algo=FP8</code>; <code>dit_layerwise_offload</code> is supported and <code>dit_cpu_offload</code> stays disabled</td>
|
||||
</tr>
|
||||
@@ -109,14 +109,14 @@ backend.
|
||||
|
||||
## Validated ModelOpt Checkpoints
|
||||
|
||||
This section is the canonical support matrix for the eight diffusion ModelOpt
|
||||
This section is the canonical support matrix for the nine diffusion ModelOpt
|
||||
checkpoints currently wired up in SGLang docs and B200 CI coverage.
|
||||
|
||||
Published checkpoints keep the serialized quantization config as
|
||||
`quant_method=modelopt`; the FP8 vs NVFP4 split below is a documentation label
|
||||
derived from `quant_algo`.
|
||||
|
||||
Seven of the eight repos live under `lmsys/*`. The FLUX.2 NVFP4 entry keeps the
|
||||
Eight of the nine repos live under `lmsys/*`. The FLUX.2 NVFP4 entry keeps the
|
||||
official `black-forest-labs/FLUX.2-dev-NVFP4` repo.
|
||||
|
||||
<table style={{width: "100%", borderCollapse: "collapse", tableLayout: "fixed"}}>
|
||||
@@ -163,6 +163,14 @@ official `black-forest-labs/FLUX.2-dev-NVFP4` repo.
|
||||
<td>primary <code>transformer</code> quantized, <code>transformer_2</code> kept BF16</td>
|
||||
<td>primary-transformer-only path; keep <code>transformer_2</code> on the base checkpoint, and do not describe this as dual-transformer full-model FP8 unless that path is validated separately</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>FP8</code></td>
|
||||
<td><code>hunyuanvideo-community/HunyuanVideo</code></td>
|
||||
<td><code>--transformer-path</code></td>
|
||||
<td><code>lmsys/hunyuanvideo-modelopt-fp8-sglang-transformer</code></td>
|
||||
<td>single-transformer override, BF16-vs-FP8 video comparison, H100 benchmark, torch-profiler trace</td>
|
||||
<td>HunyuanVideo uses different ModelOpt/diffusers and SGLang runtime module names; the converter maps those names before writing FP8 scale tensors and BF16 fallback ignores</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>FP8</code></td>
|
||||
<td><code>Qwen/Qwen-Image</code></td>
|
||||
@@ -176,7 +184,7 @@ official `black-forest-labs/FLUX.2-dev-NVFP4` repo.
|
||||
<td><code>Qwen/Qwen-Image-Edit-2511</code></td>
|
||||
<td><code>--transformer-path</code></td>
|
||||
<td><code>lmsys/qwen-image-edit-modelopt-fp8-sglang-transformer</code></td>
|
||||
<td>TI2I edit smoke, BF16-vs-FP8 image comparison, H100 benchmark</td>
|
||||
<td>TI2I edit path, BF16-vs-FP8 image comparison, H100 benchmark</td>
|
||||
<td>shares <code>QwenImageTransformer2DModel</code> with Qwen Image and uses the same Qwen Image FP8 fallback preset</td>
|
||||
</tr>
|
||||
<tr>
|
||||
@@ -206,7 +214,7 @@ official `black-forest-labs/FLUX.2-dev-NVFP4` repo.
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
These eight checkpoints are also the intended case set for the B200 diffusion CI
|
||||
These nine checkpoints are also the intended case set for the B200 diffusion CI
|
||||
job (`multimodal-gen-test-1-b200`).
|
||||
|
||||
## ModelOpt FP8
|
||||
@@ -233,6 +241,15 @@ sglang generate \
|
||||
--save-output
|
||||
```
|
||||
|
||||
```bash
|
||||
sglang generate \
|
||||
--model-path hunyuanvideo-community/HunyuanVideo \
|
||||
--transformer-path lmsys/hunyuanvideo-modelopt-fp8-sglang-transformer \
|
||||
--height 544 --width 960 --num-frames 17 \
|
||||
--prompt "A cinematic shot of a red sports car driving through rain at night" \
|
||||
--save-output
|
||||
```
|
||||
|
||||
```bash
|
||||
sglang generate \
|
||||
--model-path Qwen/Qwen-Image \
|
||||
|
||||
Reference in New Issue
Block a user