minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

[diffusion] chore: use lmsys as org for modelopt checkpoints (#23924)

Browse Source
This commit is contained in:
Xiaoyu Zhang
2026-05-02 17:18:58 +08:00
committed by GitHub
parent f3dbadb82b
commit 589f90b368
4 changed files with 26 additions and 26 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/diffusion/quantization.md
+10 -10
View File
@@ -57,17 +57,17 @@ Published checkpoints keep the serialized quantization config as
`quant_method=modelopt`; the FP8 vs NVFP4 split below is a documentation label
derived from `quant_algo`.
Five of the six repos live under `BBuf/*`. The FLUX.2 NVFP4 entry keeps the
Five of the six repos live under `lmsys/*`. The FLUX.2 NVFP4 entry keeps the
official `black-forest-labs/FLUX.2-dev-NVFP4` repo.
| Quant Algo | Base Model | Preferred CLI | HF Repo | Current Scope | Notes |
| --- | --- | --- | --- | --- | --- |
| `FP8` | `black-forest-labs/FLUX.1-dev` | `--transformer-path` | `BBuf/flux1-dev-modelopt-fp8-sglang-transformer` | single-transformer override, deterministic latent/image comparison, H100 benchmark, torch-profiler trace | SGLang converter keeps a validated BF16 fallback set for modulation and FF projection layers; use `--model-id FLUX.1-dev` for local mirrors |
| `FP8` | `black-forest-labs/FLUX.2-dev` | `--transformer-path` | `BBuf/flux2-dev-modelopt-fp8-sglang-transformer` | single-transformer override load and generation path | published SGLang-ready transformer override |
| `FP8` | `Wan-AI/Wan2.2-T2V-A14B-Diffusers` | `--transformer-path` | `BBuf/wan22-t2v-a14b-modelopt-fp8-sglang-transformer` | primary `transformer` quantized, `transformer_2` kept BF16 | primary-transformer-only path; keep `transformer_2` on the base checkpoint, and do not describe this as dual-transformer full-model FP8 unless that path is validated separately |
| `NVFP4` | `black-forest-labs/FLUX.1-dev` | `--transformer-path` | `BBuf/flux1-dev-modelopt-nvfp4-sglang-transformer` | mixed BF16+NVFP4 transformer override, correctness validation, 4x RTX 5090 benchmark, torch-profiler trace | use `build_modelopt_nvfp4_transformer.py`; validated builder keeps selected FLUX.1 modules in BF16 and sets `swap_weight_nibbles=false` |
| `FP8` | `black-forest-labs/FLUX.1-dev` | `--transformer-path` | `lmsys/flux1-dev-modelopt-fp8-sglang-transformer` | single-transformer override, deterministic latent/image comparison, H100 benchmark, torch-profiler trace | SGLang converter keeps a validated BF16 fallback set for modulation and FF projection layers; use `--model-id FLUX.1-dev` for local mirrors |
| `FP8` | `black-forest-labs/FLUX.2-dev` | `--transformer-path` | `lmsys/flux2-dev-modelopt-fp8-sglang-transformer` | single-transformer override load and generation path | published SGLang-ready transformer override |
| `FP8` | `Wan-AI/Wan2.2-T2V-A14B-Diffusers` | `--transformer-path` | `lmsys/wan22-t2v-a14b-modelopt-fp8-sglang-transformer` | primary `transformer` quantized, `transformer_2` kept BF16 | primary-transformer-only path; keep `transformer_2` on the base checkpoint, and do not describe this as dual-transformer full-model FP8 unless that path is validated separately |
| `NVFP4` | `black-forest-labs/FLUX.1-dev` | `--transformer-path` | `lmsys/flux1-dev-modelopt-nvfp4-sglang-transformer` | mixed BF16+NVFP4 transformer override, correctness validation, 4x RTX 5090 benchmark, torch-profiler trace | use `build_modelopt_nvfp4_transformer.py`; validated builder keeps selected FLUX.1 modules in BF16 and sets `swap_weight_nibbles=false` |
| `NVFP4` | `black-forest-labs/FLUX.2-dev` | `--transformer-weights-path` | `black-forest-labs/FLUX.2-dev-NVFP4` | packed-QKV load path | official raw export repo; validated packed export detection and runtime layout handling |
| `NVFP4` | `Wan-AI/Wan2.2-T2V-A14B-Diffusers` | `--transformer-path` | `BBuf/wan22-t2v-a14b-modelopt-nvfp4-sglang-transformer` | primary `transformer` quantized with ModelOpt NVFP4, `transformer_2` kept BF16 | primary-transformer-only path; keep `transformer_2` on the base checkpoint, and current B200/Blackwell bring-up uses `SGLANG_DIFFUSION_FLASHINFER_FP4_GEMM_BACKEND=cudnn` |
| `NVFP4` | `Wan-AI/Wan2.2-T2V-A14B-Diffusers` | `--transformer-path` | `lmsys/wan22-t2v-a14b-modelopt-nvfp4-sglang-transformer` | primary `transformer` quantized with ModelOpt NVFP4, `transformer_2` kept BF16 | primary-transformer-only path; keep `transformer_2` on the base checkpoint, and current B200/Blackwell bring-up uses `SGLANG_DIFFUSION_FLASHINFER_FP4_GEMM_BACKEND=cudnn` |
These six checkpoints are also the intended case set for the B200 diffusion CI
job (`multimodal-gen-test-1-b200`).
@@ -83,7 +83,7 @@ overrides. If the repo or local directory already contains `config.json`, use
```bash
sglang generate \
--model-path black-forest-labs/FLUX.2-dev \
--transformer-path BBuf/flux2-dev-modelopt-fp8-sglang-transformer \
--transformer-path lmsys/flux2-dev-modelopt-fp8-sglang-transformer \
--prompt "A Logo With Bold Large Text: SGL Diffusion" \
--save-output
```
@@ -91,7 +91,7 @@ sglang generate \
```bash
sglang generate \
--model-path Wan-AI/Wan2.2-T2V-A14B-Diffusers \
--transformer-path BBuf/wan22-t2v-a14b-modelopt-fp8-sglang-transformer \
--transformer-path lmsys/wan22-t2v-a14b-modelopt-fp8-sglang-transformer \
--prompt "a fox walking through neon rain" \
--save-output
```
@@ -126,7 +126,7 @@ For mixed ModelOpt NVFP4 transformer overrides that already contain
```bash
sglang generate \
--model-path black-forest-labs/FLUX.1-dev \
--transformer-path BBuf/flux1-dev-modelopt-nvfp4-sglang-transformer \
--transformer-path lmsys/flux1-dev-modelopt-nvfp4-sglang-transformer \
--prompt "A Logo With Bold Large Text: SGL Diffusion" \
--save-output
```
@@ -159,7 +159,7 @@ was quantized:
SGLANG_DIFFUSION_FLASHINFER_FP4_GEMM_BACKEND=cudnn \
sglang generate \
--model-path Wan-AI/Wan2.2-T2V-A14B-Diffusers \
--transformer-path BBuf/wan22-t2v-a14b-modelopt-nvfp4-sglang-transformer \
--transformer-path lmsys/wan22-t2v-a14b-modelopt-nvfp4-sglang-transformer \
--prompt "a fox walking through neon rain" \
--save-output
```
docs_new/docs/sglang-diffusion/quantization.mdx
+10 -10
View File
@@ -116,7 +116,7 @@ Published checkpoints keep the serialized quantization config as
`quant_method=modelopt`; the FP8 vs NVFP4 split below is a documentation label
derived from `quant_algo`.
Five of the six repos live under `BBuf/*`. The FLUX.2 NVFP4 entry keeps the
Five of the six repos live under `lmsys/*`. The FLUX.2 NVFP4 entry keeps the
official `black-forest-labs/FLUX.2-dev-NVFP4` repo.
<table style={{width: "100%", borderCollapse: "collapse", tableLayout: "fixed"}}>
@@ -143,7 +143,7 @@ official `black-forest-labs/FLUX.2-dev-NVFP4` repo.
<td><code>FP8</code></td>
<td><code>black-forest-labs/FLUX.1-dev</code></td>
<td><code>--transformer-path</code></td>
<td><code>BBuf/flux1-dev-modelopt-fp8-sglang-transformer</code></td>
<td><code>lmsys/flux1-dev-modelopt-fp8-sglang-transformer</code></td>
<td>single-transformer override, deterministic latent/image comparison, H100 benchmark, torch-profiler trace</td>
<td>SGLang converter keeps a validated BF16 fallback set for modulation and FF projection layers; use <code>--model-id FLUX.1-dev</code> for local mirrors</td>
</tr>
@@ -151,7 +151,7 @@ official `black-forest-labs/FLUX.2-dev-NVFP4` repo.
<td><code>FP8</code></td>
<td><code>black-forest-labs/FLUX.2-dev</code></td>
<td><code>--transformer-path</code></td>
<td><code>BBuf/flux2-dev-modelopt-fp8-sglang-transformer</code></td>
<td><code>lmsys/flux2-dev-modelopt-fp8-sglang-transformer</code></td>
<td>single-transformer override load and generation path</td>
<td>published SGLang-ready transformer override</td>
</tr>
@@ -159,7 +159,7 @@ official `black-forest-labs/FLUX.2-dev-NVFP4` repo.
<td><code>FP8</code></td>
<td><code>Wan-AI/Wan2.2-T2V-A14B-Diffusers</code></td>
<td><code>--transformer-path</code></td>
<td><code>BBuf/wan22-t2v-a14b-modelopt-fp8-sglang-transformer</code></td>
<td><code>lmsys/wan22-t2v-a14b-modelopt-fp8-sglang-transformer</code></td>
<td>primary <code>transformer</code> quantized, <code>transformer_2</code> kept BF16</td>
<td>primary-transformer-only path; keep <code>transformer_2</code> on the base checkpoint, and do not describe this as dual-transformer full-model FP8 unless that path is validated separately</td>
</tr>
@@ -167,7 +167,7 @@ official `black-forest-labs/FLUX.2-dev-NVFP4` repo.
<td><code>NVFP4</code></td>
<td><code>black-forest-labs/FLUX.1-dev</code></td>
<td><code>--transformer-path</code></td>
<td><code>BBuf/flux1-dev-modelopt-nvfp4-sglang-transformer</code></td>
<td><code>lmsys/flux1-dev-modelopt-nvfp4-sglang-transformer</code></td>
<td>mixed BF16+NVFP4 transformer override, correctness validation, 4x RTX 5090 benchmark, torch-profiler trace</td>
<td>use <code>build_modelopt_nvfp4_transformer.py</code>; validated builder keeps selected FLUX.1 modules in BF16 and sets <code>swap_weight_nibbles=false</code></td>
</tr>
@@ -183,7 +183,7 @@ official `black-forest-labs/FLUX.2-dev-NVFP4` repo.
<td><code>NVFP4</code></td>
<td><code>Wan-AI/Wan2.2-T2V-A14B-Diffusers</code></td>
<td><code>--transformer-path</code></td>
<td><code>BBuf/wan22-t2v-a14b-modelopt-nvfp4-sglang-transformer</code></td>
<td><code>lmsys/wan22-t2v-a14b-modelopt-nvfp4-sglang-transformer</code></td>
<td>primary <code>transformer</code> quantized with ModelOpt NVFP4, <code>transformer_2</code> kept BF16</td>
<td>primary-transformer-only path; keep <code>transformer_2</code> on the base checkpoint, and current B200/Blackwell bring-up uses <code>SGLANG_DIFFUSION_FLASHINFER_FP4_GEMM_BACKEND=cudnn</code></td>
</tr>
@@ -204,7 +204,7 @@ overrides. If the repo or local directory already contains `config.json`, use
```bash
sglang generate \
--model-path black-forest-labs/FLUX.2-dev \
--transformer-path BBuf/flux2-dev-modelopt-fp8-sglang-transformer \
--transformer-path lmsys/flux2-dev-modelopt-fp8-sglang-transformer \
--prompt "A Logo With Bold Large Text: SGL Diffusion" \
--save-output
```
@@ -212,7 +212,7 @@ sglang generate \
```bash
sglang generate \
--model-path Wan-AI/Wan2.2-T2V-A14B-Diffusers \
--transformer-path BBuf/wan22-t2v-a14b-modelopt-fp8-sglang-transformer \
--transformer-path lmsys/wan22-t2v-a14b-modelopt-fp8-sglang-transformer \
--prompt "a fox walking through neon rain" \
--save-output
```
@@ -247,7 +247,7 @@ For mixed ModelOpt NVFP4 transformer overrides that already contain
```bash
sglang generate \
--model-path black-forest-labs/FLUX.1-dev \
--transformer-path BBuf/flux1-dev-modelopt-nvfp4-sglang-transformer \
--transformer-path lmsys/flux1-dev-modelopt-nvfp4-sglang-transformer \
--prompt "A Logo With Bold Large Text: SGL Diffusion" \
--save-output
```
@@ -280,7 +280,7 @@ was quantized:
SGLANG_DIFFUSION_FLASHINFER_FP4_GEMM_BACKEND=cudnn \
sglang generate \
--model-path Wan-AI/Wan2.2-T2V-A14B-Diffusers \
--transformer-path BBuf/wan22-t2v-a14b-modelopt-nvfp4-sglang-transformer \
--transformer-path lmsys/wan22-t2v-a14b-modelopt-nvfp4-sglang-transformer \
--prompt "a fox walking through neon rain" \
--save-output
```
python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-modelopt-quant/SKILL.md
+1 -1
View File
@@ -67,7 +67,7 @@ Validated documentation and CI coverage currently center on six ModelOpt diffusi
- NVFP4: FLUX.1-dev, FLUX.2-dev, Wan2.2
Treat a new family, a new precision, or a new checkpoint layout as unsupported until it has a documented matrix row and a matching validation story.
Before writing CLI examples, re-read the active branch's `docs/diffusion/quantization.md`: FLUX.2 NVFP4 is an official `black-forest-labs/*` repo rather than a `BBuf/*` converted repo, and its preferred flag depends on the current documented loader flow. Use `--transformer-path` for a component override directory with `config.json`; use `--transformer-weights-path` when the repo or path should be probed as raw weights.
Before writing CLI examples, re-read the active branch's `docs/diffusion/quantization.md`: FLUX.2 NVFP4 is an official `black-forest-labs/*` repo rather than a `lmsys/*` converted repo, and its preferred flag depends on the current documented loader flow. Use `--transformer-path` for a component override directory with `config.json`; use `--transformer-weights-path` when the repo or path should be probed as raw weights.
B200 CI coverage can include loose BF16-vs-quantized quality checks. Inspect the active branch's `run_suite.py` before assuming they are part of the suite; mainline and feature branches may differ. Those checks are intended to catch blank, corrupted, or obviously divergent images, not exact image parity.
python/sglang/multimodal_gen/test/server/testcase_configs.py
+5 -5
View File
@@ -431,13 +431,13 @@ HUNYUAN3D_SHAPE_sampling_params = DiffusionSamplingParams(
image_path="https://raw.githubusercontent.com/sgl-project/sgl-test-files/main/diffusion-ci/consistency_gt/1-gpu/hunyuan3d_2_0/hunyuan3d.png",
)
MODELOPT_FLUX1_FP8_TRANSFORMER = "BBuf/flux1-dev-modelopt-fp8-sglang-transformer"
MODELOPT_FLUX2_FP8_TRANSFORMER = "BBuf/flux2-dev-modelopt-fp8-sglang-transformer"
MODELOPT_WAN22_FP8_TRANSFORMER = "BBuf/wan22-t2v-a14b-modelopt-fp8-sglang-transformer"
MODELOPT_FLUX1_NVFP4_TRANSFORMER = "BBuf/flux1-dev-modelopt-nvfp4-sglang-transformer"
MODELOPT_FLUX1_FP8_TRANSFORMER = "lmsys/flux1-dev-modelopt-fp8-sglang-transformer"
MODELOPT_FLUX2_FP8_TRANSFORMER = "lmsys/flux2-dev-modelopt-fp8-sglang-transformer"
MODELOPT_WAN22_FP8_TRANSFORMER = "lmsys/wan22-t2v-a14b-modelopt-fp8-sglang-transformer"
MODELOPT_FLUX1_NVFP4_TRANSFORMER = "lmsys/flux1-dev-modelopt-nvfp4-sglang-transformer"
MODELOPT_FLUX2_NVFP4_WEIGHTS = "black-forest-labs/FLUX.2-dev-NVFP4"
MODELOPT_WAN22_NVFP4_TRANSFORMER = (
"BBuf/wan22-t2v-a14b-modelopt-nvfp4-sglang-transformer"
"lmsys/wan22-t2v-a14b-modelopt-nvfp4-sglang-transformer"
)
MODELOPT_NVFP4_B200_ENV_VARS = {"SGLANG_DIFFUSION_FLASHINFER_FP4_GEMM_BACKEND": "cudnn"}