diff --git a/docs/diffusion/quantization.md b/docs/diffusion/quantization.md index 970b1ee5d..b88f4f827 100644 --- a/docs/diffusion/quantization.md +++ b/docs/diffusion/quantization.md @@ -57,17 +57,17 @@ Published checkpoints keep the serialized quantization config as `quant_method=modelopt`; the FP8 vs NVFP4 split below is a documentation label derived from `quant_algo`. -Five of the six repos live under `BBuf/*`. The FLUX.2 NVFP4 entry keeps the +Five of the six repos live under `lmsys/*`. The FLUX.2 NVFP4 entry keeps the official `black-forest-labs/FLUX.2-dev-NVFP4` repo. | Quant Algo | Base Model | Preferred CLI | HF Repo | Current Scope | Notes | | --- | --- | --- | --- | --- | --- | -| `FP8` | `black-forest-labs/FLUX.1-dev` | `--transformer-path` | `BBuf/flux1-dev-modelopt-fp8-sglang-transformer` | single-transformer override, deterministic latent/image comparison, H100 benchmark, torch-profiler trace | SGLang converter keeps a validated BF16 fallback set for modulation and FF projection layers; use `--model-id FLUX.1-dev` for local mirrors | -| `FP8` | `black-forest-labs/FLUX.2-dev` | `--transformer-path` | `BBuf/flux2-dev-modelopt-fp8-sglang-transformer` | single-transformer override load and generation path | published SGLang-ready transformer override | -| `FP8` | `Wan-AI/Wan2.2-T2V-A14B-Diffusers` | `--transformer-path` | `BBuf/wan22-t2v-a14b-modelopt-fp8-sglang-transformer` | primary `transformer` quantized, `transformer_2` kept BF16 | primary-transformer-only path; keep `transformer_2` on the base checkpoint, and do not describe this as dual-transformer full-model FP8 unless that path is validated separately | -| `NVFP4` | `black-forest-labs/FLUX.1-dev` | `--transformer-path` | `BBuf/flux1-dev-modelopt-nvfp4-sglang-transformer` | mixed BF16+NVFP4 transformer override, correctness validation, 4x RTX 5090 benchmark, torch-profiler trace | use `build_modelopt_nvfp4_transformer.py`; validated builder keeps selected FLUX.1 modules in BF16 and sets `swap_weight_nibbles=false` | +| `FP8` | `black-forest-labs/FLUX.1-dev` | `--transformer-path` | `lmsys/flux1-dev-modelopt-fp8-sglang-transformer` | single-transformer override, deterministic latent/image comparison, H100 benchmark, torch-profiler trace | SGLang converter keeps a validated BF16 fallback set for modulation and FF projection layers; use `--model-id FLUX.1-dev` for local mirrors | +| `FP8` | `black-forest-labs/FLUX.2-dev` | `--transformer-path` | `lmsys/flux2-dev-modelopt-fp8-sglang-transformer` | single-transformer override load and generation path | published SGLang-ready transformer override | +| `FP8` | `Wan-AI/Wan2.2-T2V-A14B-Diffusers` | `--transformer-path` | `lmsys/wan22-t2v-a14b-modelopt-fp8-sglang-transformer` | primary `transformer` quantized, `transformer_2` kept BF16 | primary-transformer-only path; keep `transformer_2` on the base checkpoint, and do not describe this as dual-transformer full-model FP8 unless that path is validated separately | +| `NVFP4` | `black-forest-labs/FLUX.1-dev` | `--transformer-path` | `lmsys/flux1-dev-modelopt-nvfp4-sglang-transformer` | mixed BF16+NVFP4 transformer override, correctness validation, 4x RTX 5090 benchmark, torch-profiler trace | use `build_modelopt_nvfp4_transformer.py`; validated builder keeps selected FLUX.1 modules in BF16 and sets `swap_weight_nibbles=false` | | `NVFP4` | `black-forest-labs/FLUX.2-dev` | `--transformer-weights-path` | `black-forest-labs/FLUX.2-dev-NVFP4` | packed-QKV load path | official raw export repo; validated packed export detection and runtime layout handling | -| `NVFP4` | `Wan-AI/Wan2.2-T2V-A14B-Diffusers` | `--transformer-path` | `BBuf/wan22-t2v-a14b-modelopt-nvfp4-sglang-transformer` | primary `transformer` quantized with ModelOpt NVFP4, `transformer_2` kept BF16 | primary-transformer-only path; keep `transformer_2` on the base checkpoint, and current B200/Blackwell bring-up uses `SGLANG_DIFFUSION_FLASHINFER_FP4_GEMM_BACKEND=cudnn` | +| `NVFP4` | `Wan-AI/Wan2.2-T2V-A14B-Diffusers` | `--transformer-path` | `lmsys/wan22-t2v-a14b-modelopt-nvfp4-sglang-transformer` | primary `transformer` quantized with ModelOpt NVFP4, `transformer_2` kept BF16 | primary-transformer-only path; keep `transformer_2` on the base checkpoint, and current B200/Blackwell bring-up uses `SGLANG_DIFFUSION_FLASHINFER_FP4_GEMM_BACKEND=cudnn` | These six checkpoints are also the intended case set for the B200 diffusion CI job (`multimodal-gen-test-1-b200`). @@ -83,7 +83,7 @@ overrides. If the repo or local directory already contains `config.json`, use ```bash sglang generate \ --model-path black-forest-labs/FLUX.2-dev \ - --transformer-path BBuf/flux2-dev-modelopt-fp8-sglang-transformer \ + --transformer-path lmsys/flux2-dev-modelopt-fp8-sglang-transformer \ --prompt "A Logo With Bold Large Text: SGL Diffusion" \ --save-output ``` @@ -91,7 +91,7 @@ sglang generate \ ```bash sglang generate \ --model-path Wan-AI/Wan2.2-T2V-A14B-Diffusers \ - --transformer-path BBuf/wan22-t2v-a14b-modelopt-fp8-sglang-transformer \ + --transformer-path lmsys/wan22-t2v-a14b-modelopt-fp8-sglang-transformer \ --prompt "a fox walking through neon rain" \ --save-output ``` @@ -126,7 +126,7 @@ For mixed ModelOpt NVFP4 transformer overrides that already contain ```bash sglang generate \ --model-path black-forest-labs/FLUX.1-dev \ - --transformer-path BBuf/flux1-dev-modelopt-nvfp4-sglang-transformer \ + --transformer-path lmsys/flux1-dev-modelopt-nvfp4-sglang-transformer \ --prompt "A Logo With Bold Large Text: SGL Diffusion" \ --save-output ``` @@ -159,7 +159,7 @@ was quantized: SGLANG_DIFFUSION_FLASHINFER_FP4_GEMM_BACKEND=cudnn \ sglang generate \ --model-path Wan-AI/Wan2.2-T2V-A14B-Diffusers \ - --transformer-path BBuf/wan22-t2v-a14b-modelopt-nvfp4-sglang-transformer \ + --transformer-path lmsys/wan22-t2v-a14b-modelopt-nvfp4-sglang-transformer \ --prompt "a fox walking through neon rain" \ --save-output ``` diff --git a/docs_new/docs/sglang-diffusion/quantization.mdx b/docs_new/docs/sglang-diffusion/quantization.mdx index 62d336f34..659164ca8 100644 --- a/docs_new/docs/sglang-diffusion/quantization.mdx +++ b/docs_new/docs/sglang-diffusion/quantization.mdx @@ -116,7 +116,7 @@ Published checkpoints keep the serialized quantization config as `quant_method=modelopt`; the FP8 vs NVFP4 split below is a documentation label derived from `quant_algo`. -Five of the six repos live under `BBuf/*`. The FLUX.2 NVFP4 entry keeps the +Five of the six repos live under `lmsys/*`. The FLUX.2 NVFP4 entry keeps the official `black-forest-labs/FLUX.2-dev-NVFP4` repo. @@ -143,7 +143,7 @@ official `black-forest-labs/FLUX.2-dev-NVFP4` repo. - + @@ -151,7 +151,7 @@ official `black-forest-labs/FLUX.2-dev-NVFP4` repo. - + @@ -159,7 +159,7 @@ official `black-forest-labs/FLUX.2-dev-NVFP4` repo. - + @@ -167,7 +167,7 @@ official `black-forest-labs/FLUX.2-dev-NVFP4` repo. - + @@ -183,7 +183,7 @@ official `black-forest-labs/FLUX.2-dev-NVFP4` repo. - + @@ -204,7 +204,7 @@ overrides. If the repo or local directory already contains `config.json`, use ```bash sglang generate \ --model-path black-forest-labs/FLUX.2-dev \ - --transformer-path BBuf/flux2-dev-modelopt-fp8-sglang-transformer \ + --transformer-path lmsys/flux2-dev-modelopt-fp8-sglang-transformer \ --prompt "A Logo With Bold Large Text: SGL Diffusion" \ --save-output ``` @@ -212,7 +212,7 @@ sglang generate \ ```bash sglang generate \ --model-path Wan-AI/Wan2.2-T2V-A14B-Diffusers \ - --transformer-path BBuf/wan22-t2v-a14b-modelopt-fp8-sglang-transformer \ + --transformer-path lmsys/wan22-t2v-a14b-modelopt-fp8-sglang-transformer \ --prompt "a fox walking through neon rain" \ --save-output ``` @@ -247,7 +247,7 @@ For mixed ModelOpt NVFP4 transformer overrides that already contain ```bash sglang generate \ --model-path black-forest-labs/FLUX.1-dev \ - --transformer-path BBuf/flux1-dev-modelopt-nvfp4-sglang-transformer \ + --transformer-path lmsys/flux1-dev-modelopt-nvfp4-sglang-transformer \ --prompt "A Logo With Bold Large Text: SGL Diffusion" \ --save-output ``` @@ -280,7 +280,7 @@ was quantized: SGLANG_DIFFUSION_FLASHINFER_FP4_GEMM_BACKEND=cudnn \ sglang generate \ --model-path Wan-AI/Wan2.2-T2V-A14B-Diffusers \ - --transformer-path BBuf/wan22-t2v-a14b-modelopt-nvfp4-sglang-transformer \ + --transformer-path lmsys/wan22-t2v-a14b-modelopt-nvfp4-sglang-transformer \ --prompt "a fox walking through neon rain" \ --save-output ``` diff --git a/python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-modelopt-quant/SKILL.md b/python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-modelopt-quant/SKILL.md index 4c396991c..8fbcbef12 100644 --- a/python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-modelopt-quant/SKILL.md +++ b/python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-modelopt-quant/SKILL.md @@ -67,7 +67,7 @@ Validated documentation and CI coverage currently center on six ModelOpt diffusi - NVFP4: FLUX.1-dev, FLUX.2-dev, Wan2.2 Treat a new family, a new precision, or a new checkpoint layout as unsupported until it has a documented matrix row and a matching validation story. -Before writing CLI examples, re-read the active branch's `docs/diffusion/quantization.md`: FLUX.2 NVFP4 is an official `black-forest-labs/*` repo rather than a `BBuf/*` converted repo, and its preferred flag depends on the current documented loader flow. Use `--transformer-path` for a component override directory with `config.json`; use `--transformer-weights-path` when the repo or path should be probed as raw weights. +Before writing CLI examples, re-read the active branch's `docs/diffusion/quantization.md`: FLUX.2 NVFP4 is an official `black-forest-labs/*` repo rather than a `lmsys/*` converted repo, and its preferred flag depends on the current documented loader flow. Use `--transformer-path` for a component override directory with `config.json`; use `--transformer-weights-path` when the repo or path should be probed as raw weights. B200 CI coverage can include loose BF16-vs-quantized quality checks. Inspect the active branch's `run_suite.py` before assuming they are part of the suite; mainline and feature branches may differ. Those checks are intended to catch blank, corrupted, or obviously divergent images, not exact image parity. diff --git a/python/sglang/multimodal_gen/test/server/testcase_configs.py b/python/sglang/multimodal_gen/test/server/testcase_configs.py index c25bb2d2f..9dcbff969 100644 --- a/python/sglang/multimodal_gen/test/server/testcase_configs.py +++ b/python/sglang/multimodal_gen/test/server/testcase_configs.py @@ -431,13 +431,13 @@ HUNYUAN3D_SHAPE_sampling_params = DiffusionSamplingParams( image_path="https://raw.githubusercontent.com/sgl-project/sgl-test-files/main/diffusion-ci/consistency_gt/1-gpu/hunyuan3d_2_0/hunyuan3d.png", ) -MODELOPT_FLUX1_FP8_TRANSFORMER = "BBuf/flux1-dev-modelopt-fp8-sglang-transformer" -MODELOPT_FLUX2_FP8_TRANSFORMER = "BBuf/flux2-dev-modelopt-fp8-sglang-transformer" -MODELOPT_WAN22_FP8_TRANSFORMER = "BBuf/wan22-t2v-a14b-modelopt-fp8-sglang-transformer" -MODELOPT_FLUX1_NVFP4_TRANSFORMER = "BBuf/flux1-dev-modelopt-nvfp4-sglang-transformer" +MODELOPT_FLUX1_FP8_TRANSFORMER = "lmsys/flux1-dev-modelopt-fp8-sglang-transformer" +MODELOPT_FLUX2_FP8_TRANSFORMER = "lmsys/flux2-dev-modelopt-fp8-sglang-transformer" +MODELOPT_WAN22_FP8_TRANSFORMER = "lmsys/wan22-t2v-a14b-modelopt-fp8-sglang-transformer" +MODELOPT_FLUX1_NVFP4_TRANSFORMER = "lmsys/flux1-dev-modelopt-nvfp4-sglang-transformer" MODELOPT_FLUX2_NVFP4_WEIGHTS = "black-forest-labs/FLUX.2-dev-NVFP4" MODELOPT_WAN22_NVFP4_TRANSFORMER = ( - "BBuf/wan22-t2v-a14b-modelopt-nvfp4-sglang-transformer" + "lmsys/wan22-t2v-a14b-modelopt-nvfp4-sglang-transformer" ) MODELOPT_NVFP4_B200_ENV_VARS = {"SGLANG_DIFFUSION_FLASHINFER_FP4_GEMM_BACKEND": "cudnn"}
FP8 black-forest-labs/FLUX.1-dev --transformer-pathBBuf/flux1-dev-modelopt-fp8-sglang-transformerlmsys/flux1-dev-modelopt-fp8-sglang-transformer single-transformer override, deterministic latent/image comparison, H100 benchmark, torch-profiler trace SGLang converter keeps a validated BF16 fallback set for modulation and FF projection layers; use --model-id FLUX.1-dev for local mirrors
FP8 black-forest-labs/FLUX.2-dev --transformer-pathBBuf/flux2-dev-modelopt-fp8-sglang-transformerlmsys/flux2-dev-modelopt-fp8-sglang-transformer single-transformer override load and generation path published SGLang-ready transformer override
FP8 Wan-AI/Wan2.2-T2V-A14B-Diffusers --transformer-pathBBuf/wan22-t2v-a14b-modelopt-fp8-sglang-transformerlmsys/wan22-t2v-a14b-modelopt-fp8-sglang-transformer primary transformer quantized, transformer_2 kept BF16 primary-transformer-only path; keep transformer_2 on the base checkpoint, and do not describe this as dual-transformer full-model FP8 unless that path is validated separately
NVFP4 black-forest-labs/FLUX.1-dev --transformer-pathBBuf/flux1-dev-modelopt-nvfp4-sglang-transformerlmsys/flux1-dev-modelopt-nvfp4-sglang-transformer mixed BF16+NVFP4 transformer override, correctness validation, 4x RTX 5090 benchmark, torch-profiler trace use build_modelopt_nvfp4_transformer.py; validated builder keeps selected FLUX.1 modules in BF16 and sets swap_weight_nibbles=false
NVFP4 Wan-AI/Wan2.2-T2V-A14B-Diffusers --transformer-pathBBuf/wan22-t2v-a14b-modelopt-nvfp4-sglang-transformerlmsys/wan22-t2v-a14b-modelopt-nvfp4-sglang-transformer primary transformer quantized with ModelOpt NVFP4, transformer_2 kept BF16 primary-transformer-only path; keep transformer_2 on the base checkpoint, and current B200/Blackwell bring-up uses SGLANG_DIFFUSION_FLASHINFER_FP4_GEMM_BACKEND=cudnn