@@ -50,28 +66,33 @@ The `quality` field in a **video response** body is unrelated. It is Sora-compat
Do not combine request-gated DiT fusions with `--enable-breakable-cuda-graph`.
-BCG warmup captures the lossless module branches before a high-quality request
-mounts its DiT fusions, so replay would bypass the requested kernels. SGLang
-rejects this combination for models with eligible DiT quality sites. Models
-whose high-quality path changes only VAE decode remain allowed because BCG
-captures the DiT only.
+BCG warmup captures the lossless module branches before an `extra-high` or
+`high` request mounts its DiT fusions, so replay would bypass the requested
+kernels. SGLang rejects this combination for models with eligible DiT quality
+sites. Models whose request-gated path changes only VAE decode remain allowed
+because BCG captures the DiT only.
-These fusion families mount under `quality="high"`:
+These fusion families mount under both `quality="extra-high"` and
+`quality="high"`:
| Fusion | What it folds |
| --- | --- |
| Linear + tanh-GELU | Bias-add and GELU into the GEMM epilogue (cublasLt), removing the `[tokens, 4*dim]` intermediate round trip |
+| Wan NVFP4 linear + GELU | Bias-add and GELU fused into Wan's NVFP4 FFN projection output |
+| Qwen-Image added-QKV | Added Q/K/V projections fused into joint-buffer production |
| LayerNorm + modulate | `layer_norm(x, weight=(1 + scale), bias=shift)` in place of affine-free LN plus a separate modulate |
| LTX-2 RMSNorm + modulate | `rms_norm(x) * (1 + scale) + shift` in one launch |
| Gate RMSNorm (BF16-native) | `RMSNorm + tanh + mul + add` in one pass |
| HunyuanVideo strided QK RMSNorm | Per-head QK RMSNorm over the packed QKV layout |
| LingBot Video fused RMSNorm | Replaces the handwritten cast, square, mean, rsqrt, and multiply chain with existing Triton RMSNorm kernels |
| SANA-Video BF16-input linear attention | Keeps the first linear-attention GEMM's inputs in BF16 with FP32 accumulation/output; the second GEMM remains FP32 |
+| FLUX-family VAE fast paths | Channels-last decode, GroupNorm(+SiLU), upsample, and attention replacements for FLUX.2 and AutoencoderKL-based FLUX.1, Z-Image, and SD3 pipelines |
+| Wan VAE RMSNorm + SiLU | Replaces the channel-first RMSNorm/SiLU chain while keeping the decode in `channels_last_3d` |
## Kernel inventory
-34 operators are registered in the kernel registry across 38 implementations (some operators carry several backends). Backends are named by provenance, not device: `JIT` compiles under nvcc *and* hipcc, `TRITON` runs on CUDA and ROCm, `CUTE_DSL` needs CUTLASS, `FLYDSL` is ROCm gfx950 only, `AOT` comes from the `sgl_kernel` wheel.
+43 operators are registered in the kernel registry across 47 implementations (some operators carry several backends). Backends are named by provenance, not device: `JIT` compiles under nvcc *and* hipcc, `TRITON` runs on CUDA and ROCm, `CUTE_DSL` needs CUTLASS, `FLYDSL` is ROCm gfx950 only, `AOT` comes from the `sgl_kernel` wheel.
### Normalization
@@ -79,6 +100,7 @@ These fusion families mount under `quality="high"`:
| --- | --- | --- | --- |
| `rmsnorm_scale_shift` | Triton | bit-exact | RMSNorm + `* (1 + scale) + shift` (4 kernels) |
| `scale_residual_norm_scale_shift` | Triton / CuTe-DSL / FlyDSL | bit-exact (Triton) | the above plus the preceding `residual + gate * update` |
+| `scale_residual_norm_scale_shift_nvfp4` | JIT CUDA | matches the selected NVFP4 producer contract | Qwen residual LayerNorm/modulation + FC1 NVFP4 quantization |
| `layernorm_modulate` | Triton | bit-exact | affine-free LayerNorm + adaLN modulate |
| `qk_head_layernorm` | Triton | bit-exact | per-head LayerNorm on q/k |
| `qk_rmsnorm_native` | Triton | bit-exact | Z-Image per-head QK RMSNorm |
@@ -107,8 +129,11 @@ These fusion families mount under `quality="high"`:
| Operator | Backend | Contract | Replaces |
| --- | --- | --- | --- |
| `fused_inplace_qknorm_rope` | JIT CUDA | one bf16 rounding step vs the split baseline; exact with `round_norm_before_rope=True` | separate QK-norm kernel + RoPE |
+| `flux2_qkv_epilogue` | JIT CUDA | bit-exact against its selected BF16 reference chain | FLUX.2 QK RMSNorm + RoPE + joint text/image QKV packing |
+| `qwen_qkv_epilogue` | JIT CUDA | bit-exact against its selected BF16 reference chain | Qwen-Image QK RMSNorm + RoPE + joint QKV writes on SM100+ |
| `rope_rotate_half` | Triton | bit-exact | `chunk` → `cat(-x2, x1)` → two muls + add → `cat(tail)`, about 7 kernels per projection |
| `interleaved_rope_fp64` | JIT CUDA | bit-exact | paired SANA-Video Q/K RoPE with fp64 tables, about 14 eager kernels |
+| `helios_qk_rope` | JIT CUDA | bit-exact | paired in-place Helios Q/K RoPE with transposed frequency layout |
| `ltx2_qknorm_split_rope` | JIT CUDA | close (validated on B200) | LTX-2 QK-norm + split RoPE |
| `ltx25_decoder_rope` | JIT CUDA | bit-exact | paired LTX-2.5 decoder 3D RoPE from cached compact axis tables |
| `hunyuan_qkv_rope_pack` | Triton | bit-exact | QKV pack and RoPE in one pass |
@@ -137,10 +162,20 @@ Every kernel here only moves values (plus zero fill, plus at most one same-order
| `usp_merge_heads` | JIT CUDA | USP all-to-all output head merge (`permute` + `contiguous`) |
| `pack_qkv_destination_major` | Triton | Ulysses destination-major QKV pack |
| `varlen_pack_qkv`, `varlen_scatter_to_padded` | Triton | varlen gather/scatter around the masked attention path |
+| `varlen_pack_segmented_qkv` | Triton | varlen gather from a virtual prefix/main Q/K/V sequence |
| `causal_conv3d_cat_pad` | JIT CUDA / Triton | causal Conv3d `cat` + `pad` |
| `cat_pad_channels_last_3d` | Triton | Wan causal VAE `cat + F.pad + contiguous` (three passes plus cache bookkeeping) in one pass |
| `dup_up3d_add` | Triton | `repeat_interleave + permute().contiguous() + add` |
+### Quantized layout producers
+
+These kernels preserve the quantized checkpoint path's selected reference operation. They are not a claim that FP8 or NVFP4 is equivalent to an unquantized BF16 checkpoint.
+
+| Operator | Backend | Replaces |
+| --- | --- | --- |
+| `flux2_token_cat_fp8` | Triton | FLUX.2 single-block attention/MLP concatenation plus static FP8 quantization |
+| `flux2_token_cat_nvfp4` | JIT CUDA | FLUX.2 single-block attention/MLP concatenation plus NVFP4 quantization |
+
## Coverage by model
Kernels are written against a specific eager chain in a specific model, so coverage is per-model rather than universal.
@@ -148,18 +183,18 @@ Kernels are written against a specific eager chain in a specific model, so cover
| Model | Fused paths |
| --- | --- |
| FLUX.1 | LN+modulate, modulate, residual-gate add, linear+GELU |
-| FLUX.2 | LN+modulate, packed SwiGLU, residual-gate add |
-| Qwen-Image | linear+GELU, select-0/1 LN modulation |
+| FLUX.2 | LN+modulate, packed SwiGLU, gated residual/norm, residual-gate add, QK RMSNorm+RoPE+joint QKV packing, FP8/NVFP4 token-cat producers |
+| Qwen-Image | linear+GELU, select-0/1 LN modulation, added-QKV fusion, QK RMSNorm+RoPE+joint QKV writes, residual norm/modulate+NVFP4 producer |
| GLM-Image | LN+modulate, per-head qk LN, residual-gate add, linear+GELU |
| ERNIE-Image | RMSNorm+scale/shift, residual-gated variant, rotate-half RoPE, residual-gate add |
| Z-Image | BF16-native RMSNorm scale / tanh-residual, per-head QK RMSNorm |
| Ideogram 4 | gate RMSNorm, SwiGLU, rotate-half RoPE, modulate, residual-gate add |
| LTX-2 | QK-norm + split RoPE, ada-values split, RMSNorm+modulate, modulate, residual-gate add, linear+GELU |
| LTX-2.5 decoder | paired 3D RoPE with shared axis-table cache |
-| HunyuanVideo | QKV+RoPE pack, strided QK RMSNorm, linear+GELU |
-| LingBot Video MoE | Fused RMSNorm at `quality=high` |
+| HunyuanVideo / Helios | QKV+RoPE pack, strided QK RMSNorm, linear+GELU; Helios also has paired in-place Q/K RoPE |
+| LingBot Video MoE | Fused RMSNorm at `quality=extra-high` or `quality=high` |
| Sana | LN+modulate, GLUMB bias+SiLU / bias+GLU, residual-gate add |
-| SANA-Video | Packed QKV/KV; paired fp64 interleaved RoPE; LN+modulate, GLUMB bias+SiLU / bias+GLU, and residual-gate add during BCG; BF16-input linear attention at `quality=high` |
+| SANA-Video | Packed QKV/KV; paired fp64 interleaved RoPE; LN+modulate, GLUMB bias+SiLU / bias+GLU, and residual-gate add during BCG; BF16-input linear attention at `quality=extra-high` or `quality=high` |
| Sana-WM | bidirectional gated delta-net, fused QK inverse-RMS |
| Wan | temb table slices; VAE cat+pad and DupUp3D add, `channels_last_3d` RMSNorm+SiLU |
| Cosmos3 / Krea2 / MiniMax-H3 | QK-norm + RoPE (Krea2 also CuTe-DSL norm+scale/shift; MiniMax-H3 also indexed modulation) |
@@ -201,9 +236,9 @@ The package `README.md` carries a selection matrix for the cases where several k
## References
-- [Performance Optimization](./performance-optimization)
-- [Attention Backends](./attention_backends)
-- [Quantization](./quantization)
-- [Profiling](./profiling)
+- [Performance Optimization](/docs/sglang-diffusion/performance-optimization)
+- [Attention Backends](/docs/sglang-diffusion/attention_backends)
+- [Quantization](/docs/sglang-diffusion/quantization)
+- [Profiling](/docs/sglang-diffusion/profiling)
- [`sglang/kernels/ops/diffusion`](https://github.com/sgl-project/sglang/tree/main/python/sglang/kernels/ops/diffusion) — source and selection matrix
- [RFC #29630](https://github.com/sgl-project/sglang/issues/29630) — the unified `sglang.kernels` namespace
diff --git a/docs/docs/sglang-diffusion/performance-optimization.mdx b/docs/docs/sglang-diffusion/performance-optimization.mdx
index 049936e88..3999bbfc6 100644
--- a/docs/docs/sglang-diffusion/performance-optimization.mdx
+++ b/docs/docs/sglang-diffusion/performance-optimization.mdx
@@ -12,11 +12,28 @@ The docs use "output-preserving" instead of promising bit-exact "lossless" becau
## Start Here
-1. Pick a serving or generation mode from [Deployment and Performance Modes](./deployment_cookbook). `--performance-mode auto` is the default; use `speed` when the model fits in GPU memory and latency matters most, `memory` when GPU memory is the bottleneck, and `manual` when every performance flag should be explicit.
-2. Choose the right attention backend from [Attention Backends](./attention_backends).
-3. Use [Sequence Parallelism](./ring_sp_performance) only when the model and video shape benefit from sequence splitting.
-4. Use [Inference Batching](./dynamic_batching) for concurrent compatible requests during serving.
-5. Use [Profiling](./profiling) before changing several levers at once.
+1. Pick a serving or generation mode from [Deployment and Performance Modes](/docs/sglang-diffusion/deployment_cookbook). `--performance-mode auto` is the default; use `speed` when the model fits in GPU memory and latency matters most, `memory` when GPU memory is the bottleneck, and `manual` when every performance flag should be explicit.
+2. Choose the right attention backend from [Attention Backends](/docs/sglang-diffusion/attention_backends).
+3. Use [Sequence Parallelism](/docs/sglang-diffusion/ring_sp_performance) only when the model and video shape benefit from sequence splitting.
+4. Use [Inference Batching](/docs/sglang-diffusion/dynamic_batching) for concurrent compatible requests during serving.
+5. Use [Profiling](/docs/sglang-diffusion/profiling) before changing several levers at once.
+
+## Choose a request quality tier
+
+`--quality` is cumulative: a broader tier never drops an optimization from a
+stricter tier.
+
+| Tier | Optimization boundary |
+| --- | --- |
+| `lossless` (default) | The selected deployment's reference execution plus all unconditional bit-exact replacements |
+| `extra-high` | Everything in `lossless`, plus only request-gated DiT/VAE kernel fusions; the tier does not itself enable sparse, caching, or other approximate paths |
+| `high` | Everything in `extra-high`, plus model-owned high-only paths such as an audited Cache-DiT policy or lower-precision VAE decode |
+
+Use `extra-high` when you want to isolate fusion wins from approximate
+acceleration. A tier may be a no-op when the active model has no eligible path.
+Separately configured quantization, attention, or caching options still apply.
+See [Fused Kernels](/docs/sglang-diffusion/fused_kernels) for the current
+request-gated families and their numerical contracts.
## Output-Preserving / Lossless-Style Levers
@@ -39,42 +56,42 @@ These settings should preserve model behavior while changing residency, parallel
--performance-mode |
You want a safe preset for speed or memory without overriding explicit flags. |
- Deployment and Performance Modes |
+ Deployment and Performance Modes |
| Breakable CUDA graph |
A supported pipeline serves a fixed set of shapes and eager execution is launch-bound. |
- CLI reference |
+ CLI reference |
| Offload, FSDP, CFG parallelism |
GPU memory, multi-GPU residency, or CFG branch splitting is the main bottleneck. |
- Deployment and Performance Modes |
+ Deployment and Performance Modes |
| Sequence parallelism |
Long image/video sequences need sequence-level parallelism. |
- Sequence Parallelism |
+ Sequence Parallelism |
--encoder-parallel |
Text/image encoding is a visible share of the request and the DiT replica sits idle during it. |
- Encoder Parallelism |
+ Encoder Parallelism |
| Attention backend |
Kernel choice dominates DiT latency or memory. |
- Attention Backends |
+ Attention Backends |
| Fused kernels |
You want to know which elementwise chains are already fused, or to opt into the request-gated set. |
- Fused Kernels |
+ Fused Kernels |
| Dynamic batching |
Serving many compatible requests concurrently. |
- Inference Batching |
+ Inference Batching |
@@ -100,22 +117,22 @@ These techniques can change the denoising path, numerical representation, or gen
| Cache-DiT |
Skips selected DiT block or step computation based on cache decisions. |
- Cache-DiT |
+ Cache-DiT |
| TeaCache |
Reuses residuals when consecutive denoising steps are similar enough. |
- TeaCache |
+ TeaCache |
| Progressive resolution |
Runs early denoising at lower latent resolution for supported pipelines. |
- Progressive Resolution Generation |
+ Progressive Resolution Generation |
| Quantization |
Uses lower-precision transformer weights or activations. |
- Quantization |
+ Quantization |
@@ -124,20 +141,21 @@ These techniques can change the denoising path, numerical representation, or gen
1. Establish a baseline with the target model, resolution, frame count, step count, and GPU type.
2. Select `--performance-mode` and explicit residency or parallelism flags.
-3. Compare breakable CUDA graph against eager execution for supported fixed-shape pipelines. Pass every served resolution to `--warmup-resolutions` and confirm capture in the server log.
-4. Tune attention backend and batching for the deployment pattern.
-5. Profile if the bottleneck is unclear.
-6. Add caching, progressive resolution, or quantization only after comparing output quality against your acceptance target.
+3. Compare `quality=lossless` with `quality=extra-high` to isolate the request-gated fusion set.
+4. Compare breakable CUDA graph against eager execution for supported fixed-shape pipelines. Pass every served resolution to `--warmup-resolutions` and confirm capture in the server log. Models with request-gated DiT fusions cannot combine those fusions with a graph captured from the lossless branches.
+5. Tune attention backend and batching for the deployment pattern.
+6. Profile if the bottleneck is unclear.
+7. Add `quality=high`, caching, progressive resolution, or quantization only after comparing output quality against your acceptance target.
## Diagnostics
-[Profiling](./profiling) is not an optimization technique by itself. It belongs in the performance workflow because it tells you which stage, kernel, or denoising step is worth optimizing before you change multiple levers.
+[Profiling](/docs/sglang-diffusion/profiling) is not an optimization technique by itself. It belongs in the performance workflow because it tells you which stage, kernel, or denoising step is worth optimizing before you change multiple levers.
## References
-- [Deployment and Performance Modes](./deployment_cookbook)
-- [Attention Backends](./attention_backends)
-- [Fused Kernels](./fused_kernels)
-- [Sequence Parallelism](./ring_sp_performance)
-- [Caching Strategies](./caching-acceleration)
-- [Profiling](./profiling)
+- [Deployment and Performance Modes](/docs/sglang-diffusion/deployment_cookbook)
+- [Attention Backends](/docs/sglang-diffusion/attention_backends)
+- [Fused Kernels](/docs/sglang-diffusion/fused_kernels)
+- [Sequence Parallelism](/docs/sglang-diffusion/ring_sp_performance)
+- [Caching Strategies](/docs/sglang-diffusion/caching-acceleration)
+- [Profiling](/docs/sglang-diffusion/profiling)
diff --git a/docs/src/snippets/configs/MiniMaxAI/minimax-h3.jsx b/docs/src/snippets/configs/MiniMaxAI/minimax-h3.jsx
index 8758ebb49..5e113201f 100644
--- a/docs/src/snippets/configs/MiniMaxAI/minimax-h3.jsx
+++ b/docs/src/snippets/configs/MiniMaxAI/minimax-h3.jsx
@@ -486,7 +486,7 @@ return {
title: "Quality",
scope: "request",
docsHref: "/docs/sglang-diffusion/cache_dit",
- description: "Reference execution or the audited Cache-DiT acceleration preset.",
+ description: "Cumulative reference, fusion-only, or audited Cache-DiT execution.",
quality: "Sampling policy",
learnMore: "#choose-the-quality-level",
default: "lossless",
@@ -497,6 +497,11 @@ return {
recommended: true,
description: "Reference-exact denoising without Cache-DiT approximation.",
},
+ {
+ id: "extra-high",
+ label: "Extra high",
+ description: "Includes fusion-only request paths but not Cache-DiT; MiniMax-H3 currently follows its lossless denoise path at this tier.",
+ },
{
id: "high",
label: "Audited high",
@@ -669,7 +674,7 @@ return {
|| (s.execution === "bcg" && ["b200", "h200"].includes(s.hw) && s.weights === "ref2va");
const serveVerified = topologyVerified && encoderVerified && attentionVerified
&& precisionVerified && executionVerified;
- const requestVerified = topologyVerified && (s.quality === "lossless"
+ const requestVerified = topologyVerified && (["lossless", "extra-high"].includes(s.quality)
|| (s.quality === "high" && highAudited && s.execution === "eager"));
const topologyParts = [];
diff --git a/python/sglang/kernels/ops/diffusion/README.md b/python/sglang/kernels/ops/diffusion/README.md
index 5a3b879e5..7802b9118 100644
--- a/python/sglang/kernels/ops/diffusion/README.md
+++ b/python/sglang/kernels/ops/diffusion/README.md
@@ -45,7 +45,7 @@ ext/ JIT C++/CUDA extensions (Hunyuan3D raster/inpaint) — NOT kernels
../../kda_kernels/ agent-generated implementations and their JIT CUDA sources
```
-## The two numerical contracts
+## Numerical contracts and quality policy
**Bit-exact (`torch.equal` vs the eager chain) → mounted unconditionally.**
These kernels reproduce every aten rounding boundary, sometimes down to the
@@ -59,11 +59,18 @@ themselves against the live eager chain on first sight via
dispatch they replicate can change under them.
**Not bit-exact → quality-gated.** Mounted onto marked `nn.Module` sites only
-for `quality="high"` requests, at batch boundaries, all-or-nothing per
-transformer (`sites/quality_gate.py`). A plain fp32 single-pass norm fusion
-looks harmless and is not: on ERNIE-Image it moved the 50-step trajectory to
-PSNR 18.83 dB at `quality=high`, which is what motivated the bit-exact
-rewrite.
+for `quality="extra-high"` and `quality="high"` requests, at batch boundaries,
+all-or-nothing per transformer (`sites/quality_gate.py`). `extra-high` adds
+only these request-gated DiT/VAE fusions; `high` is cumulative and may also
+enable model-owned approximate paths such as Cache-DiT or a lower-precision
+decode. A plain fp32 single-pass norm fusion looks harmless and is not: on
+ERNIE-Image it moved the 50-step trajectory to PSNR 18.83 dB, which is what
+motivated the bit-exact rewrite.
+
+**Model/checkpoint-native.** Generic close-contract kernels, sparse operators,
+and FP8/NVFP4 producers can belong to the selected model or deployment path.
+The request `quality` tier neither selects nor disables those independent
+choices.
SANA-Video's quality-gated linear-attention site keeps BF16 inputs for the
first GEMM while requesting FP32 accumulation/output, then runs the second
@@ -98,6 +105,7 @@ Several norms look interchangeable and are not. Start here.
| `fused_layernorm_modulate` | Triton | bit-exact vs aten `vectorized_layer_norm` | bf16, `N % 4 == 0`, 16B-aligned |
| `fused_norm_scale_shift` / `fused_scale_residual_norm_scale_shift` | CuTe-DSL | fp32 statistics, close | fp16/bf16/fp32, LN or RMS, many broadcast modes |
| `flydsl_norm_scale_shift` / `flydsl_fused_residual_norm_scale_shift` | FlyDSL | close | **ROCm gfx950 only** |
+| `try_fused_scale_residual_norm_scale_shift_nvfp4` | JIT CUDA | matches the selected NVFP4 producer contract | Qwen residual LayerNorm/modulation + FC1 NVFP4 quantization |
| `fuse_layernorm_scale_shift_gate_select01_kernel` | Triton | close | per-token select between two modulation rows (Qwen-Image) |
| `norm_infer` / `rms_norm_fn` | Triton (+torch/NPU/MPS fallbacks) | close | the generic entry point; use when nothing above fits |
@@ -131,6 +139,8 @@ tensor copy per residual site.
|---|---|---|
| `fused_inplace_qknorm_rope` | JIT CUDA | one bf16 rounding step vs split baseline; `round_norm_before_rope=True` makes it exact; supports compact and full-width NeoX/interleaved caches |
| `fused_qknorm_rope_pack_kv` | JIT CUDA | as above, also packs prefix K/V |
+| `try_fused_flux2_qkv_epilogue` | JIT CUDA | bit-exact vs the selected BF16 chain | FLUX.2 QK RMSNorm + RoPE + joint QKV packing |
+| `try_fused_qwen_qkv_epilogue` | JIT CUDA | bit-exact vs the selected BF16 chain | Qwen-Image QK RMSNorm + RoPE + joint QKV writes; SM100+ |
| `fused_rope_rotate_half_bitexact` | Triton | bit-exact (elementwise only) |
| `fused_interleaved_rope_fp64` | JIT CUDA | bit-exact vs paired SANA-Video fp64 RoPE |
| `fused_inplace_helios_qk_rope` | JIT CUDA | bit-exact paired in-place RoPE for Helios' transposed frequency layout |
@@ -139,13 +149,16 @@ tensor copy per residual site.
| `apply_rotary_embedding` | Triton (+fallbacks) | close; the generic entry point |
| `hunyuan_qkv_rope_pack` | Triton | bit-exact; packs QKV and applies RoPE in one pass |
-### Data movement (all bit-exact by construction)
+### Data movement and quantized layout producers
`usp_merge_heads`, `pack_qkv_destination_major`, `fused_pack_qkv`,
`fused_pack_segmented_qkv`, `fused_scatter_to_padded`,
`fused_causal_conv3d_cat_pad_cuda`,
`cat_pad_channels_last_3d`, `dup_up3d_add`, `fused_temb_table_slices`,
-`ltx2_ada_values9`.
+and `ltx2_ada_values9` are bit-exact data movement or same-order arithmetic.
+`try_flux2_token_cat_fp8` and `try_flux2_token_cat_nvfp4` fuse branch
+concatenation directly into the quantized representation selected by the
+FLUX.2 checkpoint path.
`fused_temb_table_slices` is worth knowing about: the eager
`(table + temb.float()).chunk(6, dim=2)` materializes ~8 GB of fp32 at
@@ -172,7 +185,7 @@ inspecting model modules is its whole job.
3. Give it a `can_use_*` predicate; raise, don't return `None`.
4. State the numerical contract in the module docstring, including which
shapes it was verified on.
-5. If it is not bit-exact, gate it through `sites/`. Do not mount it by
- default.
+5. If it is not bit-exact, gate it through `sites/`. It must mount for both
+ `extra-high` and `high`, never for the default `lossless` path.
6. Test it in the domain suite (`test/registered/kernels/ops/diffusion/`), and
the model wiring in `test_model_fast_paths.py`.
diff --git a/python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-add-model/SKILL.md b/python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-add-model/SKILL.md
index 2d07edac5..d43593bdd 100644
--- a/python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-add-model/SKILL.md
+++ b/python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-add-model/SKILL.md
@@ -656,15 +656,17 @@ BCG validation must prove all of the following:
For a non-bit-exact optimization, integrate through the request-scoped site
framework under `sglang.kernels.ops.diffusion.sites`. Mark sites during model
-construction and let `QualityGatedFusion` mount them only for
-`quality="high"`; `quality="lossless"` must keep the original code path.
+construction and let `QualityGatedFusion` mount them for both
+`quality="extra-high"` and `quality="high"`; `quality="lossless"` must keep
+the original code path. A high-only sparse, caching, or other approximate
+path must remain outside this fusion gate.
Eligibility must be all-or-nothing for coupled sites and fail closed on dtype,
shape, layout, backend, BCG, or compile incompatibility. Add clean site-level
guard/parity tests and a model wiring test instead of embedding request-policy
branches throughout the DiT.
Finally, use the benchmark/profile skill's `--quality-bcg-matrix` to run
-same-GPU ABBA pairs for Eager/BCG at lossless/high. Report denoise and saved
+same-GPU ABBA pairs for Eager/BCG at lossless/extra-high/high. Report denoise and saved
request e2e separately, require at least 1.5% repeated mean e2e improvement for
an optimization PR, attach profile and generated-media A/B evidence, then
delete the task-owned checkpoint cache and verify zero residual weight files
diff --git a/python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-benchmark-profile/SKILL.md b/python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-benchmark-profile/SKILL.md
index 3bc260378..f84be1ac6 100644
--- a/python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-benchmark-profile/SKILL.md
+++ b/python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-benchmark-profile/SKILL.md
@@ -9,7 +9,7 @@ Use this skill when measuring denoise performance, finding the slow op, checking
This skill is diagnosis-first. It owns:
- checked-in denoise benchmark presets
-- same-GPU quality/BCG applicability checks with repeated lossless and high rows
+- same-GPU quality/BCG applicability checks with repeated lossless, extra-high, and high rows
- perf dump collection and before/after comparison
- `torch.profiler` trace capture and quick hotspot ranking
- mapping hot kernels back to known fast paths and fusion families
@@ -65,7 +65,7 @@ Always rule out these existing families first:
- Z-Image bf16-native Triton RMSNorm scale/tanh-residual modulation
- SANA packed self-attention Q/K/V and cross-attention K/V GEMMs
- SANA-Video's packed projections and request-scoped BF16-input linear
- attention at `quality=high`; keep the second attention GEMM in FP32 and
+ attention at `quality=extra-high` or `quality=high`; keep the second attention GEMM in FP32 and
compare against `quality=lossless` before changing its precision further
- SANA-Video reuse of SANA's bit-exact bias/activation, residual-gate, and
LayerNorm-modulation fast paths before adding video-only kernels
@@ -73,7 +73,7 @@ Always rule out these existing families first:
USP relayout, and batched TP AdaLN collectives
- bit-exact diffusion adaLN modulation and fused LayerNorm + modulation for
FLUX.1, GLM-Image, and SANA
-- request-scoped `quality=high` DiT and VAE fast paths
+- request-scoped DiT and VAE fast paths at `quality=extra-high` or `quality=high`
- Wan causal-VAE cache/padding and DupUp3D data-movement fusions
- fused diffusion `QK norm + RoPE`
- LTX2 split RoPE
@@ -94,9 +94,9 @@ controlled comparator, never for the eager ground truth. The legacy
`--no-torch-compile` spelling remains accepted but is redundant.
For kernel/BCG discovery, run `--quality-bcg-matrix`. It executes Eager/BCG as
-A-B-B-A at `lossless`, then repeats the pair at `high`, on one locked GPU set
-and one isolated checkpoint cache. The high+BCG rows are applicability checks,
-not presumed-valid performance cells. A BCG row is invalid unless the log
+A-B-B-A at `lossless`, then repeats the pair at `extra-high` and `high`, on
+one locked GPU set and one isolated checkpoint cache. The extra-high/high+BCG
+rows are applicability checks, not presumed-valid performance cells. A BCG row is invalid unless the log
contains `[Diffusion BCG] captured` and contains no support-disable,
capture-failure, serving-signature-miss, or late quality-fusion marker. In
particular, a request-scoped DiT fusion mounted after lossless warmup capture
@@ -120,7 +120,7 @@ its normal cleanup finally block without modifying the seed cache.
Keep prompt, negative prompt, seed, shape, steps, guidance, dtype, topology,
and residency fixed. Lossless comparisons require byte-identical artifacts.
-For `quality=high`, report aggregate and worst-frame SSIM/PSNR; the repository
+For `quality=extra-high` and `quality=high`, report aggregate and worst-frame SSIM/PSNR; the repository
defaults are 0.95/28 dB for images and 0.92/24 dB for video unless the model's
checked-in consistency metadata defines a different threshold. A performance
PR needs repeated saved-request e2e improvement of at least 1.5%, a
diff --git a/python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-benchmark-profile/benchmark-and-profile.md b/python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-benchmark-profile/benchmark-and-profile.md
index fa8422623..b6095a058 100644
--- a/python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-benchmark-profile/benchmark-and-profile.md
+++ b/python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-benchmark-profile/benchmark-and-profile.md
@@ -139,9 +139,9 @@ The helper defaults to eager. Add `--torch-compile` only for a labeled compile
control. `--no-torch-compile` remains accepted for compatibility but is no
longer required.
-Run one explicit quality or BCG comparator with `--quality lossless|high` and
+Run one explicit quality or BCG comparator with `--quality {lossless,extra-high,high}` and
`--breakable-cuda-graph`. BCG and `torch.compile` are intentionally mutually
-exclusive in this helper. A high+BCG command is only a compatibility probe:
+exclusive in this helper. An extra-high/high+BCG command is only a compatibility probe:
it is invalid if request-scoped DiT fusions mount after the lossless warmup
graphs were captured. When a preset has explicit width and height, the helper
declares that same `--warmup-resolutions` value automatically. Video presets
@@ -150,21 +150,22 @@ with an explicit frame count also declare the matching `--warmup-num-frames`:
```bash
PYTHONPATH=python python3 "$BENCH_PY" \
--model longcat-image \
- --quality high \
+ --quality extra-high \
--breakable-cuda-graph \
- --label bcg-high \
+ --label bcg-extra-high \
--output-dir "${BENCH_DIR}"
```
For optimization discovery, use the full repeated matrix. It runs
-Eager/BCG/BCG/Eager at `lossless`, then the same sequence at `high`, while
-holding one GPU set and one isolated checkpoint cache. The high+BCG cells test
+Eager/BCG/BCG/Eager at `lossless`, then the same sequence at `extra-high` and
+`high`, while holding one GPU set and one isolated checkpoint cache. The
+extra-high/high+BCG cells test
whether the combination is actually supported; do not average them when the
runtime rejects the combination or the helper detects a late quality-fusion
mount. The helper hashes every generated image, video, audio, or 3D mesh
artifact. It first requires the two Eager rows at each quality to agree, then
rejects any BCG row whose hash differs from that Eager reference. Cleanup occurs
-only after all eight runs, including on failure or interruption:
+only after all twelve runs, including on failure or interruption:
```bash
MODEL_CACHE_ROOT=/path/to/task-owned/model-caches
@@ -181,7 +182,7 @@ Before starting, confirm the chosen GPU set has no foreign process and remains
unchanged through every run boundary. The helper rejects a BCG row unless its
log contains `[Diffusion BCG] captured` and contains none of: support-gate
disable, capture failure, `serving signature MISSED`, a message that no graph
-will be captured, or a request-scoped high-quality DiT fusion mounted after
+will be captured, or a request-scoped quality-gated DiT fusion mounted after
capture. Do not average rejected rows with valid results.
BCG signatures include more than width and height. The helper maps an explicit
@@ -319,13 +320,13 @@ Use the preset categories this way:
| `qwen-edit-base` | `Qwen/Qwen-Image-Edit` | No | Covers the original native `QwenImageEditPipelineConfig`, which is distinct from the 2509/2511 edit-plus paths; public SGLang edit fixture, 1024x1024. |
| `qwen-image-layered` | `Qwen/Qwen-Image-Layered` | No | Native layered-image path using the same public reference image and four-frame request as the GPU server case, at the registered 640x640 canvas. |
| `stable-diffusion-3.5-medium` | `stabilityai/stable-diffusion-3.5-medium-diffusers` | No | Representative native `StableDiffusion3PipelineConfig` path at 1024x1024. The repository is gated, so export `HF_TOKEN`; an unauthenticated run is a recorded access blocker, not model evidence. |
-| `sana-video` | `Efficient-Large-Model/SANA-Video_2B_480p_diffusers` | No | CI-sized T2V baseline: 832x480, 17 frames, 8 steps, guidance 6.0. The BCG comparator declares the same 17-frame warmup shape. Compare `quality=lossless` and `quality=high`; high enables the BF16-input first linear-attention GEMM while retaining FP32 output and the FP32 second GEMM. |
+| `sana-video` | `Efficient-Large-Model/SANA-Video_2B_480p_diffusers` | No | CI-sized T2V baseline: 832x480, 17 frames, 8 steps, guidance 6.0. The BCG comparator declares the same 17-frame warmup shape. Compare all three tiers; `extra-high` and `high` enable the BF16-input first linear-attention GEMM while retaining FP32 output and the FP32 second GEMM. |
| `sana-wm-bidirectional` | `Efficient-Large-Model/SANA-WM_bidirectional` | No | Dense two-stage TI2V baseline at the native 1280x704 shape, 49 frames, 16 fps, 20 steps, guidance 4.5, and a 48-frame forward/left action program. Uses the shared cat fixture. |
| `sana-wm-streaming` | `Efficient-Large-Model/SANA-WM_streaming` | No | Matching offline chunk-causal two-stage baseline with the streaming DiT and chunked refiner enabled; uses the same shape, fixture, seed, and camera action for comparison. |
| `lingbot-video-moe` | `robbyant/lingbot-video-moe-30b-a3b` | No | One-GPU eager baseline using the CI structured-JSON caption, 384x640, 17 frames, 12 steps, and text-encoder CPU offload. |
| `lingbot-world` | `robbyant/lingbot-world-fast-diffusers` | No | One-H200 offline single-chunk profile for the registered causal DMD path: 832x480x9, four steps, guidance 1.0, the shared image fixture, and forward-camera actions for all nine frames. Keep stateful websocket latency as a separate metric. |
| `lingbot-world-v2` | `robbyant/lingbot-world-v2-14b-causal-fast-diffusers` | No | Matching controlled single-chunk profile for the separately registered v2 checkpoint. The fixed shape, action program, and schedule make v1/v2 hotspot comparisons reproducible without presenting one-chunk e2e as stateful realtime latency. |
-| `fastwan21-t2v-1.3b` | `FastVideo/FastWan2.1-T2V-1.3B-Diffusers` | No | One-GPU 832x480, 61-frame, 3-step DMD baseline. The preset pins manual mode with a resident DiT so lossless/high comparisons do not measure an offload-policy change. |
+| `fastwan21-t2v-1.3b` | `FastVideo/FastWan2.1-T2V-1.3B-Diffusers` | No | One-GPU 832x480, 61-frame, 3-step DMD baseline. The preset pins manual mode with a resident DiT so lossless/extra-high/high comparisons do not measure an offload-policy change. |
| `wan21-t2v-1.3b` | `Wan-AI/Wan2.1-T2V-1.3B-Diffusers` | No | Registered one-GPU 832x480, 81-frame Wan2.1 baseline at 50 steps and guidance 3.0. Keep it separate from FastWan and TurboWan because the longer schedule changes the end-to-end weight of VAE optimizations. |
| `wan21-t2v-14b` | `Wan-AI/Wan2.1-T2V-14B-Diffusers` | No | Cookbook-aligned four-GPU CFG/Ulysses baseline at 832x480, 81 frames, 50 steps, and guidance 5.0. Text encoding stays CPU-offloaded as in the documented deployment command. |
| `wan21-i2v-14b-480p` | `Wan-AI/Wan2.1-I2V-14B-480P-Diffusers` | No | Four-GPU CFG/Ulysses image-conditioned baseline at 832x480, 81 frames, 50 steps, and guidance 5.0. Uses the shared cat fixture and its motion prompt. |
@@ -668,7 +669,8 @@ some generation failures are reported through the response payload without a
nonzero process exit.
For `quality=lossless`, compare saved artifact hashes and require byte equality
-for a claimed lossless fast path or BCG change. For `quality=high`, keep the
+for a claimed lossless fast path or BCG change. For `quality=extra-high` and
+`quality=high`, keep the
lossless artifact as ground truth and report both aggregate and worst-frame
SSIM/PSNR. Repository defaults are SSIM 0.95 / PSNR 28 dB for images and SSIM
0.92 / PSNR 24 dB for videos; checked-in model/hardware consistency metadata
@@ -788,7 +790,7 @@ the known mainline families.
| `to_q -> to_k -> to_v` on NVFP4 or Nunchaku FLUX-family checkpoints | Treat as a packed-QKV fast-path miss or checkpoint-format mismatch |
| `rmsnorm_scale` or `rmsnorm_tanh_residual` missing on Z-Image | Check the bf16-native Triton eligibility guards before proposing a new fusion |
| FLUX.1, GLM-Image, or SANA shows separate LayerNorm plus adaLN elementwise kernels | Check the bit-exact `modulate_scale_shift` and `fused_layernorm_modulate` guards/self-test before proposing another norm fusion |
-| `quality=high` shows the same FLUX/GLM DiT or FLUX-family/Wan VAE chain as `lossless` | Check whether the request-scoped quality gate mounted and whether every site passed its all-or-nothing compatibility checks |
+| `quality=extra-high` or `quality=high` shows the same FLUX/GLM DiT or FLUX-family/Wan VAE chain as `lossless` | Check whether the request-scoped quality gate mounted and whether every site passed its all-or-nothing compatibility checks |
| LTX-2 split RoPE appears as a long PyTorch elementwise chain | Check the `apply_ltx2_split_rotary_emb` Triton path and its shape guards |
| Wan decode is dominated by causal `cat + pad + contiguous`, feature-cache copies, or `repeat_interleave + permute + add` | Check the bit-exact Wan causal-cache and DupUp3D data-movement kernels before writing a new decoder kernel |
| masked attention spends time packing/unpacking Q/K/V | Check whether fused varlen USP pack/scatter should have engaged |
@@ -826,7 +828,7 @@ This skill intentionally stops here. It tells you whether you are looking at:
- [ ] `compare_perf.py` table generated
- [ ] one representative `torch.profiler` trace saved
- [ ] hotspot classified against `existing-fast-paths.md`
-- [ ] lossless artifact hash is exact; high-quality aggregate and worst-frame SSIM/PSNR pass the checked-in threshold
+- [ ] lossless artifact hash is exact; extra-high/high aggregate and worst-frame SSIM/PSNR pass the checked-in threshold
- [ ] reference image or start/middle/end video contact sheet checked visually
- [ ] any PR claim has repeated saved-request e2e improvement >= 1.5%
- [ ] task-owned checkpoint cache cleaned and ledger shows zero residual weight files
diff --git a/python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-benchmark-profile/existing-fast-paths.md b/python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-benchmark-profile/existing-fast-paths.md
index c627f1065..d997f1f3f 100644
--- a/python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-benchmark-profile/existing-fast-paths.md
+++ b/python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-benchmark-profile/existing-fast-paths.md
@@ -114,17 +114,20 @@ framework-specific optimization workflow.
check dtype, alignment, shape, BCG/compile context, and the one-time equality
self-test before proposing another fusion.
-4. Request-scoped `quality=high` fusion gates
+4. Request-scoped fusion gates at `quality=extra-high` or `quality=high`
- Locations: `quality_gate.py`, `fused_ln_modulate.py`, `denoising.py`,
`decoding.py`, `fast_path_gate.py`, `flux2_vae_cuda_opt.py`, and
`wan_vae_cuda_opt.py`.
- Behavior: `quality="lossless"` is the default exact reference path.
- `quality="high"` may mount model-owned, validated but non-bit-exact DiT
- fusions and decode-scoped VAE rewrites. Mounting is all-or-nothing per
+ `quality="extra-high"` and `quality="high"` mount the same validated but
+ non-bit-exact DiT fusions and decode-scoped VAE rewrites. `high` is
+ cumulative and may additionally enable model-owned approximate paths.
+ Mounting is all-or-nothing per
transformer/fusion family; VAE gates reset after every decode.
- Current families include FLUX affine-folded LN+modulate / fused GELU sites,
- GLM/Qwen/Hunyuan/LTX fused GELU sites, LTX RMSNorm+modulate, Hunyuan QK
- RMSNorm, Ideogram gated RMSNorm, SANA-Video linear attention, generic KL VAE
+ Wan cublasLt/NVFP4 GELU, Qwen added-QKV, GLM/Qwen/Hunyuan/LTX fused GELU,
+ LTX RMSNorm+modulate, Hunyuan QK RMSNorm, Ideogram gated RMSNorm,
+ LingBot RMSNorm, SANA-Video linear attention, generic KL VAE
decoder rewrites used by FLUX.1/FLUX.2/Z-Image/SD3, and Wan VAE
RMSNorm+SiLU.
- Do not confuse request `--quality` with `--output-quality`, which controls
@@ -218,7 +221,7 @@ framework-specific optimization workflow.
one channels-last-3D pass, and fuse `main + DupUp3D(src)` without
materializing `repeat_interleave + permute().contiguous()` intermediates.
- Numerical contract: these are bit-exact data-movement / same-order-add
- replacements and run independently of the `quality=high` Wan RMSNorm+SiLU
+ replacements and run independently of the request-gated Wan RMSNorm+SiLU
path. Unsupported layouts or padding fall back to the aten chain.
- Validation: `test/registered/kernels/ops/diffusion/test_wan_causal_cache.py`.
@@ -328,16 +331,16 @@ framework-specific optimization workflow.
**Request-Scoped DiT Fusions with Breakable CUDA Graphs**
-- `quality=high` DiT sites are mounted at a request boundary. BCG warmup uses
+- DiT sites at `quality=extra-high` or `quality=high` are mounted at a request boundary. BCG warmup uses
the model's lossless sampling default unless a quality-aware graph variant
was captured explicitly.
-- A graph captured before the high-quality mount retains the lossless module
- branches. Replaying it after the mount silently bypasses the requested high
+- A graph captured before the request-quality mount retains the lossless module
+ branches. Replaying it after the mount silently bypasses the requested fused
kernels even when the tensor signature matches.
-- Workflow rule: a high+BCG cell is valid only when the model has no
+- Workflow rule: an extra-high/high+BCG cell is valid only when the model has no
request-scoped DiT quality sites, or when logs prove those sites were mounted
before the matching graph capture. A mount after `[Diffusion BCG] captured`
- invalidates the row; do not use its latency or output as high-quality
+ invalidates the row; do not use its latency or output as request-quality
evidence.
**Recent Model Audit Boundaries**
@@ -363,10 +366,10 @@ framework-specific optimization workflow.
- LingBot Video MoE's router implements sigmoid+bias grouped top-k in
`multimodal_gen/runtime/layers/moe.py`. Check parameter and output-order
compatibility with `srt/layers/moe/topk.py::biased_grouped_topk` before
- writing a new router kernel. Its released eager path still expands RMSNorm
- into `pow/mean/rsqrt` chains. #35969 is a measured `quality=high` candidate
- that dispatches existing Triton row kernels by weight dtype and hidden size;
- it is not current-main behavior until the PR merges.
+ writing a new router kernel. Current main mounts fused Triton RMSNorm row
+ kernels by weight dtype and hidden size for `quality=extra-high` and
+ `quality=high`; check the quality-site guards before treating an expanded
+ `pow/mean/rsqrt` chain as a new opportunity.
- LTX-2.5 reuses the mature LTX-2 DiT paths. Treat the optional diffusion
decoder separately: confirm NATTEN `na3d` is active, then inspect its
per-block 3D RoPE construction and split QKV/SwiGLU projections.
@@ -379,7 +382,7 @@ framework-specific optimization workflow.
- AdaLN modulation: `LayerNormScaleShift`, `RMSNormScaleShift`, `ScaleResidual*` in `layernorm.py`.
- Bit-exact adaLN modulation / LayerNorm folding: `modulate_scale_shift` and
`fused_layernorm_modulate` through `flux.py`, `glm_image.py`, and `sana.py`.
-- Request-scoped high-quality acceleration: `QualityGatedFusion` in
+- Request-scoped extra-high/high acceleration: `QualityGatedFusion` in
`quality_gate.py`, `_maybe_toggle_quality_fusions` in `denoising.py`, and
`use_vae_fast_path` in `decoding.py`.
- Bit-exact first-sight verify/disable: `BitExactFusionGate` in
@@ -394,7 +397,7 @@ framework-specific optimization workflow.
- QK norm: `apply_qk_norm` used in `flux.py`, `flux_2.py`, `qwen_image.py`, `zimage.py`, `wanvideo.py`, `ltx_2.py`, `hunyuanvideo.py`.
- QK norm + RoPE: `apply_qk_norm_rope` in `layernorm.py`; use this path when the model wants fused attention prep instead of separate QK norm and RoPE calls.
- LTX2 split RoPE: `apply_ltx2_split_rotary_emb` in `ltx_2.py`.
-- LTX2 RMSNorm+modulate and FFN GELU epilogue under `quality="high"`:
+- LTX2 RMSNorm+modulate and FFN GELU epilogue under `quality="extra-high"` and `quality="high"`:
`mark_ltx2_rms_norm_modulate_site` / `fused_ltx2_rms_norm_modulate` in
`kernels/ops/diffusion/sites/ltx2_rmsnorm_modulate_site.py` (mount-based
`QualityGatedFusion`, not a first-sight `BitExactFusionGate` — the fused
@@ -468,8 +471,9 @@ relying on any file path, flag, or claim about whether the work has merged.
- #34584 Wan TI2V modulation/RoPE; #34616 FLUX2; #34617 Hunyuan;
#34619 GLM; #34620 ERNIE; #34928 SANA; #34932 Cosmos3; #35728
SANA-Video linear attention.
- - #35961 SANA-Video lossless shared-kernel reuse and #35969 LingBot
- `quality=high` RMSNorm are open candidates, not current-main fast paths.
+ - SANA-Video shared-kernel reuse and LingBot request-gated RMSNorm are now
+ current-main fast paths; verify the source tree before treating their
+ historical PRs as open work.
- VAE and decode-side acceleration:
- #22531 LTX2 parallel VAE support and #20927 batched tiled VAE decode (draft).
- Attention, communication, and runtime scheduling:
@@ -492,7 +496,7 @@ relying on any file path, flag, or claim about whether the work has merged.
**Constraints and Fallbacks**
- `scale_shift` Triton requires CUDA + contiguous `x`. NPU swaps to native.
- Bit-exact BF16 LayerNorm+modulate requires the guarded aten-compatible shape
- and a successful live equality check; `quality=high` affine folding is a
+ and a successful live equality check; request-gated affine folding is a
separate non-bit-exact path.
- CuTe DSL fused norms require `D % 256 == 0` and `D <= 8192`.
- Triton norm kernels error on feature size >= 64KB.
diff --git a/python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-benchmark-profile/scripts/bench_diffusion_denoise.py b/python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-benchmark-profile/scripts/bench_diffusion_denoise.py
index 5a199aa11..b9269b5b9 100755
--- a/python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-benchmark-profile/scripts/bench_diffusion_denoise.py
+++ b/python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-benchmark-profile/scripts/bench_diffusion_denoise.py
@@ -16,8 +16,9 @@ Usage:
# Opt in to a compile control (presets are eager by default)
python3 python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-benchmark-profile/scripts/bench_diffusion_denoise.py --model flux --torch-compile
- # Check Eager/BCG at lossless/high on one GPU set; high+BCG is invalid when
- # request-scoped DiT fusions mount only after lossless graph capture.
+ # Check Eager/BCG at every request quality on one GPU set; extra-high/high
+ # + BCG are invalid when request-scoped DiT fusions mount only after the
+ # lossless graph capture.
python3 python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-benchmark-profile/scripts/bench_diffusion_denoise.py --model sana-video --quality-bcg-matrix --model-cache-root /task/model-caches --cleanup-model-cache
# Clean an isolated model cache even if the run fails or is interrupted
@@ -80,14 +81,14 @@ DIFFUSERS_FALLBACK_SIGNALS = (
"using diffusers backend",
"loaded diffusers pipeline",
)
-BENCHMARK_QUALITY_LEVELS = ("lossless", "high")
+BENCHMARK_QUALITY_LEVELS = ("lossless", "extra-high", "high")
BCG_CAPTURE_SIGNAL = "[diffusion bcg] captured"
BCG_INVALID_SIGNALS = (
"[diffusion bcg] capture failed",
"[diffusion bcg] disabled",
"[diffusion bcg] serving signature missed",
"no graph will be captured",
- "quality='high' cannot be used with breakable cuda graphs",
+ "cannot be used with breakable cuda graphs",
)
BCG_LATE_QUALITY_FUSION_SIGNAL = "quality fusion mounted after BCG capture"
QUALITY_BCG_ABBA_MATRIX = (
@@ -95,6 +96,10 @@ QUALITY_BCG_ABBA_MATRIX = (
("bcg-lossless-a", "lossless", True),
("bcg-lossless-b", "lossless", True),
("eager-lossless-b", "lossless", False),
+ ("eager-extra-high-a", "extra-high", False),
+ ("bcg-extra-high-a", "extra-high", True),
+ ("bcg-extra-high-b", "extra-high", True),
+ ("eager-extra-high-b", "extra-high", False),
("eager-high-a", "high", False),
("bcg-high-a", "high", True),
("bcg-high-b", "high", True),
@@ -1838,11 +1843,11 @@ def _run_benchmark_once_impl(
if BCG_CAPTURE_SIGNAL in lower_line:
bcg_capture_detected = True
if (
- quality == "high"
+ quality in {"extra-high", "high"}
and breakable_cuda_graph
and bcg_capture_detected
and "mounted " in lower_line
- and "for quality=high" in lower_line
+ and f"for quality={quality}" in lower_line
):
bcg_invalid_signals.add(BCG_LATE_QUALITY_FUSION_SIGNAL)
bcg_invalid_signals.update(
@@ -2256,7 +2261,8 @@ def main():
"--quality-bcg-matrix",
action="store_true",
help=(
- "Run lossless/high Eager-vs-BCG as two ABBA pairs on one GPU set "
+ "Run lossless/extra-high/high Eager-vs-BCG as three ABBA pairs "
+ "on one GPU set "
"and one task-owned model cache."
),
)
diff --git a/python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-performance/SKILL.md b/python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-performance/SKILL.md
index 4cd1d86ab..93f1438a1 100644
--- a/python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-performance/SKILL.md
+++ b/python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-performance/SKILL.md
@@ -62,7 +62,7 @@ These options **trade output quality** for speed or VRAM savings. Results will d
| Option | CLI Flag / Env Var | What It Does | Speedup | Quality Impact / Limitations |
|---|---|---|---|---|
-| **Request Quality Fast Paths** | `--quality high` (`lossless` is default) | Mounts model-owned accelerated DiT/VAE paths that are validated for high quality but are not bit-exact to the reference path. | Model- and shape-specific | Support is per model and may be a no-op. Keep `--quality lossless` as the A/B ground truth. Report aggregate and worst-frame SSIM/PSNR; defaults are 0.95/28 dB for images and 0.92/24 dB for video unless checked-in model metadata overrides them. Do not confuse this with `--output-quality`, which controls file compression. |
+| **Request Quality Fast Paths** | `--quality {extra-high,high}` (`lossless` is default) | `extra-high` mounts only request-gated DiT/VAE fusions. `high` includes that complete set and may add model-owned approximate paths such as Cache-DiT or lower-precision decode. | Model- and shape-specific | Support is per model and may be a no-op. Keep `--quality lossless` as the A/B ground truth, then compare `extra-high` before `high` to isolate fusion wins. Report aggregate and worst-frame SSIM/PSNR for every non-bit-exact path; defaults are 0.95/28 dB for images and 0.92/24 dB for video unless checked-in model metadata overrides them. Do not confuse this with `--output-quality`, which controls file compression. |
| **Approximate Attention** | Server-wide: `--attention-backend sage_attn` / `sage_attn_3` / `sliding_tile_attn` / `video_sparse_attn` / `sparse_video_gen_2_attn` / `vmoba_attn` / `sla_attn` / `sage_sla_attn`. Per-request (dense drop-ins only): `--attention-backend-override sage_attn` sampling param / API `extra_body` — valid values `fa`, `torch_sdpa`, `sage_attn`, `sage_attn_3`; rejected (with a log) under BCG, torch.compile, sparse server backends, or a non-ring-capable target with ring parallelism. | Replaces exact attention with approximate or sparse variants. `sage_attn`: INT8/FP8 quantized Q·K; `sliding_tile_attn`: spatial-temporal tile skipping; others: model-specific sparse patterns. | ~1.5–2x on attention (varies by backend) | Quality degradation varies by backend and model. `sage_attn` is the most general; sparse backends (`sliding_tile_attn`, `video_sparse_attn`, etc.) are video-model-specific, may require config files (e.g. `--mask-strategy-file-path` for STA), and are server-level only. Requires corresponding packages installed. |
| **Cache-DiT** | Native: per-request `--enable-cache-dit true\|false` + `--cache-dit-params ` (sampling params; also via API `extra_body`). `SGLANG_CACHE_DIT_ENABLED` / `SGLANG_CACHE_DIT_*` env vars are the server-wide defaults for requests that leave them unset. Diffusers backend: `--backend diffusers --cache-dit-config ` | Caches intermediate residuals across denoising steps and skips redundant computations via DBCache, TaylorSeer, and optional SCM. | ~1.5-2x on supported models | Quality depends on cache policy. Compatible with `--dit-layerwise-offload`: skipped blocks are not streamed, and the first layer after a skip may sync-load. Models that touch every layer before the block loop (for example a full-stack AdaLN prepass) must keep that prepass off while caching. Do not pass `--cache-dit-config` for native SGLang tuning unless you are intentionally using the diffusers backend flow. |
| **CFG Gating** | Per-request `--cfg-gate-step 0.5` (sampling param; also via API `extra_body`). `SGLANG_DIFFUSION_CFG_GATE_STEP` is the server-wide default (1.0 = off). | After the given fraction of denoising steps, reuses the cached cond-uncond residual instead of running the unconditional branch each step. | Up to ~2x on the gated tail of CFG models (skips one of two branches) | Lossy; no-op without classifier-free guidance or with `--enable-cfg-parallel`. Lower fractions gate earlier and drift more. |
@@ -252,7 +252,7 @@ For video, also match the captured frame and conditioning shape; `WxH` alone
does not prove replay.
For a repeated discovery sweep, use the benchmark/profile helper. This runs
-lossless and high-quality Eager/BCG ABBA pairs on one GPU set, then deletes the
+lossless, extra-high, and high Eager/BCG ABBA pairs on one GPU set, then deletes the
model group cache once:
```bash
@@ -262,20 +262,25 @@ python3 python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-benchmark-p
--cleanup-model-cache
```
-### Compare request-scoped high-quality fast paths
+### Compare cumulative request-quality fast paths
```bash
sglang generate --model-path \
--quality lossless --prompt "..." --seed 42 \
--perf-dump-path baseline.json --save-output
+sglang generate --model-path \
+ --quality extra-high --prompt "..." --seed 42 \
+ --perf-dump-path quality-extra-high.json --save-output
+
sglang generate --model-path \
--quality high --prompt "..." --seed 42 \
--perf-dump-path quality-high.json --save-output
```
Keep every other flag fixed and compare the generated artifact as well as the
-perf dumps. If the model has no registered quality-gated sites, `high` may be a
+perf dumps. `high` must retain every fusion observed under `extra-high`. If the
+model has no registered request-gated or high-only sites, either tier may be a
no-op.
### Image-edit baselines: JoyAI and FireRed
@@ -384,7 +389,7 @@ Use these as first commands to benchmark, not as universal winners.
| Z-Image / Z-Image-Turbo | 1024x1024, runtime-default steps/guidance, 1 GPU | `--enable-torch-compile --warmup-mode request` | Keep base Z-Image separate from Turbo: base uses 50-step CFG defaults, Turbo uses 9-step zero-CFG defaults. Mainline has bf16-native Triton RMSNorm scale and tanh-residual fusions. |
| Wan2.2 A14B T2V/I2V | 1280x720, 81 frames | Nightly: `--num-gpus 4 --enable-cfg-parallel --ulysses-degree 2 --text-encoder-cpu-offload --pin-cpu-memory` | For lowest latency, also benchmark pure Ulysses on the same GPUs. |
| Wan2.2 TI2V 5B | 1280x720, 81 frames, 1 GPU | `--enable-torch-compile --warmup-mode request` | Keep the input image and motion prompt fixed when comparing sparse attention or Cache-DiT. |
-| Wan2.1 / FastWan / TurboWan variants | 480p or 720p video, family defaults | Compare `--quality lossless` with `--quality high`, then try `--enable-torch-compile --warmup-mode request`; add `--ulysses-degree` / CFG parallel only after measuring | `quality=high` mounts the Wan FFN cublasLt GELU epilogue and the Wan VAE RMSNorm+SiLU fast path when their guards pass; validate video quality against lossless. Current registry includes Wan2.1, FastWan2.1, FastWan2.2 TI2V, TurboWan2.1, TurboWan2.2 I2V, and Wan2.1-Fun InP. Use the compatibility matrix and benchmark presets before choosing topology. |
+| Wan2.1 / FastWan / TurboWan variants | 480p or 720p video, family defaults | Compare `--quality lossless`, `--quality extra-high`, and `--quality high`, then try `--enable-torch-compile --warmup-mode request`; add `--ulysses-degree` / CFG parallel only after measuring | `extra-high` and `high` mount the Wan FFN cublasLt/NVFP4 GELU epilogues and the Wan VAE RMSNorm+SiLU fast path when their guards pass; validate video quality against lossless. Current registry includes Wan2.1, FastWan2.1, FastWan2.2 TI2V, TurboWan2.1, TurboWan2.2 I2V, and Wan2.1-Fun InP. Use the compatibility matrix and benchmark presets before choosing topology. |
| Cosmos3 Nano / Super | T2I: 1024x1024 with `--num-frames 1`; T2V/I2V: 480p/720p video | Start with `--performance-mode auto --warmup-mode request`; use `SGLANG_DISABLE_COSMOS3_GUARDRAILS=1` only for benchmark isolation, and compare compile separately | One checkpoint serves T2I/T2V/I2V. Mode is request-driven: `num_frames == 1` means T2I, `--image-path` means I2V. On GPUs with at least 120 GiB available, auto mode keeps the Cosmos3 DiT and VAE resident for every checkpoint in the family; a 1xH200 832x480x9f, 4-step eager ABBA reduced e2e from 1.576 to 0.428 seconds with exact output parity. Cosmos3 runs one DiT per pipeline, so component offload above that threshold only buys a DiT copy out to host memory and back per request -- it cost Cosmos3-Super 720p 81f T2V ~4s of ~115s on 2xH200. |
| Cosmos3 Edge / distilled Super | Edge T2I: 640x640, 35 steps, 1 GPU; distilled Super T2I: 640x640, fixed 4-step schedule, 4 GPUs | Start eager with `--performance-mode manual`; use `SGLANG_DISABLE_COSMOS3_GUARDRAILS=1` only for benchmark isolation | Edge is trained for 256p/480p shapes. Distilled checkpoints own their sigma schedule and force guidance 1.0; do not override steps or flow shift. Do not retry the closed experimental Cosmos BCG path without a new lifecycle design. |
| Ideogram 4 FP8/NVFP4 | 1024x1024, native preset defaults | `--enable-torch-compile --warmup-mode request` | Do not set `--num-inference-steps` or `--guidance-scale` directly unless you also update the Ideogram preset; sampling params derive them from `preset`. |
@@ -397,7 +402,7 @@ Use these as first commands to benchmark, not as universal winners.
| JoyAI-Image-Edit | 1024-class TI2I, 40 steps, guidance 4.0 | `--backend=sglang --num-gpus 2 --enable-cfg-parallel --ulysses-degree 1 --enable-torch-compile --warmup-mode request --dit-layerwise-offload false --dit-cpu-offload false` | Newly supported image-edit path. Keep the input image, prompt, seed, and output size fixed; 2-GPU CFG parallel is the validated H100 starting point. |
| FireRed-Image-Edit 1.0 / 1.1 | 1024x1024 image edit, 40 steps, guidance 4.0 | `--backend=sglang --num-gpus 2 --enable-cfg-parallel --ulysses-degree 1 --enable-torch-compile --warmup-mode request --dit-layerwise-offload false --dit-cpu-offload false` | Uses the native `QwenImageEditPlusPipeline` path. 2-GPU CFG parallel is the validated H100 starting point; benchmark 1.0 and 1.1 separately because checkpoint differences can change denoise latency. |
| Hunyuan3D-2 shape | Shape generation, 50 steps, guidance 5.0 | `--backend=sglang --enable-torch-compile --warmup-mode request --dit-layerwise-offload false --dit-cpu-offload false` | Focus on `Hunyuan3DShapeDenoisingStage`; keep mesh export/paint timings separate from denoise. |
-| LingBot Video MoE 30B | 384x640, 17 frames, 12 steps for the current GPU case | `--model-path robbyant/lingbot-video-moe-30b-a3b --text-encoder-cpu-offload` | Native T2V path. Prompts are structured JSON captions, not raw free text; keep that contract when comparing latency or quality. Main still expands RMSNorm into PyTorch reduction chains; #35969 is an open `quality=high` Triton-dispatch candidate, not a current-main option until merged. |
+| LingBot Video MoE 30B | 384x640, 17 frames, 12 steps for the current GPU case | `--model-path robbyant/lingbot-video-moe-30b-a3b --text-encoder-cpu-offload` | Native T2V path. Prompts are structured JSON captions, not raw free text; keep that contract when comparing latency or quality. Current main can mount the fused Triton RMSNorm path at `quality=extra-high` or `quality=high`; keep `lossless` as the reference. |
| MOVA / Helios / LingBot World | Use the benchmark/profile presets or server test cases first | `--enable-torch-compile --warmup-mode request`; pin offload and topology flags explicitly | These video/realtime families have model-specific stages and condition handling. For LingBot World causal serving, keep `--kv-cache-quant off` as the exact cache baseline before testing INT4/INT2. |
## Historical PR Watchlist
diff --git a/python/sglang/multimodal_gen/apps/ComfyUI_SGLDiffusion/core/server_api.py b/python/sglang/multimodal_gen/apps/ComfyUI_SGLDiffusion/core/server_api.py
index 0b3f92bec..602eca0c9 100644
--- a/python/sglang/multimodal_gen/apps/ComfyUI_SGLDiffusion/core/server_api.py
+++ b/python/sglang/multimodal_gen/apps/ComfyUI_SGLDiffusion/core/server_api.py
@@ -99,7 +99,8 @@ class SGLDiffusionServerAPI:
seed: Random seed for reproducible generation
enable_teacache: Enable TEA cache acceleration
response_format: Response format ("b64_json" or "url")
- quality: Image quality ("auto", "standard", "hd") - only for generation
+ quality: Request optimization tier ("auto", "lossless",
+ "extra-high", "high") - only for generation
style: Image style ("vivid" or "natural") - only for generation
background: Background type ("auto", "transparent", "opaque")
output_format: Output format ("png", "jpeg", "webp")
diff --git a/python/sglang/multimodal_gen/configs/sample/sampling_params.py b/python/sglang/multimodal_gen/configs/sample/sampling_params.py
index 68bca19ca..d169d7ddc 100644
--- a/python/sglang/multimodal_gen/configs/sample/sampling_params.py
+++ b/python/sglang/multimodal_gen/configs/sample/sampling_params.py
@@ -51,10 +51,17 @@ def generate_request_id() -> str:
return str(uuid.uuid4())
-# Validated request-level quality levels. "lossless" is the exact reference
-# path (bit-exact against the CI golden outputs); "high" opts into validated
-# accelerated paths whose quality is guaranteed but not bit-exact.
-QUALITY_LEVELS: tuple[str, ...] = ("lossless", "high")
+# Validated request-level quality levels, ordered from the strictest numerical
+# contract to the broadest optimization set. "lossless" keeps the exact
+# reference path; "extra-high" adds only request-gated kernel fusions; "high"
+# is cumulative and may also enable model-owned approximate optimizations.
+QUALITY_LEVELS: tuple[str, ...] = ("lossless", "extra-high", "high")
+KERNEL_FUSION_QUALITY_LEVELS = frozenset({"extra-high", "high"})
+
+
+def quality_allows_kernel_fusions(quality: str) -> bool:
+ """Return whether a quality level includes request-gated kernel fusions."""
+ return quality in KERNEL_FUSION_QUALITY_LEVELS
def _sanitize_filename(name: str, replacement: str = "_", max_length: int = 150) -> str:
@@ -141,15 +148,15 @@ class SamplingParams:
# - "lossless" (default): the exact reference path. Output is expected to
# be bit-identical to the HF reference implementation and to pass the
# CI golden/ground-truth comparisons.
- # - "high": opt into validated accelerated paths. Quality stays
- # guaranteed (the intent is to back every such path with mathematical
- # acceptance thresholds, e.g. PSNR > 25 against the reference), but
- # the output is no longer bit-exact versus the HF reference or the CI
- # ground truth.
+ # - "extra-high": add only validated kernel fusions. These may change
+ # half-precision rounding order, so output is not bit-exact versus the
+ # reference, but this tier does not itself enable sparse or approximate
+ # optimizations.
+ # - "high": include every "extra-high" fusion and allow model-owned
+ # approximate optimizations such as sparse computation or feature
+ # caching. These paths require model-specific quality validation.
#
- # Models that support "high" must validate the deployment and workload
- # explicitly. It intentionally participates in the dynamic-batch
- # signature.
+ # It intentionally participates in the dynamic-batch signature.
quality: str = "lossless"
# Frame interpolation
@@ -1120,10 +1127,12 @@ class SamplingParams:
help=(
"Request-level quality: 'lossless' (default) keeps the exact "
"reference path, bit-exact against the reference "
- "implementation; 'high' opts into the model-owned validated "
- "accelerated path, whose quality stays guaranteed but is not "
- "bit-exact. Support and validated deployment constraints are "
- "model-specific."
+ "implementation; 'extra-high' adds only request-gated kernel "
+ "fusions and does not itself enable sparse or approximate "
+ "optimization; 'high' includes every extra-high fusion and "
+ "may also enable "
+ "model-owned approximate paths. Support and validated "
+ "deployment constraints are model-specific."
),
)
add_argument(
diff --git a/python/sglang/multimodal_gen/runtime/models/dits/flux.py b/python/sglang/multimodal_gen/runtime/models/dits/flux.py
index 1e328fe91..ff5c6d186 100644
--- a/python/sglang/multimodal_gen/runtime/models/dits/flux.py
+++ b/python/sglang/multimodal_gen/runtime/models/dits/flux.py
@@ -169,10 +169,11 @@ def _flux_norm_modulate(
"""``norm(x) * (1 + scale) + shift`` for the FLUX adaLN sites.
Priority: (1) the bit-exact single-kernel LN+modulate -- lossless, so it
- needs no quality gate and also supersedes the ``quality="high"`` affine
+ needs no quality gate and also supersedes the request-gated affine
fold wherever it verifies; (2) when the site is mounted
- (``quality="high"``) and the bit-exact kernel is unavailable, the
- modulate folded into the LN affine (one aten kernel; not bit-exact);
+ (``quality="extra-high"`` or ``"high"``) and the bit-exact kernel is
+ unavailable, the modulate folded into the LN affine (one aten kernel; not
+ bit-exact);
(3) affine-free LayerNorm + the bit-exact fused modulate.
"""
out = _flux_fused_ln_modulate(norm, x, scale, shift)
@@ -387,7 +388,7 @@ class FluxGELU(nn.Module):
prefix=f"{prefix}.proj" if prefix else "proj",
)
self.gelu = nn.GELU(approximate="tanh")
- # quality="high" fusion site: up-proj GEMM + tanh-GELU in the cublasLt
+ # extra-high/high fusion site: up-proj GEMM + tanh-GELU in the cublasLt
# epilogue. Off by default; mounted per batch by the denoising stage.
mark_fused_gelu_site(self, "proj")
@@ -407,7 +408,7 @@ class FluxFusedGELUProj(nn.Module):
``approximate="tanh"`` that keeps the ``net.0.proj`` parameter path. The
default path is the bit-exact reference (plain Linear + tanh-GELU); the
cublasLt GELU epilogue is mounted per batch by the denoising stage for
- quality="high" requests only.
+ requests with ``quality="extra-high"`` or ``quality="high"`` only.
"""
def __init__(self, proj: nn.Linear):
@@ -790,7 +791,7 @@ class FluxSingleTransformerBlock(nn.Module):
prefix=f"{prefix}.proj_mlp" if prefix else "proj_mlp",
)
self.act_mlp = nn.GELU(approximate="tanh")
- # quality="high" fusion site: proj_mlp GEMM + tanh-GELU in the
+ # extra-high/high fusion site: proj_mlp GEMM + tanh-GELU in the
# cublasLt epilogue (mounted per batch by the denoising stage).
mark_fused_gelu_site(self, "proj_mlp")
proj_out_cls = (
@@ -954,7 +955,7 @@ class FluxTransformerBlock(nn.Module):
self.norm2 = LayerNorm(dim, eps=1e-6, elementwise_affine=False)
self.norm2_context = LayerNorm(dim, eps=1e-6, elementwise_affine=False)
- # quality="high" site: the norm2/norm2_context modulate folds into the
+ # extra-high/high site: norm2/norm2_context modulate folds into the
# LN affine when mounted.
mark_fused_ln_modulate_site(self)
@@ -1009,7 +1010,7 @@ class FluxTransformerBlock(nn.Module):
activation_fn="gelu-approximate",
)
# Re-home each FF's tanh-GELU up-projection onto a marked
- # quality="high" fusion site (bit-exact reference by default).
+ # extra-high/high fusion site (bit-exact reference by default).
self.ff.net[0] = FluxFusedGELUProj(self.ff.net[0].proj)
self.ff_context.net[0] = FluxFusedGELUProj(self.ff_context.net[0].proj)
diff --git a/python/sglang/multimodal_gen/runtime/models/dits/glm_image.py b/python/sglang/multimodal_gen/runtime/models/dits/glm_image.py
index 45647eb78..97fd79a73 100644
--- a/python/sglang/multimodal_gen/runtime/models/dits/glm_image.py
+++ b/python/sglang/multimodal_gen/runtime/models/dits/glm_image.py
@@ -447,7 +447,7 @@ class GlmImageGELU(nn.Module):
quant_config=quant_config,
prefix=f"{prefix}.proj" if prefix else "proj",
)
- # quality="high" fusion site: up-proj GEMM + tanh-GELU in the cublasLt
+ # extra-high/high fusion site: up-proj GEMM + tanh-GELU in cublasLt
# epilogue. Off by default; mounted per batch by the denoising stage.
mark_fused_gelu_site(self, "proj")
diff --git a/python/sglang/multimodal_gen/runtime/models/dits/ideogram.py b/python/sglang/multimodal_gen/runtime/models/dits/ideogram.py
index 30cecf1cb..691283302 100644
--- a/python/sglang/multimodal_gen/runtime/models/dits/ideogram.py
+++ b/python/sglang/multimodal_gen/runtime/models/dits/ideogram.py
@@ -421,7 +421,7 @@ def _norm_scale(
norm: Ideogram4RMSNorm,
enable_fused: bool,
) -> torch.Tensor:
- """``RMSNorm(x) * (1 + scale)``, fused for ``quality="high"`` batches."""
+ """``RMSNorm(x) * (1 + scale)``, fused at extra-high or high quality."""
if enable_fused:
y = fused_rmsnorm_scale(
x,
@@ -455,7 +455,7 @@ def _gate_residual(
norm: Ideogram4RMSNorm,
enable_fused: bool,
) -> torch.Tensor:
- """``residual + tanh(gate) * RMSNorm(x)``, fused for ``quality="high"``."""
+ """``residual + tanh(gate) * RMSNorm(x)``, fused at extra-high or high."""
if enable_fused:
y = fused_rmsnorm_tanh_residual(
x,
@@ -511,7 +511,7 @@ class Ideogram4TransformerBlock(nn.Module):
self.ffn_norm1 = Ideogram4RMSNorm(hidden_size, eps=norm_eps)
self.attention_norm2 = Ideogram4RMSNorm(hidden_size, eps=norm_eps)
self.ffn_norm2 = Ideogram4RMSNorm(hidden_size, eps=norm_eps)
- # quality="high" fusion sites: each RMSNorm modulate/gate chain
+ # extra-high/high fusion sites: each RMSNorm modulate/gate chain
# collapses into one Triton kernel (Z-Image bf16-native suite). Off by
# default (bit-exact reference path); mounted per batch by the
# denoising stage.
diff --git a/python/sglang/multimodal_gen/runtime/models/dits/longcat_image.py b/python/sglang/multimodal_gen/runtime/models/dits/longcat_image.py
index b2d5ba293..eb85876cd 100644
--- a/python/sglang/multimodal_gen/runtime/models/dits/longcat_image.py
+++ b/python/sglang/multimodal_gen/runtime/models/dits/longcat_image.py
@@ -222,7 +222,7 @@ class _LongCatFFN(nn.Module):
]
)
self.act = nn.GELU(approximate="tanh")
- # quality="high" site: up-proj GEMM + tanh-GELU cublasLt epilogue. Off by
+ # extra-high/high site: up-proj GEMM + tanh-GELU epilogue. Off by
# default; the denoising stage mounts it per batch. The ModuleDict holds
# `proj` in _modules, so getattr resolves it for the fusion helper.
mark_fused_gelu_site(self.net[0], "proj")
@@ -511,7 +511,7 @@ class _SingleTransformerBlock(nn.Module):
prefix=f"{prefix}.proj_mlp",
)
self.act_mlp = nn.GELU(approximate="tanh")
- # quality="high" site: proj_mlp GEMM + tanh-GELU cublasLt epilogue,
+ # extra-high/high site: proj_mlp GEMM + tanh-GELU epilogue,
# mounted per batch by the denoising stage; off (bit-exact) by default.
mark_fused_gelu_site(self, "proj_mlp")
# proj_out: RowParallelLinear reduces sharded [attn | mlp] concat via
diff --git a/python/sglang/multimodal_gen/runtime/models/dits/ltx_2.py b/python/sglang/multimodal_gen/runtime/models/dits/ltx_2.py
index bc40d0901..8e25ddb05 100644
--- a/python/sglang/multimodal_gen/runtime/models/dits/ltx_2.py
+++ b/python/sglang/multimodal_gen/runtime/models/dits/ltx_2.py
@@ -198,7 +198,7 @@ def _ltx2_rms_norm_modulate(
"""``rms_norm(x) * (1 + scale) + shift`` for the LTX-2 adaLN sites.
Folds the weightless RMSNorm and the modulate into one kernel when the
- ``quality="high"`` fusion is mounted on ``block`` and the per-call guard
+ request-gated fusion is mounted on ``block`` and the per-call guard
passes; otherwise the verbatim eager reference chain (the ``lossless``
default). The fused kernel is not bit-exact (<=1 bf16 ULP) so it is gated
on the request-scoped mount rather than a runtime self-check.
diff --git a/python/sglang/multimodal_gen/runtime/models/dits/qwen_image.py b/python/sglang/multimodal_gen/runtime/models/dits/qwen_image.py
index 40ab1ab4f..944b67161 100644
--- a/python/sglang/multimodal_gen/runtime/models/dits/qwen_image.py
+++ b/python/sglang/multimodal_gen/runtime/models/dits/qwen_image.py
@@ -824,7 +824,7 @@ class QwenImageCrossAttention(nn.Module):
)
if self._unquantized_added_qkv_is_packed:
# Packing changes BF16 GEMM reduction association. Keep it
- # off for lossless requests and mount it for quality=high.
+ # off for lossless and mount it at extra-high or high.
mark_qwen_image_added_qkv_site(self)
else:
self.add_q_proj = ColumnParallelLinear(
@@ -1171,7 +1171,7 @@ class QwenImageGELU(nn.Module):
quant_config=quant_config,
prefix=f"{prefix}.proj",
)
- # quality="high" fusion site: up-proj GEMM + tanh-GELU in the cublasLt
+ # Extra-high-or-higher fusion site: up-proj GEMM + tanh-GELU in cublasLt
# epilogue. Off by default; mounted per batch by the denoising stage.
mark_fused_gelu_site(self, "proj")
diff --git a/python/sglang/multimodal_gen/runtime/models/vaes/flux2_vae_cuda_opt.py b/python/sglang/multimodal_gen/runtime/models/vaes/flux2_vae_cuda_opt.py
index 73fb7905c..4bf19a53a 100644
--- a/python/sglang/multimodal_gen/runtime/models/vaes/flux2_vae_cuda_opt.py
+++ b/python/sglang/multimodal_gen/runtime/models/vaes/flux2_vae_cuda_opt.py
@@ -8,8 +8,9 @@ decoder module family (``ResnetBlock2D`` GroupNorm+SiLU chains,
All rewrites are mathematically exact re-associations of the original
operators. Wrappers are installed once at VAE load and dispatch on a
-decode-scoped :class:`VaeFastPathGate`: ``quality == "high"`` runs the fast
-paths, the ``"lossless"`` default runs the original module path bit-for-bit.
+decode-scoped :class:`VaeFastPathGate`: ``quality="extra-high"`` and
+``quality="high"`` run the fast paths, while the ``"lossless"`` default runs
+the original module path bit-for-bit.
- channels_last: run the decoder in NHWC so cuDNN convs skip the transpose
kernels; parameter layout is swapped at decode entry to match the gate.
diff --git a/python/sglang/multimodal_gen/runtime/models/vaes/wan_vae_cuda_opt.py b/python/sglang/multimodal_gen/runtime/models/vaes/wan_vae_cuda_opt.py
index 3044762bd..eb53a88f7 100644
--- a/python/sglang/multimodal_gen/runtime/models/vaes/wan_vae_cuda_opt.py
+++ b/python/sglang/multimodal_gen/runtime/models/vaes/wan_vae_cuda_opt.py
@@ -3,10 +3,10 @@
Fuses every decoder ``WanRMS_norm -> SiLU`` chain into one Triton kernel on
the channels_last_3d layout. Wrappers are installed once at VAE load and
-dispatch on a decode-scoped :class:`VaeFastPathGate`: ``quality == "high"``
-runs the fused kernel (not bitwise-identical to aten, hence gated), the
-``"lossless"`` default runs the original module path bit-for-bit. Install is
-all-or-nothing and fail-closed.
+dispatch on a decode-scoped :class:`VaeFastPathGate`: ``quality="extra-high"``
+and ``quality="high"`` run the fused kernel (not bitwise-identical to aten,
+hence gated), while the ``"lossless"`` default runs the original module path
+bit-for-bit. Install is all-or-nothing and fail-closed.
"""
import torch
diff --git a/python/sglang/multimodal_gen/runtime/pipelines_core/stages/decoding.py b/python/sglang/multimodal_gen/runtime/pipelines_core/stages/decoding.py
index 53cc5ed56..ca65b357d 100644
--- a/python/sglang/multimodal_gen/runtime/pipelines_core/stages/decoding.py
+++ b/python/sglang/multimodal_gen/runtime/pipelines_core/stages/decoding.py
@@ -10,6 +10,9 @@ import weakref
import torch
import torch.nn as nn
+from sglang.multimodal_gen.configs.sample.sampling_params import (
+ quality_allows_kernel_fusions,
+)
from sglang.multimodal_gen.runtime.distributed import (
get_decode_parallel_world_size,
get_local_torch_device,
@@ -333,7 +336,10 @@ class DecodingStage(PipelineStage):
assert vae is not None
self.vae = vae
- with use_vae_fast_path(vae, batch.sampling_params.quality == "high"):
+ with use_vae_fast_path(
+ vae,
+ quality_allows_kernel_fusions(batch.sampling_params.quality),
+ ):
frames = self.decode(batch.latents, server_args, vae_dtype=vae_dtype)
# decode trajectory latents if needed
diff --git a/python/sglang/multimodal_gen/runtime/pipelines_core/stages/denoising.py b/python/sglang/multimodal_gen/runtime/pipelines_core/stages/denoising.py
index 1d79a4a47..03cc84fa0 100644
--- a/python/sglang/multimodal_gen/runtime/pipelines_core/stages/denoising.py
+++ b/python/sglang/multimodal_gen/runtime/pipelines_core/stages/denoising.py
@@ -49,6 +49,9 @@ from sglang.multimodal_gen.configs.pipeline_configs.flux import (
FluxPipelineConfig,
)
from sglang.multimodal_gen.configs.pipeline_configs.zimage import ZImagePipelineConfig
+from sglang.multimodal_gen.configs.sample.sampling_params import (
+ quality_allows_kernel_fusions,
+)
from sglang.multimodal_gen.runtime.breakable_cuda_graph import (
prompt_padding as bcg_utils,
)
@@ -326,7 +329,7 @@ class DenoisingStage(PipelineStage, RolloutDenoisingMixin):
self._cache_dit_request_overrides: dict[str, Any] = {}
# Overrides key the mounted hooks were built from; None when unmounted.
self._cache_dit_active_key: tuple | None = None
- # Whether request-scoped quality="high" fusions are currently mounted.
+ # Whether request-scoped extra-high-or-higher fusions are mounted.
self._quality_fusions_mounted = False
self._torch_compile_registry = CompiledModuleRegistry()
# Breakable CUDA graph runners, one per transformer module (lazy).
@@ -666,17 +669,18 @@ class DenoisingStage(PipelineStage, RolloutDenoisingMixin):
return stage_backend
def _maybe_toggle_quality_fusions(self, batch: Req) -> None:
- """Mount/unmount the ``quality="high"`` fusions for this batch.
+ """Mount/unmount request-gated kernel fusions for this batch.
These fusions are numerically equivalent only at half-precision
- rounding level (not bit-exact), so they are mounted for
- ``quality="high"`` requests and unmounted otherwise. The
- ``"lossless"`` default runs the reference path bit-for-bit. ``quality``
- participates in the dynamic-batch signature, making this transition
- safe at the batch boundary. Mounting is all-or-nothing per transformer
- and fusion family; models without marked sites are no-ops.
+ rounding level (not bit-exact), so they are mounted for both
+ ``quality="extra-high"`` and ``quality="high"``. The ``"lossless"``
+ default runs the reference path bit-for-bit. ``quality`` participates
+ in the dynamic-batch signature, making this transition safe at the
+ batch boundary. Mounting is all-or-nothing per transformer and fusion
+ family; models without marked sites are no-ops.
"""
- want = getattr(batch.sampling_params, "quality", "lossless") == "high"
+ quality = getattr(batch.sampling_params, "quality", "lossless")
+ want = quality_allows_kernel_fusions(quality)
if want == self._quality_fusions_mounted:
return
mounted_fusions: set[str] = set()
@@ -694,7 +698,7 @@ class DenoisingStage(PipelineStage, RolloutDenoisingMixin):
unmount(transformer)
descriptions = ", ".join(sorted(mounted_fusions))
raise ValueError(
- "quality='high' cannot be used with breakable CUDA graphs for "
+ f"quality={quality!r} cannot be used with breakable CUDA graphs for "
f"this model because its request-scoped DiT fusions "
f"({descriptions}) do not match the lossless warmup graphs. "
"Disable breakable CUDA graphs or use quality='lossless'."
@@ -702,7 +706,7 @@ class DenoisingStage(PipelineStage, RolloutDenoisingMixin):
self._quality_fusions_mounted = want
for description in sorted(mounted_fusions):
- logger.info("Mounted %s for quality=high", description)
+ logger.info("Mounted %s for quality=%s", description, quality)
def _cache_dit_dual_model_name(self) -> str:
return "wan2.2"
diff --git a/python/sglang/multimodal_gen/runtime/platforms/cuda.py b/python/sglang/multimodal_gen/runtime/platforms/cuda.py
index 1c1e54294..46ad5b686 100644
--- a/python/sglang/multimodal_gen/runtime/platforms/cuda.py
+++ b/python/sglang/multimodal_gen/runtime/platforms/cuda.py
@@ -685,8 +685,8 @@ class CudaPlatformBase(Platform):
"""Install the quality-gated FLUX.2 / AutoencoderKL / Wan VAE decoder
fast paths.
- Requests with quality == "high" run the fast paths; the "lossless"
- default runs the original module path bit-for-bit. See
+ Requests with quality="extra-high" or "high" run the fast paths; the
+ "lossless" default runs the original module path bit-for-bit. See
flux2_vae_cuda_opt and wan_vae_cuda_opt for details.
"""
try:
diff --git a/python/sglang/multimodal_gen/test/unit/test_diffusion_bcg_padding.py b/python/sglang/multimodal_gen/test/unit/test_diffusion_bcg_padding.py
index 79e00feb3..9e45604d6 100644
--- a/python/sglang/multimodal_gen/test/unit/test_diffusion_bcg_padding.py
+++ b/python/sglang/multimodal_gen/test/unit/test_diffusion_bcg_padding.py
@@ -70,22 +70,26 @@ class TestQualityFusionBCGCompatibility(unittest.TestCase):
def _batch(quality: str):
return SimpleNamespace(sampling_params=SimpleNamespace(quality=quality))
- def test_rejects_high_when_dit_fusion_would_replace_captured_graph(self):
- unmounted = []
- handlers = (
- (
- "test fusion",
- lambda _: True,
- lambda transformer: unmounted.append(transformer),
- ),
- )
+ def test_rejects_fusion_levels_when_they_would_replace_captured_graph(self):
+ for quality in ("extra-high", "high"):
+ with self.subTest(quality=quality):
+ unmounted = []
+ handlers = (
+ (
+ "test fusion",
+ lambda _: True,
+ lambda transformer: unmounted.append(transformer),
+ ),
+ )
- with patch.object(denoising_module, "_QUALITY_FUSION_HANDLERS", handlers):
- with self.assertRaisesRegex(ValueError, "lossless warmup graphs"):
- self.stage._maybe_toggle_quality_fusions(self._batch("high"))
+ with patch.object(
+ denoising_module, "_QUALITY_FUSION_HANDLERS", handlers
+ ):
+ with self.assertRaisesRegex(ValueError, "lossless warmup graphs"):
+ self.stage._maybe_toggle_quality_fusions(self._batch(quality))
- self.assertEqual(unmounted, [self.stage.transformer])
- self.assertFalse(self.stage._quality_fusions_mounted)
+ self.assertEqual(unmounted, [self.stage.transformer])
+ self.assertFalse(self.stage._quality_fusions_mounted)
def test_allows_high_when_model_has_no_dit_quality_fusions(self):
handlers = (("test fusion", lambda _: False, lambda _: None),)
@@ -95,6 +99,21 @@ class TestQualityFusionBCGCompatibility(unittest.TestCase):
self.assertTrue(self.stage._quality_fusions_mounted)
+ def test_high_keeps_extra_high_fusions_mounted(self):
+ self.stage.server_args.enable_breakable_cuda_graph = False
+ mounted = []
+ handlers = (
+ ("test fusion", lambda _: mounted.append(True) or True, lambda _: None),
+ )
+
+ with patch.object(denoising_module, "_QUALITY_FUSION_HANDLERS", handlers):
+ self.stage._maybe_toggle_quality_fusions(self._batch("extra-high"))
+ self.assertTrue(self.stage._quality_fusions_mounted)
+ self.stage._maybe_toggle_quality_fusions(self._batch("high"))
+
+ self.assertEqual(mounted, [True])
+ self.assertTrue(self.stage._quality_fusions_mounted)
+
def _fake_cache_dit_batch(*, is_warmup: bool) -> SimpleNamespace:
return SimpleNamespace(
diff --git a/python/sglang/multimodal_gen/test/unit/test_diffusion_benchmark_skill.py b/python/sglang/multimodal_gen/test/unit/test_diffusion_benchmark_skill.py
index c0b233391..795626b82 100644
--- a/python/sglang/multimodal_gen/test/unit/test_diffusion_benchmark_skill.py
+++ b/python/sglang/multimodal_gen/test/unit/test_diffusion_benchmark_skill.py
@@ -257,6 +257,12 @@ class TestDiffusionBenchmarkSkill(unittest.TestCase):
self.assertIn("--quality=high", high_cmd)
self.assertNotIn("--enable-breakable-cuda-graph", high_cmd)
+ extra_high_cmd = module.build_sglang_cmd(
+ "longcat-image", quality="extra-high"
+ )
+ self.assertIn("--quality=extra-high", extra_high_cmd)
+ self.assertNotIn("--enable-breakable-cuda-graph", extra_high_cmd)
+
bcg_cmd = module.build_sglang_cmd(
"longcat-image",
breakable_cuda_graph=True,
@@ -538,6 +544,37 @@ class TestDiffusionBenchmarkSkill(unittest.TestCase):
[module.BCG_LATE_QUALITY_FUSION_SIGNAL],
)
+ def test_extra_high_bcg_rejects_quality_fusion_mounted_after_capture(self):
+ with tempfile.TemporaryDirectory() as tmpdir:
+ temp_root = Path(tmpdir)
+ module = _load_benchmark_module(temp_root)
+ output_dir = temp_root / "outputs"
+ output_dir.mkdir()
+
+ with patch.object(module.subprocess, "Popen") as popen:
+ popen.return_value.stdout = iter(
+ (
+ "[Diffusion BCG] captured 3 segment(s)\n",
+ "Mounted Qwen fused added-QKV for quality=extra-high\n",
+ )
+ )
+ popen.return_value.wait.return_value = 0
+ result = module._run_benchmark_once_impl(
+ "longcat-image",
+ "bcg-extra-high",
+ output_dir,
+ warmup=False,
+ quality="extra-high",
+ breakable_cuda_graph=True,
+ cuda_visible_devices="0",
+ )
+
+ self.assertTrue(result["error"])
+ self.assertEqual(
+ result["bcg_invalid_signals"],
+ [module.BCG_LATE_QUALITY_FUSION_SIGNAL],
+ )
+
def test_quality_bcg_matrix_reuses_one_gpu_set_and_cleans_once(self):
with tempfile.TemporaryDirectory() as tmpdir:
temp_root = Path(tmpdir)
@@ -564,7 +601,7 @@ class TestDiffusionBenchmarkSkill(unittest.TestCase):
cleanup_model_cache=True,
)
- self.assertEqual(len(results), 8)
+ self.assertEqual(len(results), 12)
self.assertEqual(
[
(call[2]["quality"], call[2]["breakable_cuda_graph"])
diff --git a/python/sglang/multimodal_gen/test/unit/test_minimax_h3_admission.py b/python/sglang/multimodal_gen/test/unit/test_minimax_h3_admission.py
index 0daf8e023..ec74ed9cd 100644
--- a/python/sglang/multimodal_gen/test/unit/test_minimax_h3_admission.py
+++ b/python/sglang/multimodal_gen/test/unit/test_minimax_h3_admission.py
@@ -23,6 +23,7 @@ from sglang.multimodal_gen.runtime.managers.memory_managers.component_residency
LAYERWISE_OFFLOAD,
RESIDENT,
)
+from sglang.multimodal_gen.runtime.pipelines_core.stages.denoising import DenoisingStage
from sglang.multimodal_gen.runtime.pipelines_core.stages.model_specific_stages.minimax_h3.release_metadata import (
MiniMaxH3PartitionAdmissionStage,
MiniMaxH3ReleaseMetadata,
@@ -351,6 +352,31 @@ def test_high_quality_request_warns_when_bcg_suppresses_cache_dit():
)
+def test_extra_high_quality_does_not_enable_h3_cache_dit():
+ stage = MiniMaxH3DenoisingStage.__new__(MiniMaxH3DenoisingStage)
+ stage.server_args = SimpleNamespace(enable_breakable_cuda_graph=False)
+ stage._cache_dit_enabled = False
+ stage._minimax_h3_cache_mode = None
+ stage._minimax_h3_quality = "lossless"
+ batch = SimpleNamespace(
+ sampling_params=SimpleNamespace(
+ quality="extra-high",
+ _explicit_fields={"quality"},
+ enable_cache_dit=None,
+ cache_dit_params=None,
+ )
+ )
+
+ # Even a server-wide generic Cache-DiT default must not turn an explicit
+ # fusion-only quality tier into an approximate H3 request.
+ with patch.object(DenoisingStage, "_cache_dit_requested", return_value=True):
+ stage._maybe_enable_cache_dit(50, batch)
+
+ assert stage._minimax_h3_quality == "extra-high"
+ assert stage._minimax_h3_cache_mode is None
+ assert not stage._cache_dit_enabled
+
+
def test_quality_admission_fails_closed_outside_validated_request():
metadata = MiniMaxH3ReleaseMetadata.from_model_index(
{
@@ -405,6 +431,9 @@ def test_quality_admission_fails_closed_outside_validated_request():
server_args.attention_backend = "sage_attn"
assert stage.forward(batch, server_args) is batch
+ batch.sampling_params.quality = "extra-high"
+ assert stage.forward(batch, server_args) is batch
+
batch.sampling_params.quality = "ultra"
server_args.attention_backend = None
with pytest.raises(ValueError, match="quality must be one of"):
diff --git a/python/sglang/multimodal_gen/test/unit/test_openai_image_api.py b/python/sglang/multimodal_gen/test/unit/test_openai_image_api.py
index 9cd67f7a3..81ee23cd2 100644
--- a/python/sglang/multimodal_gen/test/unit/test_openai_image_api.py
+++ b/python/sglang/multimodal_gen/test/unit/test_openai_image_api.py
@@ -58,6 +58,7 @@ def test_runtime_sampling_quality_preserves_the_openai_default():
assert _runtime_sampling_quality(None) is None
assert _runtime_sampling_quality("auto") is None
assert _runtime_sampling_quality("lossless") == "lossless"
+ assert _runtime_sampling_quality("extra-high") == "extra-high"
assert _runtime_sampling_quality("high") == "high"
diff --git a/python/sglang/multimodal_gen/test/unit/test_precision_consistency.py b/python/sglang/multimodal_gen/test/unit/test_precision_consistency.py
index 18de56c2a..ca6efaa80 100644
--- a/python/sglang/multimodal_gen/test/unit/test_precision_consistency.py
+++ b/python/sglang/multimodal_gen/test/unit/test_precision_consistency.py
@@ -149,6 +149,13 @@ class TestDiffusionPrecisionConsistency(unittest.TestCase):
),
torch.bfloat16,
)
+ self.assertEqual(
+ resolve_decode_precision(
+ self._server_args(vae_decode_precision_high="bf16"),
+ quality="extra-high",
+ ),
+ torch.float16,
+ )
self.assertEqual(
resolve_decode_precision(
self._server_args(vae_decode_precision_high="bf16"),
diff --git a/python/sglang/multimodal_gen/test/unit/test_sampling_params.py b/python/sglang/multimodal_gen/test/unit/test_sampling_params.py
index 9b322079e..47862045e 100644
--- a/python/sglang/multimodal_gen/test/unit/test_sampling_params.py
+++ b/python/sglang/multimodal_gen/test/unit/test_sampling_params.py
@@ -26,8 +26,10 @@ from sglang.multimodal_gen.configs.sample.glmimage import (
)
from sglang.multimodal_gen.configs.sample.qwenimage import QwenImageSamplingParams
from sglang.multimodal_gen.configs.sample.sampling_params import (
+ QUALITY_LEVELS,
SamplingParams,
_json_safe,
+ quality_allows_kernel_fusions,
)
from sglang.multimodal_gen.configs.sample.spectrum import SpectrumParams
from sglang.multimodal_gen.configs.sample.teacache import TeaCacheParams
@@ -52,9 +54,15 @@ class TestSamplingParamsValidate(unittest.TestCase):
def test_quality_defaults_to_lossless(self):
self.assertEqual(SamplingParams().quality, "lossless")
- def test_quality_accepts_the_two_validated_levels(self):
- self.assertEqual(SamplingParams(quality="lossless").quality, "lossless")
- self.assertEqual(SamplingParams(quality="high").quality, "high")
+ def test_quality_levels_are_cumulative(self):
+ self.assertEqual(QUALITY_LEVELS, ("lossless", "extra-high", "high"))
+ for quality in QUALITY_LEVELS:
+ with self.subTest(quality=quality):
+ self.assertEqual(SamplingParams(quality=quality).quality, quality)
+
+ self.assertFalse(quality_allows_kernel_fusions("lossless"))
+ self.assertTrue(quality_allows_kernel_fusions("extra-high"))
+ self.assertTrue(quality_allows_kernel_fusions("high"))
def test_quality_rejects_invalid_values(self):
for bad in ("ultra", "draft", "fast", "", True, 1):
@@ -338,9 +346,11 @@ class TestSamplingParamsCliArgs(unittest.TestCase):
def test_quality_is_request_scoped_cli_arg(self):
self.assertNotIn("quality", self._parse_cli_kwargs([]))
- self.assertEqual(
- self._parse_cli_kwargs(["--quality", "high"])["quality"], "high"
- )
+ for quality in ("extra-high", "high"):
+ with self.subTest(quality=quality):
+ self.assertEqual(
+ self._parse_cli_kwargs(["--quality", quality])["quality"], quality
+ )
def test_get_cli_args_maps_spectrum_prefixed_flags(self):
kwargs = self._parse_cli_kwargs(