docs(diffusion): refresh skills for latest runtime (#34143)

2026-08-09 10:30:46 +08:00
parent 38c007dfe5
commit f6cbdc1dd1
7 changed files with 224 additions and 26 deletions
@@ -64,7 +64,12 @@ points cannot silently diverge.
 | Adding a variant of an existing model that shares most logic | **Modular** — reuse existing stages, customize via PipelineConfig callbacks |
 | A specific pre-processing step needs special parallelism or profiling isolation | **Modular** — extract that step as a dedicated stage |

-**Key principle (both styles)**: The stage(s) before `DenoisingStage` must produce a `Req` batch object with all the standard tensor fields that `DenoisingStage` expects (latents, timesteps, prompt_embeds, etc.). As long as this contract is met, the pipeline remains composable regardless of which style you use.
+**Key principle (standard-denoise styles)**: For Hybrid and Modular pipelines,
+the stage(s) before `DenoisingStage` must produce a `Req` batch object with all
+the standard tensor fields that `DenoisingStage` expects (latents, timesteps,
+prompt embeds, and model-specific conditioning). Native task-contract pipelines
+may own a different denoise/decode contract; keep that divergence explicit and
+covered by request-contract tests.

 ---

@@ -115,7 +120,7 @@ Once you have the reference code, study it thoroughly:
 **Before creating any new files, check whether an existing pipeline or stage can be reused or extended.** Only create new pipelines/stages when the existing ones would require extensive modifications or when no similar implementation exists.

 Specifically:
-1. **Compare the new model's architecture against existing pipelines** before creating files. Current native families include MiniMax-H3, LTX-2/2.3, HunyuanVideo/FastHunyuan, Wan/FastWan/TurboWan/LingBot World, MOVA, FLUX/FLUX.2/Klein, Z-Image, Qwen-Image/edit/layered, GLM-Image, SD3, Hunyuan3D, Helios, Cosmos3, SANA/SANA-WM, FireRed, ERNIE-Image, JoyAI, and Ideogram4. If the new model shares most of its structure with an existing one (e.g., same text encoders, similar latent format, compatible denoising loop), prefer:
+1. **Compare the new model's architecture against existing pipelines** before creating files. Current native families include MiniMax-H3, Krea-2, LTX-2/2.3, HunyuanVideo/FastHunyuan, Wan/FastWan/TurboWan/LingBot World/LingBot Video MoE, MOVA, FLUX/FLUX.2/Klein, Z-Image, Qwen-Image/edit/layered, GLM-Image, SD3, Hunyuan3D, Helios, Cosmos3, SANA/SANA-WM, FireRed, ERNIE-Image, JoyAI, and Ideogram4. If the new model shares most of its structure with an existing one (e.g., same text encoders, similar latent format, compatible denoising loop), prefer:
   - Adding a new config variant to the existing pipeline rather than creating a new pipeline class
   - Reusing the existing `BeforeDenoisingStage` with minor parameter differences
   - Using `add_standard_t2i_stages()` / `add_standard_ti2i_stages()` / `add_standard_ti2v_stages()` if the model fits standard patterns
@@ -582,6 +587,7 @@ After implementation, **you must verify that the generated output is not noise**
 | Hunyuan3D | `runtime/pipelines/hunyuan3d_pipeline.py` | `stages/model_specific_stages/hunyuan3d/` | `configs/pipeline_configs/hunyuan3d.py` |
 | SANA-WM | `runtime/pipelines/sana_wm_pipeline.py`, `sana_wm_realtime_pipeline.py` | `stages/model_specific_stages/sana_wm/` | `configs/pipeline_configs/sana_wm.py` |
 | LingBot World realtime | `runtime/pipelines/lingbot_world_causal_dmd_pipeline.py` | `stages/model_specific_stages/lingbot_world/` | `configs/pipeline_configs/lingbot_world.py` |
+| Krea-2 | `runtime/pipelines/krea2.py` | `stages/model_specific_stages/krea2.py` | `configs/pipeline_configs/krea2.py` |

 ### Modular Style (when standard stages fit well)

@@ -599,6 +605,7 @@ After implementation, **you must verify that the generated output is not noise**
 | Helios | `runtime/pipelines/helios_pipeline.py` | Video pipeline family with custom denoising and decoding stages |
 | FireRed/JoyAI image edit | `runtime/pipelines/qwen_image.py`, `runtime/pipelines/joy_image.py` | FireRed reuses Qwen edit-plus config; JoyAI has its own edit pipeline |
 | Wan | `runtime/pipelines/wan_pipeline.py` | Uses `add_standard_ti2v_stages()` |
+| LingBot Video MoE 30B | `runtime/pipelines/lingbot_video_moe.py` | Uses a model-specific structured-JSON text-encoding stage, then standard latent/timestep preparation, denoising, and decoding |

 ### Native Task-Contract Style (coupled multimodal requests)

@@ -612,7 +619,7 @@ After implementation, **you must verify that the generated output is not noise**

 Before submitting, verify:

-**Common (both styles):**
+**Common (all styles):**
 - [ ] **Pipeline file** exists at `runtime/pipelines/{model_name}.py` with `EntryClass`
 - [ ] **PipelineConfig** at `configs/pipeline_configs/{model_name}.py`
 - [ ] **SamplingParams** at `configs/sample/{model_name}.py`
@@ -23,6 +23,9 @@ Before running any benchmark, profiler, or kernel-validation command:
 - verify the repo is writable
 - export `HF_TOKEN` before using gated Hugging Face models such as `black-forest-labs/FLUX.*`
 - export `FLASHINFER_DISABLE_VERSION_CHECK=1`
+- set `SGLANG_DIFFUSION_SYNC_STAGE_PROFILING=1` when comparing stage-level
+  denoise/decode timings; the preset helper sets it by default unless the
+  caller explicitly overrides it
 - choose idle GPU(s) before starting perf work

 ## Native Backend Gate
@@ -45,7 +48,7 @@ If any benchmark, perf-dump, or `torch.profiler` command prints one of those sig
 - [benchmark-and-profile.md](benchmark-and-profile.md) — canonical denoise benchmark, perf dump, and `torch.profiler` workflow; uses checked-in nightly-aligned presets plus current-source extras such as MiniMax-H3 joint video/audio T2VA, FLUX.2 Klein, Cosmos3, Ideogram4, ERNIE/GLM/SANA image models, FastWan2.2, `LTX-2.3` one-stage/two-stage/HQ, HunyuanVideo, MOVA, Helios, JoyAI/FireRed image edit, and Hunyuan3D shape
 - [existing-fast-paths.md](existing-fast-paths.md) — map bottlenecks to existing fused kernels, packed QKV paths, fused `QK norm + RoPE`, distributed overlap patterns, and open optimization PRs before proposing new code
 - [scripts/diffusion_skill_env.py](scripts/diffusion_skill_env.py) — preflight helper: repo root discovery via `sglang.__file__`, write-access probe, benchmark/profile output directories, idle GPU selection
- [scripts/bench_diffusion_denoise.py](scripts/bench_diffusion_denoise.py) — end-to-end denoise benchmark preset runner via `sglang generate`; supports `--no-torch-compile`, forces the H3 preset to its eager consistency mode, validates nightly preset drift with `--validate-nightly-alignment`, and saves perf dumps by label for `compare_perf.py`
+- [scripts/bench_diffusion_denoise.py](scripts/bench_diffusion_denoise.py) — end-to-end denoise benchmark preset runner via `sglang generate`; supports `--no-torch-compile`, forces the H3 preset to its eager consistency mode, enables synchronized stage attribution for perf dumps, validates nightly preset drift with `--validate-nightly-alignment`, and saves perf dumps by label for `compare_perf.py`

 ## Opportunity Discovery Rule

@@ -58,6 +61,10 @@ Always rule out these existing families first:
 - SANA packed self-attention Q/K/V and cross-attention K/V GEMMs
 - MiniMax-H3 indexed modulation, fused QK norm + RoPE, packed Ulysses QKV,
  USP relayout, and batched TP AdaLN collectives
+- bit-exact diffusion adaLN modulation and fused LayerNorm + modulation for
+  FLUX.1, GLM-Image, and SANA
+- request-scoped `quality=high` DiT and VAE fast paths
+- Wan causal-VAE cache/padding and DupUp3D data-movement fusions
 - fused diffusion `QK norm + RoPE`
 - LTX2 split RoPE
 - LTX2 residual-gate add
@@ -67,6 +74,7 @@ Always rule out these existing families first:
 - Ulysses / USP attention overlap
 - turbo-layer async all-to-all overlap
 - `torch.compile` compute / communication reorder
+- breakable CUDA graph capture for supported fixed-resolution pipelines
 - dual-stream diffusion execution

 If the user explicitly requires `torch.compile` to stay off, do not use the
@@ -37,6 +37,9 @@ python3 "$ENV_PY" check-write-access >/dev/null

 export HF_TOKEN=<your_hf_token>  # required for gated repos such as black-forest-labs/FLUX.*
 export FLASHINFER_DISABLE_VERSION_CHECK=1
+# Required for correctly attributed stage-level denoise/decode timings. The
+# checked-in benchmark helper sets this by default unless you explicitly set 0.
+export SGLANG_DIFFUSION_SYNC_STAGE_PROFILING=1
 # Leave CUDA_VISIBLE_DEVICES unset to let the preset helper select the number
 # of idle GPUs it requires. For manual runs, set --count to that command's
 # exact --num-gpus value.
@@ -83,6 +86,8 @@ Environment notes:
 - all commands below assume you are inside the configured diffusion container shell
 - export `HF_TOKEN` before any gated Hugging Face model run
 - export `FLASHINFER_DISABLE_VERSION_CHECK=1` before any benchmark or profiler run
+- keep `SGLANG_DIFFUSION_SYNC_STAGE_PROFILING=1` for stage-level comparisons;
+  without it, asynchronous GPU work can be charged to a later stage
 - re-run `print-idle-gpus` before each perf command if GPU availability may have changed
 - keep benchmark commands within 4 GPUs or fewer

@@ -130,6 +135,10 @@ PYTHONPATH=python python3 "$BENCH_PY" \
  --output-dir "${BENCH_DIR}"
 ```

+The helper sets `SGLANG_DIFFUSION_SYNC_STAGE_PROFILING=1` for accurate stage
+attribution. Set it to `0` explicitly only when collecting an e2e-only run and
+do not compare its per-stage values with synchronized results.
+
 Keep `torch.compile` off when the task requires it:

 ```bash
@@ -500,10 +509,17 @@ Always keep:
 - denoise latency
 - end-to-end latency
 - peak GPU memory
- exact command line, model shape, dtype, and GPU topology
+- exact command line, model shape, dtype, request `quality`, GPU topology, and
+  whether synchronized stage profiling was enabled

 Never keep a perf dump produced after a diffusers-backend fallback.

+Stage durations are host wall times around asynchronous GPU launches unless
+`SGLANG_DIFFUSION_SYNC_STAGE_PROFILING=1`. Without the sync, queued denoise
+work can leak into the next blocking stage and inflate `DecodingStage` by 2-3x.
+Use synchronized dumps for denoise/decode attribution and keep the setting
+identical in every before/after pair.
+
 ## `torch.profiler` Workflow

 ### 1. Establish the baseline
@@ -604,9 +620,13 @@ the known mainline families.
 | `fused_inplace_qknorm_rope` missing, but separate qk norm plus rope show up | Check whether the fused diffusion `QK norm + RoPE` path should have engaged |
 | `to_q -> to_k -> to_v` on NVFP4 or Nunchaku FLUX-family checkpoints | Treat as a packed-QKV fast-path miss or checkpoint-format mismatch |
 | `zimage_rmsnorm_scale` or `zimage_rmsnorm_tanh_residual` missing on Z-Image | Check the bf16-native Triton eligibility guards before proposing a new fusion |
+| FLUX.1, GLM-Image, or SANA shows separate LayerNorm plus adaLN elementwise kernels | Check the bit-exact `modulate_scale_shift` and `fused_layernorm_modulate` guards/self-test before proposing another norm fusion |
+| `quality=high` shows the same FLUX/GLM DiT or FLUX-family/Wan VAE chain as `lossless` | Check whether the request-scoped quality gate mounted and whether every site passed its all-or-nothing compatibility checks |
 | LTX-2 split RoPE appears as a long PyTorch elementwise chain | Check the `apply_ltx2_split_rotary_emb` Triton path and its shape guards |
+| Wan decode is dominated by causal `cat + pad + contiguous`, feature-cache copies, or `repeat_interleave + permute + add` | Check the bit-exact Wan causal-cache and DupUp3D data-movement kernels before writing a new decoder kernel |
 | masked attention spends time packing/unpacking Q/K/V | Check whether fused varlen USP pack/scatter should have engaged |
 | `all_to_all`, ring attention, or async A2A dominate | Classify against Ulysses, USP, or turbo-layer overlap first |
+| Fixed-resolution image/video traces show many small launch gaps | Check supported breakable CUDA graph capture, declared warmup resolutions, and text buckets before adding a new graph mechanism |
 | H3 shows separate indexed gather + scale/shift, QK norm + RoPE, or three Q/K/V Ulysses relayouts | Check H3's indexed-modulation, fused QK-norm+RoPE, packed Ulysses-QKV, and USP relayout guards before writing a new kernel |
 | H3 TP traces show one AdaLN collective per block | Check the batched TP AdaLN projection/all-gather path in `minimax_h3.py` before attempting communication overlap |
 | split `fc1 -> gelu -> quant -> fc2.lora_down` on Nunchaku FLUX | Treat as a missing fused GELU MLP path |
@@ -633,6 +653,7 @@ This skill intentionally stops here. It tells you whether you are looking at:

 - [ ] fixed-shape baseline perf dump saved
 - [ ] fixed-shape new perf dump saved
+- [ ] request `quality` and `SGLANG_DIFFUSION_SYNC_STAGE_PROFILING` match
 - [ ] `compare_perf.py` table generated
 - [ ] one representative `torch.profiler` trace saved
 - [ ] hotspot classified against `existing-fast-paths.md`
@@ -11,10 +11,16 @@ framework-specific optimization workflow.
 - `python/sglang/multimodal_gen/runtime/layers/fused_scale_shift_gate.py`
 - `python/sglang/multimodal_gen/runtime/layers/rotary_embedding/utils.py`
 - `python/sglang/kernels/ops/diffusion/triton/scale_shift.py`
+- `python/sglang/kernels/ops/diffusion/modulate_scale_shift.py`
+- `python/sglang/kernels/ops/diffusion/fused_ln_modulate.py`
+- `python/sglang/kernels/ops/diffusion/quality_gate.py`
 - `python/sglang/kernels/ops/diffusion/group_norm_silu.py`
 - `python/sglang/kernels/ops/diffusion/triton/group_norm_silu.py`
+- `python/sglang/kernels/ops/diffusion/triton/group_norm_silu_twopass.py`
 - `python/sglang/kernels/ops/diffusion/triton/norm.py`
 - `python/sglang/kernels/ops/diffusion/triton/rmsnorm_onepass.py`
+- `python/sglang/kernels/ops/diffusion/triton/layernorm_modulate.py`
+- `python/sglang/kernels/ops/diffusion/triton/native_bf16_rmsnorm.py`
 - `python/sglang/kernels/ops/diffusion/triton/zimage_native_norm.py`
 - `python/sglang/kernels/ops/diffusion/triton/rotary.py`
 - `python/sglang/kernels/ops/diffusion/triton/ltx2_rotary.py`
@@ -26,13 +32,24 @@ framework-specific optimization workflow.
 - `python/sglang/kernels/ops/diffusion/residual_gate_add.py`
 - `python/sglang/kernels/jit/csrc/diffusion/residual_gate_add.cuh`
 - `python/sglang/kernels/ops/diffusion/triton/varlen_pack_pad.py`
+- `python/sglang/kernels/ops/diffusion/triton/wan_causal_cache.py`
 - `python/sglang/kernels/ops/diffusion/cutedsl/scale_residual_norm_scale_shift.py`
+- `python/sglang/multimodal_gen/runtime/models/vaes/fast_path_gate.py`
+- `python/sglang/multimodal_gen/runtime/models/vaes/flux2_vae_cuda_opt.py`
+- `python/sglang/multimodal_gen/runtime/models/vaes/wan_vae_cuda_opt.py`
+- `python/sglang/multimodal_gen/runtime/breakable_cuda_graph/runner.py`
 - `test/registered/kernels/ops/diffusion/test_qwen_image_modulation.py`
 - `test/registered/kernels/ops/diffusion/test_group_norm_silu.py`
 - `test/registered/kernels/ops/diffusion/test_residual_gate_add.py`
 - `test/registered/kernels/ops/diffusion/test_varlen_pack_pad.py`
 - `test/registered/kernels/ops/diffusion/test_varlen_uspattn_equivalence.py`
- `test/registered/kernels/ops/diffusion/test_zimage_native_norm.py`
+- `test/registered/kernels/ops/diffusion/test_native_bf16_rmsnorm.py`
+- `test/registered/kernels/ops/diffusion/test_flux_ln_modulate.py`
+- `test/registered/kernels/ops/diffusion/test_glm_image_ln_modulate.py`
+- `test/registered/kernels/ops/diffusion/test_sana_ln_modulate.py`
+- `test/registered/kernels/ops/diffusion/test_quality_gate.py`
+- `test/registered/kernels/ops/diffusion/test_wan_causal_cache.py`
+- `test/registered/kernels/ops/diffusion/test_stage_profiler_sync.py`
 - `test/registered/kernels/benchmark/diffusion/bench_qwen_image_modulation.py`
 - `test/registered/kernels/benchmark/diffusion/bench_group_norm_silu.py`
 - `test/registered/kernels/benchmark/diffusion/bench_residual_gate_add.py`
@@ -60,33 +77,72 @@ framework-specific optimization workflow.
 - Constraints: `D % 256 == 0` and `D <= 8192`. `x/residual/gate/scale/shift` must pass shape and stride validation. Dtypes limited to fp16/bf16/fp32.
 - Behavior: CuTe DSL compilation cached by `(dtype, ndim, D, norm_type)`. `None` tensors replaced by scalar placeholders. If constraints fail, `layernorm.py` warns and falls back to native PyTorch.

-3. Z-Image bf16-native RMSNorm modulation (Triton)
+3. Bit-exact adaLN modulation and LayerNorm + modulation
+- Kernels: `modulate_scale_shift`, `fused_layernorm_modulate`, and
+  `fused_qk_head_layernorm`.
+- Locations: `modulate_scale_shift.py`, `triton/layernorm_modulate.py`,
+  `runtime/models/dits/flux.py`, `glm_image.py`, and `sana.py`.
+- Use cases:
+  - `x * (1 + scale[:, None]) + shift[:, None]` as one JIT CUDA launch.
+  - BF16 `LayerNorm(x) * (1 + scale) + shift` as one Triton launch that
+    reproduces the active aten BF16 reduction and rounding order.
+  - GLM-Image per-head Q/K LayerNorm with the same aten-compatible reduction.
+- Constraints: the JIT modulation path requires aligned contiguous CUDA
+  fp16/bf16 BLC inputs with `[B, D]` scale/shift. The Triton LayerNorm path is
+  BF16-specific and only claims bit-exactness for its guarded aten dispatch;
+  FLUX/GLM/SANA run a live eager equality check and fail closed on mismatch.
+- Validation: `test_flux_ln_modulate.py`, `test_glm_image_ln_modulate.py`,
+  `test_sana_ln_modulate.py`, `test_modulate_scale_shift.py`, and
+  `test_fused_ln_modulate.py`.
+- Workflow rule: if these models show separate norm and modulation kernels,
+  check dtype, alignment, shape, BCG/compile context, and the one-time equality
+  self-test before proposing another fusion.
+
+4. Request-scoped `quality=high` fusion gates
+- Locations: `quality_gate.py`, `fused_ln_modulate.py`, `denoising.py`,
+  `decoding.py`, `fast_path_gate.py`, `flux2_vae_cuda_opt.py`, and
+  `wan_vae_cuda_opt.py`.
+- Behavior: `quality="lossless"` is the default exact reference path.
+  `quality="high"` may mount model-owned, validated but non-bit-exact DiT
+  fusions and decode-scoped VAE rewrites. Mounting is all-or-nothing per
+  transformer/fusion family; VAE gates reset after every decode.
+- Current families include FLUX affine-folded LN+modulate / fused GELU sites,
+  GLM-Image fused GELU sites, generic KL VAE decoder rewrites used by
+  FLUX.1/FLUX.2/Z-Image/SD3, and Wan VAE RMSNorm+SiLU.
+- Do not confuse request `--quality` with `--output-quality`, which controls
+  output-file compression rather than model math.
+- Validation: `test_quality_gate.py`, `test_fused_ln_modulate.py`,
+  `test_flux2_vae_fastpath.py`, `test_wan_vae_fastpath.py`, and
+  `test_vae_fast_path_gate.py`.
+
+5. Z-Image bf16-native RMSNorm modulation (Triton)
 - Kernels: `zimage_rmsnorm_scale`, `zimage_rmsnorm_tanh_residual`
- Locations: `triton/zimage_native_norm.py`, `zimage.py`
+- Locations: `triton/native_bf16_rmsnorm.py`, compatibility exports in
+  `triton/zimage_native_norm.py`, and `zimage.py`
 - Use cases:
  - `y = rmsnorm(x) * scale`
  - `y = residual + tanh(gate) * rmsnorm(x)`
 - Constraints: CUDA bf16 tensors, contiguous weights, flattenable row strides,
  compatible modulation row counts, and `D <= 8192`.
- Validation: `test/registered/kernels/ops/diffusion/test_zimage_native_norm.py`
+- Validation: `test/registered/kernels/ops/diffusion/test_native_bf16_rmsnorm.py`
 - Behavior: the kernels preserve Z-Image's native bf16 arithmetic. They return
  `None` when an eligibility guard fails, and the runtime wrapper executes the
  native PyTorch formula.

-4. Triton LayerNorm/RMSNorm fusion
+6. Triton LayerNorm/RMSNorm fusion
 - Kernels: `rms_norm_fn`, `layer_norm_fn`, `norm_infer`
 - Locations: `triton/norm.py`, `layernorm.py`
 - Use cases: fp32 RMSNorm with residual/dropout/rowscale/x1 branches, and inference-friendly `norm_infer`.
 - Constraints: last dim must be contiguous, and `N * element_size < 64KB`.
 - Validation: `test/registered/kernels/ops/layernorm/test_rmsnorm.py`.

-5. Triton one-pass RMSNorm (small hidden size fast path)
+7. Triton one-pass RMSNorm (small hidden size fast path)
 - Kernel: `triton_one_pass_rms_norm`
 - Locations: `triton/rmsnorm_onepass.py`, `layernorm.py`
 - Use case: `hidden_size <= 128` in `RMSNorm.forward_cuda`.
 - `torch.compile` note: keep this path behind the custom-op wrapper in `rmsnorm_onepass.py`; direct `wrap_triton` can recompile on dynamic row counts.

-6. Triton RoPE fusion
+8. Triton RoPE fusion
 - Kernel: `apply_rotary_embedding`
 - Locations: `triton/rotary.py`, `rotary_embedding/utils.py`
 - Use case: GPT-J style RoPE when not Neox.
@@ -94,14 +150,14 @@ framework-specific optimization workflow.
 - NPU fallback: `npu_fallback.apply_rotary_embedding_native`.
 - Validation: `test/registered/kernels/ops/attention/test_rope.py`.

-7. LTX2 split RoPE fusion
+9. LTX2 split RoPE fusion
 - Kernel: `apply_ltx2_split_rotary_emb`
 - Locations: `triton/ltx2_rotary.py`, `runtime/models/dits/ltx_2.py`
 - Use case: LTX-2 split rotary embedding over `[B, S, num_heads * head_dim]` with separate `cos` and `sin` tensors.
 - Constraints: `cos` and `sin` shapes must match `[B, H, S, head_dim / 2]`, and `inner_dim == H * head_dim`.
 - Workflow rule: if LTX-2 traces show a large split-RoPE PyTorch chain, check whether the LTX2-specific Triton path was disabled by shape or dtype before proposing a new RoPE kernel.

-8. LTX2 residual-gate add fusion
+10. LTX2 residual-gate add fusion
 - Kernel: `diffusion_residual_gate_add`
 - Locations: `diffusion/residual_gate_add.py`, `csrc/diffusion/residual_gate_add.cuh`, `runtime/models/dits/ltx_2.py`
 - Use case: `residual + update * gate` in LTX2 self-attention, prompt cross-attention, audio/video cross-attention, and feed-forward residual updates.
@@ -111,7 +167,7 @@ framework-specific optimization workflow.
 - Microbench: `test/registered/kernels/benchmark/diffusion/bench_residual_gate_add.py`.
 - Workflow rule: if LTX2 traces show repeated elementwise `mul` + `add` ladders around attention or MLP residuals, check whether this existing CUDA path was disabled by shape, dtype, contiguity, or a prior runtime failure before proposing another elementwise fusion.

-9. MiniMax-H3 indexed AdaLN modulation and gated residual fusion
+11. MiniMax-H3 indexed AdaLN modulation and gated residual fusion
 - Kernels: `indexed_scale_shift_bf16_`, `indexed_gate_bf16_`
 - Locations: `triton/indexed_modulation.py`, `runtime/models/dits/minimax_h3.py`
 - Use cases: H3's packed video/audio/text rows select per-token modulation with `combined_indices`; the Triton paths replace `index_select` plus scale/shift or gated residual chains in place.
@@ -119,7 +175,7 @@ framework-specific optimization workflow.
 - Numerical contract: the kernels explicitly reproduce H3's eager BF16 rounding boundaries. Do not replace them with a mathematically equivalent contraction without the H3 consistency check.
 - Workflow rule: if H3 traces show `index_select` plus elementwise ladders around every block, check dtype, contiguity, and input-reuse eligibility before designing another modulation kernel.

-10. MiniMax-H3 packed Ulysses QKV and output relayout
+12. MiniMax-H3 packed Ulysses QKV and output relayout
 - Kernels: `pack_qkv_destination_major`, `usp_merge_heads`
 - Locations: `triton/ulysses_qkv.py`, `usp_relayout.py`, `runtime/layers/usp.py`, `runtime/models/dits/minimax_h3.py`
 - Use cases: one destination-major QKV pack plus one collective replaces three separately prepared Ulysses input exchanges; the output JIT kernel replaces `permute(...).contiguous()` when merging gathered heads.
@@ -127,7 +183,7 @@ framework-specific optimization workflow.
 - Related transport: 2-rank, peer-accessible CUDA groups can use the existing IPC A2A transport; larger or unsupported groups fall back to the normal collective path.
 - Workflow rule: if an H3 Ulysses trace has three Q/K/V preparation ladders or a large output `permute + contiguous`, first prove why these existing guards missed.

-11. HunyuanVideo / LTX upsampler GroupNorm + SiLU fusion
+13. HunyuanVideo / LTX upsampler GroupNorm + SiLU fusion
 - Kernel: `triton_group_norm_silu`
 - Locations: `diffusion/group_norm_silu.py`, `triton/group_norm_silu.py`, `runtime/models/vaes/hunyuanvae.py`, `runtime/models/upsampler/latent_upsampler.py`
 - Use case: `activation(group_norm(x))` when the activation is non-inplace `nn.SiLU` and the GroupNorm is affine.
@@ -136,6 +192,18 @@ framework-specific optimization workflow.
 - Validation: `test/registered/kernels/ops/diffusion/test_group_norm_silu.py`.
 - Microbench: `test/registered/kernels/benchmark/diffusion/bench_group_norm_silu.py`.

+14. Wan causal-VAE data-movement fusion
+- Kernels: `cat_pad_channels_last_3d` and `dup_up3d_add`.
+- Locations: `triton/wan_causal_cache.py` and
+  `runtime/models/vaes/wanvae.py`.
+- Use cases: build causal Conv3d input plus the next compact feature cache in
+  one channels-last-3D pass, and fuse `main + DupUp3D(src)` without
+  materializing `repeat_interleave + permute().contiguous()` intermediates.
+- Numerical contract: these are bit-exact data-movement / same-order-add
+  replacements and run independently of the `quality=high` Wan RMSNorm+SiLU
+  path. Unsupported layouts or padding fall back to the aten chain.
+- Validation: `test/registered/kernels/ops/diffusion/test_wan_causal_cache.py`.
+
 **Faster CUDA Kernel Usage Points**

 1. sgl-kernel RMSNorm and fused add RMSNorm
@@ -218,10 +286,15 @@ framework-specific optimization workflow.

 **Common Entry Points in Diffusion Models**
 - AdaLN modulation: `LayerNormScaleShift`, `RMSNormScaleShift`, `ScaleResidual*` in `layernorm.py`.
+- Bit-exact adaLN modulation / LayerNorm folding: `modulate_scale_shift` and
+  `fused_layernorm_modulate` through `flux.py`, `glm_image.py`, and `sana.py`.
+- Request-scoped high-quality acceleration: `QualityGatedFusion` in
+  `quality_gate.py`, `_maybe_toggle_quality_fusions` in `denoising.py`, and
+  `use_vae_fast_path` in `decoding.py`.
 - Qwen-Image gating: `fuse_layernorm_scale_shift_gate_select01_kernel` and `fuse_residual_layernorm_scale_shift_gate_select01_kernel` through `fused_scale_shift_gate.py` and `qwen_image.py`.
 - Z-Image native norm modulation: `zimage_rmsnorm_scale` and
-  `zimage_rmsnorm_tanh_mul_add` in `zimage.py`, backed by
-  `triton/zimage_native_norm.py`.
+  `zimage_rmsnorm_tanh_mul_add` in `zimage.py`, backed by the shared
+  `triton/native_bf16_rmsnorm.py` kernels.
 - HunyuanVideo VAE and LTX upsampler GroupNorm+SiLU: `apply_group_norm_silu` in `hunyuanvae.py` and `latent_upsampler.py`; default-eligible when wrapper guards pass.
 - MiniMax-H3 indexed modulation: `_modulate_scale_shift` and `_modulate_gate` in `minimax_h3.py`, backed by `triton/indexed_modulation.py`.
 - MiniMax-H3 Ulysses relayout: `_usp_input_all_to_all_packed_qkv` and `usp_merge_heads` through `runtime/layers/usp.py`.
@@ -229,6 +302,8 @@ framework-specific optimization workflow.
 - QK norm + RoPE: `apply_qk_norm_rope` in `layernorm.py`; use this path when the model wants fused attention prep instead of separate QK norm and RoPE calls.
 - LTX2 split RoPE: `apply_ltx2_split_rotary_emb` in `ltx_2.py`.
 - LTX2 residual-gate add: `_ltx2_residual_gate_add` in `ltx_2.py` wraps the CUDA `diffusion_residual_gate_add` custom op for attention, cross-attention, and MLP residual updates.
+- Wan causal VAE: `cat_pad_channels_last_3d` and `dup_up3d_add` in
+  `wanvae.py`, backed by `triton/wan_causal_cache.py`.
 - Varlen USP attention: `fused_pack_qkv` and `fused_scatter_to_padded` in `attention/layer.py`.
 - SANA packed projections: `to_qkv` and `to_kv` in `sana.py`.
 - Nunchaku fused GELU MLP: `_fused_gelu_mlp` in `flux.py` for quantized FLUX-family checkpoints.
@@ -238,10 +313,20 @@ framework-specific optimization workflow.
 **Existing Overlap / Communication Families**

 - Ulysses / USP attention: treat `all_to_all`, `ring_attn`, and head / sequence reshards as an existing distributed attention family, not a new overlap idea.
+- Cross-node SP: current server args support `--nnodes`, `--node-rank`, and
+  `--dist-init-addr`. Prefer node-local Ulysses multiplied by cross-node Ring;
+  keep encoders replicated and verify each model's Ring admission before
+  treating cross-node transport as a new framework gap.
 - MiniMax-H3 TP AdaLN: the DiT stacks every block's TP-local AdaLN projection and performs one batched all-gather before the block loop when `_can_batch_block_adaln()` passes. One all-gather per block indicates that this existing batching path missed.
 - MiniMax-H3 final projections: H3 removes dead text/padding rows before the final TP column gathers and combines video/audio for the SP row gather. Preserve that ordering when optimizing output communication.
 - Turbo-layer async all-to-all: `all_to_all_single(..., async_op=True)` plus staged waits already form an existing overlap family in `turbo_layer.py`.
 - TorchInductor compute / communication reorder: `torch._inductor.config.reorder_for_compute_comm_overlap = True` can already partially overlap compiled denoise traces.
+- Breakable CUDA graph: `runtime/breakable_cuda_graph/runner.py` captures
+  fixed-resolution DiT segments around eager attention/collectives for
+  supported pipelines. It is mutually exclusive with `torch.compile` and
+  Cache-DiT, requires every served resolution in `--warmup-resolutions`, and
+  uses `--bcg-text-buckets` for prompt signatures. Check this path before
+  proposing a second graph-capture mechanism for launch-bound traces.
 - Dual-stream diffusion models: `use_dual_stream = True` in models such as `hunyuan3d.py` is an existing overlap family.
 - Workflow rule: if a hotspot is communication-heavy, rule out these in-repo overlap families before proposing a brand new overlap design.

@@ -278,6 +363,9 @@ relying on any file path, flag, or claim about whether the work has merged.

 **Constraints and Fallbacks**
 - `scale_shift` Triton requires CUDA + contiguous `x`. NPU swaps to native.
+- Bit-exact BF16 LayerNorm+modulate requires the guarded aten-compatible shape
+  and a successful live equality check; `quality=high` affine folding is a
+  separate non-bit-exact path.
 - CuTe DSL fused norms require `D % 256 == 0` and `D <= 8192`.
 - Triton norm kernels error on feature size >= 64KB.
 - FlashAttention requires fp16/bf16 and SM80+; otherwise SDPA.
@@ -665,7 +665,11 @@ def _expected_nightly_cli_args(case: dict) -> dict[str, str]:
    serve_args = shlex.split(case["frameworks"]["sglang"].get("serve_args", ""))
    parsed_serve_args = _parse_cli_args(serve_args)
    for flag, value in parsed_serve_args.items():
-        if flag in {"enable-torch-compile", "warmup-mode"}:
+        # Nightly's comparison driver still owns its legacy ``--warmup``
+        # switch.  It is not a valid ``sglang generate`` flag after the
+        # warmup-mode migration, so exclude both spellings from preset drift
+        # validation.
+        if flag in {"enable-torch-compile", "warmup", "warmup-mode"}:
            continue
        expected[flag] = _normalize_cli_value(value)

@@ -834,6 +838,11 @@ def run_benchmark_once(

    env = os.environ.copy()
    env.setdefault("FLASHINFER_DISABLE_VERSION_CHECK", "1")
+    # Perf dumps are consumed as stage-attributed denoise measurements.  Drain
+    # the device queue at stage boundaries so asynchronous denoise work cannot
+    # leak into a later stage (most visibly DecodingStage).  An explicit 0 in
+    # the caller's environment still opts out for e2e-only experiments.
+    env.setdefault("SGLANG_DIFFUSION_SYNC_STAGE_PROFILING", "1")
    cfg = MODELS[model_key]
    for key, value in cfg.get("env", {}).items():
        env.setdefault(key, str(value))
@@ -23,7 +23,12 @@ This skill owns the ModelOpt-to-SGLang bridge. It is not a generic kernel-tuning
 - Benchmark only when BF16 and quantized commands are identical except for the checkpoint override being tested.
 - For diffusion FP8, keep `dit_cpu_offload=false`. `dit_layerwise_offload=true` is valid on the fixed path when you want lower DiT residency.
 - For multi-transformer pipelines, use per-component overrides when different components need different checkpoints.
- For B200 NVFP4 validation, keep backend-sensitive environment variables explicit. Wan2.2 NVFP4 is commonly validated with `SGLANG_DIFFUSION_FLASHINFER_FP4_GEMM_BACKEND=cudnn`; benchmark the default CUTLASS path separately if that is what you are evaluating.
+- For B200 NVFP4 validation, keep backend-sensitive environment variables
+  explicit. The current default is FlashInfer TensorRT-LLM
+  (`flashinfer_trtllm`); high-resolution Qwen Image can favor
+  `SGLANG_DIFFUSION_FLASHINFER_FP4_GEMM_BACKEND=cutlass`, while 1024x1024 can
+  remain BF16-faster. Benchmark the exact shape instead of assuming one backend
+  or quantized checkpoint wins.
 - When a branch is missing the validated helper tools, refresh `python/sglang/multimodal_gen/tools/build_modelopt_fp8_transformer.py`, `python/sglang/multimodal_gen/tools/build_modelopt_nvfp4_transformer.py`, and `python/sglang/multimodal_gen/tools/compare_diffusion_trajectory_similarity.py` instead of inventing one-off scripts elsewhere.
 - After validating a new ModelOpt quant path, update the ModelOpt support matrix in `docs/docs/sglang-diffusion/quantization.mdx` before closing the task.

@@ -35,6 +40,7 @@ Read these sources before changing code:
 - ModelOpt quantization entrypoint: `examples/diffusers/quantization/quantize.py`
 - ModelOpt diffusers quant presets: `examples/diffusers/quantization/config.py`
 - SGLang diffusion quant runtime:
+  - `python/sglang/multimodal_gen/runtime/layers/quantization/modelopt_fp8.py`
  - `python/sglang/multimodal_gen/runtime/layers/quantization/modelopt_quant.py`
  - `python/sglang/multimodal_gen/runtime/utils/quantization_utils.py`
  - `python/sglang/multimodal_gen/runtime/loader/transformer_load_utils.py`
@@ -65,7 +71,8 @@ This repo now contains:
 Validated documentation and CI coverage currently center on these ModelOpt diffusion transformer override families:

 - FP8: FLUX.1-dev, FLUX.2-dev, Wan2.2, HunyuanVideo, Qwen Image, Qwen Image Edit
- NVFP4: FLUX.1-dev, FLUX.2-dev, Wan2.2
+- NVFP4: FLUX.1-dev, FLUX.2-dev, Wan2.2, Qwen Image, Qwen Image 2512,
+  Qwen Image Edit, Qwen Image Edit 2511

 Treat a new family, a new precision, or a new checkpoint layout as unsupported until it has a documented matrix row and a matching validation story.
 Current B200 CI also contains an Ideogram4 NVFP4 native load case
@@ -77,8 +84,8 @@ Before writing CLI examples, re-read the active branch's `docs/docs/sglang-diffu

 B200 CI coverage can include loose BF16-vs-quantized quality checks. Inspect the active branch's `run_suite.py` before assuming they are part of the suite; mainline and feature branches may differ. Those checks are intended to catch blank, corrupted, or obviously divergent images, not exact image parity.

-Mainline documentation now uses `lmsys/*` for the eight converted ModelOpt
-checkpoint repos; the FLUX.2 NVFP4 raw export remains
+Mainline documentation now tracks thirteen published ModelOpt checkpoints.
+Twelve live under `lmsys/*`; the FLUX.2 NVFP4 raw export remains
 `black-forest-labs/FLUX.2-dev-NVFP4`. Do not use older `BBuf/*` examples unless
 you are explicitly testing a historical branch.

@@ -305,10 +312,23 @@ sglang generate \
  --save-output
 ```

+Full ModelOpt Diffusers repo example (current Qwen Image NVFP4 path):
+
+```bash
+sglang generate \
+  --model-path lmsys/qwen-image-2512-modelopt-nvfp4-sglang \
+  --prompt "<prompt>" \
+  --seed <seed> \
+  --save-output
+```
+
 Guideline:

 - use the global `--transformer-path` only when the model effectively has one transformer override to apply
 - use per-component overrides when different backbones need different checkpoints
+- use `--model-path` directly for published full ModelOpt Diffusers repos such
+  as the Qwen Image NVFP4 family; this is different from a transformer-only
+  override
 - the preferred CLI form is `--<component>-path`
 - config-expanded forms such as `--component_paths.transformer_2=...` also resolve to the same internal override map

@@ -411,6 +431,7 @@ When documenting results:
 | File | Role |
 | --- | --- |
 | `runtime/layers/quantization/__init__.py` | registers diffusion quant methods |
+| `runtime/layers/quantization/modelopt_fp8.py` | static per-tensor ModelOpt FP8 path used by flat `quant_method=modelopt` exports |
 | `runtime/layers/quantization/modelopt_quant.py` | ModelOpt FP8 and NVFP4 runtime loading |
 | `runtime/utils/quantization_utils.py` | resolves flat ModelOpt configs and reconstructs NVFP4 config from metadata |
 | `runtime/loader/transformer_load_utils.py` | guards incompatible FP8 offload modes |
@@ -32,10 +32,13 @@ These options are intended to preserve output quality. In practice, some paths (

 | Option | CLI Flag / Env Var | What It Does | Speedup | Limitations / Notes |
 |---|---|---|---|---|
+| **Performance Mode** | `--performance-mode auto\|speed\|memory\|manual` (`--mode` alias) | Applies model-aware residency, FSDP/CFG, and compile defaults without overriding explicit flags. `auto` is the safe default; `speed` favors GPU residency; `memory` favors offload; `manual` leaves performance args explicit. | Fastest way to establish a sensible deployment baseline | `speed` may OOM and enables `torch.compile` only when the model deployment config allows it. Explicit offload/FSDP/parallelism/compile flags win. Use `manual` for controlled A/B benchmarks. |
 | **torch.compile** | `--enable-torch-compile` | Applies `torch.compile` to the DiT forward pass, fusing ops and reducing kernel launch overhead. | ~1.2–1.5x on denoising | First request is slow (compilation). May cause minor precision drifts due to [PyTorch issue #145213](https://github.com/pytorch/pytorch/issues/145213). Pair with `--warmup-mode request` for best results. |
+| **Breakable CUDA Graph** | `--enable-breakable-cuda-graph --warmup-resolutions <WxH...>` plus optional `--bcg-text-buckets ...` | Captures fixed-resolution DiT segments while leaving attention/collectives eager, reducing launch overhead on supported pipelines. | Large on launch-bound paths; merged SANA and LTX-2 cases show material e2e gains | Mutually exclusive with `torch.compile` and Cache-DiT; BCG takes priority. Every served resolution must be declared for warmup capture. Current support is model-specific (Ideogram4, LTX-2, MiniMax-H3, Qwen-Image, SANA1.5, Z-Image, GLM-Image); benchmark before keeping it. |
 | **Warmup** | `--warmup-mode request` | Runs dummy forward passes to warm up CUDA caches, JIT, and `torch.compile`. Eliminates cold-start penalty. | Removes first-request latency spike | Adds startup time. Without `--warmup-resolutions`, warmup happens on first request. |
 | **Warmup Resolutions** | `--warmup-resolutions 256x256 720x720` | Pre-compiles and warms up specific resolutions at server startup (instead of lazily on first request). | Faster first request per resolution | Each resolution adds to startup time. Serving mode only; useful when you know your target resolutions in advance. |
 | **Multi-GPU (SP)** | `--num-gpus N --ulysses-degree N` | Sequence parallelism across GPUs. Shards sequence tokens (not frames) to minimize padding. | Near-linear scaling with N GPUs | Requires NCCL; inter-GPU bandwidth matters. `ulysses_degree * ring_degree = sp_degree`. For Wan2.2 video, start by benchmarking pure Ulysses before assuming a mixed Ulysses/Ring layout is fastest. |
+| **Cross-node SP** | `--nnodes`, `--node-rank`, `--dist-init-addr` with total `--num-gpus`; combine node-local Ulysses with cross-node Ring | Extends sequence parallel groups across multiple nodes. | Capacity and long-sequence scaling beyond one host | Prefer Ulysses within a node and Ring across nodes; all-to-all is usually the least cross-node-friendly. Use `--encoder-parallel replicate` today and verify the model's Ring admission and determinism. MiniMax-H3 is the current end-to-end validated recipe. |
 | **CFG Parallel** | `--enable-cfg-parallel` | Runs conditional and unconditional CFG branches in parallel across GPUs. For CFG models on multi-GPU, benchmark this against pure Ulysses on your topology instead of assuming one always wins. | Often faster than pure SP for CFG models | Requires `num_gpus >= 2`. Halves the Ulysses group size (e.g. 8 GPU → two 4-GPU groups). Only for models that use CFG. Nightly coverage configs may intentionally use smaller Ulysses groups to keep ring behavior exercised; that does not automatically make them the lowest-latency choice. |
 | **Layerwise Offload** | `--dit-layerwise-offload` | Async layer-by-layer H2D prefetch with compute overlap. Only ~2 DiT layers reside on GPU at a time, dramatically reducing VRAM. For some video models the copy stream can be almost fully hidden behind compute. | Saves VRAM (40 GB → ~11 GB for Wan A14B); can be near-zero speed cost on the right workload | Enabled by default for Wan/MOVA video models. Incompatible with Cache-DiT. For **image models** or highly parallelized setups (many GPUs, small per-GPU compute), the copy stream may not be fully hidden and can cause slowdown. |
 | **Offload Prefetch Size** | `--dit-offload-prefetch-size F` | Fine-grained control over layerwise offload: how many layers to prefetch ahead. `0.0` = 1 layer (min VRAM), `0.1` = 10% of layers, `≥1` = absolute layer count. | Tune for cases where default offload has copy stream interference (e.g. image models). 0.05–0.1 is a good starting point. | Values ≥ 0.5 approach no-offload VRAM with worse performance. Use lower values when copy overlap is weak; disable offload when memory allows and latency dominates. |
@@ -53,8 +56,13 @@ These options **trade output quality** for speed or VRAM savings. Results will d

 | Option | CLI Flag / Env Var | What It Does | Speedup | Quality Impact / Limitations |
 |---|---|---|---|---|
+| **Request Quality Fast Paths** | `--quality high` (`lossless` is default) | Mounts model-owned accelerated DiT/VAE paths that are validated for high quality but are not bit-exact to the reference path. | Model- and shape-specific | Support is per model and may be a no-op. Keep `--quality lossless` as the A/B ground truth. Do not confuse this with `--output-quality`, which controls file compression. |
 | **Approximate Attention** | `--attention-backend sage_attn` / `sage_attn_3` / `sliding_tile_attn` / `video_sparse_attn` / `sparse_video_gen_2_attn` / `vmoba_attn` / `sla_attn` / `sage_sla_attn` | Replaces exact attention with approximate or sparse variants. `sage_attn`: INT8/FP8 quantized Q·K; `sliding_tile_attn`: spatial-temporal tile skipping; others: model-specific sparse patterns. | ~1.5–2x on attention (varies by backend) | Quality degradation varies by backend and model. `sage_attn` is the most general; sparse backends (`sliding_tile_attn`, `video_sparse_attn`, etc.) are video-model-specific and may require config files (e.g. `--mask-strategy-file-path` for STA). Requires corresponding packages installed. |
 | **Cache-DiT** | Native: `SGLANG_CACHE_DIT_ENABLED=true` plus `SGLANG_CACHE_DIT_*` env vars. Diffusers backend: `--backend diffusers --cache-dit-config <yaml-or-json>` | Caches intermediate residuals across denoising steps and skips redundant computations via DBCache, TaylorSeer, and optional SCM. | ~1.5-2x on supported models | Quality depends on cache policy. Incompatible with `--dit-layerwise-offload`. Do not pass `--cache-dit-config` for native SGLang tuning unless you are intentionally using the diffusers backend flow. |
+| **TeaCache** | `--enable-teacache` (uses model sampling presets) | Reuses residuals when adjacent denoising steps are sufficiently similar. | Model- and threshold-dependent | Approximate and model-specific. Mutually exclusive with Spectrum. Fix prompt/seed/shape/steps and validate temporal consistency, not only single frames. |
+| **Spectrum** | `--enable-spectrum` plus optional `--spectrum-*` controls | Forecasts DiT features and skips selected denoising steps. | Defaults target an accuracy/speed tradeoff; aggressive windows can be much faster | Native `sglang generate` only for FLUX.1, Wan, HunyuanVideo, and SD3; not FLUX.2 or server requests. Mutually exclusive with TeaCache. `--debug` adds shadow validation and is not representative latency. |
+| **Progressive Resolution** | `--progressive-mode dct_rewind --progressive-levels N --progressive-delta D` | Runs early denoising at lower latent resolution, then spectrally upsamples and switches to the target resolution. | Model- and schedule-dependent | Approximate and pipeline-specific. Keep the switch schedule fixed and compare detail, composition, and temporal stability. |
+| **Causal KV-Cache Quantization** | `--kv-cache-quant int4\|int2` plus optional `--kv-cache-quant-*` controls | Compresses completed causal KV-cache chunks with Quant-VideoGen PRQ while keeping the mutable/current chunk and recent chunks in BF16. | Primarily a long-session memory saving | Currently limited to LingBot World realtime causal serving; requires `quant-videogen`. INT4 is the starting point; INT2 saves more memory with more error. It quantizes cache state, not checkpoint weights. |
 | **Quantized Models (Nunchaku / SVDQuant)** | `--enable-svdquant --transformer-weights-path <path>` + optional `--quantization-precision int4\|nvfp4`, `--quantization-rank 32` | W4A4-style quantization via [Nunchaku](https://nunchaku.tech). Reduces DiT weight memory by ~4x. Precision/rank can be auto-inferred from weight filename or set explicitly. | ~1.5–2x compute speedup | Lossy quantization; quality depends on rank and precision. Requires pre-quantized weights. Ampere (SM8x) or SM12x only (no Hopper SM90). Higher rank = better quality but more memory. |
 | **Pre-quantized Transformer Override** | `--transformer-path <dir-or-repo>` / `--transformer-weights-path <path>` | Load a quantized transformer component or raw transformer weights. For converted ModelOpt FP8/NVFP4 directories, prefer `--transformer-path`; use `--transformer-weights-path` for weight-only artifacts the model loader expects. | ~1.3–1.5x compute (dtype dependent) | Requires a validated quantized transformer override, such as one produced by the ModelOpt helper tools. Quality is usually slightly worse than BF16 and depends on the format, fallback layers, and calibration scope. |
 | **Component Precision Override** | `--dit-precision fp16`, `--vae-precision fp16\|bf16` | On-the-fly dtype conversion for individual components. E.g. convert a BF16 model to FP16 at load time, or run VAE in BF16 instead of FP32. | Reduces memory; FP16 can be faster on some GPUs | May affect numerical stability. VAE is FP32 by default for accuracy; lowering it is lossy. DiT defaults to BF16. |
@@ -216,6 +224,37 @@ sglang generate --model-path <IMAGE_MODEL> \

 Note: for image models, per-layer compute is smaller, so layerwise offload may not fully hide H2D transfer. Disable DiT layerwise and CPU offload if VRAM allows; otherwise a large image DiT can stay resident on CPU and make the denoise loop H2D-bound.

+### Launch-bound fixed-resolution path: Breakable CUDA Graph
+
+```bash
+sglang serve --model-path Efficient-Large-Model/SANA1.5_1.6B_1024px_diffusers \
+  --performance-mode speed \
+  --enable-torch-compile false \
+  --enable-breakable-cuda-graph \
+  --warmup-resolutions 1024x1024 \
+  --port 30010
+```
+
+Keep `torch.compile` off, declare every production resolution, and benchmark
+the exact prompt-length distribution. Add `--bcg-text-buckets` only when the
+default buckets create excessive padding or miss a served prompt signature.
+
+### Compare request-scoped high-quality fast paths
+
+```bash
+sglang generate --model-path <MODEL> \
+  --quality lossless --prompt "..." --seed 42 \
+  --perf-dump-path baseline.json --save-output
+
+sglang generate --model-path <MODEL> \
+  --quality high --prompt "..." --seed 42 \
+  --perf-dump-path quality-high.json --save-output
+```
+
+Keep every other flag fixed and compare the generated artifact as well as the
+perf dumps. If the model has no registered quality-gated sites, `high` may be a
+no-op.
+
 ### Image-edit baselines: JoyAI and FireRed

 ```bash
@@ -317,6 +356,7 @@ Use these as first commands to benchmark, not as universal winners.
 | FLUX.1 / FLUX.2 image | 1024x1024, runtime-default steps/guidance, 1 GPU | `--enable-torch-compile --warmup-mode request --dit-layerwise-offload false` | `black-forest-labs/FLUX.*` repos are gated; for FP8/NVFP4 use validated `--transformer-path` or `--transformer-weights-path` flows from the quant skill. |
 | FLUX.2 Klein / Klein Base | 1024x1024, runtime-default steps/guidance, 1 GPU | `--enable-torch-compile --warmup-mode request --dit-layerwise-offload false` | Current registry has `black-forest-labs/FLUX.2-klein-4B`, `FLUX.2-klein-9B`, and base variants. Klein is step-distilled; Klein Base is not. |
 | Qwen-Image / Qwen-Image-Edit | 1024x1024, runtime-default steps/guidance, 1 GPU | `--enable-torch-compile --warmup-mode request`; optionally native `SGLANG_CACHE_DIT_ENABLED=true` | Cache-DiT is lossy. For edit tasks, keep reference image, seed, and output size fixed. |
+| Krea-2 | 1024x1024, distilled `oss_turbo` defaults (8 steps, guidance 1.0) | `--performance-mode speed --warmup-mode request` | Native `krea/Krea-2` text-to-image path with Qwen3-VL text conditioning. The repo may require HF access; keep the 8-step distilled baseline separate from non-turbo sampling experiments. |
 | Z-Image / Z-Image-Turbo | 1024x1024, runtime-default steps/guidance, 1 GPU | `--enable-torch-compile --warmup-mode request` | Keep base Z-Image separate from Turbo: base uses 50-step CFG defaults, Turbo uses 9-step zero-CFG defaults. Mainline has bf16-native Triton RMSNorm scale and tanh-residual fusions. |
 | Wan2.2 A14B T2V/I2V | 1280x720, 81 frames | Nightly: `--num-gpus 4 --enable-cfg-parallel --ulysses-degree 2 --text-encoder-cpu-offload --pin-cpu-memory` | For lowest latency, also benchmark pure Ulysses on the same GPUs. |
 | Wan2.2 TI2V 5B | 1280x720, 81 frames, 1 GPU | `--enable-torch-compile --warmup-mode request` | Keep the input image and motion prompt fixed when comparing sparse attention or Cache-DiT. |
@@ -329,7 +369,8 @@ Use these as first commands to benchmark, not as universal winners.
 | JoyAI-Image-Edit | 1024-class TI2I, 40 steps, guidance 4.0 | `--backend=sglang --num-gpus 2 --enable-cfg-parallel --ulysses-degree 1 --enable-torch-compile --warmup-mode request --dit-layerwise-offload false --dit-cpu-offload false` | Newly supported image-edit path. Keep the input image, prompt, seed, and output size fixed; 2-GPU CFG parallel is the validated H100 starting point. |
 | FireRed-Image-Edit 1.0 / 1.1 | 1024x1024 image edit, 40 steps, guidance 4.0 | `--backend=sglang --num-gpus 2 --enable-cfg-parallel --ulysses-degree 1 --enable-torch-compile --warmup-mode request --dit-layerwise-offload false --dit-cpu-offload false` | Uses the native `QwenImageEditPlusPipeline` path. 2-GPU CFG parallel is the validated H100 starting point; benchmark 1.0 and 1.1 separately because checkpoint differences can change denoise latency. |
 | Hunyuan3D-2 shape | Shape generation, 50 steps, guidance 5.0 | `--backend=sglang --enable-torch-compile --warmup-mode request --dit-layerwise-offload false --dit-cpu-offload false` | Focus on `Hunyuan3DShapeDenoisingStage`; keep mesh export/paint timings separate from denoise. |
-| MOVA / Helios / LingBot World | Use the benchmark/profile presets or server test cases first | `--enable-torch-compile --warmup-mode request`; pin offload and topology flags explicitly | These video/realtime families have model-specific stages and condition handling. Keep prompt/image/action inputs fixed and prefer perf dumps over wall time alone. |
+| LingBot Video MoE 30B | 384x640, 17 frames, 12 steps for the current GPU case | `--model-path robbyant/lingbot-video-moe-30b-a3b --text-encoder-cpu-offload` | Native T2V path. Prompts are structured JSON captions, not raw free text; keep that contract when comparing latency or quality. |
+| MOVA / Helios / LingBot World | Use the benchmark/profile presets or server test cases first | `--enable-torch-compile --warmup-mode request`; pin offload and topology flags explicitly | These video/realtime families have model-specific stages and condition handling. For LingBot World causal serving, keep `--kv-cache-quant off` as the exact cache baseline before testing INT4/INT2. |

 ## Historical PR Watchlist

@@ -344,10 +385,13 @@ about whether the work has merged:
 ## Tips

 - **Benchmarking**: always use `--warmup-mode request` and look for the line ending with `(with warmup excluded)` for accurate timing.
+- **Preset vs experiment control**: start with `--performance-mode auto` or
+  `speed` for deployment, but use `--performance-mode manual` and pin the
+  relevant residency/parallelism flags for controlled A/B claims.
 - **Perf dump**: use `--perf-dump-path result.json` to save structured metrics, then compare with `python python/sglang/multimodal_gen/benchmarks/compare_perf.py baseline.json result.json`.
 - **Offload tuning**: after the first request, the runtime logs peak GPU memory and which components could stay resident. Use this to decide which `--*-cpu-offload` flags to disable.
 - **Backend selection**: `--backend sglang` (default, auto-detected) enables native optimizations (fused kernels, SP, native Cache-DiT env knobs, etc.). `--backend diffusers` falls back to Diffusers pipelines and is the path that accepts `--cache-dit-config` plus diffusers attention backend names.
 - **Wan2.2-I2V sizing**: explicit `--width/--height` on `Wan2.2-I2V-A14B` control the target area while preserving the condition-image aspect ratio.
- **Mainline diffusion fast paths**: before proposing a new kernel or overlap scheme, check `sglang-diffusion-benchmark-profile/existing-fast-paths.md`. It covers H3 indexed modulation, fused QK norm + RoPE, packed Ulysses QKV/USP relayout and batched TP AdaLN, plus GroupNorm+SiLU, Z-Image bf16-native Triton norm modulation, LTX2 split RoPE, LTX2 residual-gate add, varlen USP pack/scatter, packed QKV/NVFP4 expectations, and existing multi-GPU overlap families such as Ulysses / USP and turbo-layer async all-to-all.
+- **Mainline diffusion fast paths**: before proposing a new kernel or overlap scheme, check `sglang-diffusion-benchmark-profile/existing-fast-paths.md`. It covers H3 indexed modulation, fused QK norm + RoPE, packed Ulysses QKV/USP relayout and batched TP AdaLN; FLUX/GLM/SANA bit-exact LayerNorm+modulate; request-scoped quality gates; Wan causal-VAE data movement; GroupNorm+SiLU, Z-Image bf16-native norm modulation, LTX2 split RoPE/residual-gate add, varlen USP pack/scatter, packed QKV/NVFP4, breakable CUDA graph, and existing distributed overlap families.
 - **NVFP4 trace interpretation**: on FLUX.2 NVFP4 and Nunchaku-style checkpoints, packed QKV is expected. SGLang intentionally uses fused projection modules such as `to_qkv` / `to_added_qkv` instead of separate `to_q` / `to_k` / `to_v`, so a split-QKV trace usually means the quantized path did not engage rather than a brand new fusion opportunity.
 - **Hotspot workflow split**: use `sglang-diffusion-benchmark-profile` to prove and classify a slowdown with perf dumps plus `torch.profiler`; hand concrete kernel work off with the perf/profile evidence attached instead of expanding the benchmark skill.