minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

[skill] Remove outdated llm-serving-auto-benchmark skill (#29487)

Browse Source
This commit is contained in:
Xiaoyu Zhang
2026-06-27 14:19:11 +08:00
committed by GitHub
parent 43435a2f8e
commit c36f166364
60 changed files with 563 additions and 6977 deletions
Download Patch File Download Diff File Expand all files Collapse all files
python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-benchmark-profile/SKILL.md
+1
View File
@@ -58,6 +58,7 @@ Always rule out these existing families first:
- SANA packed self-attention Q/K/V and cross-attention K/V GEMMs
- fused diffusion `QK norm + RoPE`
- LTX2 split RoPE
- LTX2 residual-gate add
- varlen USP attention pack/scatter
- NVFP4 / Nunchaku packed QKV
- Nunchaku fused GELU MLP
python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-benchmark-profile/existing-fast-paths.md
+17 -1
View File
@@ -17,14 +17,18 @@ framework-specific optimization workflow.
- `python/sglang/jit_kernel/diffusion/triton/rmsnorm_onepass.py`
- `python/sglang/jit_kernel/diffusion/triton/rotary.py`
- `python/sglang/jit_kernel/diffusion/triton/ltx2_rotary.py`
- `python/sglang/jit_kernel/diffusion/residual_gate_add.py`
- `python/sglang/jit_kernel/csrc/diffusion/residual_gate_add.cuh`
- `python/sglang/jit_kernel/diffusion/triton/varlen_pack_pad.py`
- `python/sglang/jit_kernel/diffusion/cutedsl/scale_residual_norm_scale_shift.py`
- `test/registered/jit/diffusion/test_qwen_image_modulation.py`
- `test/registered/jit/diffusion/test_group_norm_silu.py`
- `test/registered/jit/diffusion/test_residual_gate_add.py`
- `test/registered/jit/diffusion/test_varlen_pack_pad.py`
- `test/registered/jit/diffusion/test_varlen_uspattn_equivalence.py`
- `test/registered/jit/benchmark/diffusion/bench_qwen_image_modulation.py`
- `test/registered/jit/benchmark/diffusion/bench_group_norm_silu.py`
- `test/registered/jit/benchmark/diffusion/bench_residual_gate_add.py`
- `python/sglang/jit_kernel/norm.py`
- `python/sglang/multimodal_gen/runtime/platforms/cuda.py`
- `python/sglang/multimodal_gen/runtime/layers/attention/selector.py`
@@ -87,7 +91,17 @@ framework-specific optimization workflow.
- Constraints: `cos` and `sin` shapes must match `[B, H, S, head_dim / 2]`, and `inner_dim == H * head_dim`.
- Workflow rule: if LTX-2 traces show a large split-RoPE PyTorch chain, check whether the LTX2-specific Triton path was disabled by shape or dtype before proposing a new RoPE kernel.
8. HunyuanVideo / LTX upsampler GroupNorm + SiLU fusion
8. LTX2 residual-gate add fusion
- Kernel: `diffusion_residual_gate_add`
- Locations: `diffusion/residual_gate_add.py`, `csrc/diffusion/residual_gate_add.cuh`, `runtime/models/dits/ltx_2.py`
- Use case: `residual + update * gate` in LTX2 self-attention, prompt cross-attention, audio/video cross-attention, and feed-forward residual updates.
- Constraints: `residual`, `update`, and `gate` must be CUDA tensors on the same device, contiguous, same dtype (`fp16`, `bf16`, or `fp32`), with `update.shape == residual.shape`; `gate` can match `residual` or be row-broadcast with the last dimension matching.
- Behavior: `_ltx2_residual_gate_add(...)` uses the CUDA custom op while guards pass. On a runtime exception outside `torch.compile`, it logs once, disables the fast path for the process, and falls back to `residual + update * gate`.
- Validation: `test/registered/jit/diffusion/test_residual_gate_add.py`.
- Microbench: `test/registered/jit/benchmark/diffusion/bench_residual_gate_add.py`.
- Workflow rule: if LTX2 traces show repeated elementwise `mul` + `add` ladders around attention or MLP residuals, check whether this existing CUDA path was disabled by shape, dtype, contiguity, or a prior runtime failure before proposing another elementwise fusion.
9. HunyuanVideo / LTX upsampler GroupNorm + SiLU fusion
- Kernel: `triton_group_norm_silu`
- Locations: `diffusion/group_norm_silu.py`, `triton/group_norm_silu.py`, `runtime/models/vaes/hunyuanvae.py`, `runtime/models/upsampler/latent_upsampler.py`
- Use case: `activation(group_norm(x))` when the activation is non-inplace `nn.SiLU` and the GroupNorm is affine.
@@ -178,6 +192,7 @@ framework-specific optimization workflow.
- QK norm: `apply_qk_norm` used in `flux.py`, `flux_2.py`, `qwen_image.py`, `zimage.py`, `wanvideo.py`, `ltx_2.py`, `hunyuanvideo.py`.
- QK norm + RoPE: `apply_qk_norm_rope` in `layernorm.py`; use this path when the model wants fused attention prep instead of separate QK norm and RoPE calls.
- LTX2 split RoPE: `apply_ltx2_split_rotary_emb` in `ltx_2.py`.
- LTX2 residual-gate add: `_ltx2_residual_gate_add` in `ltx_2.py` wraps the CUDA `diffusion_residual_gate_add` custom op for attention, cross-attention, and MLP residual updates.
- Varlen USP attention: `fused_pack_qkv` and `fused_scatter_to_padded` in `attention/layer.py`.
- SANA packed projections: `to_qkv` and `to_kv` in `sana.py`.
- Nunchaku fused GELU MLP: `_fused_gelu_mlp` in `flux.py` for quantized FLUX-family checkpoints.
@@ -207,6 +222,7 @@ relying on any file path, flag, or claim about whether the work has merged.
- #18897 dual norm fusion for FLUX-family paths (draft).
- #20429 Qwen-Image layernorm and `fuse_scale_shift_gate_select01` work.
- #20530 MOVA fused RMSNorm + interleaved RoPE.
- #29361 LTX2 residual-gate CUDA fast path for `residual + update * gate`.
- VAE and decode-side acceleration:
- #22531 LTX2 parallel VAE support and #20927 batched tiled VAE decode (draft).
- Attention, communication, and runtime scheduling:
python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-performance/SKILL.md
+1 -1
View File
@@ -285,6 +285,6 @@ about whether the work has merged:
- **Offload tuning**: after the first request, the runtime logs peak GPU memory and which components could stay resident. Use this to decide which `--*-cpu-offload` flags to disable.
- **Backend selection**: `--backend sglang` (default, auto-detected) enables native optimizations (fused kernels, SP, native Cache-DiT env knobs, etc.). `--backend diffusers` falls back to Diffusers pipelines and is the path that accepts `--cache-dit-config` plus diffusers attention backend names.
- **Wan2.2-I2V sizing**: explicit `--width/--height` on `Wan2.2-I2V-A14B` control the target area while preserving the condition-image aspect ratio.
- **Mainline diffusion fast paths**: before proposing a new kernel or overlap scheme, check `sglang-diffusion-benchmark-profile/existing-fast-paths.md`. It covers GroupNorm+SiLU, Z-Image residual-form modulation, fused diffusion `QK norm + RoPE`, LTX2 split RoPE, varlen USP pack/scatter, packed QKV/NVFP4 expectations, and existing multi-GPU overlap families such as Ulysses / USP and turbo-layer async all-to-all.
- **Mainline diffusion fast paths**: before proposing a new kernel or overlap scheme, check `sglang-diffusion-benchmark-profile/existing-fast-paths.md`. It covers GroupNorm+SiLU, Z-Image residual-form modulation, fused diffusion `QK norm + RoPE`, LTX2 split RoPE, LTX2 residual-gate add, varlen USP pack/scatter, packed QKV/NVFP4 expectations, and existing multi-GPU overlap families such as Ulysses / USP and turbo-layer async all-to-all.
- **NVFP4 trace interpretation**: on FLUX.2 NVFP4 and Nunchaku-style checkpoints, packed QKV is expected. SGLang intentionally uses fused projection modules such as `to_qkv` / `to_added_qkv` instead of separate `to_q` / `to_k` / `to_v`, so a split-QKV trace usually means the quantized path did not engage rather than a brand new fusion opportunity.
- **Hotspot workflow split**: use `sglang-diffusion-benchmark-profile` to prove and classify a slowdown with perf dumps plus `torch.profiler`; hand concrete kernel work off with the perf/profile evidence attached instead of expanding the benchmark skill.