[diffusion] Fuse SANA-Video interleaved RoPE (#35695)
This commit is contained in:
@@ -98,6 +98,7 @@ These fusion families mount under `quality="high"`:
|
||||
| --- | --- | --- | --- |
|
||||
| `fused_inplace_qknorm_rope` | JIT CUDA | one bf16 rounding step vs the split baseline; exact with `round_norm_before_rope=True` | separate QK-norm kernel + RoPE |
|
||||
| `rope_rotate_half` | Triton | bit-exact | `chunk` → `cat(-x2, x1)` → two muls + add → `cat(tail)`, about 7 kernels per projection |
|
||||
| `interleaved_rope_fp64` | JIT CUDA | bit-exact | paired SANA-Video Q/K RoPE with fp64 tables, about 14 eager kernels |
|
||||
| `ltx2_qknorm_split_rope` | JIT CUDA | close (validated on B200) | LTX-2 QK-norm + split RoPE |
|
||||
| `ltx25_decoder_rope` | JIT CUDA | bit-exact | paired LTX-2.5 decoder 3D RoPE from cached compact axis tables |
|
||||
| `hunyuan_qkv_rope_pack` | Triton | bit-exact | QKV pack and RoPE in one pass |
|
||||
@@ -146,6 +147,7 @@ Kernels are written against a specific eager chain in a specific model, so cover
|
||||
| LTX-2 | QK-norm + split RoPE, ada-values split, RMSNorm+modulate, modulate, residual-gate add, linear+GELU |
|
||||
| LTX-2.5 decoder | paired 3D RoPE with shared axis-table cache |
|
||||
| HunyuanVideo | QKV+RoPE pack, strided QK RMSNorm, linear+GELU |
|
||||
| SANA-Video | paired fp64 interleaved RoPE |
|
||||
| Sana | LN+modulate, GLUMB bias+SiLU / bias+GLU, residual-gate add |
|
||||
| SANA-Video | Packed QKV/KV; BF16-input linear attention at `quality=high` |
|
||||
| Sana-WM | bidirectional gated delta-net, fused QK inverse-RMS |
|
||||
|
||||
Reference in New Issue
Block a user