Commit Graph
100 Commits
Author SHA1 Message Date
Xiaoyu Zhang c32c4ef79c [Kernel] Move sgl-kernel under sglang.kernels.aot (#32648) 2026-07-29 17:25:00 +08:00
Xiaoyu Zhang c9947b087b Enable multimodal prefill BCG for VL and audio models (#30872) 2026-07-29 06:47:40 +08:00
Xiaoyu Zhang 7778dd23ea [diffusion] refactor: remove stale kernels and dead code (#32651) 2026-07-29 06:23:23 +08:00
Xiaoyu Zhang 9cffc2ba52 [Kernel] Remove unused implementations and stale registry entries (#32636) 2026-07-28 18:12:10 +08:00
8d6549bc40 [Attention Backend] Extend hpc_ops dynamic-scheduled decode to bf16 (#32304)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Halcyon <56064364+VAthree@users.noreply.github.com>
2026-07-27 21:31:04 +08:00
3d91a569ce [MoE Backend] Add HPC-Ops FP8 MoE runner backend (#30541)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Halcyon <56064364+VAthree@users.noreply.github.com>
2026-07-24 19:46:11 +08:00
841fa293b5 [Fix] Reject online weight updates while the HPC-Ops router GEMM split cache is active (#31943)
Co-authored-by: Halcyon <56064364+VAthree@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 19:30:13 +08:00
Xiaoyu ZhangandClaude Fable 5 d4a0dfbc31 [Fix] Two root causes of the H100 deepep TBO CI break: scale-tensor use-after-free + missing non-finite quant sanitization (#32188)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 07:29:28 +08:00
Xiaoyu ZhangandClaude Opus 4.8 62aa85d9aa [Kernel] Sweep missed dedicated kernels into kernels.ops (moe/quant siblings + dspark) (RFC #29630) (#32160)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-23 17:07:16 +08:00
Xiaoyu ZhangandClaude Opus 4.8 11b0e5c5ad [Kernel] Classification cleanup: unify _jit_ naming, drop empty/model groups, add elementwise (RFC #29630) (#32148)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-23 13:47:02 +08:00
Xiaoyu ZhangandClaude Opus 4.8 2d1a7be8c4 [Kernel] Reclassify kernel tests by ops group + move helpers out of the package (RFC #29630) (#32128)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-23 12:18:27 +08:00
Xiaoyu ZhangandClaude Opus 4.8 99f636a86f [Kernel] RFC #29630 finale: retire sglang.jit_kernel into sglang.kernels (#32072)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-23 08:35:09 +08:00
0a6d1930c3 [Attention Backend] Add HPC-Ops attention backend (#30540)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Halcyon <56064364+VAthree@users.noreply.github.com>
2026-07-22 22:06:22 +08:00
Xiaoyu ZhangandClaude Opus 4.8 74338e94f1 [Kernel] Phase 4 batch-3: migrate tangled JIT subsystems + new groups into kernels.ops (RFC #29630) (#32045)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 21:15:03 +08:00
Xiaoyu ZhangandClaude Opus 4.8 977ea336cd [Kernel] Phase 4 batch-2: migrate JIT operator groups into kernels.ops (no shims) (RFC #29630) (#32015)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 17:49:51 +08:00
Xiaoyu ZhangandClaude Opus 4.8 246b3c3eaf [Kernel] Phase 3+4: move JIT infra + operator groups into sglang.kernels (RFC #29630) (#31666)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 11:28:27 +08:00
Xiaoyu Zhang 075bd97952 [Benchmark] Remove obsolete auto-benchmark remnants (#31941) 2026-07-21 20:44:52 +08:00
e4eea7ce2f Optimize LongCat-Flash router GEMM with the HPC-Ops bf16xfp32 kernel (#30247)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Halcyon <56064364+VAthree@users.noreply.github.com>
2026-07-21 20:05:17 +08:00
Xiaoyu Zhang 829e9ce9d5 Lower AutoRound quantization MMLU threshold (#31748) 2026-07-20 13:40:48 +08:00
Xiaoyu ZhangandClaude Opus 4.8 216b750c8f [Kernel] Sweep decoupled scattered kernels into sglang.kernels.ops (RFC #29630) (#31582)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-18 19:07:07 +08:00
Xiaoyu ZhangandClaude Opus 4.8 619609aa5a [Kernel] Simplify sglang.kernels tests to idiomatic pytest style (RFC #29630) (#31546)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 15:06:26 +08:00
Xiaoyu ZhangandClaude Opus 4.8 1ac1ffea0c [Kernel] Fill non-CUDA coverage: HIP (aiter/rocm-triton) + Ascend NPU backends (RFC #29630) (#31307)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 14:05:35 +08:00
Xiaoyu ZhangandClaude Opus 4.8 8432eafd3d [Kernel] Decouple KernelBackend from device + device-based CapabilityRequirement (RFC #29630) (#31292)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 10:35:34 +08:00
Xiaoyu Zhang e73f323464 [JIT] Reduce MoE fused gate CI test sweep (#31400) 2026-07-16 14:42:45 +08:00
Xiaoyu ZhangandClaude Fable 5 22453ca63c docker: build HPC-Ops into the GPU image (#31390)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 11:07:01 +08:00
Xiaoyu ZhangandClaude Fable 5 4aadf94146 [Kernel] Relocate vendored fla and mamba kernel trees to sglang.kernels (RFC #29630, Phase 2.5, 7/7) (#30795)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 12:52:15 +08:00
Xiaoyu ZhangandClaude Fable 5 c00131ebaa [Kernel] Migrate linear-attention, MiniMax-sparse and diffusion kernels to sglang.kernels (RFC #29630, Phase 2.5, 6/7) (#30793)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 11:21:36 +08:00
Xiaoyu ZhangandClaude Fable 5 ba5be86d42 [Kernel] Migrate DSA + DSV4 attention kernels to sglang.kernels (RFC #29630, Phase 2.5, 5/7) (#30792)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 11:11:22 +08:00
Xiaoyu ZhangandClaude Fable 5 ee000f6734 [CI] Fix SGLANG_JIT_KERNEL_RUN_FULL_TESTS never activating the nightly full jit-kernel sweep (#31042)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 17:32:29 +08:00
Xiaoyu ZhangandClaude Fable 5 1a35440c4a [Kernel] Migrate generic attention kernels to sglang.kernels (RFC #29630, Phase 2.5, 4/7) (#30789)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 16:53:46 +08:00
Xiaoyu ZhangandClaude Fable 5 e9ef06c560 [Kernel] Migrate top-level srt/layers stray kernels to sglang.kernels (RFC #29630, Phase 2.5, 3/7) (#30787)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 09:20:59 +08:00
Xiaoyu ZhangandClaude Fable 5 ee464fedc6 [Kernel] Migrate scattered MoE kernels to sglang.kernels (RFC #29630, Phase 2.5, 2/7) (#30786)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 09:03:21 +08:00
Xiaoyu ZhangandClaude Opus 4.8 4c997310f5 [Kernel] Hotfix: update sgl-kernel imports of relocated fp8_kernel (RFC #29630 #30784) (#31089)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 08:41:23 +08:00
Xiaoyu ZhangandClaude Fable 5 874fc07d9b [Kernel] Migrate scattered quantization kernels to sglang.kernels (RFC #29630, Phase 2.5, 1/7) (#30784)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 16:17:01 +08:00
Xiaoyu Zhang 65abb23842 Add diffusion BCG prompt conditioning guard (#30782) 2026-07-11 13:14:31 +08:00
Xiaoyu ZhangandClaude Opus 4.8 6ed9843b57 [Kernel] Introduce sglang.kernels namespace and migrate scattered triton_ops kernels (RFC #29630, Phase 2) (#30044)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 21:41:08 +08:00
Xiaoyu Zhang e9493a015c Fix diffusion BCG lifetime and add Z-Image-Turbo CI (#30584) 2026-07-10 21:19:59 +08:00
Xiaoyu Zhang b8ca06fdad Fix zero expert routed ids for MoE backends (#30387) 2026-07-08 21:23:25 +08:00
33c3dfd7e0 [diffusion] Enable breakable CUDA graph (BCG) for diffusion DiTs (#27436)
Co-authored-by: BBuf <xiaoyu.zhang@radixark.net>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: BBuf <bbuf@sglang.local>
2026-07-08 14:45:48 +08:00
Xiaoyu ZhangandZijie Xia ead1e490b5 [Doc] Add LongCat 2.0 FP8 cookbook (#30320)
Co-authored-by: Zijie Xia <zijie.xia@radixark.ai>
2026-07-07 11:48:13 -07:00
e339c83f82 [Model] Support LongCat 2.0 FP8 (#30275)
Co-authored-by: sunjiaqi11 <sunjiaqi11@meituan.com>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
2026-07-07 19:51:12 +08:00
Xiaoyu ZhangandClaude Opus 4.8 1da7d3a50b [MoE] Retire the AOT moe_fused_gate / kimi_k2_moe_fused_gate gate kernels (#26771) (#29997)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 13:53:17 +08:00
Xiaoyu Zhang 931b00f1b0 [diffusion] Clean up duplicate helper definitions (#30159) 2026-07-05 22:05:11 +08:00
Xiaoyu Zhang 6dd0cefb2a [CI] Revert ModelOpt NVFP4 threshold relax (#29844) 2026-07-04 21:17:32 +08:00
Xiaoyu ZhangandClaude Opus 4.8 a2d7eb303e [MoE] Consolidate ungrouped + grouped gate/topk onto one Triton router (#26771) — faster than AOT on B200/H100/H200, at parity with flashinfer (#29771)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 11:18:59 +08:00
Xiaoyu Zhang b276a9acee chore: cleanup garbage code (#29770) 2026-07-02 16:14:01 +08:00
df0dfbaa45 [Kernel] Strengthen kernel shape coverage (#29636)
Co-authored-by: Khoa Pham <khoa.pham@radixark.ai>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>
2026-07-01 15:44:20 +08:00
Xiaoyu Zhang 8205aa3603 chore: clean diffusion dead code (#29789) 2026-07-01 15:42:16 +08:00
Xiaoyu Zhang 47ae1241d3 [CI] Relax ModelOpt NVFP4 diffusion consistency thresholds (#29767) 2026-07-01 14:45:08 +08:00
Xiaoyu Zhang fcb9f229b3 [KDA-Pilot] Add LTX2 QKNorm split-RoPE CUDA fast path (#29708) 2026-07-01 14:42:07 +08:00
Xiaoyu ZhangandClaude Opus 4.8 3add35e26d [Diffusion] Reuse shared AlignedVector and tidy jit_kernel/diffusion (#29664)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-30 11:38:22 +08:00
Xiaoyu Zhang c36f166364 [skill] Remove outdated llm-serving-auto-benchmark skill (#29487) 2026-06-27 14:19:11 +08:00
Xiaoyu ZhangandClaude Opus 4.8 495f13fa12 [KDA-Pilot] Add diffusion residual-gate CUDA fast path for LTX2 (#29361)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-27 12:59:41 +08:00
Xiaoyu Zhang 8524678889 Fix MiniMax MSA fallback when fmha plan is unavailable (#29250) 2026-06-26 23:14:31 +08:00
Xiaoyu Zhang 18b0e5757e [Diffusion] Fuse LTX2 Ada values (#29390) 2026-06-26 23:13:15 +08:00
Xiaoyu ZhangandClaude Opus 4.8 5996b54bd3 [KDA-Pilot] Add diffusion causal Conv3D cat-pad CUDA fast path for Cosmos3 (#29281)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 15:06:49 +08:00
Xiaoyu Zhang 4d06d4c97f Sync Gemma4 hardware table with Blackwell recipes (#29266) 2026-06-25 22:57:33 +08:00
52c32035eb [diffusion] Add Qwen-Image ModelOpt NVFP4 support (#28928)
Co-authored-by: jingyu-ml <jingyux@nvidia.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-25 22:56:09 +08:00
Xiaoyu Zhang ddda4f9028 [Bugfix] Fix Ministral3 init argument forwarding (#29111) 2026-06-25 18:26:45 +08:00
Xiaoyu Zhang 7c9804ef21 Add MiMo V2.5 Blackwell vision FA4 recipe (#29253) 2026-06-25 13:47:32 +08:00
Xiaoyu Zhang efbe67d237 Tune Gemma4 26B-A4B B200 memory recipe (#29252) 2026-06-25 11:31:25 +08:00
Xiaoyu Zhang 26e1d4d847 [KDA-Pilot] Add B200 diffusion norm-scale-shift CUDA fast path for Qwen-Image (#27392) 2026-06-24 14:38:36 +08:00
Xiaoyu Zhang 534ac98eb2 [codex] Optimize DMD Wan auto residency on high-memory GPUs (#28780) 2026-06-24 14:03:23 +08:00
Xiaoyu Zhang 31d71c47af [Diffusion] Fix SANA VAE dtype and TurboWan backend selection (#28769) 2026-06-23 22:57:08 +08:00
Xiaoyu Zhang 6b2c730bf7 [codex] Fix DSA indexer in prefill piecewise CUDA graph (#28644) 2026-06-22 22:39:21 +08:00
Xiaoyu Zhang b43bd6824f [B300] Enable FlashInfer allreduce for Qwen3-VL MoE (#28786) 2026-06-22 22:38:45 +08:00
Xiaoyu Zhang 0c9e775f2c [Diffusion] Fix FastWan2.1 default 480p resolution (#28733) 2026-06-22 18:43:24 +08:00
Xiaoyu Zhang c1416bb3ee [Diffusion] Keep FastHunyuan VAE resident on high-memory GPUs (#28773) 2026-06-20 15:54:06 +08:00
Xiaoyu Zhang 31c0a98066 [codex] Update diffusion skills for latest main (#28711) 2026-06-19 16:37:59 +08:00
Xiaoyu Zhang b6be5dd20f [codex] Remove outdated SGLang SOTA skill (#28719) 2026-06-19 16:26:44 +08:00
Xiaoyu Zhang 3b417d3999 Revert "[Diffusion] FLUX: fuse FeedForward GELU into up-proj GEMM (cublasLt epilogue)" (#28708) 2026-06-19 13:44:57 +08:00
Xiaoyu ZhangandClaude Opus 4.8 7da92e1112 [diffusion] Sana: pack self-attn q/k/v and cross-attn k/v into single GEMMs (#28393)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-19 13:25:52 +08:00
Xiaoyu Zhang 7e20e25848 [docs] Add B300 cookbook deployment options (#28697) 2026-06-18 21:52:45 -07:00
Xiaoyu Zhang b7d7dfb4ed [Bugfix] Fix Intern-S1 FP8 expert count lookup (#28629) 2026-06-18 17:49:59 +08:00
Xiaoyu Zhang c5b9106c1a [perf] Use default torch compile mode for Wan2.2 T2V A14B (#28304) 2026-06-16 15:45:00 +08:00
07b9108348 [Diffusion] FLUX: fuse FeedForward GELU into up-proj GEMM (cublasLt epilogue) (#28166)
Co-authored-by: BBuf <xiaoyu.zhang@radixark.net>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 13:00:26 +08:00
Xiaoyu ZhangandBBuf 06e0df5899 Optimize Qwen3 Next FP8 MoE on H200 (#26204)
Co-authored-by: BBuf <xiaoyu.zhang@radixark.net>
2026-06-11 22:18:43 +08:00
Xiaoyu ZhangandBBuf 1a6b5561db Fix MLA scaling when YARN scaling is disabled (#26203)
Co-authored-by: BBuf <xiaoyu.zhang@radixark.net>
2026-06-11 22:17:55 +08:00
Xiaoyu Zhang d571e076fa [codex] Centralize more inline Triton kernels (#27429) 2026-06-11 22:17:26 +08:00
Xiaoyu ZhangandClaude Opus 4.8 aa18a68ac5 [diffusion] Run LTX-2 VAE decode in channels_last_3d (faster decode, lower peak memory) (#27431)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-09 23:26:40 +08:00
Xiaoyu Zhangandxiaoyu.zhang 6c2770149b [diffusion] Fix native text-encoder loading for T5/UMT5 encoder-decoder models (#27432)
Co-authored-by: xiaoyu.zhang <xiaoyu.zhang@radixark.net>
2026-06-08 12:23:16 +08:00
Xiaoyu Zhang 5bf7dd8e4a Update SGLang diffusion skills (#27496) 2026-06-08 12:22:45 +08:00
Xiaoyu Zhang ff8b97406d [diffusion] optimize: precompute ideogram4 denoising metadata (#27443) 2026-06-07 14:27:41 +08:00
Xiaoyu Zhang 2c3e84affe [Diffusion] Enable Cosmos3 denoising profiling (#27439) 2026-06-07 10:32:28 +08:00
bd7fea0740 [diffusion] Fix LingBot-World crash on camera control with ulysses>1 (#27437)
Co-authored-by: BBuf <xiaoyu.zhang@radixark.net>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-06 22:02:29 +08:00
Xiaoyu Zhang 7c6f9542c7 [Diffusion] Avoid GPU syncs in UniPC scheduler (#27440) 2026-06-06 22:01:41 +08:00
Xiaoyu ZhangandBBuf f57f8a8afd Optimize Gemma4 H200 MoE and extend attention (#26588)
Co-authored-by: BBuf <xiaoyu.zhang@radixark.net>
2026-06-06 14:14:25 +08:00
e513c13e2e Optimize ngram decode token table update (#24756)
Co-authored-by: Codex <codex@example.com>
Co-authored-by: BBuf <xiaoyu.zhang@radixark.net>
2026-06-06 14:13:45 +08:00
84e1108312 Optimize ngram decode id computation (#24757)
Co-authored-by: Codex <codex@example.com>
Co-authored-by: BBuf <xiaoyu.zhang@radixark.net>
2026-06-02 17:37:34 +08:00
Xiaoyu ZhangandBBuf 3ea1ba5b15 [GDN] Optimize prefill QKV split dispatch (#26206)
Co-authored-by: BBuf <xiaoyu.zhang@radixark.net>
2026-06-02 16:48:31 +08:00
Xiaoyu ZhangandBBuf 559581b383 [codex] Centralize Triton utility kernels (#26000)
Co-authored-by: BBuf <xiaoyu.zhang@radixark.net>
2026-06-02 16:47:45 +08:00
e279b0bf72 Optimize large add_constant tensors (#24755)
Co-authored-by: Codex <codex@example.com>
Co-authored-by: BBuf <xiaoyu.zhang@radixark.net>
2026-05-30 22:25:19 +08:00
Xiaoyu Zhang e60f799b40 Enable Kimi-K2.5 piecewise CUDA graph (#26382) 2026-05-27 22:51:33 -07:00
Xiaoyu ZhangandBBuf 121cc09405 [diffusion] Add CFG gating for denoising (#25848)
Co-authored-by: BBuf <bbuf@example.com>
2026-05-25 22:57:09 +08:00
Xiaoyu ZhangandBBuf 85f9522e36 [diffusion] Cache fp32 layernorm params (#25847)
Co-authored-by: BBuf <bbuf@example.com>
2026-05-25 22:56:39 +08:00
Xiaoyu Zhang 533ef41112 [Diffusion] Default NVFP4 backend to FlashInfer TRTLLM (#25523) 2026-05-25 18:14:06 +08:00
Xiaoyu Zhang 0b65588c18 [diffusion] Clean up VSA attention hot path (#25514) 2026-05-24 16:46:03 +08:00
Xiaoyu Zhang 75427c9ca4 Route concat MLA to JIT and remove unused downcast (#25843) 2026-05-23 14:30:43 +08:00
Xiaoyu Zhang ccbbae00ea [codex] Reland Wan2.2 ModelOpt CI checkpoints (#25857) 2026-05-20 22:15:25 +08:00
Xiaoyu Zhang af22390af7 [codex] Align diffusion skills with nightly Nvidia benchmarks (#25842) 2026-05-20 12:18:05 +08:00