Commit Graph
6055 Commits
Author SHA1 Message Date
Hanming Lu 297f9336c8 [Test] Enable mamba lazy extra buffer alloc edge case kl test in CI (#31991) 2026-07-21 20:45:43 -07:00
Xiaoyu ZhangandClaude Opus 4.8 246b3c3eaf [Kernel] Phase 3+4: move JIT infra + operator groups into sglang.kernels (RFC #29630) (#31666)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 11:28:27 +08:00
3217b7e3ce fix(hicache): support staged write-back for asymmetric MHA (#30981)
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
Co-authored-by: Zhangheng <hzh0425@apache.org>
2026-07-22 09:58:49 +08:00
Ma MingfeiandCopilot Autofix powered by AI 878d77929d [CPU] refactor rope kernels (#31897)
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-07-22 09:12:02 +08:00
ashwini rathi 14c0a31829 [Bugfix] Fix Cohere2MoeConfig import crash from huggingface_hub @strict (#31769) 2026-07-22 08:56:30 +08:00
DarkSharpnessandClaude Fable 5 8bb0d8d005 [JIT] Trait-driven per_token_group_quant: unify the quant kernel family (flat + masked) (#30924)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 08:46:52 +08:00
Liangsheng Yin 024639a372 [Perf] Stack dspark dense draft per-layer ctx KV projection into one GEMM (#31986) 2026-07-21 17:46:13 -07:00
Fardin Hoque b54adced46 [Quant] Support NVFP4_AWQ checkpoints in ModelOpt FP4 path (#31825) 2026-07-22 08:46:10 +08:00
03342e7732 Delete sgl-kernel AOT router GEMM and fused A GEMM (#30280)
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
Co-authored-by: root <root@sgl-b300-inference.datacrunch.io>
2026-07-22 08:44:59 +08:00
Liangsheng Yin 08cb081ed0 [Perf] Skip page-table columns past kv length in DSA draft-extend metadata kernel (#31981) 2026-07-21 16:20:14 -07:00
JD becf252e6c AutoWeightLoader support Sglang native models 1: demo (#28671) 2026-07-21 14:32:13 -07:00
Hanming Lu d03c8cee80 Negotiate PrefillDelayer only after KV-budget admission checks (#31835) 2026-07-21 12:07:30 -07:00
Hanming Lu 927979e127 Fix pad-row top-k masking with custom_routing_function under DP attention (#31838) 2026-07-21 11:49:17 -07:00
Ke Bao 604b1507d2 Add Inkling to nightly test (#31939) 2026-07-21 23:06:11 +08:00
Xiaoyu Zhang 075bd97952 [Benchmark] Remove obsolete auto-benchmark remnants (#31941) 2026-07-21 20:44:52 +08:00
e4eea7ce2f Optimize LongCat-Flash router GEMM with the HPC-Ops bf16xfp32 kernel (#30247)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Halcyon <56064364+VAthree@users.noreply.github.com>
2026-07-21 20:05:17 +08:00
YAMY 7c9257529f [Mamba] Support speculative decoding with extra_buffer_lazy (#30437) 2026-07-21 19:38:03 +08:00
df39a7b0b6 [CI][PD] Add NIXL disaggregation functional tests (#27894)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
2026-07-21 18:17:06 +08:00
Ke Bao 2cf2e7377f Fix dropped tool calls when a stream delta carries several (#31860) 2026-07-21 18:07:43 +08:00
dcd9014f15 [AMD][MXFP4] Reland "Online MXFP4 quantization 2/N - FP8 to MXFP4 requantization on AMD GPUs" (#28291)
Co-authored-by: Bowen Bao <bowenbao@amd.com>
Co-authored-by: HAI <hixiao@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-07-21 02:50:23 -07:00
Alison Shao 696e8f80d1 [CI] Graceful teardown in kl_mamba hicache tests to release pinned host pool (#31871) 2026-07-21 02:09:26 -07:00
wenxuewuhd c0ed009f5b [NPU] Fix LLaDA2 MoE OOM after the FRACTAL_NZ cast, re-enabling the NZ speedup (#31772) 2026-07-21 16:09:34 +08:00
Lianmin Zheng 429f6b6d15 Turn on breakable prefill cuda graph for dp attention by default (#31682) 2026-07-21 00:03:43 -07:00
Yuhao Yang 37a830b667 [Feature] Add DWDP (Distributed Weight Data Parallelism) for MoE prefill (#29778) 2026-07-20 23:59:54 -07:00
Michael c4c405a46b [AMD] batch 3: register newly-added JIT kernel benchmarks for jit-kernel-benchmark-test-amd (#31615) 2026-07-20 23:51:51 -07:00
iridiumine d6ef68881e [NPU] Adapt MiMo-V2.5-W8A8 (#29131) 2026-07-21 09:16:48 +08:00
Rahul Vijayaraghavan fa0ced195e [XPU] Enable breakable prefill CUDA graph on XPU (#30273) 2026-07-21 09:09:40 +08:00
paulzhang-tm d093c6a4bb [FullCG] Preserve attention LSE through the custom-op boundary (#31050) 2026-07-21 09:01:40 +08:00
Yang Liu 9462c303a5 Fix stop boundaries for grammar-constrained speculative decoding (#31738) 2026-07-20 17:55:27 -07:00
Mick 4682ded472 vlm: parallelize multimodal preprocessing with customized worker num (#31438) 2026-07-21 08:44:58 +08:00
Lianmin ZhengandJason Park e7e8aaa73c Overlap grammar (constrained decoding) with speculative decode verify (#31488)
Co-authored-by: Jason Park <jasonjk@fb.com>
2026-07-20 17:36:05 -07:00
zijiexia 8905cbd42f Fix MiniMax-M3 crash on ROCm by making its override fields resolvable (#31837) 2026-07-20 16:14:12 -07:00
Mohammad Miadh Angkad a82ead53bd Make Q contiguous before TRT-LLM MHA decode (#31667) 2026-07-20 15:21:00 -07:00
Liangsheng Yin ff6c755952 [Refactor] Unify logprob results into a single LogprobResult and rename chunk env vars (#31733) 2026-07-20 12:44:27 -07:00
Ke Bao 5ab3d90b81 Fix dropped Inkling reasoning at stream end (#31787) 2026-07-20 23:23:15 +08:00
fafa302e41 [XPU][NIGHTLY] Add 8 XPU nightly tests, enable 1-gpu suite (#30246)
Co-authored-by: arathi-hlab <arathi-hlab@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 16:15:50 +08:00
Baizhou Zhang 9668d9ea72 Support GPT-OSS zigzag CP with TRTLLM-MHA (#31732) 2026-07-20 00:48:04 -07:00
02236fa38c Add Inkling model support (#31681)
Co-authored-by: Chunan Zeng <zcnrex@gmail.com>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
Co-authored-by: Yanbin Jiang <jybsuper@gmail.com>
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: Qiaolin Yu <qiaolin.yu@radixark.ai>
Co-authored-by: Zhichen Zeng <zczeng@uw.edu>
Co-authored-by: Aurick Qiao <aurick@thinkingmachines.ai>
Co-authored-by: Joseph <jk@thinkingmachines.ai>
2026-07-19 22:57:37 -07:00
Xiaoyu Zhang 829e9ce9d5 Lower AutoRound quantization MMLU threshold (#31748) 2026-07-20 13:40:48 +08:00
Chenchen HongandXiaoyu Zhang 49b9c46f41 [dLLM] Reuse block KV/req slots in place across FDFO rounds (#27877)
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
2026-07-20 13:36:13 +08:00
Liangsheng Yin b15a83983c [Fix] Release hierarchical cache host pool on graceful shutdown (#31746) 2026-07-19 22:27:29 -07:00
Mick 2eed35d738 perf: avoid temporary VLM encoder gather padding (#31301) 2026-07-20 12:54:29 +08:00
gaopengff bab1dd0d12 [Intel XPU] Enable (biased) grouped topk for xpu (#31126) 2026-07-20 09:35:08 +08:00
Mick 6a25dd7b5f fix: warm up Kimi VLM vision encoder at startup (#31298) 2026-07-20 08:50:57 +08:00
Liangsheng Yin b3570a4531 [Refactor] Unify input logprob processing on a single chunked path (#31655) 2026-07-19 16:03:23 -07:00
Alison Shao 1a317839d7 ci: lower VL PP gsm8k threshold to 0.60 (0.65 has zero margin on H100) (#31702) 2026-07-19 15:41:12 -07:00
Mick d4801be447 fix: fix vlm cuda graph shape stability (#30868) 2026-07-19 22:35:51 +08:00
Yuhao Yang a03ca46a28 Fix KDA prefix caching under mamba extra_buffer and enable it for kimi_linear (#31474) 2026-07-19 20:09:03 +08:00
Baizhou Zhang 7a03d30149 Support MiMo V2.5 with zigzag context parallelism (#29972) 2026-07-19 02:36:40 -07:00
Feng Yao 942bf04ef9 [Scheduler] Add SGLANG_FORCE_COARSE_WAR_BARRIER opt-in for a whole-forward WAR barrier (#29353) 2026-07-19 00:28:49 -07:00