minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
16,624 Commits 3 Branches 0 Tags
5afdb1caea27eaebfa04a7490ccc4080fa89bed3
Commit Graph
4061 Commits
Author SHA1 Message Date
hanwlax 8720a72814 fix(scheduler): track max prefill batch size over recent real admissions (#34284) 2026-08-15 10:48:03 +08:00
weireweireandweireweire 6eb941a34c [Spec] Support MegaMoE for DSpark under dp attention (#34844) 
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
 2026-08-14 17:20:01 -07:00
gilfordtingandJames Liu 6f005e4da1 [Fix] Require JSON booleans for response_format json_schema.strict (#34777) 
Co-authored-by: James Liu <jamesl@modal.com>
 2026-08-14 16:46:27 -07:00
Baizhou Zhang a1844709f1 [CI] Trim Qwen3.5 FP8 GB300 performance batches (#34882) 2026-08-14 16:45:11 -07:00
gilfordtingandJames Liu 3f64f14360 [Fix] Work around xgrammar 0.2.1 negative integer minimum in Kimi-K3 structural tags (#34778) 
Co-authored-by: James Liu <jamesl@modal.com>
 2026-08-14 16:41:06 -07:00
Nan Jiang be804c1b83 [RL] Expose top-p-only sampling masks (#33593) 2026-08-14 16:40:27 -07:00
Ziang Li a9654eacc1 fix(dsa): use FlashInfer fused top-k for packed PAGED rows (#33006) 2026-08-14 15:22:18 -07:00
Mohammad Miadh Angkad d7207be156 Fix startup weight load after TorchAO removal (#34869) 2026-08-14 13:18:53 -07:00
DarkraiHLhjzhangYAMY
 		41cd5a7189 [Fix] Fix Qwen3.5 MTP startup with HiCache (#34560) 
Co-authored-by: hjzhang <76768149+1e4ves@users.noreply.github.com>
Co-authored-by: YAMY <74099316+YAMY1234@users.noreply.github.com>
 2026-08-14 10:55:41 -07:00
Ke Bao 7562e741e2 Retain SWA down to the last state checkpoint (#34729) 2026-08-15 01:12:38 +08:00
Ke Bao c20aceeb88 Fix mamba checkpoint depth under dcp (#34808) 2026-08-15 00:34:00 +08:00
Yuan Luoandluoyuan.luo b95a746948 [MoE] Fuse swiglu moe up gemm epilogue (#32944) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-08-14 22:11:34 +08:00
Yuang Chen晟海liusy58Xiaoyu Zhang
 		1a178f7c7c [EPD] Batch embedding cache host-device range copies (#31574) 
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
Co-authored-by: liusy58 <liusy58@linux.alibaba.com>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
 2026-08-14 22:01:12 +08:00
Brayden ZhongandBrayden Zhong 5e65dd01a7 Remove the torchao integration (--torchao-config) (#34304) 
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
 2026-08-14 21:49:11 +08:00
kk 65d62109dd [AMD] Fix Triton 3.7 gfx950 extend-attention spills (#34741) 2026-08-14 00:05:42 -07:00
datdo-msftandXiaoyu Zhang 827552bc1d Fix eager AMX backend probe imports (#34496) 
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
 2026-08-14 14:44:14 +08:00
zijiecjacky.chengYC Yen-Ching Tseng
 		ba1d980b35 [AMD] Accelerate AITER unified-attention decode with scaled FP8 Q (#31856) 
Co-authored-by: jacky.cheng <yichiche@amd.com>
Co-authored-by: YC Yen-Ching Tseng <yctseng@amd.com>
 2026-08-13 23:42:25 -07:00
Mick 46d84f4b48 feat(cli): add extensible serve backend plugins (#34753) 2026-08-14 13:57:59 +08:00
DarkSharpnessClaudeBBuf
 		b784726863 [kernel] Content-addressed JIT build cache, generated from our own ninja (#34274) 
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: BBuf <1182563586@qq.com>
 2026-08-14 13:41:04 +08:00
jacky.chengandsogalin_codegen 240a12b302 [AMD] Fuse shared_expert_gate GEMV into the MoE append kernel (HIP/aiter) (#28666) 
Co-authored-by: sogalin_codegen <39478626+sogalin@users.noreply.github.com>
 2026-08-13 21:32:27 -07:00
Baizhou Zhang 0a6bbbe128 [Fix] Restore layer-level DSV4 RoPE policy (#34788) 2026-08-13 21:26:45 -07:00
Xinyuan Tongandhnyls2002 85cdf1178d [CI] Prune redundant CPU test overhead (#34309) 
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
 2026-08-13 19:51:17 -07:00
ashwini rathiandarathi-hlab 456c5551cc [XPU][test] Add cache_salt=None to _make_req in test_lmcache_radix_cache.py (#34728) 
Co-authored-by: arathi-hlab <arathi.hlab@users.noreply.github.com>
 2026-08-14 10:27:15 +08:00
SII-yangdianandSII-yangdian f2b2b567aa perf(jit_kernel/deepseek_v4): optimize paged_mqa_metadata (#25855) 
Co-authored-by: SII-yangdian <yangdian@sii.edu.cn>
 2026-08-13 19:00:22 -07:00
Ziang Li 9d34c2809f [FlashInfer v0.6.16] Support FlashInfer CuTe DSL NVFP4 MoE quantization (#28354) 2026-08-13 17:33:46 -07:00
weireweireandweireweire 54c44fef73 [Perf] Skip trivial DSV4 nonpaged indexer logits (#33857) 
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
 2026-08-13 16:21:30 -07:00
Khoa PhamandClaude Opus 5 f3beb2c529 [CI] Disable the prefill CUDA graph on the P worker of test_kimi_linear_pd_dcp4 (#34779) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-13 15:26:21 -07:00
Khoa PhamandClaude Opus 5 81fe452810 [DCP] Share one pack kernel between both a2a backends (#34651) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-13 15:06:02 -07:00
jthomson04 903439044a perf(kv-events): coalesce cache events (#31479) 2026-08-13 13:36:53 -07:00
Liangsheng Yin 8554d9a5bc [Fix] Carry the backend on Kimi-K3 deferred preprocessing configs (#34766) 2026-08-13 13:30:33 -07:00
Mohammad Miadh Angkad 96db53ec70 [CI] Fix test_resolution_is_reproducible after cuda_ipc became opt-in (#34746) 2026-08-13 12:47:03 -07:00
Han YuWenhui ZhuAlex Nails
 		6b94d39f13 [Model Loading] Overlap checkpoint staging with CUDA graph capture during startup (#32017) 
Co-authored-by: Wenhui Zhu <wzhu59@asu.edu>
Co-authored-by: Alex Nails <alex.nails@radixark.ai>
 2026-08-13 12:26:25 -07:00
Shangming Cai 0772e79ee7 [CI][PD] Pin nccl rendezvous port per side to fix flaky disaggregation tests (#34755) 2026-08-14 01:44:20 +08:00
Xiaoyu Zhang ebca0bbde4 [Diffusion][ERNIE] Fuse QKNorm with full-width RoPE (#34620) 2026-08-13 23:23:21 +08:00
Xiaoyu Zhang 74c0322342 [Diffusion][FLUX.2] Fuse eager AdaLN and packed SwiGLU (#34616) 2026-08-13 19:55:53 +08:00
Xiaoyu Zhang 3c1791a7df [Diffusion][HunyuanVideo] Fuse eager QKV packing and high-quality QKNorm (#34617) 2026-08-13 19:54:00 +08:00
Baizhou Zhang 9c0d4cba3f [CI] Split Kimi K2.5 performance batches by config (#34669) 2026-08-13 03:20:50 -07:00
Rainchar9119 dbebc1deb4 [Perf] Occupancy tuning for DSA indexer fp8-quant Q kernel (#32755) 
Signed-off-by: Rainchar9119 <1134601163@qq.com>
 2026-08-13 01:15:27 -07:00
Weiweisys-lpot-valWeiwei ZhangMa Mingfei
 		fad376d3ee [CPU][QUANT] add amx cpu support for auto-round (#29593) 
Signed-off-by: WeiweiZhang1 <weiwei1.zhang@intel.com>
Signed-off-by: sys-lpot-val <sys_lpot_val@intel.com>
Co-authored-by: sys-lpot-val <sys_lpot_val@intel.com>
Co-authored-by: Weiwei Zhang <WeiweiZhang1@users.noreply.github.com>
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
 2026-08-13 15:50:59 +08:00
AMD-yanfeiwangandZhangheng a34f81251f fix(hicache/umbp): support DeepSeek-V4 hybrid HostPoolGroup (multi-po… (#30762) 
Co-authored-by: Zhangheng <hzh0425@apache.org>
 2026-08-13 15:40:07 +08:00
Ke Bao 5a5c3d309b Drop the mmlu case from the unified radix cache kit (#34667) 2026-08-13 15:16:05 +08:00
Xinguo Zhu 3f6ef01322 Optimize MiniMax-M2.7 on CPU (#31956) 2026-08-13 15:04:16 +08:00
Gabriel Wu 889c2f31aa feat(attention): add architecture-owned SM12x FA4 kernels (#32991) 2026-08-12 23:58:42 -07:00
Mick e8c7dddfa0 [VLM] add content-addressed preprocessing cache infrastructure (#34398) 2026-08-13 14:09:07 +08:00
elvischenv 035c622a14 Reenable breakable CUDA graph for NemotronH (#34538) 2026-08-13 13:58:10 +08:00
Mick 69bf601e3c fix: restore VLM nightly regression coverage (#34662) 2026-08-12 21:47:16 -07:00
Liangsheng Yin 8761b971f1 [CI] Fix nightly test failures (#34637) 2026-08-12 20:53:44 -07:00
+76
vstone-wClownBinhuangzhenyuclownbadmerLiangsheng YinYAMYShangming CaiMickBrayden ZhongBrayden ZhongJimmy ShongZijie XiaThomas WangsiyuYuang ChenYuang Chen黄孝君Xinyuan TongXiaoyu ZhangCursorClaude Fable 5TobyMintTobyMintCheng WanTan TrinhLifan ShenJustin TongQiaolin YuAMD-yanfeiwangQIN2DIMZhiyao JiangBrayden ZhongDevashishLal-CBDevashish LalAlex NailsMohammad Miadh AngkadBaizhou ZhangMichael GschwindweireweireweireweireKhoa PhammilesialelvischenvcctryJae B.MichaelforrestlEchOTanmay patilybyangHsiu-Chun, HungHungHaiShawBingxu ChenYC Yen-Ching TsengCherry_mingEven Zhousglang-npu-botZhiqiang XieTingwei HuangKaixigithub-actions[bot]Faradawn YangRyan StewartgjsheuJinyan YiKe BaohuangtingweiHanming LuJeremy ZhangDmitrii SergeevHao ZhangzhisbugDouglas Yanggongwei1027ilyasher-harmonicHanming Lusglang-botsglang-botJimmy Shonghnyls2002Meng, HengyuShu WangYanbin Jiangzijiec
 		bca8ed4afc [minimax m3][npu]Adaptation of Minimax M3(w8a8) for NPU platforms [1/2] (#32941) 
Signed-off-by: Devashish Lal <devcode@fb.com>
Signed-off-by: Alexandre Milesi <milesial@users.noreply.github.com>
Signed-off-by: Faradawn Yang <73060648+faradawn@users.noreply.github.com>
Signed-off-by: Ryan Stewart <rystewart@nvidia.com>
Co-authored-by: ClownBin <chaobin1993@126.com>
Co-authored-by: huangzhenyu <q_m_p@qq.com>
Co-authored-by: clown <17490516+ClownBin@users.noreply.github.com>
Co-authored-by: badmer <374330057@qq.com.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
Co-authored-by: YAMY <74099316+YAMY1234@users.noreply.github.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
Co-authored-by: Brayden Zhong <brayden.zhong@radixark.ai>
Co-authored-by: Jimmy Shong <jimmysh341@gmail.com>
Co-authored-by: Zijie Xia <zijie.xia@radixark.ai>
Co-authored-by: Thomas Wang <thomawan@amd.com>
Co-authored-by: siyu <liusy58@linux.alibaba.com>
Co-authored-by: Yuang Chen <cya539102@antgroup.com>
Co-authored-by: Yuang Chen <1131578721@qq.com>
Co-authored-by: 黄孝君 <dingfangsu23@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: TobyMint <130973409+TobyMint@users.noreply.github.com>
Co-authored-by: TobyMint <tobymint@users.noreply.github.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: Tan Trinh <84185999+tanth47@users.noreply.github.com>
Co-authored-by: Lifan Shen <draftbks@gmail.com>
Co-authored-by: Justin Tong <justintong0323@outlook.com>
Co-authored-by: Qiaolin Yu <liin1211@outlook.com>
Co-authored-by: AMD-yanfeiwang <yanfei.wang@amd.com>
Co-authored-by: QIN2DIM <62018067+QIN2DIM@users.noreply.github.com>
Co-authored-by: Zhiyao Jiang <jessicajiang324@gmail.com>
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
Co-authored-by: DevashishLal-CB <devashish@rivosinc.com>
Co-authored-by: Devashish Lal <devcode@fb.com>
Co-authored-by: Alex Nails <alex.nails@radixark.ai>
Co-authored-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
Co-authored-by: Michael Gschwind <mkgschwind+private@gmail.com>
Co-authored-by: weireweire <weiliangl@nvidia.com>
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
Co-authored-by: Khoa Pham <khoa.pham@radixark.ai>
Co-authored-by: milesial <milesial@users.noreply.github.com>
Co-authored-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Co-authored-by: cctry <csycfl@gmail.com>
Co-authored-by: Jae B. <jlee5814@gmail.com>
Co-authored-by: Michael <13900043+michaelzhang-ai@users.noreply.github.com>
Co-authored-by: forrestl <16055533+forrestl111@users.noreply.github.com>
Co-authored-by: EchO <117733745+CyberSecurityErial@users.noreply.github.com>
Co-authored-by: Tanmay patil <tanmaypatil3151@gmail.com>
Co-authored-by: ybyang <10629930+whybeyoung@users.noreply.github.com>
Co-authored-by: Hsiu-Chun, Hung <160560375+Emmanuel0612@users.noreply.github.com>
Co-authored-by: Hung <Emmanuel0612@users.noreply.github.com>
Co-authored-by: HaiShaw <hixiao@gmail.com>
Co-authored-by: Bingxu Chen <bingxche@amd.com>
Co-authored-by: YC Yen-Ching Tseng <yctseng@amd.com>
Co-authored-by: Cherry_ming <136634645@qq.com>
Co-authored-by: Even Zhou <even.y.zhou@outlook.com>
Co-authored-by: sglang-npu-bot <sglangnpu@163.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
Co-authored-by: Tingwei Huang <huangtingwei9988@gmail.com>
Co-authored-by: Kaixi <kaiximatteoc@nvidia.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Faradawn Yang <73060648+faradawn@users.noreply.github.com>
Co-authored-by: Ryan Stewart <rystewart@nvidia.com>
Co-authored-by: gjsheu <gjsheu@163.com>
Co-authored-by: Jinyan Yi <yjy20010615@gmail.com>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
Co-authored-by: huangtingwei <141888744+huangtingwei9988@users.noreply.github.com>
Co-authored-by: Hanming Lu <hanminglu@meta.com>
Co-authored-by: Jeremy Zhang <jeremy.zhang866@gmail.com>
Co-authored-by: Dmitrii Sergeev <dmi.sergeev@gmail.com>
Co-authored-by: Hao Zhang <zhisbug@users.noreply.github.com>
Co-authored-by: zhisbug <1654062+zhisbug@users.noreply.github.com>
Co-authored-by: Douglas Yang <dyang@college.harvard.edu>
Co-authored-by: gongwei1027 <gongwei833x@gmail.com>
Co-authored-by: ilyasher-harmonic <ilya.sherstyuk@harmonic.fun>
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: sglang-bot <sglangbot@gmail.com>
Co-authored-by: sglang-bot <232288953+sglang-bot@users.noreply.github.com>
Co-authored-by: Jimmy Shong <69131491+Jiminator@users.noreply.github.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: Meng, Hengyu <hengyu.meng@intel.com>
Co-authored-by: Shu Wang <shuw@nvidia.com>
Co-authored-by: Yanbin Jiang <jybsuper@gmail.com>
Co-authored-by: zijiec <zijie.chen@amd.com>
 2026-08-13 11:23:14 +08:00
Ke Bao 5efe9b104d Record both architectures in the bit-exact guard docstrings (#34656) 2026-08-13 10:55:35 +08:00
Liangsheng Yin 50cc1aa241 [CI] Route mmlu and GB300 MMMU-Pro evals through sgl-eval (#34477) 2026-08-12 19:33:21 -07:00