minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
12,572 Commits 2 Branches 0 Tags
c32f2dc1ac0c454e8701ca8775ac48e1114dc19c
Commit Graph
8365 Commits
Author SHA1 Message Date
Lukas HumbelandClaude Sonnet 4.6 c32f2dc1ac fix(nixl): close file descriptors after each FILE transfer (#24671) 
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-05-13 00:34:52 -07:00
Matti Varjokallio 839f7f2696 [AMD] Add _skip_rope_for_aiter_fused_mla method and check to avoid double rotating with gfx950 and Aiter backend (#24148) 2026-05-13 00:27:09 -07:00
Baizhou Zhang 51a9403104 Update flashinfer to 0.6.11.post1 (#25129) 2026-05-13 00:12:19 -07:00
Cheng WanandClaude Opus 4.7 f2a90094c9 bench: fix wrong flag names in bench_one_batch{,_server} docstrings (#25143) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-12 23:46:47 -07:00
billishyahaoandDuyi-Wang 66a9234246 [AMD] support fp8 blockwise quantization combine for mori ep (#24879) 
Co-authored-by: Duyi-Wang <duyi.wang@amd.com>
 2026-05-12 23:24:40 -07:00
Khoa Pham c665edec6e [env] Make max KV chunk capacity configurable via SGLANG_MAX_KV_CHUNK_CAPACITY (#25120) 2026-05-12 22:37:45 -07:00
Liangsheng Yin adae4042a7 spec: defer verify() idle hidden_size to worker fixup (#25109) 2026-05-12 22:13:52 -07:00
RulinJuiceandRulinJuice 3f048c80b8 Reject repetition_penalty=0 in SamplingParams.verify() (#24874) 
Co-authored-by: RulinJuice <265952454+RulinJuice@users.noreply.github.com>
 2026-05-12 21:25:23 -07:00
Taegeon UmTaegeon UmZhangheng
 		4e35c30cbe [Bugfix, NSA HiCache] Fix missing override_kv_cache_dim in attach_hybrid_nsa_pool_to_hiradix_cache (#25022) 
Co-authored-by: Taegeon Um <taegeon.um@samsung.com>
Co-authored-by: Zhangheng <hzh0425@apache.org>
 2026-05-13 11:45:08 +08:00
shuwenn 5ed9a494d0 [UnifiedTree] fix: allow partial match on evicted+backuped nodes (#24943) 2026-05-13 11:43:12 +08:00
ZhanghengandJINZ 6140e45ef3 [UnifiedTree]: Fix the leaf determination logic in _cascade_evict. (#25068) 
Co-authored-by: JINZ <1023553676@qq.com>
 2026-05-13 10:57:32 +08:00
sky 409d350fb6 Bugfix: fix symm not enabled due to incorrect registration of comm (#19329) 
Signed-off-by: wangfakang <fakangwang@gmail.com>
 2026-05-13 10:01:04 +08:00
chenxu214 642ac9c916 [NPU]pp support mla kv transfer (#23893) 2026-05-13 09:10:02 +08:00
Yuhao YangweireweireCheng Wan
 		d6d3d0f599 Optimize SWA memory preallocation for disaggregated decode (#24857) 
Co-authored-by: weireweire <weiliangl@nvidia.com>
Co-authored-by: Cheng Wan <chwan@rice.edu>
 2026-05-13 09:09:34 +08:00
Cheng WanandClaude Opus 4.7 4fb40bffac perf(nvfp4): free unused source scales after weight processing (#25107) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-12 16:53:50 -07:00
Emmanuel AcheampongandClaude Sonnet 4.6 b49d05fd0e feat: add Crusoe managed inference backend (#20475) 
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-05-12 16:23:23 -07:00
Khoa Pham 49ac447c94 [bench] Agentic support for bench_serving.py (#25016) 2026-05-12 16:00:45 -07:00
YAMY a4109e87ac Fix TRTLLM MHA routing for draft extend (#24856) 2026-05-12 15:48:32 -07:00
Brayden Zhongandb8zhong d5f3254ed1 [Dependency] Flashinfer 0.6.8post1 -> 0.6.11 (#24452) 
Co-authored-by: b8zhong <b8zhong@users.noreply.github.com>
 2026-05-12 14:38:32 -07:00
Lianmin Zheng 48fc26a814 Fix Eagle draft decode positions (#25015) 2026-05-12 14:04:58 -07:00
Kaixi Hou 486b54714b [NVIDIA] Deterministic inference backend order on Blackwell (#21450) 2026-05-12 13:35:50 -07:00
Polisetty V R K Jyothendra VarmaandBrayden Zhong 52d4c697bb Fix fused_moe import for non-NPU devices (#25076) 
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
 2026-05-12 23:05:51 +03:00
flutist-alibaba 00d8730d05 [Bug Fix] Fix broken sgemm_lora_a_graph_fwd due to invalid torch.mm() call (#24760) 2026-05-12 22:29:01 +03:00
Alison Shao 1419bcc39e [CI] Temporarily disable multimodal-gen test_update_weights_from_disk (flaky) (#25032) 2026-05-12 12:28:33 -07:00
Liangsheng YinandQiaolin Yu 538832c8b7 spec: STANDALONE skips hidden_states end-to-end (Optional schema + None-safe consumers) (#25037) 
Co-authored-by: Qiaolin Yu <qy254@cornell.edu>
 2026-05-12 12:27:21 -07:00
Krishnan PrashanthIshan DhananiClaude Opus 4.6ishandhanani
 		e86fb42736 feat: emit per-iteration forward pass metrics via ZMQ PUB (#22789) 
Co-authored-by: Ishan Dhanani <ishandhanani@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
 2026-05-12 10:28:17 -07:00
Ke Bao 6be1a45d99 Fix swa component host hit (#25085) 2026-05-12 21:03:39 +08:00
iridiumineandiridiumine 693f49793a [NPU] use causal_conv1d_update_v2 for performance (#24595) 
Co-authored-by: iridiumine <iridiumine@users.noreply.github.com>
 2026-05-12 17:04:02 +08:00
Praneth Paruchuri 1efe9e2164 [Bug Fix] Reject incompatible combination of --disable-cuda-graph-padding and --enable-torch-compile (#23903) 2026-05-12 16:18:36 +08:00
lw9527 8d27ce7371 Optimize uvicorn startup command (#25041) 2026-05-12 00:28:13 -07:00
Yihao Wang b35fd5f104 [fix] skip legacy minicpmv conv template for MiniCPM-V 4.6 (#24998) 2026-05-12 15:27:28 +08:00
Yuwei AnandClaude Opus 4.7 7582237f8e [Tiny Fix] Disable BCG when inner layer_model unresolved (#25021) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-12 14:51:27 +08:00
zhangxiaolei ca3bc05fea Deepseek-v4-Pro share expert tp1 (#24949) 2026-05-11 23:19:35 -07:00
Liangsheng Yin a72d3aef17 [Spec] Multi-layer mamba scatter cleanup; fix positional call bug (#25030) 2026-05-11 22:42:30 -07:00
Liangsheng Yin 1f985c50db [Spec] Rename accepted_indices -> accept_indices; drop _token_id suffix per Rule 5 (#25038) 2026-05-11 22:29:02 -07:00
Ke Bao d7f4761a48 [PD] Refactor hybrid state transfer (#24932) 2026-05-12 13:16:54 +08:00
Zhangheng 91907b7b93 [UnifiedTree]: Fix Unified HiCache tombstone lock release replay (#24972) 2026-05-12 13:16:06 +08:00
Liangsheng Yin 4ad63ad02f [Spec] Rename accepted_drafts -> correct_drafts for unambiguous naming (#24081) 2026-05-11 22:12:40 -07:00
tangcy98andzhangzhang 6bfb365c07 [PD] Rate limit prefill inflight polling warnings (#24967) 
Signed-off-by: zhangzhang <tangchenyu@xiaohongshu.com>
Co-authored-by: zhangzhang <tangchenyu@xiaohongshu.com>
 2026-05-12 12:50:57 +08:00
Ming Yang 6bb79c155b [Linear Attn] Add CUSTOM enum and plugin extensibility for kernel backends (#24937) 2026-05-11 21:46:53 -07:00
Qiaolin Yu cfc41d5b15 Fix kimi k2.5 mla eagle + dp attention (#25033) 2026-05-11 20:58:00 -07:00
Yihao Wang 0f3932c055 [Fix] Qwen3-ASR config: set thinker_config before super().__init__ (#24187) 2026-05-12 11:52:15 +08:00
Liangsheng Yin f526e3fa27 [Spec] Mamba scatter cleanup; fix multi-layer positional bug; dflash naming (#25029) 2026-05-11 20:36:50 -07:00
Jianhong Zhang 10375a1037 [NIXL][XPU] Fix uint64 overflow for mismatched P/D TP sizes (e.g. prefill_tp=1, decode_tp=2) (#24648) 2026-05-12 11:02:24 +08:00
R0CKSTAR 0a37d24e62 [diffusion] hardware: support sage attention backend on MUSA (attn backend, 21/N) (#24752) 
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
 2026-05-11 19:50:52 -07:00
shuwenn 5495026a3b [HiCache] feat: default storage prefetch timeout (#23309) 2026-05-11 18:49:35 -07:00
Doğaç Eldenk 186eb42459 Feat: Support SWA (Sliding Window Attention) for EAGLE-3 drafter (#24664) 2026-05-11 18:40:10 -07:00
Doğaç Eldenkandhnyls2002 a75b79e03b Feat: Support newer EAGLE-3 drafters (#24663) 
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
 2026-05-11 18:17:20 -07:00
Liangsheng Yin f3a8189e20 [Spec] Internal rename per N2 v2 naming rule (#25014) 2026-05-11 18:16:26 -07:00
popsiclexu bfc2eda42d [MUSA] Use MUSA-optimized operators in piecewise CUDA graph (#23633) 
Signed-off-by: popsiclexu <zhenxuexu@gmail.com>
 2026-05-11 17:55:27 -07:00