minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
12,554 Commits 2 Branches 0 Tags
642ac9c916d533de253f69ffe43799f5c42b4da5
Commit Graph
8353 Commits
Author SHA1 Message Date
chenxu214 642ac9c916 [NPU]pp support mla kv transfer (#23893) 2026-05-13 09:10:02 +08:00
Yuhao YangweireweireCheng Wan
 		d6d3d0f599 Optimize SWA memory preallocation for disaggregated decode (#24857) 
Co-authored-by: weireweire <weiliangl@nvidia.com>
Co-authored-by: Cheng Wan <chwan@rice.edu>
 2026-05-13 09:09:34 +08:00
Cheng WanandClaude Opus 4.7 4fb40bffac perf(nvfp4): free unused source scales after weight processing (#25107) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-12 16:53:50 -07:00
Emmanuel AcheampongandClaude Sonnet 4.6 b49d05fd0e feat: add Crusoe managed inference backend (#20475) 
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-05-12 16:23:23 -07:00
Khoa Pham 49ac447c94 [bench] Agentic support for bench_serving.py (#25016) 2026-05-12 16:00:45 -07:00
YAMY a4109e87ac Fix TRTLLM MHA routing for draft extend (#24856) 2026-05-12 15:48:32 -07:00
Brayden Zhongandb8zhong d5f3254ed1 [Dependency] Flashinfer 0.6.8post1 -> 0.6.11 (#24452) 
Co-authored-by: b8zhong <b8zhong@users.noreply.github.com>
 2026-05-12 14:38:32 -07:00
Lianmin Zheng 48fc26a814 Fix Eagle draft decode positions (#25015) 2026-05-12 14:04:58 -07:00
Kaixi Hou 486b54714b [NVIDIA] Deterministic inference backend order on Blackwell (#21450) 2026-05-12 13:35:50 -07:00
Polisetty V R K Jyothendra VarmaandBrayden Zhong 52d4c697bb Fix fused_moe import for non-NPU devices (#25076) 
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
 2026-05-12 23:05:51 +03:00
flutist-alibaba 00d8730d05 [Bug Fix] Fix broken sgemm_lora_a_graph_fwd due to invalid torch.mm() call (#24760) 2026-05-12 22:29:01 +03:00
Alison Shao 1419bcc39e [CI] Temporarily disable multimodal-gen test_update_weights_from_disk (flaky) (#25032) 2026-05-12 12:28:33 -07:00
Liangsheng YinandQiaolin Yu 538832c8b7 spec: STANDALONE skips hidden_states end-to-end (Optional schema + None-safe consumers) (#25037) 
Co-authored-by: Qiaolin Yu <qy254@cornell.edu>
 2026-05-12 12:27:21 -07:00
Krishnan PrashanthIshan DhananiClaude Opus 4.6ishandhanani
 		e86fb42736 feat: emit per-iteration forward pass metrics via ZMQ PUB (#22789) 
Co-authored-by: Ishan Dhanani <ishandhanani@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
 2026-05-12 10:28:17 -07:00
Ke Bao 6be1a45d99 Fix swa component host hit (#25085) 2026-05-12 21:03:39 +08:00
iridiumineandiridiumine 693f49793a [NPU] use causal_conv1d_update_v2 for performance (#24595) 
Co-authored-by: iridiumine <iridiumine@users.noreply.github.com>
 2026-05-12 17:04:02 +08:00
Praneth Paruchuri 1efe9e2164 [Bug Fix] Reject incompatible combination of --disable-cuda-graph-padding and --enable-torch-compile (#23903) 2026-05-12 16:18:36 +08:00
lw9527 8d27ce7371 Optimize uvicorn startup command (#25041) 2026-05-12 00:28:13 -07:00
Yihao Wang b35fd5f104 [fix] skip legacy minicpmv conv template for MiniCPM-V 4.6 (#24998) 2026-05-12 15:27:28 +08:00
Yuwei AnandClaude Opus 4.7 7582237f8e [Tiny Fix] Disable BCG when inner layer_model unresolved (#25021) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-12 14:51:27 +08:00
zhangxiaolei ca3bc05fea Deepseek-v4-Pro share expert tp1 (#24949) 2026-05-11 23:19:35 -07:00
Liangsheng Yin a72d3aef17 [Spec] Multi-layer mamba scatter cleanup; fix positional call bug (#25030) 2026-05-11 22:42:30 -07:00
Liangsheng Yin 1f985c50db [Spec] Rename accepted_indices -> accept_indices; drop _token_id suffix per Rule 5 (#25038) 2026-05-11 22:29:02 -07:00
Ke Bao d7f4761a48 [PD] Refactor hybrid state transfer (#24932) 2026-05-12 13:16:54 +08:00
Zhangheng 91907b7b93 [UnifiedTree]: Fix Unified HiCache tombstone lock release replay (#24972) 2026-05-12 13:16:06 +08:00
Liangsheng Yin 4ad63ad02f [Spec] Rename accepted_drafts -> correct_drafts for unambiguous naming (#24081) 2026-05-11 22:12:40 -07:00
tangcy98andzhangzhang 6bfb365c07 [PD] Rate limit prefill inflight polling warnings (#24967) 
Signed-off-by: zhangzhang <tangchenyu@xiaohongshu.com>
Co-authored-by: zhangzhang <tangchenyu@xiaohongshu.com>
 2026-05-12 12:50:57 +08:00
Ming Yang 6bb79c155b [Linear Attn] Add CUSTOM enum and plugin extensibility for kernel backends (#24937) 2026-05-11 21:46:53 -07:00
Qiaolin Yu cfc41d5b15 Fix kimi k2.5 mla eagle + dp attention (#25033) 2026-05-11 20:58:00 -07:00
Yihao Wang 0f3932c055 [Fix] Qwen3-ASR config: set thinker_config before super().__init__ (#24187) 2026-05-12 11:52:15 +08:00
Liangsheng Yin f526e3fa27 [Spec] Mamba scatter cleanup; fix multi-layer positional bug; dflash naming (#25029) 2026-05-11 20:36:50 -07:00
Jianhong Zhang 10375a1037 [NIXL][XPU] Fix uint64 overflow for mismatched P/D TP sizes (e.g. prefill_tp=1, decode_tp=2) (#24648) 2026-05-12 11:02:24 +08:00
R0CKSTAR 0a37d24e62 [diffusion] hardware: support sage attention backend on MUSA (attn backend, 21/N) (#24752) 
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
 2026-05-11 19:50:52 -07:00
shuwenn 5495026a3b [HiCache] feat: default storage prefetch timeout (#23309) 2026-05-11 18:49:35 -07:00
Doğaç Eldenk 186eb42459 Feat: Support SWA (Sliding Window Attention) for EAGLE-3 drafter (#24664) 2026-05-11 18:40:10 -07:00
Doğaç Eldenkandhnyls2002 a75b79e03b Feat: Support newer EAGLE-3 drafters (#24663) 
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
 2026-05-11 18:17:20 -07:00
Liangsheng Yin f3a8189e20 [Spec] Internal rename per N2 v2 naming rule (#25014) 2026-05-11 18:16:26 -07:00
popsiclexu bfc2eda42d [MUSA] Use MUSA-optimized operators in piecewise CUDA graph (#23633) 
Signed-off-by: popsiclexu <zhenxuexu@gmail.com>
 2026-05-11 17:55:27 -07:00
R0CKSTAR 74d70af09a [Apple Silicon] Add Metal kernel support in sgl-kernel (#23449) 
Signed-off-by: Xiaodong Ye <yeahdongcn@gmail.com>
 2026-05-11 17:54:27 -07:00
Ethan (Yusheng) SuCursorClaude Opus 4.7
 		e9dea79755 (3/n - prefill optimize)[LoRA][MoE] Optimize virtual experts: remove CPU-GPU sync & multi-block CUDA JIT histogram (#24262) 
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-11 16:36:57 -07:00
Liangsheng Yin 6c3541a914 spec: route idle hidden_size via EagleDraft{,Extend}Input classmethods (#25013) 2026-05-11 15:59:51 -07:00
Liangsheng Yin ce1736fcc6 [Spec] Remove dead kernel params; fix stale comment in trtllm_mla (#25010) 2026-05-11 14:50:46 -07:00
Bruce Changlong Xu d3edfb579b [connector] Add Azure Blob Storage connector (az:// and *.blob.core.windows.net URLs) (#23995) 2026-05-11 14:20:22 -07:00
Jia Guo da0eeb82f2 perf: add --prefill-only-disable-kv-cache to skip KV pool allocation (#23675) 2026-05-11 13:10:24 -07:00
Liangsheng Yin c7e53e68a2 [Spec] Fix ngram metric off-by-1 in num_accepted_drafts_per_req_cpu (#24965) 2026-05-11 12:25:18 -07:00
Yan Ru Pei 4b6f776889 feat(kv-events): publish SWA radix cache events (#24718) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-05-11 10:31:45 -07:00
TianheandClaude Sonnet 4.6 95985f983d feat(trace): support SGLANG_TRACE_LEVEL env var for startup trace level (#24716) 
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-05-11 10:31:03 -07:00
Bingxu ChenandCursor Agent aeb8fefc25 [AMD] Pin cache-dit==1.3.0 in rocm.Dockerfile + AMD CI install script (#24924) 
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
 2026-05-11 22:32:13 +08:00
Mick 6d30b571b2 [diffusion] fix: fix single-step flow-match timesteps (#24708) 2026-05-11 18:09:24 +08:00
iridiumineandiridiumine df441b8fea [NPU] Support shared expert dual stream optimization (#23827) 
Co-authored-by: iridiumine <iridiumine@users.noreply.github.com>
 2026-05-11 15:50:15 +08:00