minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
15,363 Commits 2 Branches 0 Tags
d67aa0569727fb02d5b31644f45d361181dd6921
Commit Graph
10456 Commits
Author SHA1 Message Date
Mohammad Miadh AngkadandBrayden Zhong d67aa05697 Bump FlashInfer to 0.6.15 and revert regressions (#31502) 
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
 2026-07-17 00:50:12 -07:00
huangtingweiandZhangheng 44e3dd2713 [HiCache] Optimize HiCache host pool free-list release (#30658) 
Co-authored-by: Zhangheng <hzh0425@apache.org>
 2026-07-17 14:52:08 +08:00
Liangsheng Yin 6e3be088a9 [Spec] Allocate the verify tree-mask scratch on the target backend only (#31527) 2026-07-16 23:23:21 -07:00
Junlin Wu bbd2a3fe4a ✨ [llm][npu][quant] Add W4A4 MXFP4 quantization support for Qwen3 Dense on Ascend NPU (#23795) 2026-07-17 09:06:30 +03:00
Xiaoyu ZhangandClaude Opus 4.8 1ac1ffea0c [Kernel] Fill non-CUDA coverage: HIP (aiter/rocm-triton) + Ascend NPU backends (RFC #29630) (#31307) 
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-17 14:05:35 +08:00
Khoa PhamandCursor 96dd96c02b [DCP] Auto-disable tc_piecewise and breakable prefill CUDA graphs under DCP (#31532) 
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-07-16 22:17:50 -07:00
Khoa Phamandkpham-sgl d310fce85f [Fix] Update PR #25015 revert for fused topk=1 draft postprocess (#31519) 
Co-authored-by: kpham-sgl <kpham-sgl@users.noreply.github.com>
 2026-07-16 21:51:16 -07:00
Khoa Phamandkpham-sgl ead703815e [DCP] Enable decode context parallel for Kimi K2.5 NVFP4 (#31514) 
Co-authored-by: kpham-sgl <kpham-sgl@users.noreply.github.com>
 2026-07-16 21:50:42 -07:00
AuFlowandAuFlow bf417440e9 [Scheduler] Add SGLANG_MAX_NEW_TOKENS_LIMIT to cap per-request max_new_tokens (#22591) 
Co-authored-by: AuFlow <AuFlow@users.noreply.github.com>
 2026-07-16 21:34:10 -07:00
Liangsheng Yin 45824c69ca fa3: build the topk>1 verify replay page table on-device (#31381) 2026-07-16 21:21:01 -07:00
Liangsheng Yin 27ad9d11b1 [CI] Wait for GPU memory release before each test class setUpClass (#31509) 2026-07-16 21:06:29 -07:00
Peng Xingchen 12af7e6c34 [NPU] Fix DSA top-k seed buffer shape for MTP IndexShare (#31005) 2026-07-17 12:05:57 +08:00
Liangsheng Yin dfa6278370 [Fix] Publish idle scheduler metrics immediately when the running-reqs gauge is stale (#31517) 2026-07-16 21:04:10 -07:00
Liangsheng Yin 9910ef8167 flashmla: sync-free spec via device-side draft-extend (#31090) 2026-07-16 19:38:12 -07:00
Xiaoyu ZhangandClaude Opus 4.8 8432eafd3d [Kernel] Decouple KernelBackend from device + device-based CapabilityRequirement (RFC #29630) (#31292) 
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-17 10:35:34 +08:00
amote-i 68d324d697 [PD] Fix send_multipart blocking after prefill failure (#31306) 2026-07-17 10:33:29 +08:00
Sam Shleifer 0675d3033f Fix mamba track-boundary seqlen under overlap scheduler (#31369) 2026-07-17 10:22:07 +08:00
Polisetty V R K Jyothendra VarmaandRahul Vijayaraghavan 37f94cb7a0 [Intel GPU] DeepSeek V4 13/N: use sgl-kernel implementation of kernels in V2 Compressor to run on XPU (#28439) 
Signed-off-by: P V R K Jyothendra Varma <polisettyvarma@gmail.com>
Signed-off-by: P V R K Jyothendra Varma <polisetty.v.r.k.jyothendra.varma@intel.com>
Co-authored-by: Rahul Vijayaraghavan <rahul.vijayaraghavan@intel.com>
 2026-07-17 09:16:04 +08:00
ZiruandNiu Ziru 302c3b97d2 fix: Qwen3.5-35B-A3B-AWQ w2_weight KeyError and related params for CPU (#31174) 
Co-authored-by: Niu Ziru <niuziru@a4bf018d3341.jf.intel.com>
 2026-07-17 09:14:55 +08:00
jiayisunx 27ea5ed070 [XPU] Fix dtype mismatch in MRotaryEmbedding.forward_xpu by calling _… (#31441) 2026-07-17 09:14:25 +08:00
Kaixi dc0b3eb68f [2/3] [EAGLE] perf: Fuse TP vocab-parallel embedding (#30948) 2026-07-16 15:57:51 -07:00
Kaixi d539bf2cda [1/3] [EAGLE] perf: Fuse topk=1 draft postprocess (#30947) 2026-07-16 15:57:27 -07:00
Yoray Zack 77d23a796e [1/N] elastic-ep: Add runtime EP scale-up (#30164) 2026-07-16 15:53:44 -07:00
cctryandZhiqiang Xie d28e35b1a1 Fix num_running_reqs gauge on disagg prefill servers (#31495) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2026-07-16 15:43:06 -07:00
Bob ChenandZhiqiang Xie 7cd55c6818 [HiCache] Optimize L2 mem allocation when cache miss in L3 (#19320) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2026-07-16 15:00:53 -07:00
Liangsheng Yin 9a4d640244 [Perf] Cache uniform ragged-verify layout for DSpark verify-all compact (#31434) 2026-07-16 15:00:41 -07:00
Liangsheng Yin fc1e3797b7 [Spec] Split the capture width from num_tokens_per_req and gate replay on it (#31255) 2026-07-16 14:59:11 -07:00
Alison Shao 059ac7efe8 Don't fail server startup when psutil can't parse /proc/meminfo (GB300 ShadowCallStack) (#31098) 2026-07-16 14:57:34 -07:00
yuyu5333andzekai bff489284b [Feature] Support DeepSeek-V4 Wint4Abf16 and Win4Afp8. (#25763) 
Co-authored-by: zekai <zekai01@antgroup.com>
 2026-07-16 14:24:34 -07:00
Liangsheng Yin 44f4ea917c fa3: sync-free eagle spec via fixed-window draft-extend metadata (#31364) 2026-07-16 12:31:50 -07:00
Serge Panev 1f34911de7 [NemotronH] Load shared embed_tokens/lm_head in MTP draft weights (#30456) 2026-07-16 17:20:38 +00:00
Elizaveta Martirosianandronnie_zheng 3bc7c54111 [NPU] Updated baselines for GLM-Image (#31467) 
Co-authored-by: ronnie_zheng <zl19940307@163.com>
 2026-07-16 20:16:07 +03:00
pllimax 4ad418d2c3 Push test case scripts from test repo to main upstream community repository (#31114) 2026-07-16 21:24:01 +08:00
Mick 7d0fd5101d optimization: shard kimi dp image feature transport and misc optimizations (#31227) 2026-07-16 20:31:42 +08:00
jojoandhzh0425 b296e1a503 [hicache]: add mamba concurrency io transfer kernel (#30535) 
Co-authored-by: hzh0425 <hzh0425@apache.org>
 2026-07-16 18:13:02 +08:00
Wang, FangYuan e2d021d4ab [AMD] Support two batch overlap with MTP on DeepSeekV4 (#30238) 2026-07-16 02:34:04 -07:00
Brayden ZhongrootBrayden Zhong
 		e5f9804e26 Refining fused A GEMM dispatch (#31241) 
Co-authored-by: root <root@sgl-b300-inference.datacrunch.io>
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
 2026-07-16 16:57:01 +08:00
Yuan Luoandluoyuan.luo a798a2aeea [GDN/KDA] Fuse SM100 CuteDSL prefill state I/O into the chunk h kernel (#30169) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-07-16 16:40:33 +08:00
hjzhanghjzhanghnyls2002
 		db7e6807de [BugFix] Preserve tokenizer worker fanout when skip_tokenizer_init is enabled (#30682) 
Co-authored-by: hjzhang <zhanghjzzz@qq.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
 2026-07-16 01:29:46 -07:00
zhengyao 01b003255a [AMD]Reuse fused FP8 KV cache write on standard aiter prefill/decode (#26852) 2026-07-16 01:05:04 -07:00
Liangsheng Yin b55228cfdb [Spec] Consolidate the verify step into eagle_worker_common.run_eagle_verify (#31380) 2026-07-16 00:56:54 -07:00
王鹤男whn09Claude Opus 4.8
 		5cbea10e2f Fix LongCat n-gram embedding in PD-disaggregated scheduler loops (#31134) 
Co-authored-by: whn09 <whn09@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-16 15:42:51 +08:00
Ethan ZHUClaude Opus 4.7vuuihcZhiqiang Xie
 		095a817612 [Bugfix][HiCache] measure load-back duration with CUDA events (#26411) 
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: vuuihc <vuuihc@users.noreply.github.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2026-07-16 00:11:04 -07:00
kangwangamd f8eac995aa [diffusion] test: fix GLM-Image AR model-path to resolve local snapshot subfolder (#31313) 2026-07-15 23:57:34 -07:00
王鹤男whn09Claude Opus 4.8
 		0a64139c94 Fix --moe-a2a-backend silently ignored for LongCat-2.0 (moe_topk missing from gate) (#30975) 
Co-authored-by: whn09 <whn09@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-16 14:40:01 +08:00
Ryan Zzzandzhujunyu 5af65d8542 [DeepSeek-V4] Support BF16 Compress State for Online C128 (#29609) 
Co-authored-by: zhujunyu <zhujunyu.666@bytedance.com>
 2026-07-15 23:17:29 -07:00
Khoa PhamandClaude Fable 5 dc60f65661 chore: bump tokenspeed_mla to 0.1.8 (#31385) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-15 22:37:54 -07:00
weireweireandweireweire ac4fa6496c Skip no-op EAGLE sampling renormalization (#31294) 
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
 2026-07-15 22:37:10 -07:00
Liangsheng Yin b871a509e6 [Fix] Wire the detokenizer soft watchdog into the multi-http-worker event loop (#31392) 2026-07-15 21:55:46 -07:00
guzekai01 dee91c51cf perf(deepseek_v4): enable SGLANG_OPT_FP8_WO_A_GEMM on sm90 (Hopper) (#28983) 2026-07-16 11:16:32 +08:00