minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
18,477 Commits 2 Branches 0 Tags
81421b91e9ad02c526bb3025f66f42e2a7c52059
Commit Graph
18477 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Cheng Wan 81421b91e9 One read path for every parallel name (#40069) 2026-09-18 17:43:56 -07:00
Cheng Wan afe71f4b9e Read process groups through the runtime context (#40068) 2026-09-18 17:40:32 -07:00
Yonghao ZhuangyhzhuangLianmin ZhengYonghao ZhuangCheng Wan
 		5931fd60ee Support unified memory page-envelope transfers in PD (#39477) 
Co-authored-by: yhzhuang <yhzhuang@fb.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: Yonghao Zhuang <yhzhuang@users.noreply.github.com>
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
 2026-09-18 17:39:50 -07:00
Cheng Wan d0730a0e8b Give the attention-DP width and rank one home (#40067) 2026-09-18 17:34:33 -07:00
Lifan Shen 8ea0ee300d perf(sampling): avoid GPU syncs when applying custom logit processors (#39234) 2026-09-18 17:09:30 -07:00
Liangsheng Yin b876213548 [Test] Add a ci-test-audit skill cataloging CI and test audit patterns (#40257) 2026-09-18 16:35:49 -07:00
Alison ShaoandXinyuan Tong 21e6c98ccb Fix corrupted chat prompts on mistral_common tokenizers (tool_choice auto never fires) (#39773) 
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2026-09-18 16:12:06 -07:00
Alison ShaoXinyuan TongXinyuan Tong
 		2394b231c2 Fix Mistral3 retaining every vision-tower layer to read one (#39185) 
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2026-09-18 16:11:54 -07:00
metamergebotmetamergebotJialin OuyangLianmin Zheng
 		fa521e2758 [MM] Copy placeholder ids to CUDA asynchronously (#40010) 
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
 2026-09-18 16:10:37 -07:00
Zhiqiang Xieandcctry ceb1d2e580 [PD] Enable optimistic prefill with buffer-only L3 write-through HiCache (#40043) 
Co-authored-by: cctry <csycfl@gmail.com>
 2026-09-18 16:04:15 -07:00
metamergebotmetamergebotJialin OuyangLianmin Zheng
 		5e4b94b134 [MM] Skip VMM error gathers for text-only requests (#40005) 
Co-authored-by: metamergebot <324680979+metamergebot@users.noreply.github.com>
Co-authored-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
 2026-09-18 16:00:43 -07:00
hunhokimandHun-ho Kim 803f0c93d2 Fix DSA partial DP-TP mode log to use derived attn_tp_size (#39871) 
Co-authored-by: Hun-ho Kim <hunho.kim@samsung.com>
 2026-09-18 15:54:45 -07:00
Jialin Ouyang f3851486cb [Perf] Fuse SWA page lookup and mapping clear (#38948) 2026-09-18 15:51:07 -07:00
metamergebotPranjal ShankhdharJialin Ouyang
 		81a199f56a [HiCache] Read the in-flight buffer backup's node id from its snapshot in sanity_check (#40013) 
Co-authored-by: Pranjal Shankhdhar <pranjalssh@meta.com>
Co-authored-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
 2026-09-18 15:47:56 -07:00
metamergebotmetamergebotXingyu Liupranjalssh
 		0e5347db82 Support MXFP8 and deferred route weighting in DeepEP v2 (#40030) 
Co-authored-by: metamergebot <324680979+metamergebot@users.noreply.github.com>
Co-authored-by: Xingyu Liu <38244988+charlotte12l@users.noreply.github.com>
Co-authored-by: pranjalssh <14260275+pranjalssh@users.noreply.github.com>
 2026-09-18 15:40:38 -07:00
Liangsheng Yin 6cc9090d1f [mem_cache] Release up to owned_kv_len on radix cache insert (#40075) 2026-09-18 15:37:52 -07:00
Zhiqiang Xie a0534f8cca [HiCache] Stop arming a prefetch retry for a too-short storage span (#40042) 2026-09-18 15:33:58 -07:00
Sam (Kesen Li) d346b214fb feat(kv-cache): support SM100 NVFP4 GenMHA and speculative decoding (#36340) 2026-09-18 14:50:46 -07:00
Lianmin ZhengandJialin Ouyang f5a1434700 [HiCache] Document transfer arguments (#40239) 
Clarify the legacy host_indices argument and label Mamba test arguments,
including the current staging_tokens parameter. Executable code is unchanged.

Co-authored-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
 2026-09-18 14:08:19 -07:00
amd-danli103 cd4dd81c22 [AMD][DSV4] fix: drop shadowing local get_exec import that breaks model startup on ROCm (#40186) 2026-09-18 13:28:38 -07:00
metamergebotandcctry 6a9c7001d3 [Logprob] Borrow graph-pool memory for input logprob logits construction (#40007) 
Co-authored-by: cctry <csycfl@gmail.com>
 2026-09-18 13:18:01 -07:00
Jialin Ouyang da2f434951 [Spec] Add explicit prefill shared-read capability for plugins (#39502) 2026-09-18 13:11:01 -07:00
Kangyan-ZhouKangyan ZhouClaude Opus 5Shangming Cai
 		aed3fb1cdd [Router] Log every request at one site; derive its outcome from the final status (3/3) (#39465) 
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2026-09-19 03:11:00 +08:00
Lianmin Zhengandraghotham 248c202b46 Use runtime token widths for Triton speculative verification (#39859) 
Co-authored-by: raghotham <853234+raghotham@users.noreply.github.com>
 2026-09-18 11:07:55 -07:00
Lianmin Zheng 6bd1a0af1d Add registration for external model configurations (#39452) 2026-09-18 10:06:24 -07:00
Byron Hsu 50a7de47d5 [Benchmark] Add agentic rollout simulator and offline explorer (#40034) 2026-09-18 10:06:13 -07:00
Kangyan-ZhouKangyan ZhouClaude Opus 5
 		4e0b56c811 [Router] Treat an upstream 503/429 as backpressure, not a breaker fault (2/3) (#39464) 
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-19 00:46:29 +08:00
Shuwen Wang 191172fa74 [Unified Tree] fix: exempt host-locked aux nodes from the sanity_check host-LRU check (#39980) 2026-09-18 15:29:29 +00:00
ChangLiu0709 45938a24ae [AMD] GLM-5.2 MI355X MXFP4: bump image to 20260916, use HIP Top-K (#40148) 2026-09-18 23:21:58 +08:00
Xiaoyu Zhang 7714b182f2 [Bugfix] Fix top-1 MoE routing with non-unit scaling (#40187) 2026-09-18 22:53:13 +08:00
DarkSharpnessClaude Opus 5BBuf
 		81363bf8cb [kernel] Share the warp vectorized copy and enforce its alignment (#36176) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: BBuf <1182563586@qq.com>
 2026-09-18 22:40:54 +08:00
MickandMick Qian 9784d5f979 [diffusion] doc: sync CFG and tracing documentation (#39883) 
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
 2026-09-18 18:03:40 +08:00
Liangsheng YinBBufClaude Opus 5Xiaoyu ZhangYuwei AnKhoa PhamYuhao YangZhichen ZengKe Bao
 		a6cf05817f dsv4.1: remaining model and runtime integration (#38798) 
Co-authored-by: BBuf <1182563586@qq.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Xiaoyu Zhang <xiaoyu.zhang@radixark.ai>
Co-authored-by: Yuwei An <ayw.sirius19@gmail.com>
Co-authored-by: Khoa Pham <khoa.pham@radixark.ai>
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: Zhichen Zeng <zczeng@uw.edu>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
 2026-09-18 02:55:30 -07:00
Liangsheng Yin 1b200ffaaa [Quant] Serve 32-wide-K ue8m0 block-FP8 linears through the FlashInfer MXFP8 GEMMs (#40039) 2026-09-18 02:51:12 -07:00
iridiumine 6c7c5e78de [NPU] Run arch35 block-FP8 dense linears on the native MXFP8 GEMM (#39823) 2026-09-18 17:19:31 +08:00
iridiumine 7e6d5cbfac [NPU] Gate DFlash replay metadata refresh behind spec_algorithm check (#39879) 2026-09-18 17:07:07 +08:00
Cheng Wan bbfcda48ce [CI] Add a unified-memory rerun test group (#40147) 2026-09-18 01:59:07 -07:00
iridiumine d6090f92bf [NPU] Fuse MXFP4 W4A8 MoE gmm1 + swiglu + requant into one kernel (#39881) 2026-09-18 16:50:49 +08:00
Gurpreet Singh Dhamijacky.chengClaude
 		2dee23a876 [ROCm][diffusion] Enable fused qk norm and rope on ROCm (#35573) 
Co-authored-by: jacky.cheng <yichiche@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-09-18 16:40:32 +08:00
Shu Wang 1e8699fda3 [NVIDIA][comm] Merge EP+MoE-TP post-experts all-reduces into one _TP reduction (#32963) 2026-09-18 01:35:10 -07:00
zhaozx-cn 8ac39c66d8 [NPU] support kimi k3 on A5 and improve performance (#39589) 2026-09-18 16:33:54 +08:00
Kangyan-ZhouKangyan ZhouClaude Opus 5
 		6de4666e43 [Router] Derive error status from a failure class; preserve the worker's status (1/3) (#39463) 
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-18 16:32:45 +08:00
Rumit DesaiandXiaoyu Zhang 1fdd6c8921 [Runtime] Let out-of-tree platforms provide full graph backends (#37969) 
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
 2026-09-18 15:42:46 +08:00
Ziang Li c46bf5e990 [MoE] Disable FlashInfer fused finalize by default for numerical accuracy (#40105) 2026-09-18 00:15:57 -07:00
Ke Bao 956d414dad Update ci permission (#40121) 2026-09-18 15:12:36 +08:00
Shuwen Wang 0be8a0af0e [DSV4] fix: keep the TileLang JIT cache under SGLANG_CACHE_DIR (#39364) 2026-09-18 00:06:35 -07:00
Kangyan-ZhouKangyan ZhouClaude Opus 5Shangming Cai
 		3bf243d6a9 [Router] Shard the cache-aware KV tree by chain root (#39167) 
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2026-09-18 15:00:17 +08:00
iridiumine f86f60081d [NPU] Adapt hicache for K3 hybrid models (#39415) 2026-09-18 14:48:20 +08:00
amote-i 6952538980 [NPU] [DOC] delete unsupported api --disable-hybrid-swa-memory for npu (#40058) 2026-09-18 14:31:05 +08:00
Xinyuan Tong 4dbba37965 Verify the Ling-3.0-flash-VL FP4 lane on H200 and disable shared-expert fusion in quant recipes (#39419) 2026-09-17 23:20:18 -07:00