minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 10 Packages Projects Releases Wiki Activity
18,464 Commits 2 Branches 0 Tags
81a199f56ada4d5b26928aa807ad1aabd89c0d0f
Commit Graph
18464 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
metamergebotPranjal ShankhdharJialin Ouyang
 		81a199f56a [HiCache] Read the in-flight buffer backup's node id from its snapshot in sanity_check (#40013) 
Co-authored-by: Pranjal Shankhdhar <pranjalssh@meta.com>
Co-authored-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
 2026-09-18 15:47:56 -07:00
metamergebotmetamergebotXingyu Liupranjalssh
 		0e5347db82 Support MXFP8 and deferred route weighting in DeepEP v2 (#40030) 
Co-authored-by: metamergebot <324680979+metamergebot@users.noreply.github.com>
Co-authored-by: Xingyu Liu <38244988+charlotte12l@users.noreply.github.com>
Co-authored-by: pranjalssh <14260275+pranjalssh@users.noreply.github.com>
 2026-09-18 15:40:38 -07:00
Liangsheng Yin 6cc9090d1f [mem_cache] Release up to owned_kv_len on radix cache insert (#40075) 2026-09-18 15:37:52 -07:00
Zhiqiang Xie a0534f8cca [HiCache] Stop arming a prefetch retry for a too-short storage span (#40042) 2026-09-18 15:33:58 -07:00
Sam (Kesen Li) d346b214fb feat(kv-cache): support SM100 NVFP4 GenMHA and speculative decoding (#36340) 2026-09-18 14:50:46 -07:00
Lianmin ZhengandJialin Ouyang f5a1434700 [HiCache] Document transfer arguments (#40239) 
Clarify the legacy host_indices argument and label Mamba test arguments,
including the current staging_tokens parameter. Executable code is unchanged.

Co-authored-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
 2026-09-18 14:08:19 -07:00
amd-danli103 cd4dd81c22 [AMD][DSV4] fix: drop shadowing local get_exec import that breaks model startup on ROCm (#40186) 2026-09-18 13:28:38 -07:00
metamergebotandcctry 6a9c7001d3 [Logprob] Borrow graph-pool memory for input logprob logits construction (#40007) 
Co-authored-by: cctry <csycfl@gmail.com>
 2026-09-18 13:18:01 -07:00
Jialin Ouyang da2f434951 [Spec] Add explicit prefill shared-read capability for plugins (#39502) 2026-09-18 13:11:01 -07:00
Kangyan-ZhouKangyan ZhouClaude Opus 5Shangming Cai
 		aed3fb1cdd [Router] Log every request at one site; derive its outcome from the final status (3/3) (#39465) 
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2026-09-19 03:11:00 +08:00
Lianmin Zhengandraghotham 248c202b46 Use runtime token widths for Triton speculative verification (#39859) 
Co-authored-by: raghotham <853234+raghotham@users.noreply.github.com>
 2026-09-18 11:07:55 -07:00
Lianmin Zheng 6bd1a0af1d Add registration for external model configurations (#39452) 2026-09-18 10:06:24 -07:00
Byron Hsu 50a7de47d5 [Benchmark] Add agentic rollout simulator and offline explorer (#40034) 2026-09-18 10:06:13 -07:00
Kangyan-ZhouKangyan ZhouClaude Opus 5
 		4e0b56c811 [Router] Treat an upstream 503/429 as backpressure, not a breaker fault (2/3) (#39464) 
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-19 00:46:29 +08:00
Shuwen Wang 191172fa74 [Unified Tree] fix: exempt host-locked aux nodes from the sanity_check host-LRU check (#39980) 2026-09-18 15:29:29 +00:00
ChangLiu0709 45938a24ae [AMD] GLM-5.2 MI355X MXFP4: bump image to 20260916, use HIP Top-K (#40148) 2026-09-18 23:21:58 +08:00
Xiaoyu Zhang 7714b182f2 [Bugfix] Fix top-1 MoE routing with non-unit scaling (#40187) 2026-09-18 22:53:13 +08:00
DarkSharpnessClaude Opus 5BBuf
 		81363bf8cb [kernel] Share the warp vectorized copy and enforce its alignment (#36176) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: BBuf <1182563586@qq.com>
 2026-09-18 22:40:54 +08:00
MickandMick Qian 9784d5f979 [diffusion] doc: sync CFG and tracing documentation (#39883) 
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
 2026-09-18 18:03:40 +08:00
Liangsheng YinBBufClaude Opus 5Xiaoyu ZhangYuwei AnKhoa PhamYuhao YangZhichen ZengKe Bao
 		a6cf05817f dsv4.1: remaining model and runtime integration (#38798) 
Co-authored-by: BBuf <1182563586@qq.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Xiaoyu Zhang <xiaoyu.zhang@radixark.ai>
Co-authored-by: Yuwei An <ayw.sirius19@gmail.com>
Co-authored-by: Khoa Pham <khoa.pham@radixark.ai>
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: Zhichen Zeng <zczeng@uw.edu>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
 2026-09-18 02:55:30 -07:00
Liangsheng Yin 1b200ffaaa [Quant] Serve 32-wide-K ue8m0 block-FP8 linears through the FlashInfer MXFP8 GEMMs (#40039) 2026-09-18 02:51:12 -07:00
iridiumine 6c7c5e78de [NPU] Run arch35 block-FP8 dense linears on the native MXFP8 GEMM (#39823) 2026-09-18 17:19:31 +08:00
iridiumine 7e6d5cbfac [NPU] Gate DFlash replay metadata refresh behind spec_algorithm check (#39879) 2026-09-18 17:07:07 +08:00
Cheng Wan bbfcda48ce [CI] Add a unified-memory rerun test group (#40147) 2026-09-18 01:59:07 -07:00
iridiumine d6090f92bf [NPU] Fuse MXFP4 W4A8 MoE gmm1 + swiglu + requant into one kernel (#39881) 2026-09-18 16:50:49 +08:00
Gurpreet Singh Dhamijacky.chengClaude
 		2dee23a876 [ROCm][diffusion] Enable fused qk norm and rope on ROCm (#35573) 
Co-authored-by: jacky.cheng <yichiche@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-09-18 16:40:32 +08:00
Shu Wang 1e8699fda3 [NVIDIA][comm] Merge EP+MoE-TP post-experts all-reduces into one _TP reduction (#32963) 2026-09-18 01:35:10 -07:00
zhaozx-cn 8ac39c66d8 [NPU] support kimi k3 on A5 and improve performance (#39589) 2026-09-18 16:33:54 +08:00
Kangyan-ZhouKangyan ZhouClaude Opus 5
 		6de4666e43 [Router] Derive error status from a failure class; preserve the worker's status (1/3) (#39463) 
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-18 16:32:45 +08:00
Rumit DesaiandXiaoyu Zhang 1fdd6c8921 [Runtime] Let out-of-tree platforms provide full graph backends (#37969) 
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
 2026-09-18 15:42:46 +08:00
Ziang Li c46bf5e990 [MoE] Disable FlashInfer fused finalize by default for numerical accuracy (#40105) 2026-09-18 00:15:57 -07:00
Ke Bao 956d414dad Update ci permission (#40121) 2026-09-18 15:12:36 +08:00
Shuwen Wang 0be8a0af0e [DSV4] fix: keep the TileLang JIT cache under SGLANG_CACHE_DIR (#39364) 2026-09-18 00:06:35 -07:00
Kangyan-ZhouKangyan ZhouClaude Opus 5Shangming Cai
 		3bf243d6a9 [Router] Shard the cache-aware KV tree by chain root (#39167) 
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2026-09-18 15:00:17 +08:00
iridiumine f86f60081d [NPU] Adapt hicache for K3 hybrid models (#39415) 2026-09-18 14:48:20 +08:00
amote-i 6952538980 [NPU] [DOC] delete unsupported api --disable-hybrid-swa-memory for npu (#40058) 2026-09-18 14:31:05 +08:00
Xinyuan Tong 4dbba37965 Verify the Ling-3.0-flash-VL FP4 lane on H200 and disable shared-expert fusion in quant recipes (#39419) 2026-09-17 23:20:18 -07:00
Jensen 3ce3b4969f [NPU] Avoid repeated BF16 wo_a weight transposes in DeepSeek-V4 decode (#39919) 2026-09-18 09:10:43 +03:00
黄孝君 d337b865af Update NPU tag to post4 version and related scripts (#39953) 2026-09-18 14:03:07 +08:00
Bojiang LiandXiaoyu Zhang 3d1b9e7549 [Bugfix] Include SM121 in DeepGEMM packed-scale selection (#39482) 
Signed-off-by: bojiang-li <327132355+bojiang-li@users.noreply.github.com>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
 2026-09-18 13:57:14 +08:00
maithilijoshi20 0dad91d50f Fix int4 MoE tuner config filename (#35260) 
Signed-off-by: maithilijoshi20 <maithilij2003@gmail.com>
 2026-09-18 13:56:38 +08:00
Hanming Luandhanminglu 6e1d9c1464 [profiler] Ignore PREBUILT batches in profile-by-stage (#40098) 
Co-authored-by: hanminglu <hanminglu@fb.com>
 2026-09-17 22:33:55 -07:00
Chi McIsaacandMick Qian 6215aecd51 [diffusion] feat: add metrics support (#19084) 
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
 2026-09-18 13:22:03 +08:00
Yuwei An 65ef55e2a8 [Scheduler] Add shortest-prefill-first scheduling (#40024) 2026-09-17 21:18:58 -07:00
Mohammad Miadh AngkadandMohammad Angkad c055dc6ff6 [CI] Check B200 NUMA mapping against sysfs numa_node (#40055) 
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
 2026-09-17 21:14:47 -07:00
Xiaojia WuCursorXiaoyu Zhang
 		826d5170ae [Bug] Guard FlashInfer CUTLASS MoE against 0-token inputs (#38780) 
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
 2026-09-18 11:03:27 +08:00
Zixin HuangandClaude Fable 5.1 f447bb7080 [Kernel] Coalesce the KDA CuTe DSL decode state transpose: ~3x faster, bit-identical (#39680) 
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-18 10:53:13 +08:00
gilfordtingJames Liuhnyls2002Liangsheng Yin
 		db39b7f961 [Fix] Guard conditional top-logprob keys in the completions echo path (#34776) 
Co-authored-by: James Liu <jamesl@modal.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
 2026-09-17 19:47:26 -07:00
Nan Jiang 740f57a02c [Spec] Fix CDF boundary handling in TreeSpeculativeSamplingTargetOnly (#35798) 2026-09-17 19:41:52 -07:00
William Arnoldandishandhanani 0214954f26 [gRPC] Stream engine state changes (#39915) 
Signed-off-by: William Arnold <7565007+Aphoh@users.noreply.github.com>
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
 2026-09-17 19:17:25 -07:00