minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
18,487 Commits 2 Branches 0 Tags
929230a6f015dcc4042ef8d06ef10c0c81f17327
Commit Graph
18487 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Yuxuan ZhangXinyuan TongXinyuan Tong
 		929230a6f0 Fix GLM-OCR MTP multimodal embeddings and positions (#39088) 
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2026-09-18 21:10:04 -07:00
Mind Lab 111b905bd1 fix(openai): recover logprobs token bytes from token_id (UTF-8 fragments) (#38604) 2026-09-18 20:30:04 -07:00
DefTruthandcopilot-swe-agent[bot] f1fbbd17bb [diffusion] chore: update Cache-DiT to 1.5.1 for DMD Calibrator, SVDQuant DQ, etc (#40104) 
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
 2026-09-19 11:14:33 +08:00
Vincent Liu 090263eff6 [VLM] avoid CUDA placement on non-CUDA platforms (#38750) 2026-09-19 11:13:22 +08:00
Alexandkpham-sgl c3aa09b0db [Kernel] Fuse hc_combine_norm for mid-size verify batches (9-96 rows) (#40208) 
Co-authored-by: kpham-sgl <khoa.pham@radixark.ai>
 2026-09-18 20:03:49 -07:00
hmalgewattaandMick Qian c475ac5eaf [diffusion] feat: out of tree platform support (#37547) 
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
 2026-09-19 09:35:58 +08:00
Xiaoyu Zhang 986959e3c4 [Refactor] Deduplicate kernel helpers and remove unused code (#40197) 2026-09-19 09:27:30 +08:00
Khoa PhamandCursor 10b0bcfd18 [PD] Allow decode radix cache and HiCache L1/L2 with DCP (#40263) 
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-09-18 17:54:35 -07:00
Liangsheng Yin d507accadc [Test] Drop dead and strictly-subsumed CI test registrations (#40264) 2026-09-18 17:49:13 -07:00
Cheng Wan fa7e83fd09 Name the two widths of the WORLD group (#40070) 2026-09-18 17:47:02 -07:00
Cheng Wan 81421b91e9 One read path for every parallel name (#40069) 2026-09-18 17:43:56 -07:00
Cheng Wan afe71f4b9e Read process groups through the runtime context (#40068) 2026-09-18 17:40:32 -07:00
Yonghao ZhuangyhzhuangLianmin ZhengYonghao ZhuangCheng Wan
 		5931fd60ee Support unified memory page-envelope transfers in PD (#39477) 
Co-authored-by: yhzhuang <yhzhuang@fb.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: Yonghao Zhuang <yhzhuang@users.noreply.github.com>
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
 2026-09-18 17:39:50 -07:00
Cheng Wan d0730a0e8b Give the attention-DP width and rank one home (#40067) 2026-09-18 17:34:33 -07:00
Lifan Shen 8ea0ee300d perf(sampling): avoid GPU syncs when applying custom logit processors (#39234) 2026-09-18 17:09:30 -07:00
Liangsheng Yin b876213548 [Test] Add a ci-test-audit skill cataloging CI and test audit patterns (#40257) 2026-09-18 16:35:49 -07:00
Alison ShaoandXinyuan Tong 21e6c98ccb Fix corrupted chat prompts on mistral_common tokenizers (tool_choice auto never fires) (#39773) 
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2026-09-18 16:12:06 -07:00
Alison ShaoXinyuan TongXinyuan Tong
 		2394b231c2 Fix Mistral3 retaining every vision-tower layer to read one (#39185) 
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2026-09-18 16:11:54 -07:00
metamergebotmetamergebotJialin OuyangLianmin Zheng
 		fa521e2758 [MM] Copy placeholder ids to CUDA asynchronously (#40010) 
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
 2026-09-18 16:10:37 -07:00
Zhiqiang Xieandcctry ceb1d2e580 [PD] Enable optimistic prefill with buffer-only L3 write-through HiCache (#40043) 
Co-authored-by: cctry <csycfl@gmail.com>
 2026-09-18 16:04:15 -07:00
metamergebotmetamergebotJialin OuyangLianmin Zheng
 		5e4b94b134 [MM] Skip VMM error gathers for text-only requests (#40005) 
Co-authored-by: metamergebot <324680979+metamergebot@users.noreply.github.com>
Co-authored-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
 2026-09-18 16:00:43 -07:00
hunhokimandHun-ho Kim 803f0c93d2 Fix DSA partial DP-TP mode log to use derived attn_tp_size (#39871) 
Co-authored-by: Hun-ho Kim <hunho.kim@samsung.com>
 2026-09-18 15:54:45 -07:00
Jialin Ouyang f3851486cb [Perf] Fuse SWA page lookup and mapping clear (#38948) 2026-09-18 15:51:07 -07:00
metamergebotPranjal ShankhdharJialin Ouyang
 		81a199f56a [HiCache] Read the in-flight buffer backup's node id from its snapshot in sanity_check (#40013) 
Co-authored-by: Pranjal Shankhdhar <pranjalssh@meta.com>
Co-authored-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
 2026-09-18 15:47:56 -07:00
metamergebotmetamergebotXingyu Liupranjalssh
 		0e5347db82 Support MXFP8 and deferred route weighting in DeepEP v2 (#40030) 
Co-authored-by: metamergebot <324680979+metamergebot@users.noreply.github.com>
Co-authored-by: Xingyu Liu <38244988+charlotte12l@users.noreply.github.com>
Co-authored-by: pranjalssh <14260275+pranjalssh@users.noreply.github.com>
 2026-09-18 15:40:38 -07:00
Liangsheng Yin 6cc9090d1f [mem_cache] Release up to owned_kv_len on radix cache insert (#40075) 2026-09-18 15:37:52 -07:00
Zhiqiang Xie a0534f8cca [HiCache] Stop arming a prefetch retry for a too-short storage span (#40042) 2026-09-18 15:33:58 -07:00
Sam (Kesen Li) d346b214fb feat(kv-cache): support SM100 NVFP4 GenMHA and speculative decoding (#36340) 2026-09-18 14:50:46 -07:00
Lianmin ZhengandJialin Ouyang f5a1434700 [HiCache] Document transfer arguments (#40239) 
Clarify the legacy host_indices argument and label Mamba test arguments,
including the current staging_tokens parameter. Executable code is unchanged.

Co-authored-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
 2026-09-18 14:08:19 -07:00
amd-danli103 cd4dd81c22 [AMD][DSV4] fix: drop shadowing local get_exec import that breaks model startup on ROCm (#40186) 2026-09-18 13:28:38 -07:00
metamergebotandcctry 6a9c7001d3 [Logprob] Borrow graph-pool memory for input logprob logits construction (#40007) 
Co-authored-by: cctry <csycfl@gmail.com>
 2026-09-18 13:18:01 -07:00
Jialin Ouyang da2f434951 [Spec] Add explicit prefill shared-read capability for plugins (#39502) 2026-09-18 13:11:01 -07:00
Kangyan-ZhouKangyan ZhouClaude Opus 5Shangming Cai
 		aed3fb1cdd [Router] Log every request at one site; derive its outcome from the final status (3/3) (#39465) 
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2026-09-19 03:11:00 +08:00
Lianmin Zhengandraghotham 248c202b46 Use runtime token widths for Triton speculative verification (#39859) 
Co-authored-by: raghotham <853234+raghotham@users.noreply.github.com>
 2026-09-18 11:07:55 -07:00
Lianmin Zheng 6bd1a0af1d Add registration for external model configurations (#39452) 2026-09-18 10:06:24 -07:00
Byron Hsu 50a7de47d5 [Benchmark] Add agentic rollout simulator and offline explorer (#40034) 2026-09-18 10:06:13 -07:00
Kangyan-ZhouKangyan ZhouClaude Opus 5
 		4e0b56c811 [Router] Treat an upstream 503/429 as backpressure, not a breaker fault (2/3) (#39464) 
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-19 00:46:29 +08:00
Shuwen Wang 191172fa74 [Unified Tree] fix: exempt host-locked aux nodes from the sanity_check host-LRU check (#39980) 2026-09-18 15:29:29 +00:00
ChangLiu0709 45938a24ae [AMD] GLM-5.2 MI355X MXFP4: bump image to 20260916, use HIP Top-K (#40148) 2026-09-18 23:21:58 +08:00
Xiaoyu Zhang 7714b182f2 [Bugfix] Fix top-1 MoE routing with non-unit scaling (#40187) 2026-09-18 22:53:13 +08:00
DarkSharpnessClaude Opus 5BBuf
 		81363bf8cb [kernel] Share the warp vectorized copy and enforce its alignment (#36176) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: BBuf <1182563586@qq.com>
 2026-09-18 22:40:54 +08:00
MickandMick Qian 9784d5f979 [diffusion] doc: sync CFG and tracing documentation (#39883) 
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
 2026-09-18 18:03:40 +08:00
Liangsheng YinBBufClaude Opus 5Xiaoyu ZhangYuwei AnKhoa PhamYuhao YangZhichen ZengKe Bao
 		a6cf05817f dsv4.1: remaining model and runtime integration (#38798) 
Co-authored-by: BBuf <1182563586@qq.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Xiaoyu Zhang <xiaoyu.zhang@radixark.ai>
Co-authored-by: Yuwei An <ayw.sirius19@gmail.com>
Co-authored-by: Khoa Pham <khoa.pham@radixark.ai>
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: Zhichen Zeng <zczeng@uw.edu>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
 2026-09-18 02:55:30 -07:00
Liangsheng Yin 1b200ffaaa [Quant] Serve 32-wide-K ue8m0 block-FP8 linears through the FlashInfer MXFP8 GEMMs (#40039) 2026-09-18 02:51:12 -07:00
iridiumine 6c7c5e78de [NPU] Run arch35 block-FP8 dense linears on the native MXFP8 GEMM (#39823) 2026-09-18 17:19:31 +08:00
iridiumine 7e6d5cbfac [NPU] Gate DFlash replay metadata refresh behind spec_algorithm check (#39879) 2026-09-18 17:07:07 +08:00
Cheng Wan bbfcda48ce [CI] Add a unified-memory rerun test group (#40147) 2026-09-18 01:59:07 -07:00
iridiumine d6090f92bf [NPU] Fuse MXFP4 W4A8 MoE gmm1 + swiglu + requant into one kernel (#39881) 2026-09-18 16:50:49 +08:00
Gurpreet Singh Dhamijacky.chengClaude
 		2dee23a876 [ROCm][diffusion] Enable fused qk norm and rope on ROCm (#35573) 
Co-authored-by: jacky.cheng <yichiche@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-09-18 16:40:32 +08:00
Shu Wang 1e8699fda3 [NVIDIA][comm] Merge EP+MoE-TP post-experts all-reduces into one _TP reduction (#32963) 2026-09-18 01:35:10 -07:00