minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
15,687 Commits 2 Branches 0 Tags
72e415dfc81fb031dba32bc3c2ce0d890cb4eae5
Commit Graph
15687 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
silencejade 72e415dfc8 [Bugfix] Fix prefill suspension caused by delayed negotiate_should_allow_prefill invocation (#32389) 2026-07-26 14:59:26 +08:00
YC Yen-Ching Tseng 1d0cd2e473 [AMD] Nightly Test Coverage - Minimax-M3-MXFP8 Accuracy Test (#30613) 2026-07-25 23:52:22 -07:00
ormandjandMohammad Miadh Angkad 2cbddb842d [DSV4/SM120] Allow fused MHC opt-in with standalone TileLang pre disabled (#30954) 
Signed-off-by: David Orman <ormandj@corenode.com>
Co-authored-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>
 2026-07-26 09:02:54 +08:00
Ming YangClaudeLianmin Zheng
 		55c4853487 [comm] Enable multi-node custom-AR v2 on a single NVLink clique (#32339) 
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
 2026-07-25 17:25:18 -07:00
cctrycctryZhiqiang Xie
 		2c63a2f12b Fix --hicache-size allocating ~2x host memory on hybrid SWA (#32373) 
Co-authored-by: cctry <cctry@fb.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2026-07-25 17:19:44 -07:00
Lianmin ZhengandAlec S 9989077f24 Use native batched llguidance mask generation (#32412) 
Co-authored-by: Alec S <10566873+alecsolder@users.noreply.github.com>
 2026-07-25 16:36:32 -07:00
Lianmin ZhengandXingyu Liu fae84ac0f9 Fix token count localization for replicated attention-TP forwards (#32411) 
Co-authored-by: Xingyu Liu <38244988+charlotte12l@users.noreply.github.com>
 2026-07-25 16:36:15 -07:00
Lianmin Zheng 5f330004bd Fix flaky test_sampling_mask: mask length can legitimately be top_k + 1 (#32410) 2026-07-25 16:05:57 -07:00
Liangsheng Yin 3da1071d56 [Spec] Hold the grammar bitmask in one GrammarMask type across all decode paths (#32409) 2026-07-25 15:27:43 -07:00
Lianmin Zheng d3cf4dfbaa Update audio container test time estimate (#32408) 
Signed-off-by: Lianmin Zheng <lianminzheng@gmail.com>
 2026-07-25 14:45:08 -07:00
Jialin Ouyang cd145f840f Radix Cache Split: Spin off TreeCore (#29901) 2026-07-25 14:31:59 -07:00
Kangrui DuandYihao Wang a23f6ea090 [Diffusion] offload rollout weights to pinned host memory (#32032) 
Co-authored-by: Yihao Wang <42559837+AgainstEntropy@users.noreply.github.com>
 2026-07-25 13:35:53 -07:00
YAMY 91f386a5b2 fix(disagg): support pipeline-parallel hybrid-linear transfer (#32270) 2026-07-25 13:34:38 -07:00
Cheng WanandShu Wang 659d349b61 [core/loader] Add presharded load format (#24256) 
Co-authored-by: Shu Wang <shuwanguc@google.com>
 2026-07-25 13:03:39 -07:00
Mohammad Miadh Angkad 9791fc7090 Add configurable FlashInfer autotune skips (#31389) 2026-07-25 11:17:58 -07:00
Mohammad Miadh Angkad 953c587adf [Docs] Add Qwen3.6 35B NVFP4 to cookbook (#31413) 2026-07-25 11:16:53 -07:00
Ke Bao 69a3c54c70 Fix SWA admission livelock on cached-prefix resumes (#32379) 2026-07-25 22:37:35 +08:00
shanemort1982shanemort1982hnyls2002
 		e943e609dc [DSPARK] Grammar-constrained decoding, incl. tool_choice=auto (#31753) 
Co-authored-by: shanemort1982 <shanemort1982@users.noreply.github.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
 2026-07-25 05:20:25 -07:00
yyqjwyyyuyanqiClaude Opus 4.6Xinyuan Tong
 		a678a42033 [KDA] Add target_verify support for speculative decoding (#26888) 
Co-authored-by: yuyanqi <yuyanqi@meituan.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2026-07-25 19:52:44 +08:00
Jincong ChenKe BaoZhangheng
 		7c4b22fae5 [Hicache][1/2]Support Mamba branching in Unified Radix Cache with HiCache (#31181) 
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
Co-authored-by: Zhangheng <hzh0425@apache.org>
 2026-07-25 19:44:40 +08:00
Mick 1054060ef1 perf: speed up marlin moe with occupancy-aware launch specialization (#31552) 2026-07-25 19:38:11 +08:00
Hồ Sỹ Thếandhnyls2002 d021990bf5 [DFLASH] Support grammar-constrained decoding in speculative verify (#30096) 
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
 2026-07-25 04:36:46 -07:00
Liangsheng Yin 17afd8421f [Spec] Share the grammar mask build and verify-tree staging across spec workers (#32393) 2026-07-25 03:40:21 -07:00
Liangsheng Yin 3c5bf1f6d2 [Spec] Derive NGRAM grammar tree links on the host instead of reading back retrive_next_token (#32380) 2026-07-25 02:25:16 -07:00
Yuang Chenand晟海 f5155d9602 [EPD] Fix HTTP dispatch lock blocking cross-request encoder batching (#31275) 
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
 2026-07-25 14:47:58 +08:00
Xun SunandShangming Cai 9eb2dccbb7 [Elastic EP] Fix recovery lifecycle and add manual coverage (#31744) 
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2026-07-25 14:32:47 +08:00
Mike Qiuliusy58Alex Nails
 		f9c14e6bd4 [FEAT] Support fast engine recovery through weight cache (#27139) 
Signed-off-by: Michael Qiu <qiudayu.qdy@antgroup.com>
Co-authored-by: liusy58 <liusy58@linux.alibaba.com>
Co-authored-by: Alex Nails <alex.nails@radixark.ai>
 2026-07-25 14:31:21 +08:00
Shangming Cai bdd0698541 chore: bump mooncake version to 0.3.12.post1 (#32302) 2026-07-25 14:12:46 +08:00
SovietPowerandShangming Cai 6a046fad09 [PD] Prevent decode scheduler from blocking on ZMQ sends to a stalled prefill peer (#31144) 
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2026-07-25 12:53:41 +08:00
Bill DuJimmy ShongZhangheng
 		ebcb74abd4 feat(hicache): Add shared memory allocator for host KV cache (#29326) 
Co-authored-by: Jimmy Shong <69131491+Jiminator@users.noreply.github.com>
Co-authored-by: Zhangheng <hzh0425@apache.org>
 2026-07-25 12:05:37 +08:00
Alex NailsandClaude Opus 4.8 b83041c3cc Migrate CompressedTensorsW4A4Nvfp4MoE TRT-LLM path onto MoeRunner (#32248) 
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-24 20:57:38 -07:00
Liangsheng Yin cff20a2fbb [Spec] Consolidate the grammar sync decision into ScheduleBatch.grammar_needs_sync (#32353) 2026-07-24 20:42:38 -07:00
Yihao Wang 95865de24f [diffusion] CI: read consistency GT from ci-data-diffusion at per-platform commit (#32297) 2026-07-25 09:55:57 +08:00
Yihao Wang ebf5330f71 [diffusion] CI: enhance diffusion GT publishing with per-platform directory resolution (#32282) 2026-07-25 09:54:46 +08:00
cctryandJialin Ouyang a690e5e0b3 Add stream label to TTFT metrics (#32363) 
Co-authored-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
 2026-07-24 17:44:04 -07:00
cctryandYinghai Lu ce705bb6dc Report accelerator type in /v1/loads (#32348) 
Co-authored-by: Yinghai Lu <yinghai@meta.com>
 2026-07-24 17:31:28 -07:00
sglang-botandsglang-bot 1ef2b85ef7 chore: bump docs install version to 0.5.16 (#32347) 
Co-authored-by: sglang-bot <sglang-bot@users.noreply.github.com>
 2026-07-24 17:19:39 -07:00
DarkSharpnessandClaude Fable 5 9402012f0f [Perf] Halve the non-finite sanitization overhead in per_token_group_quant (#32296) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-25 08:17:12 +08:00
danielafrimiandDaniel Afrimi 7de8792758 Fix FP8 Triton dtype selection on A100 (#31340) 
Co-authored-by: Daniel Afrimi <dafrimi@aws-dfw-cs-001-login-01.cm.cluster>
 2026-07-24 17:03:19 -07:00
Kangyan-ZhouKangyan ZhouClaude Opus 5
 		3079157175 Fix PyPI release: drop the git-only sgl-eval dep from packaged metadata (#32354) 
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-07-24 20:00:57 -04:00
Pranjal Shankhdharandpranjalssh 9c483cccfe Support a same-size mixed q dtype in the fused RoPE kernels (#31834) 
Co-authored-by: pranjalssh <pranjalssh@fb.com>
 2026-07-24 16:19:08 -07:00
Shiyan DengLianmin ZhengLucia Fang
 		962c076934 Decode input_audio media containers with PyAV & Update memory profiler (#31832) 
Signed-off-by: Shiyan Deng <dsy842974287@meta.com>
Signed-off-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: Lucia Fang <116399278+luccafong@users.noreply.github.com>
 2026-07-24 16:18:45 -07:00
Zhihao Wang 14d6e1d3b1 [RL] Support FlashInfer TRT-LLM NVFP4 MoE in the RL weight checker (#31085) 
Signed-off-by: zhihaow6 <zhihaow6@illinois.edu>
 2026-07-24 16:01:54 -07:00
Zhihao Wang 0a212c6119 [RL] DSV4: add env to quantize SWA KV cache from bf16-rounded values (#31086) 
Signed-off-by: zhihaow6 <zhihaow6@illinois.edu>
 2026-07-24 15:52:00 -07:00
Qiaolin Yu 4ececf2b1d [chore] Add verbose flag to twine upload command (#32349) 2026-07-24 14:54:07 -07:00
Zhihao Wang f7986c8603 [RL] DSV4: dispatch indexer topk_transform_512 through DSATopKBackend (#31087) 
Signed-off-by: zhihaow6 <zhihaow6@illinois.edu>
 2026-07-24 14:45:44 -07:00
cctry 8727d105db Add prefill and decode load counters to LoadSnapshot (#32245) 2026-07-24 14:04:54 -07:00
Baizhou Zhang f15b43242b Bump sgl-deep-gemm to 0.1.5 (#32345) 2026-07-24 14:03:38 -07:00
Rain Jiang be7cc17307 sglang rust server environ fsm error id gen (#32240) 2026-07-24 20:55:10 +00:00
Qiaolin Yu 82fe0f041a Fix stale flashinfer-MLA fallback poisoning spec verify capture (trtllm_mla + tc_piecewise) (#32288) 2026-07-24 13:24:26 -07:00