minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
11,712 Commits 2 Branches 0 Tags
db7a751d4869b516ab3510919a597bf1815d534f
Commit Graph
7779 Commits
Author SHA1 Message Date
Liangsheng Yin db7a751d48 refactor: extract FanOutCommunicator and use declarative spec table (#22967) 2026-04-16 15:37:19 -07:00
mqhc2020andHubert Lu 52f0b86f5d [AMD] Qwen3.5 MXFP4 breaks after shared expert fusion is enabled (#22948) 
Co-authored-by: Hubert Lu <55214931+hubertlu-tw@users.noreply.github.com>
 2026-04-16 15:25:33 -07:00
Liangsheng Yin c83ef4fdb6 use envs in server_args (#22994) 2026-04-16 15:01:33 -07:00
Xinyu Zhangandxyuzh c0172aef6e [Ray] Bind scheduler actors to GPU-local NUMA node (#22989) 
Co-authored-by: xyuzh <xyuzh@users.noreply.github.com>
 2026-04-16 14:52:15 -07:00
Xinyu Zhang d430034bde [Ray] Support multi-replica serving by making scheduler actor names unique (#22917) 2026-04-16 14:51:01 -07:00
Qiaolin Yu a87806a65f [misc] refine outdated comments for chain-style multi-layer MTP (#22996) 2026-04-16 14:49:43 -07:00
ybyang 41258f874d [PD]feat(bench): add --fake-prefill flag for decode-only stress testing (#22973) 2026-04-16 13:57:55 -07:00
Yuhao Yang 9da998a882 [diffusion] feat: disaggregated diffusion (#21701) 2026-04-16 23:51:32 +08:00
Liangsheng Yin 62309f09db fix(loads): preserve include filtering after watching mode switch (#22959) 2026-04-16 03:04:53 -07:00
ybyang 03fef357a6 fix(loads): switch get_loads_communicator to watching mode (#22919) 2026-04-16 02:12:22 -07:00
ybyang fbd6dc3565 fix: normalize tool message content for GLM5.1 chat template (#22595) 2026-04-16 16:48:38 +08:00
Aleksi Vesanto aaa682346e [diffusion] model: Properly validate device for Mistral 3 attention (#22690) 2026-04-16 00:29:23 -07:00
Lianmin Zheng 35da90cb76 [misc] Configure logging before ServerArgs.__post_init__ (#22926) 2026-04-15 23:53:15 -07:00
yuefeng Wuandgemini-code-assist[bot] 65bc839a5f [Fix] eagle/eagle3 speculative decoding conflicts with xgrammar in NPU (#20989) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-04-15 23:34:23 -07:00
Bi Xue c43716a357 [sgl] provide an option to send control req to all dp ranks rank0 (#22758) 2026-04-16 14:24:26 +08:00
Byron Hsu 3600465e81 [Bug Fix] Remove follow_bootstrap_room fast path in PD disaggregation DP rank resolution (#22901) 2026-04-15 22:53:29 -07:00
LHXuuu e7ad7c587a [EPD][VLM] Support Kimi VL EPD (#22490) 
Signed-off-by: LHXuuu <xulianhao.xlh@antgroup.com>
 2026-04-16 12:40:02 +08:00
CYYYC0310andcyy 58c6b871b2 Remove compatibility restriction between Pipeline Parallelism and Mixed Chunked Prefill (#22920) 
Co-authored-by: cyy <cy02433585@alibaba-inc.com>
 2026-04-16 11:25:31 +08:00
Xinyuan Tong 34fef07a15 Upgrade transformers to 5.5.3 and refactor hf_transformers_utils into subpackage (#21569) 2026-04-15 20:03:44 -07:00
JINZandZhangheng 14e122cdee [BugFix][RadixTree]:Fix stale eviction assertion in HiMambaRadixCache host eviction path (#22592) 
Co-authored-by: Zhangheng <hzh0425@apache.org>
 2026-04-16 10:49:30 +08:00
Yuhao Yang b8794baa6d [Step3p5] Optimize allreduce in MoE layers (#22773) 2026-04-16 09:33:12 +08:00
Liangsheng Yin a4cf2ea128 streaming session: spec v2 bonus accounting + comprehensive test matrix (#22651) 2026-04-15 17:12:41 -07:00
Xinyu Zhang e8c6e5466c [Ray] Auto-create placement group in RayEngine when none is detected (#22898) 2026-04-15 15:17:52 -07:00
Qiaolin Yu 0b1b07db72 [misc] fix ray folder lint (#22905) 2026-04-15 15:08:18 -07:00
Liangsheng Yin f9792166c3 trim_overshoot: cap swa_evicted_seqlen + unit test (#22900) 2026-04-15 15:05:35 -07:00
Xinyu Zhangandxyuzh 13a2cd748d [Ray] Add data parallel (DP) and DP attention support to RayEngine (#21887) 
Co-authored-by: xyuzh <xyuzh@users.noreply.github.com>
 2026-04-15 15:00:48 -07:00
Sundara Raman Ramachandran 4927975427 [Score API] Add return_pooled_hidden_states to Scoring API for SequenceClassification / RewardModel (#22427) 2026-04-15 14:58:56 -07:00
Lee Nau 4e480d5785 Harden FlashInfer FP4 imports in standard dispatcher (#21776) 2026-04-15 14:54:49 -07:00
Liangsheng Yin efc267ca29 streaming session: trim spec v2 overshoot in cache_finished_req (#22897) 2026-04-15 14:15:46 -07:00
Lianmin Zheng 43925d179d [Speculative] Fix Eagle3/DFLASH aux hidden state capture during CUDA graph init (#22836) 2026-04-15 14:04:54 -07:00
Kurt Shuster 32d9fe5a32 [lora] Speedup triton backend sgemm calls with better grid (#22386) 2026-04-15 13:47:07 -07:00
Jimmy Shong 28e915b474 [Bugfix] Preserve auto-detected quant_config for GLM NextN draft model (#22823) 2026-04-15 13:25:36 -07:00
Yuhao Yang 8686f42acb [VLM] Enable per-image ViT cache and avoid TP CUDA context creation for Kimi-K2.5 (#22858) 2026-04-16 01:14:24 +08:00
huangtingweiandhzh0425 7d7fdc1309 [HiCache]Fix CP support for hybrid model (#22782) 
Co-authored-by: hzh0425 <hzh0425@apache.org>
 2026-04-15 23:50:29 +08:00
Xiaoyu Zhang 695ab705cb [diffusion] quant: update modelopt quantization docs and CI coverage (#22772) 2026-04-15 21:30:28 +08:00
Mick 80718492dd [diffusion] CI: reset thresholds (#22854) 2026-04-15 21:11:00 +08:00
Zhangheng 0a5c9728a1 [HiSparse][BugFix]: Fix the memory leak issue during health checks. (#22882) 2026-04-15 19:49:54 +08:00
Liangsheng Yin ce31934ca8 Streaming session: fix retract tail leak via _free_tail (#22862) 2026-04-15 01:44:27 -07:00
huangtingweiandhzh0425 3511c2deb4 [HiCache] Fix memory host free logic when share_indices_with_anchor enabled (#22767) 
Co-authored-by: hzh0425 <hzh0425@apache.org>
 2026-04-15 16:31:18 +08:00
Liangsheng Yin aa78564e1a Refactor streaming session abort handling (#22790) 2026-04-15 00:13:05 -07:00
Hubert LuandHaiShaw b2af34be54 [AMD] Optimize _append_shared_to_topk_output by a single fused Triton kernel for Qwen3.5 (#22844) 
Co-authored-by: HaiShaw <hixiao@gmail.com>
 2026-04-14 23:50:32 -07:00
Mick e95c2e73bd [diffusion] CI: refactor diffusion ci and reduce redundancy (#22810) 2026-04-15 10:12:29 +08:00
Kangrui DuMikukuOvOClaude Opus 4.6
 		47ac830c07 [diffusion] rl: support standalone rollout api, denoising environment backpass and sp-aligned log-prob for T2I post-training (#22604) 
Co-authored-by: MikukuOvO <mikukuovo@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-15 10:10:38 +08:00
Lianmin Zheng adb310b976 Cleanup server_args.py and minor code tidying (#22820) 2026-04-14 18:52:41 -07:00
Chen, ZhentaoChen, Toddjacky.cheng
 		ea05ea5abe [AMD] Enable share expert fusion with router experts for Qwen3.5 BF16 & FP8 (#20736) 
Co-authored-by: Chen, Todd <zhenchen@amd.com>
Co-authored-by: jacky.cheng <yichiche@amd.com>
 2026-04-14 18:52:36 -07:00
Piotr Mazurek 46c8a597ef [VLM] fix LFM2-VL offline inference and GPU JPEG decode (#22448) 2026-04-15 09:13:25 +08:00
Alex NailsClaude Opus 4.6gemini-code-assist[bot]
 		8092431316 [serving] replace O(n²) stream_buffer string concat with integer offset (#22606) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-04-14 14:48:44 -07:00
Liangsheng Yin 36891ab514 Rename _alive_streaming_session_count; use _is_streaming helper (#22755) 2026-04-14 13:26:03 -07:00
Liangsheng Yin 0cb7295698 Fix streaming session busy-check double-counting via active_pool_idxs (#22753) 2026-04-14 13:11:06 -07:00
mingyue300 b4616dcbf5 [BugFix] Fix EAGLE speculative decoding missing grammar-based finish … (#21723) 2026-04-14 12:43:50 -07:00