minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
18,341 Commits 2 Branches 0 Tags
5aaa18207cbac182ad607021de593137791ff86f
Commit Graph
12737 Commits
Author SHA1 Message Date
ymren cc171fbad0 feat: support custom OTLP trace service name (#35802) 2026-09-16 22:37:42 +08:00
pllimax 00a9a81b67 [NPU][CI] Fail fast and speed up long-running qwen3.6 accuracy cases (#39813) 2026-09-16 20:20:37 +08:00
ShunkangzandZhangheng e7f7447333 [kv-shard 2/4] Sharded pools (#37615) 
Co-authored-by: Zhangheng <hzh0425@apache.org>
 2026-09-16 19:22:23 +08:00
Aurick QiaoandAurick Qiao 7b7620774c Fix device context during NIXL backend initialization (#38774) 
Co-authored-by: Aurick Qiao <6137920+aurickq@users.noreply.github.com>
 2026-09-16 17:59:24 +08:00
inkcherryandDuyi-Wang a3bf25dc62 [AMD] Clamp MORI intranode grid GPUs (#39763) 
Co-authored-by: Duyi-Wang <duyi.wang@amd.com>
 2026-09-16 02:13:59 -07:00
+4
Liangsheng YinCheng WanClaude Opus 5CursorDarkSharpnessDarkSharpnessKe BaoKhoa PhamXiaoyu ZhangXiaoyu ZhangYuhao YangYuwei AnZhichen ZengZiyi Xu
 		7d5696b3a1 dsv4.1: communication kernels and wrappers (#39653) 
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: DarkSharpness <2040703891@qq.com>
Co-authored-by: DarkSharpness <76582120+DarkSharpness@users.noreply.github.com>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
Co-authored-by: Khoa Pham <khoa.pham@radixark.ai>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
Co-authored-by: Xiaoyu Zhang <xiaoyu.zhang@radixark.ai>
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: Yuwei An <ayw.sirius19@gmail.com>
Co-authored-by: Zhichen Zeng <zczeng@uw.edu>
Co-authored-by: Ziyi Xu <ziyi.xu@radixark.ai>
 2026-09-16 02:09:40 -07:00
Bingxu Chen 5a0c1e21e9 [AMD] Preserve deterministic inference when Lean Attention is enabled (#37740) 2026-09-16 02:09:08 -07:00
Bingxu Chen 2cbfaefbf9 [AMD] Avoid the FP8 wo_a path when the weight is BF16 (#38453) 
Signed-off-by: bingxche <bingxche@amd.com>
 2026-09-16 01:57:15 -07:00
Liangsheng YinandBBuf fc5a979f21 [misc] Merge FlashInfer autotune caches across spec workers, pad MXFP4 TP shards, drop dead ngram attrs (#39678) 
Co-authored-by: BBuf <1182563586@qq.com>
 2026-09-16 01:50:29 -07:00
vstone-w 0e528dc9ff bugfix:fix unifiedcache c128 radix cache management (#39426) 2026-09-16 16:47:42 +08:00
Void 9c8d4641ff [Fix] Fix GLM5 mHC PP forward (#39720) 2026-09-16 00:32:53 -07:00
Jensen 5beb2fd552 [NPU] Avoid device synchronization in Ascend sampling (#39404) 2026-09-16 10:15:16 +03:00
+4
Liangsheng YinBBufCheng WanClaude Opus 5CursorDarkSharpnessDarkSharpnessKe BaoKhoa PhamXiaoyu ZhangYuhao YangYuwei AnZhichen ZengZiyi Xu
 		faaff1eca8 dsv4.1: Top-k kernels and candidate selection (#39648) 
Co-authored-by: BBuf <1182563586@qq.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: DarkSharpness <2040703891@qq.com>
Co-authored-by: DarkSharpness <76582120+DarkSharpness@users.noreply.github.com>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
Co-authored-by: Khoa Pham <khoa.pham@radixark.ai>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
Co-authored-by: Xiaoyu Zhang <xiaoyu.zhang@radixark.ai>
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: Yuwei An <ayw.sirius19@gmail.com>
Co-authored-by: Zhichen Zeng <zczeng@uw.edu>
Co-authored-by: Ziyi Xu <ziyi.xu@radixark.ai>
 2026-09-15 23:54:35 -07:00
Hank Han a64be2e430 [Kernel] Add H20 block-FP8 MoE configs for GLM-5.3-Flash EP4/EP8 (#38913) 2026-09-15 23:42:18 -07:00
Shuwen WangandZhangheng 6b33338242 [HiCache] Forward prefix metadata to v2 storage calls (#39567) 
Co-authored-by: Zhangheng <hzh0425@apache.org>
 2026-09-16 14:05:42 +08:00
Liangsheng Yin 91f691c490 dsv4.1: standalone kernels and Python wrappers (#39646) 2026-09-15 22:31:18 -07:00
Shangming CaiandClaude Opus 5 3f8eb35ead [PD] Do not admit intake-rejected requests to a PD handoff (#38935) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 12:44:49 +08:00
Dayananda VandClaude Opus 5 100e1cd0d9 [Fix] Spawn, don't fork, the benchmark server process (#34712) 
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-16 12:44:09 +08:00
Xinguo ZhuandAKatydid 4678536df6 [CPU] Add fp8_per_tensor_scaled_mm_cpu kernel (#32618) 
Co-authored-by: AKatydid <xinguojoe@gmail.com>
 2026-09-16 12:31:05 +08:00
Xiaoyu Zhang 444b29c932 [Diffusion] Clean up obsolete worker plumbing, dead helpers, and tests (#39293) 2026-09-16 12:11:45 +08:00
Xinyi Song 954bb6804a [AMD][bugfix] Fix DSV4 MTP crash (#39547) 2026-09-15 21:10:23 -07:00
Xiaoyu Zhang 537ac52c5b [SRT] Clean up no-op compiler pass, dead helpers, and migration tests (#39295) 2026-09-16 12:04:21 +08:00
Zhang, Jiejing 6331e43081 [AMD] Prefer HIP Top-K for GLM-5.x on ROCm (#39631) 2026-09-15 20:55:57 -07:00
chx96642264 5f6dd44edc [NPU] [bugfix] Fix undefined require_attn_tp_allgather (#39561) 2026-09-16 11:02:40 +08:00
Mohammad Miadh AngkadandMohammad Angkad 935cbf24ec Accept MXFP8 dispatch in FlashInfer A2A TRT-LLM MoE (#39613) 
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
 2026-09-15 19:23:15 -07:00
Rahul Vijayaraghavan d4ad368ed9 [Intel XPU] Enable fused_moe_triton tuning on XPU and add tuned DeepSeek-OCR-2 configs (#28723) 2026-09-16 10:20:22 +08:00
Cao E f11cd8ab0e [XPU] Use torch scaled_mm for XPU block FP8 linear (#35605) 2026-09-16 10:10:31 +08:00
Yuxingwang-intelandMa Mingfei 2cb51f5d22 [CPU] [Diffusion] Add fused scale-shift and norm kernels for CPU (#33452) 
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
 2026-09-16 10:01:29 +08:00
Zhang, Jiejingandfanxingran f920be4b09 [AMD] GLM-5.2 NextN: cast draft fused MoE to per-channel FP8 (#39155) 
Co-authored-by: fanxingran <xingran.fan@amd.com>
 2026-09-15 18:56:48 -07:00
xiaobochen-amdJohnQinAMDJiejing Zhang
 		7eedd57ab0 [ROCm] Fix EAGLE spec-decode verify silently sampling greedy on HIP (#37134) 
Co-authored-by: JohnQinAMD <yanyuan.qin@amd.com>
Co-authored-by: Jiejing Zhang <jiejing.zhang@amd.com>
 2026-09-15 18:49:49 -07:00
Jinyan Yi dbd7281b06 fix(npu): fix hybrid KV transfer with PP prefill in PD disaggregation (#38402) 2026-09-16 09:10:17 +08:00
Zhiqiang Xie 4e9e407d37 [HiCache] Label radix-cache metrics per rank and split the "shrunk" prefetch reason (#39280) 2026-09-15 17:54:04 -07:00
Siyuan ChenandYuwei An c9a8fba991 [DSV4][BCG] Optimize the heavy memory use of C4 Indexer when BCG is enabled (#36534) 
Co-authored-by: Yuwei An <ayw.sirius19@gmail.com>
 2026-09-15 17:23:11 -07:00
Liangsheng Yin 0dabef3d30 [misc] Fix tool-call index, graph padded-row count, and prefill-graph input_embeds refresh (#39574) 2026-09-15 16:37:22 -07:00
Lianmin Zheng b803cfa0c4 Add external multimodal processors to the Rust frontend (#39329) 2026-09-15 15:38:47 -07:00
Yonghao ZhuangyhzhuangCheng Wan
 		2929a39927 Use a shared byte budget for unified hybrid-SWA memory (#36729) 
Co-authored-by: yhzhuang <yhzhuang@fb.com>
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
 2026-09-15 15:27:00 -07:00
Lianmin ZhengJinghui ZhangLucia Fang
 		4da5599e93 Fix first-token metadata and reused attention-layer indexing (#39328) 
Co-authored-by: Jinghui Zhang <jinghui@meta.com>
Co-authored-by: Lucia Fang <116399278+luccafong@users.noreply.github.com>
 2026-09-15 15:22:26 -07:00
Yash Agarwalkpham-sglClaude Fable 5.1
 		08b1922405 fix(gemma4): set lm_head_is_tied for Gemma4UnifiedForConditionalGeneration (#35809) 
Co-authored-by: kpham-sgl <khoa.pham@radixark.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-15 15:16:39 -07:00
Liangsheng Yin 87c9f78e99 [Fix][PD] Give prefill and decode their own RDMA NICs in disaggregation tests (#39584) 2026-09-15 15:07:42 -07:00
Cheng Wan 406c9c71d8 [Feature] Support BF16 and batch-invariant inference with DeepEP v2 (#38160) 2026-09-15 14:30:50 -07:00
Liangsheng Yin d58342deab [Fix] Release NCCL on scheduler exit and let the ASGI server own shutdown (#39560) 2026-09-15 13:39:15 -07:00
jasonjk-parkandjasonjk 073fdf97cc Generalize auxiliary outputs (#39164) 
Co-authored-by: jasonjk <jasonjk@twshared0085.36.lco2.facebook.com>
 2026-09-15 12:05:13 -07:00
Zhiqiang Xie 49f540f3c0 [HiCache] Optimize buffer-mode storage existence bookkeeping (#39480) 2026-09-15 10:49:58 -07:00
Zhiqiang Xie 7f5dd19256 [HiCache] Rework the buffer-mode storage prefetch pipeline and retry bookkeeping (#39283) 2026-09-15 10:48:49 -07:00
pllimax 1aeeb25e86 [NPU][CI] Scope NPU nightly artifact dirs by image; fix stale glm5_2 case (#39585) 2026-09-15 22:55:53 +08:00
Siyuan ChenandBJWang-ant 832ec39cc0 [Intra-node PD][DSV4] Pack all layers into one batch for INTRA_NODE_NVLINK path (#38984) 
Co-authored-by: BJWang-ant <wangbingjia.wbj@ant-intl.com>
 2026-09-15 21:08:33 +08:00
Yoray ZackandPo-Han Huang 1895cabfa8 Fix mooncake scale joiner groups (#38503) 
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
 2026-09-15 19:07:25 +08:00
Yuxuan ZhangandXinyuan Tong 17ba2c2e7c Support non-strict GLM47 tool calls with EBNF constraints (#38890) 
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2026-09-15 16:41:48 +08:00
Qiaolin Yu 2c0a70960c [qwen 3.8 next] reuse old cuda stream instead of endlessly creating streams (#39474) 2026-09-15 01:36:14 -07:00
Carrie ChenBrayden ZhongBrayden Zhong
 		4f52c9948b keeping router GEMM in fp32 for deterministic inference (DeepSeek V3/V4) (#38176) 
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
 2026-09-15 16:16:09 +08:00