minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
18,363 Commits 2 Branches 0 Tags
4fb9b5b5bad41ac7de5586ec812c421fe3aa2b8b
Commit Graph
18363 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
qyb233 4fb9b5b5ba feat(hicache): support NPU Mamba states with FIA and async IO (#32500) 2026-09-17 10:12:26 +08:00
Shuwen Wang 4c85172f3a [HiCache] Back up MXFP8 KV scales in the host pool (#39089) 2026-09-17 09:40:26 +08:00
William Hu 0443e3179f fix(kda_prefill): fence shared writes before async proxy reads (#39124) 2026-09-16 18:37:19 -07:00
Xinyi Song 7780882f18 [AMD][bugfix] Fix dsv4 server launch (#39875) 2026-09-16 18:28:02 -07:00
Liangsheng YinBBufYuhao Yang
 		c89c63fa38 dsv4.1: vision tower and image preprocessing (#39668) 
Co-authored-by: BBuf <1182563586@qq.com>
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com>
 2026-09-16 18:11:16 -07:00
Liangsheng Yin 408d2334c3 dsv4.1: mHC computation and compensated projections (#39664) 
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
  Co-authored-by: BBuf <1182563586@qq.com>
  Co-authored-by: zijiexia <37504505+zijiexia@users.noreply.github.com>
 2026-09-16 17:11:55 -07:00
Liangsheng YinBBufYuhao Yang
 		46ae84df15 dsv4.1: Hopper FP8 matmul kernels and tuning (#39657) 
Co-authored-by: BBuf <1182563586@qq.com>
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com>
 2026-09-16 16:54:19 -07:00
Liangsheng Yin dc067c7d8c [Fix] Allow closed object schemas in Outlines prevalidation (#39869) 2026-09-16 16:42:17 -07:00
Chunan Zeng d78b35076a [CI] Fix missing elfutils headers in CUDA 13.4 DeepGEMM build (#39855) 2026-09-16 16:41:29 -07:00
Liangsheng YinDarkSharpnessXiaoyu Zhang
 		c2443458e1 dsv4.1: RoPE and FP4 packing kernels (#39656) 
Co-authored-by: DarkSharpness <2040703891@qq.com>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
Co-authored-by: BBuf <1182563586@qq.com>
 2026-09-16 16:17:12 -07:00
William Arnoldandishandhanani f45aad44bd [gRPC] Expose native pause status (#37488) 
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
 2026-09-16 15:16:52 -07:00
Liangsheng YinDarkSharpnessXiaoyu Zhang
 		35b7589e1a dsv4.1: candidate indexer library (#39671) 
Co-authored-by: DarkSharpness <76582120+DarkSharpness@users.noreply.github.com>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
 2026-09-16 15:15:18 -07:00
Liangsheng YinBBufClaude Opus 5DarkSharpness
 		13d593b6cf dsv4.1: compression, KV I/O, and metadata kernels (#39652) 
Co-authored-by: BBuf <1182563586@qq.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: DarkSharpness <76582120+DarkSharpness@users.noreply.github.com>
 2026-09-16 13:54:07 -07:00
Justin Perlman 869674b3a7 [Fix] Prevalidate JSON Schema support per grammar backend (#37839) 2026-09-16 13:10:15 -07:00
Qiaolin Yu 43390f63f5 [qwen 3.8 next] change the testing model in test_qwen4_exp_models.py (#39662) 2026-09-16 12:47:45 -07:00
Kangyan-ZhouKangyan ZhouClaude Opus 5
 		b02e16a895 [Router] Fleet-wide sampling contract 3/3: splice injection without re-serializing (#39002) 
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 09:59:48 -07:00
Kangyan-ZhouKangyan ZhouClaude Opus 5
 		3e03879f68 [Router] Drain readiness before SIGTERM shutdown so k8s deregisters the pod first (#39016) 
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 09:59:26 -07:00
Kan Wu 279339f113 [sgl-router] Share model-file discovery for chat formatters (#39485) 2026-09-16 09:36:11 -07:00
Xinyuan Tong f0bf652534 Add Ling-3.0-flash-VL model support (#38526) 2026-09-17 00:19:26 +08:00
cctry 1b78083b42 [PD] Add optional KV transfer checksums (#39500) 2026-09-16 23:53:30 +08:00
AMD-yanfeiwangandDuyi-Wang a813224e78 [ROCm][DSV4] Enable breakable CUDA graph prefill (#37810) 
Co-authored-by: Duyi-Wang <duyi.wang@amd.com>
 2026-09-16 08:52:11 -07:00
Kan WuandClaude Fable 5.1 ad94978adf [sgl-router] Rename chat encoder to chat formatter (#39459) 
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-16 08:46:11 -07:00
Ke Bao 5aaa18207c Revert "[CI] Add e2e test for dp-attention local control broadcast" (#39828) 2026-09-16 23:41:50 +08:00
Kan WuandClaude Fable 5.1 e41026f434 [sgl-router] Forward input_ids only for string content; count tokenize errors only when forwardable (#39458) 
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-16 22:49:41 +08:00
ymren cc171fbad0 feat: support custom OTLP trace service name (#35802) 2026-09-16 22:37:42 +08:00
pllimax 00a9a81b67 [NPU][CI] Fail fast and speed up long-running qwen3.6 accuracy cases (#39813) 2026-09-16 20:20:37 +08:00
ShunkangzandZhangheng e7f7447333 [kv-shard 2/4] Sharded pools (#37615) 
Co-authored-by: Zhangheng <hzh0425@apache.org>
 2026-09-16 19:22:23 +08:00
Sam ShleiferSam ShleiferClaude Fable 5.1
 		76e06febab [CI] Add e2e test for dp-attention local control broadcast (#39437) 
Co-authored-by: Sam Shleifer <sam@thinkingmachines.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-16 18:11:40 +08:00
Aurick QiaoandAurick Qiao 7b7620774c Fix device context during NIXL backend initialization (#38774) 
Co-authored-by: Aurick Qiao <6137920+aurickq@users.noreply.github.com>
 2026-09-16 17:59:24 +08:00
inkcherryandDuyi-Wang a3bf25dc62 [AMD] Clamp MORI intranode grid GPUs (#39763) 
Co-authored-by: Duyi-Wang <duyi.wang@amd.com>
 2026-09-16 02:13:59 -07:00
+4
Liangsheng YinCheng WanClaude Opus 5CursorDarkSharpnessDarkSharpnessKe BaoKhoa PhamXiaoyu ZhangXiaoyu ZhangYuhao YangYuwei AnZhichen ZengZiyi Xu
 		7d5696b3a1 dsv4.1: communication kernels and wrappers (#39653) 
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: DarkSharpness <2040703891@qq.com>
Co-authored-by: DarkSharpness <76582120+DarkSharpness@users.noreply.github.com>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
Co-authored-by: Khoa Pham <khoa.pham@radixark.ai>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
Co-authored-by: Xiaoyu Zhang <xiaoyu.zhang@radixark.ai>
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: Yuwei An <ayw.sirius19@gmail.com>
Co-authored-by: Zhichen Zeng <zczeng@uw.edu>
Co-authored-by: Ziyi Xu <ziyi.xu@radixark.ai>
 2026-09-16 02:09:40 -07:00
Bingxu Chen 5a0c1e21e9 [AMD] Preserve deterministic inference when Lean Attention is enabled (#37740) 2026-09-16 02:09:08 -07:00
Bingxu Chen 2cbfaefbf9 [AMD] Avoid the FP8 wo_a path when the weight is BF16 (#38453) 
Signed-off-by: bingxche <bingxche@amd.com>
 2026-09-16 01:57:15 -07:00
Jensenandronnie_zheng 5212797880 [NPU] fix npu docker workspace directory (#39732) 
Signed-off-by: xiedeyantu <czjourney@163.com>
Co-authored-by: ronnie_zheng <zl19940307@163.com>
 2026-09-16 11:51:38 +03:00
Liangsheng YinandBBuf fc5a979f21 [misc] Merge FlashInfer autotune caches across spec workers, pad MXFP4 TP shards, drop dead ngram attrs (#39678) 
Co-authored-by: BBuf <1182563586@qq.com>
 2026-09-16 01:50:29 -07:00
vstone-w 0e528dc9ff bugfix:fix unifiedcache c128 radix cache management (#39426) 2026-09-16 16:47:42 +08:00
Kangyan-ZhouKangyan ZhouClaude Opus 5
 		8baeded6f3 [Router] Pin to the prefix owner when the whole fleet is queueing (--saturation-queue-floor) (#39169) 
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:16:24 +08:00
Alison Shao ad28b91fae ci: fix always-failing coverage job, add by-GPU-count view (#39697) 2026-09-16 01:13:08 -07:00
Void 9c8d4641ff [Fix] Fix GLM5 mHC PP forward (#39720) 2026-09-16 00:32:53 -07:00
Kangyan-ZhouKangyan ZhouClaude Opus 5
 		54f0d72adf [Router] Fleet-wide sampling contract 2/3: enforce and inject per request (#39001) 
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 00:29:33 -07:00
Kangyan-ZhouKangyan ZhouClaude Opus 5
 		7835f1de9a [Router] Add the shutdown-drain configuration surface (#39015) 
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 00:26:50 -07:00
Jensen 5beb2fd552 [NPU] Avoid device synchronization in Ascend sampling (#39404) 2026-09-16 10:15:16 +03:00
Kangyan-ZhouKangyan ZhouClaude Opus 5
 		e2d56bbbfc [Router] Bound the e2e worker memory budget so prefill graph capture stops OOMing (#39713) 
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 00:13:09 -07:00
Kangyan-ZhouKangyan ZhouClaude Opus 5
 		d634320e48 [Router] Count open HTTP exchanges until their response body finishes (#39014) 
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 00:04:38 -07:00
+4
Liangsheng YinBBufCheng WanClaude Opus 5CursorDarkSharpnessDarkSharpnessKe BaoKhoa PhamXiaoyu ZhangYuhao YangYuwei AnZhichen ZengZiyi Xu
 		faaff1eca8 dsv4.1: Top-k kernels and candidate selection (#39648) 
Co-authored-by: BBuf <1182563586@qq.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: DarkSharpness <2040703891@qq.com>
Co-authored-by: DarkSharpness <76582120+DarkSharpness@users.noreply.github.com>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
Co-authored-by: Khoa Pham <khoa.pham@radixark.ai>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
Co-authored-by: Xiaoyu Zhang <xiaoyu.zhang@radixark.ai>
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: Yuwei An <ayw.sirius19@gmail.com>
Co-authored-by: Zhichen Zeng <zczeng@uw.edu>
Co-authored-by: Ziyi Xu <ziyi.xu@radixark.ai>
 2026-09-15 23:54:35 -07:00
Hank Han a64be2e430 [Kernel] Add H20 block-FP8 MoE configs for GLM-5.3-Flash EP4/EP8 (#38913) 2026-09-15 23:42:18 -07:00
Shuwen WangandZhangheng 6b33338242 [HiCache] Forward prefix metadata to v2 storage calls (#39567) 
Co-authored-by: Zhangheng <hzh0425@apache.org>
 2026-09-16 14:05:42 +08:00
Liangsheng Yin 91f691c490 dsv4.1: standalone kernels and Python wrappers (#39646) 2026-09-15 22:31:18 -07:00
Kangyan-ZhouKangyan ZhouClaude Opus 5
 		afde31a2f5 [router] Speak cleartext h2c on both edges: serve it inbound, forward it outbound (#39006) 
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-15 22:30:31 -07:00
Kangyan-ZhouKangyan ZhouClaude Opus 5
 		c1f5b4736a [Router] Add --worker-queue-limit: stop sending cache-affinity traffic to a queueing worker (#39168) 
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 13:24:48 +08:00