minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
17,790 Commits 2 Branches 0 Tags
85da5457deec247d333380a86e98f3fe95ef2e35
Commit Graph
17790 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Xiaoyu Zhang 85da5457de [diffusion] auto-keep video DiT resident on high-memory GPUs (#38001) 2026-09-05 08:54:09 +08:00
Clintclintg6Thomas WangHAIkk
 		3c2724c48d [AMD] Optimize Kimi-K3 Triton MLA prefill on gfx950 (#35770) 
Co-authored-by: clintg6 <7388379+clintg6@users.noreply.github.com>
Co-authored-by: Thomas Wang <thomawan@amd.com>
Co-authored-by: HAI <hixiao@gmail.com>
Co-authored-by: kk <43161300+kkHuang-amd@users.noreply.github.com>
 2026-09-04 17:44:30 -07:00
zijiexiaandClaude Opus 5 3b64169f9d [Cookbook] Kimi-K3: add measured B300 1x8 Unified 8k/1k speed numbers (#37878) 
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-04 17:44:00 -07:00
Alex Nails a44bb397a9 [Perf] Vectorize alloc_extend_naive to remove the per-request Python loop (#37938) 2026-09-04 17:26:25 -07:00
Lianmin Zheng 613d87becd [Memory] Reuse output storage across full prefill CUDA graphs (#38038) 2026-09-04 17:23:31 -07:00
YAMY 65f7957142 fix: gather CP-sharded tokens before TP-sharded dense MLP under prefill CP (#38078) 2026-09-04 17:14:31 -07:00
Amy Chang 8a98f11078 [feature] Add response-level input/output token ids to chat completions via SglExt (#34488) 2026-09-04 16:45:13 -07:00
PengYuanandMick Qian 9c254df8cd [diffusion] fix: preserve mapped courier tensor lifetime (#37965) 
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
 2026-09-05 07:41:47 +08:00
YAMY db89f639ef [GDN] Amortize ReplaySSM checkpoint materialization (#35544) 2026-09-04 15:13:20 -07:00
Hanming Lu e3eeabbbfa [Profiler] Add SGLANG_PROFILE_BY_STAGE_DECODE_MIN_BS to defer the decode-stage capture (#38067) 2026-09-04 15:12:55 -07:00
cctry 0eff0f7460 Add num_prealloc_ready_tokens to decode load snapshot (#38065) 2026-09-04 15:05:57 -07:00
Martin HickeymottopanikeikualpXinyuan Tong
 		320bdd1ee2 [Docs] Document --retraction-policy, --return-hidden-states-mode, --language-model-only (#37989) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: mottopanikeiku <fcetin@hawk.iit.edu>
Co-authored-by: alp <falpercetin@gmail.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2026-09-05 05:26:28 +08:00
alpandXinyuan Tong 3e873c2110 [Docs] Clarify OpenAI chat template defaults (#32172) 
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2026-09-05 05:22:05 +08:00
Yongji WuandLianmin Zheng fbf8f1dbf6 [Fix] Coordinate FullCG prefix variants across DP ranks (#37888) 
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
 2026-09-04 11:29:30 -07:00
Faradawn Yang c8ba8996c4 Update DeepSeek-V4 Pro for B200 FP4 agentic HiCache DSpark (#38026) 2026-09-04 11:09:24 -07:00
Jialin OuyangandPranjal Shankhdhar 010dc955be [Metrics] Export scheduler stage wall time (#37636) 
Co-authored-by: Pranjal Shankhdhar <pranjal.ssh@gmail.com>
 2026-09-04 10:45:53 -07:00
YAMY 07199fa220 [Performance] Optimize Qwen3.5 GDN prefill projection layouts (#36267) 2026-09-04 10:32:44 -07:00
Cherry_mingEven Zhousglang-npu-bot
 		55509b3f42 [NPU] Optimize the execution logic of NPU pr‑test tasks (#38015) 
Co-authored-by: Even Zhou <even.y.zhou@outlook.com>
Co-authored-by: sglang-npu-bot <sglangnpu@163.com>
 2026-09-05 00:19:50 +08:00
Shuwen WangandClaude Fable 5.1 01e19ddf55 chore: add code owners for the Rust radix tree core (#38021) 
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-04 23:40:34 +08:00
Shuwen WangandClaude Fable 5.1 5cd2a7661d fix: reformat cosmos3_edge.py to satisfy ruff-format (#38023) 
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-04 23:39:36 +08:00
huangtingweiandZhiqiang Xie 0d0e2f92be [HiCache] Buffer mode support sidecar pool (#37424) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2026-09-04 23:24:18 +08:00
Jialin OuyangLianmin Zhengpranjalssh
 		8b1d8c1703 [Metrics] Add rolling scheduler utilization counters (#37461) 
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: pranjalssh <pranjalssh@fb.com>
 2026-09-04 08:06:17 -07:00
Zhylko DimaDawid MajchrowskiKedi WuKedi WuJoel Gustafson
 		4349538c02 [model] add cosmos3 reasoner to llm only inference (#33572) 
Signed-off-by: joeltg <joel@reflection.ai>
Signed-off-by: Joe Rowell <joe@poolside.ai>
Co-authored-by: Dawid Majchrowski <dmajchrowski@nvidia.com>
Co-authored-by: Kedi Wu <kediw@nvidia.com>
Co-authored-by: Kedi Wu <31940276+kediwu0331@users.noreply.github.com>
Co-authored-by: Joel Gustafson <joelgustafson@protonmail.com>
 2026-09-04 22:11:58 +08:00
Shuwen Wang 19b46863f3 fix: align write-through pending across tree cores (#37278) 2026-09-04 22:08:25 +08:00
Mick 88021b0734 [diffusion] chore: make nightly performance measurements robust (#37915) 2026-09-04 22:03:25 +08:00
hhhh1252023 dc6b5d1f5a [CI][NPU] Remove model tests from PR-test pipeline (#38011) 2026-09-04 21:26:52 +08:00
Lianmin Zheng 978cc228ca [Rust] Bound multimodal media ingress (#37967) 2026-09-04 05:22:44 -07:00
Shuwen WangandClaude Opus 5 e4adf63275 [Fix] Vacuous marker writes in the cache tests, and an undebited Mamba admission slot (#36415) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-04 19:43:17 +08:00
Kevin MiandClaude Fable 5.1 3b678549c3 [diffusion] chore: warm up minimax-h3 at the served clip shape (#37945) 
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-04 19:18:02 +08:00
Xiaoyu Zhang f3b2725609 sm120 32GB mem-tier: raise decode cuda-graph max_bs 24->48 + chunked_prefill 2k->4k (#37898) 2026-09-04 18:47:26 +08:00
Kan WuRain JiangCursorClaude Fable 5
 		12735c2d76 [mm] refactor mm code for rust tokenizer manager (#34660) 
Co-authored-by: Rain Jiang <rain-jiang@outlook.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-09-04 03:03:25 -07:00
CSWYF3634076androot b42569a0f1 [Quant][ue8m0 fix] group requant_weight_ue8m0 reduce reserved gpu memory (#31755) 
Co-authored-by: root <root@johor-edge-gpu-a17-13.bec-host.baidu.com>
 2026-09-04 10:03:21 +00:00
Lianmin ZhengandShiyan Deng d7f235daca [Memory] Retire graph borrow pool before updating static runs (#37966) 
Co-authored-by: Shiyan Deng <dsy842974287@meta.com>
 2026-09-04 02:59:52 -07:00
Brian 2216697f90 [Docs] Refresh TPU model list and link cookbooks (#37750) 2026-09-04 17:28:57 +08:00
Ming YangClaudeLianmin Zheng
 		44c786679f [sp] Make attention-TP sequence sharding a per-forward batch property (#37546) 
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
 2026-09-04 02:11:03 -07:00
Liangsheng YinweireweireSam Shleifer
 		67248e04b4 [mem_cache] Route hybrid SWA full-side kv-row frees through free_segment (#37876) 
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
Co-authored-by: Sam Shleifer <sshleifer@gmail.com>
 2026-09-04 01:48:33 -07:00
karverma-amdandwunhuang dae126d510 [AMD][DSv4] Fuse inverse-RoPE into the fp8 wo_a quant (stacked on #37423) (#37658) 
Co-authored-by: wunhuang <wunhuang@amd.com>
 2026-09-04 00:51:29 -07:00
Ke Baoandadityakamat24 0b57847ebf Fix mamba radix cache ssm state indexing (#37836) 
Co-authored-by: adityakamat24 <adityakamat007@gmail.com>
 2026-09-04 15:46:33 +08:00
Xiaoyu Zhang 01e66a62db [Diffusion] Improve BCG warmup frame-count diagnostics for video models (#37890) 2026-09-04 15:05:24 +08:00
AMD-yanfeiwangandkk 7f89cc5286 [AMD] Skip unused TOPK v2 plan kernel on ROCm (#37580) 
Co-authored-by: kk <43161300+kkHuang-amd@users.noreply.github.com>
 2026-09-04 00:00:38 -07:00
Hemanth AcharyaClaude Opus 5kk
 		1bda9694b7 [AMD][DSv4] Switch output projection gemm (oproj_a) to fp8 (#37423) 
Signed-off-by: Hemanth Acharya <heachary@amd.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: kk <43161300+kkHuang-amd@users.noreply.github.com>
 2026-09-03 23:56:07 -07:00
Polisetty V R K Jyothendra Varma b168f905c8 [Intel GPU] Align XPU toml file for rust support (#31031) 
Signed-off-by: P V R K Jyothendra Varma <polisetty.v.r.k.jyothendra.varma@intel.com>
 2026-09-04 14:48:30 +08:00
Hemanth Acharya 31ebd8f437 [AMD][DSv4] Fuse the DSv4 FP4 indexer prefill-schedule preamble into one kernel (#37764) 2026-09-03 23:44:49 -07:00
Xiaoyu Zhang 54c2c99feb [Diffusion] Fuse LingBot per-token gated residual and RMSNorm modulate (#37910) 2026-09-04 14:35:13 +08:00
Yu-Yun ChangandHAI cb32dbc9e0 [AMD] [Kimi-K3] Fuse the KDA input projection into a single GEMM on ROCm (#35176) 
Co-authored-by: HAI <hixiao@gmail.com>
 2026-09-03 23:30:08 -07:00
Juan MunetonMeng, HengyuClaude Opus 5
 		72078cd7f5 [XPU] Support GPT-OSS MXFP4 checkpoints on Intel XPU (#35751) 
Co-authored-by: Meng, Hengyu <hengyu.meng@intel.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-04 14:20:28 +08:00
Po-Han Huang (NVIDIA)andkpham-sgl d122ca99b2 [Bugfix] Fix Llama 4 FA3 local attention with paged KV cache (#32902) 
Co-authored-by: kpham-sgl <khoa.pham@radixark.ai>
 2026-09-03 23:15:23 -07:00
mqhc2020andBingxu Chen 4e756ecc4a [AMD] CI: fix Lean decode crash on the EAGLE path (#37119) 
Co-authored-by: Bingxu Chen <bingxche@amd.com>
 2026-09-03 23:04:21 -07:00
AMD-yanfeiwangandkk 7825e5ffca [AMD] Fix DSV4 unified attention sink TP slice (#35092) 
Co-authored-by: kk <43161300+kkHuang-amd@users.noreply.github.com>
 2026-09-03 22:59:53 -07:00
Alex NailsandClaude Opus 5 ebae8ee21e [Fix] Register triton.runtime.cache.triton_key in the MPS stub so torch.compile keeps working (#37937) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-03 22:56:14 -07:00