Commit Graph
19118 Commits
Author SHA1 Message Date
24ee67d2fa Readme refresh (#41787)
Co-authored-by: zijiexia <37504505+zijiexia@users.noreply.github.com>
Co-authored-by: Zijie Xia <zijie.xia@radixark.ai>
2026-09-29 23:48:38 -07:00
eb9c9ee99d [Qwen4-Exp] Optional fp8 (e4m3) storage for the compressed QSA indexer cache (#39614)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: YAMY1234 <74099316+YAMY1234@users.noreply.github.com>
2026-09-29 22:11:46 -07:00
Qiaolin Yu 88f95f4b87 [qwen next] Fuse QSA KV preparation and sparse block expansion (#40972) 2026-09-29 21:54:34 -07:00
Qiaolin Yu b660100c00 [qwen 3.8 next] Fuse QSA graph replay metadata across draft steps (#41173) 2026-09-29 21:52:28 -07:00
JohnQinAMDandClaude Opus 5.5 964fd6af92 [ROCm] GLM-5.2: gfx950 four-kernel fused DSA indexer decode path (#38583)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 21:52:00 -07:00
hhhh1252023 be66c1b460 [NPU][CI] Fix stale multimodal-gen job name in fast-fail health check (#41824) 2026-09-30 12:39:23 +08:00
Cheng Wan d41904f05e [Refactor] Build seven more decoders from stage boundaries (#41779) 2026-09-29 21:16:45 -07:00
Cheng Wan 113e6a1496 [Refactor] Simplify layer boundary internals (#41778) 2026-09-29 21:15:58 -07:00
Wenfei (Charles) Qi 7fb85a9d88 [MLX] Fix startup after logical token capacity change (#41314) 2026-09-29 21:15:56 -07:00
Cheng Wan 9ee065e49a [Fix] Step-3.5: build the shared expert without TP under all-to-all MoE backends (#41754) 2026-09-29 21:11:37 -07:00
Cheng Wan faa5064f6e [Refactor] Send producer-written residual streams through to_pp (#41753) 2026-09-29 21:11:02 -07:00
Cheng Wan 843f50f5c1 [Fix] Reduce the Sarvam dense MLP through its row-parallel projection (#41752) 2026-09-29 21:10:17 -07:00
Cheng Wan ea855f0daf [Refactor] Leave decoder stacks through final_norm(skip_empty=True) (#41751) 2026-09-29 21:08:19 -07:00
Cheng Wan 01b5f572b7 [Refactor] Rename layer boundary protocols, contracts and helpers (#41750) 2026-09-29 21:05:48 -07:00
cb3ba2f2b7 [XPU] Add dynamic_expert_bias and track_state support (#39618)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Singh <rohitsi2@iil-login.iind.intel.com>
Co-authored-by: Singh <rohitsi2@iil-gnrap01.iind.intel.com>
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
2026-09-30 12:05:35 +08:00
Alison Shao 8854857a99 [CI] Bound maintenance API requests to 30 seconds (#41800) 2026-09-29 20:02:02 -07:00
Mohammad Miadh AngkadandMohammad Angkad 7b30ff26ee [CI] Evaluate the maintenance gate with github-script instead of jq (#41625)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-09-29 18:47:51 -07:00
fc9bdc8a3e [MoE] Keep the moe_align_block_size pad fill inside sorted_token_ids (#41759)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: oulgen <oulgen@users.noreply.github.com>
2026-09-29 17:43:17 -07:00
Baizhou Zhang e5cec303ae [Fix] Restore deferred layer dumps and pin SentencePiece for InternVL (#41777) 2026-09-29 17:28:30 -07:00
sglang-bot cae69be584 chore: bump sgl-kernel version to 0.4.8 (#41784) 2026-09-29 17:19:21 -07:00
DarkSharpness 57d4adeb83 [Kernel] Expose FlashMLA kv_format in sgl-kernel sparse decode (#41336) 2026-09-29 17:16:52 -07:00
Jason Mancuso ba3e04c063 [CI] Grant CI permissions to four contributors (#41775) 2026-09-29 16:42:01 -07:00
Cheng Wan 8d2d874758 [Fix] Release consumed residual contributions (#41749) 2026-09-29 16:38:05 -07:00
Jiajun LiandShi Dong 3d4953839c [Fix] MoE: require TopK layer_id to ensure routed expert captures (#40980)
Co-authored-by: Shi Dong <hello@dongshi.me>
2026-09-29 16:29:27 -07:00
Jiajun Li f0e4001930 [Fix] Allocate SWA KV pools inside the memory-saver region (#40979) 2026-09-29 16:29:02 -07:00
Yuwei An d3f8a8f4f5 [Spec/Prefill Coordination] Dspark support (#41623) 2026-09-29 16:08:01 -07:00
Liangsheng Yin 211b1d9784 [Refactor] Share PD routing fields across request models (#41756) 2026-09-29 15:11:18 -07:00
37a47737c8 [Perf] Tune SM90 GDN recurrent verify launch for small batches (#41486)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
2026-09-29 15:03:33 -07:00
Mohammad Miadh Angkadandhnyls2002 4885b563c3 [Test] Demote PD test RDMA openability check to a warning (#41681)
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-09-29 14:52:31 -07:00
84523d6785 [Spec] Reuse K3 auxiliary outputs across decode CUDA graph sizes (#40159)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-09-29 14:24:51 -07:00
jain-ria 46beacf13d [grpc rust] Define semantic frontend contract (#41709)
Signed-off-by: jain-ria <riajain@NVIDIA.com>
2026-09-29 13:40:04 -07:00
James LiuandClaude Opus 5.5 77a81996ce [PD] Preserve bootstrap metadata in native Messages requests (#39711)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 13:32:54 -07:00
3c4653b6a3 [Metrics] Fix PD latency histogram accounting (#39706)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
2026-09-29 13:12:09 -07:00
zijiexiaandClaude Sonnet 5.5 f884231f5a [Docs] Add IQuestLab card logo to cookbook landing page (#41746)
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-29 12:47:21 -07:00
51ace48387 [Metrics] Add request-level TPOT histogram (sglang:request_time_per_output_token_seconds) (#40275)
Co-authored-by: Gilford Ting <gilford@modal.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-29 12:20:22 -07:00
ishandhanani 9a16e47437 chore: grant jain-ria CI permissions (#41728) 2026-09-29 11:56:51 -07:00
ashwini rathi bf80732b92 Fix pip install: exclude multimodal_gen/.claude symlink from package data (#41703) 2026-09-29 11:35:11 -07:00
8f2a9638f7 [metrics] Fix non-streaming TTFT by flushing the first output through detokenization (#38600)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-09-29 11:24:52 -07:00
Colin Z 875dd41e6f [AMD] Fix Load and Inference of MLA models with Quark PTPC FP8 attention on ROCm (#28734) 2026-09-29 11:20:10 -07:00
Chunan Zeng affc602f51 [CI] Extend DeepGEMM GB300 validation timeout to three hours (#41720) 2026-09-29 10:57:20 -07:00
ced5e9fe49 [Kimi] Enable GB300 TP4 and GB200/GB300 TP16 SP collectives (#35330)
Co-authored-by: Ajit Mistry <ajmistry@amistry.nvidia.com>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
2026-09-29 10:18:11 -07:00
CYJiang fa090f7755 [MUSA] Fix fused MoE GEMV registration and torchada pin (#41444) 2026-09-29 09:51:08 -07:00
ashwini rathi 98fce73d5b [XPU] publish nightly docker image with sgl-kernel-xpu built from main (#41650) 2026-09-29 20:09:53 +08:00
4a4e1ce014 [Router] Hold a booting rank's batches and graft a snapshot on the pump (7/13) (#40693)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-29 20:07:37 +08:00
8d1643b21e [Router] Vet a peer's snapshot before it may touch the tree (6/13) (#40692)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-29 19:58:11 +08:00
8b3a4cad8b [Router] Track a booting rank's bootstrap and fetch a peer's snapshot (5/13) (#40691)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-29 19:06:13 +08:00
7bcfcf5aa0 [Router] Watch EndpointSlices for sibling router replicas (4/13) (#40690)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-29 18:56:32 +08:00
c5a38d3998 [sgl-router] Add --tokenizer-backend fast and --tokenizer-l1-cache-mb (#41268)
Co-authored-by: Kan Wu <kan.wu@MacBook-Pro.local>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:55:58 +08:00
MichaelandClaude 5828cd4927 [AMD] Add GLM-5.3 MI30x and MI35x nightly accuracy tests (#41602)
Co-authored-by: Claude <noreply@anthropic.com>
2026-09-29 02:49:01 -07:00
Rumit DesaiandXiaoyu Zhang f731e82f09 Fix MUSA detection under torch.compile fullgraph (#41065)
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
2026-09-29 17:26:27 +08:00