Commit Graph
14622 Commits
Author SHA1 Message Date
Yuwei An d3f8a8f4f5 [Spec/Prefill Coordination] Dspark support (#41623) 2026-09-29 16:08:01 -07:00
Liangsheng Yin 211b1d9784 [Refactor] Share PD routing fields across request models (#41756) 2026-09-29 15:11:18 -07:00
37a47737c8 [Perf] Tune SM90 GDN recurrent verify launch for small batches (#41486)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
2026-09-29 15:03:33 -07:00
Mohammad Miadh Angkadandhnyls2002 4885b563c3 [Test] Demote PD test RDMA openability check to a warning (#41681)
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-09-29 14:52:31 -07:00
84523d6785 [Spec] Reuse K3 auxiliary outputs across decode CUDA graph sizes (#40159)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-09-29 14:24:51 -07:00
James LiuandClaude Opus 5.5 77a81996ce [PD] Preserve bootstrap metadata in native Messages requests (#39711)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 13:32:54 -07:00
3c4653b6a3 [Metrics] Fix PD latency histogram accounting (#39706)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
2026-09-29 13:12:09 -07:00
51ace48387 [Metrics] Add request-level TPOT histogram (sglang:request_time_per_output_token_seconds) (#40275)
Co-authored-by: Gilford Ting <gilford@modal.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-29 12:20:22 -07:00
ashwini rathi bf80732b92 Fix pip install: exclude multimodal_gen/.claude symlink from package data (#41703) 2026-09-29 11:35:11 -07:00
8f2a9638f7 [metrics] Fix non-streaming TTFT by flushing the first output through detokenization (#38600)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-09-29 11:24:52 -07:00
Colin Z 875dd41e6f [AMD] Fix Load and Inference of MLA models with Quark PTPC FP8 attention on ROCm (#28734) 2026-09-29 11:20:10 -07:00
ced5e9fe49 [Kimi] Enable GB300 TP4 and GB200/GB300 TP16 SP collectives (#35330)
Co-authored-by: Ajit Mistry <ajmistry@amistry.nvidia.com>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
2026-09-29 10:18:11 -07:00
CYJiang fa090f7755 [MUSA] Fix fused MoE GEMV registration and torchada pin (#41444) 2026-09-29 09:51:08 -07:00
Rumit DesaiandXiaoyu Zhang f731e82f09 Fix MUSA detection under torch.compile fullgraph (#41065)
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
2026-09-29 17:26:27 +08:00
Carrie ChenandBrayden Zhong 0e586fd12d fuse shared experts with routed experts in MegaMoE's DeepGEMM (#39313)
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
2026-09-29 17:25:10 +08:00
c9cb32e310 [AMD] fix kda decode flydsl import (#40546)
Co-authored-by: Bingxu Chen <bingxche@amd.com>
Co-authored-by: Michael <13900043+michaelzhang-ai@users.noreply.github.com>
2026-09-29 02:20:42 -07:00
Liang Gong 34a1234d21 chore: expose agent skills via .agents directories (#40515) 2026-09-29 17:12:28 +08:00
Jiacong Fang bd78095030 [Fix] Add name mapping in load_weights of nvidia/LocateAnything-3B (#41059) 2026-09-29 00:38:58 -07:00
MickandMick Qian dc1bd46802 [diffusion] feat: support bounded exact conditioning cache across native models (#40470)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-09-29 15:27:22 +08:00
eeecho c7be3e935b [SM120] Add optional FlashInfer PCIe-IPC all-reduce for switch-free hosts (#34528) 2026-09-28 23:08:44 -07:00
Anupa SajikumarandMa Mingfei 2ff52e3ceb [Intel][XPU]Enable chunked prefill scnearios for XPU with UT (#33804)
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
2026-09-29 13:28:25 +08:00
9ae43b0eee [XPU] Support compressed-tensors W4A16 by reusing the torch int4pack path (#40828)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Singh <rohitsi2@iil-gnrap01.iind.intel.com>
Co-authored-by: Singh <rohitsi2@iil-login.iind.intel.com>
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
2026-09-29 13:16:03 +08:00
Cheng Wan 5781206833 [PD] Give FakeKVReceiver ensure_abort_notified (#41618) 2026-09-28 21:59:17 -07:00
Cheng Wan 6dad152490 [Refactor] Document layer boundary contracts and integration (#41557) 2026-09-28 20:18:27 -07:00
Cheng Wan de9f850b36 [Refactor] Rename the module to layer_boundary (#41555) 2026-09-28 20:17:50 -07:00
Cheng Wan 701213c867 [Refactor] Retire the layer facade and simplify boundary internals (#41554) 2026-09-28 20:17:33 -07:00
Cheng Wan 9b89b96230 [Refactor] Migrate specialized decoder and overlap boundaries (#41553) 2026-09-28 20:17:17 -07:00
Cheng Wan 49b5e76e45 [Refactor] Construct independent decoder stage boundaries (#41552) 2026-09-28 20:17:00 -07:00
Cheng Wan acffb0d20d [Refactor] Select reduction fusion at the consumer (#41551) 2026-09-28 20:16:44 -07:00
Cheng Wan cb1fa3c399 [Refactor] Capture auxiliary states at residual reads (#41550) 2026-09-28 20:16:26 -07:00
Cheng Wan 42f58b4da7 [Refactor] Carry residual state across stage boundaries (#41549) 2026-09-28 20:15:03 -07:00
Cheng Wan 6f72cc3b64 [Refactor] Centralize decoder output access (#41548) 2026-09-28 20:14:04 -07:00
Cheng Wan 0a8ca85565 [Refactor] Group communicator fusion and CP adapters (#41547) 2026-09-28 20:10:50 -07:00
Liangsheng Yin d9074f7943 [Test] Fail fast when PD test RDMA devices are not openable by ibverbs (#41600) 2026-09-28 19:34:58 -07:00
James 6e1df69623 [npu]support NPU 910C L2 memcache offload (#41527) 2026-09-29 10:30:01 +08:00
77091cea68 [Radix Cache] Sync Rust TreeCore and make it the default (#39627)
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
Co-authored-by: Zhangheng <hzh0425@apache.org>
Co-authored-by: alphabetc1 <alphabetc1@users.noreply.github.com>
2026-09-29 10:18:57 +08:00
WenhaoZhang 49d26ff0c2 [diffusion] model: support flux 3 action robot policies (#41066) 2026-09-29 10:17:19 +08:00
79cafec013 [Model] Add IQuest Q1 support and MTP draft (#41590)
Co-authored-by: zelong huang <yzhu@ubiquant.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: zelong518 <zelonghuang05@gmail.com>
Co-authored-by: chenwenxiaolive <cocoshirolive@gmail.com>
2026-09-28 17:21:31 -07:00
7889fbab9a [PD] Keep EAGLE DP graph and token metadata consistent (#32196)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
Co-authored-by: Khoa Pham <khoa.pham@radixark.ai>
2026-09-28 17:15:51 -07:00
Shuwen Wang 89e1316eae [mem cache] refactor: remove the index-K continuous getters orphaned by the CP v1 removal (#41469) 2026-09-28 16:41:07 -07:00
Hrithvik AlexandHrithvik Alex f124c5ceec [Model Loader] Stop checkpoint prefetch after iterator completion (#41588)
Co-authored-by: Hrithvik Alex <hrithvik@baseten.co>
2026-09-28 16:27:09 -07:00
9654e5c4bc dsv4.1-amd: fused mHC boundary and all-reduce + mHC post kernels (#41021)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Kevin Mi <kevin.mi@radixark.ai>
2026-09-28 16:15:08 -07:00
Kevin MiandClaude Opus 5.5 096b066fb4 dsv4.1-amd: gfx950 sparse decode attention and sorted top-k (#41020)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 16:14:28 -07:00
Nan Jiang 6fa3fe69e2 [PD] Keep the sampling mask of a replayed rebootstrap token (#41235) 2026-09-28 12:02:27 -07:00
Shangming CaiandClaude Fable 5 2a1c477648 [PD] Defer decode KV release on every transfer failure, not only decode-initiated aborts (#41404)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-09-29 02:19:51 +08:00
YAMY f209e67767 [Spec] Model-agnostic last-stage draft embedding under pipeline parallelism (#39643) 2026-09-28 10:22:40 -07:00
Kurkur 6624999385 [Fix][NPU] fix dp-attn hang when pin_mem is True on NPU (#40446) 2026-09-29 00:41:43 +08:00
triple-mu 2e7e0802f4 [diffusion] Qwen-Image 2.1: fuse Q/K RMSNorm + RoPE + KV packing into one CUDA kernel and project Q/K/V with one packed GEMM (#41339) 2026-09-28 21:45:39 +08:00
DarkSharpnessandClaude Opus 5.5 8151378c1a [DeepSelect] Add page-table transform to top-k and tighten the layout contract (#41364)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 21:43:57 +08:00
Shuwen Wang da3eb6db32 [Unified Memory] Fix Inkling conv-checkpoint track ids written to virtual slot numbers (#41144) 2026-09-28 05:19:19 -07:00