Commit Graph
13281 Commits
Author SHA1 Message Date
9654e5c4bc dsv4.1-amd: fused mHC boundary and all-reduce + mHC post kernels (#41021)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Kevin Mi <kevin.mi@radixark.ai>
2026-09-28 16:15:08 -07:00
Kevin MiandClaude Opus 5.5 096b066fb4 dsv4.1-amd: gfx950 sparse decode attention and sorted top-k (#41020)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 16:14:28 -07:00
Nan Jiang 6fa3fe69e2 [PD] Keep the sampling mask of a replayed rebootstrap token (#41235) 2026-09-28 12:02:27 -07:00
Shangming CaiandClaude Fable 5 2a1c477648 [PD] Defer decode KV release on every transfer failure, not only decode-initiated aborts (#41404)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-09-29 02:19:51 +08:00
YAMY f209e67767 [Spec] Model-agnostic last-stage draft embedding under pipeline parallelism (#39643) 2026-09-28 10:22:40 -07:00
Kurkur 6624999385 [Fix][NPU] fix dp-attn hang when pin_mem is True on NPU (#40446) 2026-09-29 00:41:43 +08:00
triple-mu 2e7e0802f4 [diffusion] Qwen-Image 2.1: fuse Q/K RMSNorm + RoPE + KV packing into one CUDA kernel and project Q/K/V with one packed GEMM (#41339) 2026-09-28 21:45:39 +08:00
DarkSharpnessandClaude Opus 5.5 8151378c1a [DeepSelect] Add page-table transform to top-k and tighten the layout contract (#41364)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 21:43:57 +08:00
Shuwen Wang da3eb6db32 [Unified Memory] Fix Inkling conv-checkpoint track ids written to virtual slot numbers (#41144) 2026-09-28 05:19:19 -07:00
Shuwen Wang bddeb61d94 [Unified Memory] fix: preserve FP8 dtype in unified MHA pool (#38133) 2026-09-28 05:19:03 -07:00
MickandMick Qian 090439bf8f [diffusion] docs: consolidate Qwen-Image 2.1 guidance in its cookbook (#41540)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-09-28 20:04:32 +08:00
Raiden MakotoandRaiden-Makoto fa443412c5 [AMD] [GLM5] Fuse shared expert into AITER MoE on gfx950 (#41161)
Signed-off-by: Raiden-Makoto <Raiden-Makoto@users.noreply.github.com>
Co-authored-by: Raiden-Makoto <Raiden-Makoto@users.noreply.github.com>
2026-09-28 03:40:09 -07:00
karverma-amd 85be04978d [AMD][DSV4] moe: enable shared-expert fusion on the grouped-topk path (megamoe) (#40943) 2026-09-28 03:21:57 -07:00
Denji-kk 6caf0ff4ae perf(multimodal): offload CPU feature hashing with bounded admission (#39539) 2026-09-28 17:51:05 +08:00
MickandMick Qian c02eaadeb2 [diffusion] optimization: populate cpu weight stores before host registration to speedup layerwise-offload initialization (#40439)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-09-28 17:45:58 +08:00
Peng Wu 6582425854 Let predicate-registered linear-attention models carry the mamba radix-cache leaves (#41165) 2026-09-28 17:07:35 +08:00
Peng Wu 546bfa7221 [Kimi-K3] Merge fused_qkvg_proj into the loader-seeded packed_modules_mapping (#41164) 2026-09-28 16:46:11 +08:00
ashwini rathiandPramod Kumar e8eeff2956 [XPU] Bump sglang-kernel-xpu wheel to v0.3.0 (#41220)
Co-authored-by: Pramod Kumar <144990617+pramodkumar-habanalabs@users.noreply.github.com>
2026-09-28 16:15:12 +08:00
Ke Bao 6405cf2881 Fix chat template cache key order (#41517) 2026-09-28 15:17:09 +08:00
flb_ f4de6abee6 feat(npu): Support returning indexer top-k results (#39060) 2026-09-28 14:46:45 +08:00
Rain Jiang 6c4f36e47a Rust server unify datapath for mm and generate requests (#39679) 2026-09-27 23:36:07 -07:00
e75e3b8a98 [Bugfix] fix(hicache): wait for decode offload before retraction (#30899)
Co-authored-by: agent <agent@local>
Co-authored-by: Kevin Flansburg <6134007+kflansburg@users.noreply.github.com>
Co-authored-by: Jimmy Shong <69131491+Jiminator@users.noreply.github.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-09-27 23:24:19 -07:00
6be9c78cf4 [Intel GPU] Xpu/weekly simple model enablement 2026 09 21 (#40664)
Co-authored-by: Amrutha M <amrutha.m@intel.com>
Co-authored-by: charlesxu91 <charlesxu.mi@gmail.com>
Co-authored-by: KMS07 <meher.sai.kotthagattu@intel.com>
Co-authored-by: Avi Fenesh <aviarchi1994@gmail.com>
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
2026-09-28 13:49:18 +08:00
Khoa Pham 55cc90b533 [CI] Real-model Kimi-Linear PD parity at page, DCP virtual-page, chunk and cached-prefix boundaries (#41378) 2026-09-27 20:23:07 -07:00
Yihao WangandMick Qian 8b2ca8ecc2 [diffusion] feat: support multiple task types for pipelines (#38762)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-09-28 10:00:45 +08:00
Kurkur 6cbdfec453 [Fix][NPU] Fix performance degradation caused by serial execution of two-stage ACL op calls on graph (#40814) 2026-09-28 09:32:16 +08:00
Cheng Wan 81f27fb3a7 [Refactor] Replace LayerScatterModes with LayerFacts and remove ScatterMode (#41443) 2026-09-27 16:56:11 -07:00
Cheng Wan 06d012eaca [Refactor] Give a layer its CuTe DSL kernels at construction instead of a subclass (#41442) 2026-09-27 16:55:47 -07:00
Cheng Wan 341d5273bd [Refactor] Build the boundary into any stage with one construction (#41441) 2026-09-27 16:55:25 -07:00
Cheng Wan 644014bc98 [Refactor] Declare each stage's residual read and update, and give each stage its own entry (#41440) 2026-09-27 16:55:04 -07:00
Cheng Wan 0e907b755d [Refactor] Split the layer communicator into a package (move only) (#41439) 2026-09-27 16:54:41 -07:00
Cheng Wan 427c9e05c1 [Refactor] Choose every layer's boundaries from declarations and remove the scatter-mode selection (#41438) 2026-09-27 16:54:18 -07:00
Cheng Wan 55d2ee5546 [Refactor] Step-3.5: complete the dense MLP's sum through ffn_exit (#41437) 2026-09-27 16:53:57 -07:00
Cheng Wan 0971450c16 [Fix] LongCat-Flash under attention DP: branch and merge the dense FFNs through the communicators (#41436) 2026-09-27 16:53:36 -07:00
Cheng Wan 4fac02fe5d [Refactor] Choose MoE layers' boundaries from declarations when moe_dp_size equals attn_cp_size (#41435) 2026-09-27 16:53:15 -07:00
Cheng Wan ecb8e9fc8f [Refactor] Falcon-H1: complete the FFN's sum through ffn_exit (#41434) 2026-09-27 16:52:53 -07:00
Cheng Wan 19eb56fa57 [Fix] Falcon-H1: count the Mamba mixer's output once under tensor parallelism (#41433) 2026-09-27 16:52:31 -07:00
Cheng Wan 31329c222c [Fix] Run MoE layers under attention DP and GQA prefill CP on the declared DP × CP gather (#41432) 2026-09-27 16:52:10 -07:00
Cheng Wan f0ecf15c84 [Refactor] Move CuTe DSL-fused layers onto the declared boundaries and FFN-exit kernel entries (#41431) 2026-09-27 16:51:48 -07:00
Cheng Wan 8c43c667cb [Refactor] Nemotron-H: build each layer's boundaries from its stage and the previous one (#41430) 2026-09-27 16:51:27 -07:00
Cheng Wan 6f370c3bcd [Refactor] Build each decoder boundary from the declarations of its two sides (#41429) 2026-09-27 16:51:05 -07:00
Cheng Wan 2e7f0bb28c [Refactor] Publish the LoRA token layout from the batch's FFN input rows (#41428) 2026-09-27 16:50:43 -07:00
Cheng Wan c289656434 [Refactor] Let the MoE declare whether its skipped reduction is one TP all-reduce (#41427) 2026-09-27 16:50:23 -07:00
Cheng Wan deaa7f1463 [Refactor] Run two-batch-overlap layers on the declared boundaries (#41426) 2026-09-27 16:50:03 -07:00
Cheng Wan 6f2d2437b6 [Refactor] Run MHC layers on the shared boundary steps with MHC's residual operations (#41425) 2026-09-27 16:49:42 -07:00
Cheng Wan f5ae989a0e [Refactor] Choose fully-DP dense and DSA / MLA prefill CP layers' steps from declarations (#41424) 2026-09-27 16:49:21 -07:00
Cheng Wan cad9eaf27d [Fix] Gather a dense FFN's input across attention DP and CP in one DP sum (#41423) 2026-09-27 16:48:59 -07:00
Cheng Wan 1c919e401d [Fix] Keep one copy of CP-replicated rows in the DP gather (#41422) 2026-09-27 16:48:39 -07:00
Cheng Wan 7a833ad0a2 [Refactor] Choose a GQA prefill CP extend's steps from declarations (#41421) 2026-09-27 16:48:17 -07:00
Cheng Wan 380a4d0332 [Refactor] Run every batch of a layer from one BoundarySteps (#41420) 2026-09-27 16:47:55 -07:00