Commit Graph
14312 Commits
Author SHA1 Message Date
flb_ f4de6abee6 feat(npu): Support returning indexer top-k results (#39060) 2026-09-28 14:46:45 +08:00
Rain Jiang 6c4f36e47a Rust server unify datapath for mm and generate requests (#39679) 2026-09-27 23:36:07 -07:00
e75e3b8a98 [Bugfix] fix(hicache): wait for decode offload before retraction (#30899)
Co-authored-by: agent <agent@local>
Co-authored-by: Kevin Flansburg <6134007+kflansburg@users.noreply.github.com>
Co-authored-by: Jimmy Shong <69131491+Jiminator@users.noreply.github.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-09-27 23:24:19 -07:00
6be9c78cf4 [Intel GPU] Xpu/weekly simple model enablement 2026 09 21 (#40664)
Co-authored-by: Amrutha M <amrutha.m@intel.com>
Co-authored-by: charlesxu91 <charlesxu.mi@gmail.com>
Co-authored-by: KMS07 <meher.sai.kotthagattu@intel.com>
Co-authored-by: Avi Fenesh <aviarchi1994@gmail.com>
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
2026-09-28 13:49:18 +08:00
Khoa Pham 55cc90b533 [CI] Real-model Kimi-Linear PD parity at page, DCP virtual-page, chunk and cached-prefix boundaries (#41378) 2026-09-27 20:23:07 -07:00
Yihao WangandMick Qian 8b2ca8ecc2 [diffusion] feat: support multiple task types for pipelines (#38762)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-09-28 10:00:45 +08:00
Kurkur 6cbdfec453 [Fix][NPU] Fix performance degradation caused by serial execution of two-stage ACL op calls on graph (#40814) 2026-09-28 09:32:16 +08:00
Cheng Wan 81f27fb3a7 [Refactor] Replace LayerScatterModes with LayerFacts and remove ScatterMode (#41443) 2026-09-27 16:56:11 -07:00
Cheng Wan 06d012eaca [Refactor] Give a layer its CuTe DSL kernels at construction instead of a subclass (#41442) 2026-09-27 16:55:47 -07:00
Cheng Wan 341d5273bd [Refactor] Build the boundary into any stage with one construction (#41441) 2026-09-27 16:55:25 -07:00
Cheng Wan 644014bc98 [Refactor] Declare each stage's residual read and update, and give each stage its own entry (#41440) 2026-09-27 16:55:04 -07:00
Cheng Wan 0e907b755d [Refactor] Split the layer communicator into a package (move only) (#41439) 2026-09-27 16:54:41 -07:00
Cheng Wan 427c9e05c1 [Refactor] Choose every layer's boundaries from declarations and remove the scatter-mode selection (#41438) 2026-09-27 16:54:18 -07:00
Cheng Wan 55d2ee5546 [Refactor] Step-3.5: complete the dense MLP's sum through ffn_exit (#41437) 2026-09-27 16:53:57 -07:00
Cheng Wan 0971450c16 [Fix] LongCat-Flash under attention DP: branch and merge the dense FFNs through the communicators (#41436) 2026-09-27 16:53:36 -07:00
Cheng Wan 4fac02fe5d [Refactor] Choose MoE layers' boundaries from declarations when moe_dp_size equals attn_cp_size (#41435) 2026-09-27 16:53:15 -07:00
Cheng Wan ecb8e9fc8f [Refactor] Falcon-H1: complete the FFN's sum through ffn_exit (#41434) 2026-09-27 16:52:53 -07:00
Cheng Wan 19eb56fa57 [Fix] Falcon-H1: count the Mamba mixer's output once under tensor parallelism (#41433) 2026-09-27 16:52:31 -07:00
Cheng Wan 31329c222c [Fix] Run MoE layers under attention DP and GQA prefill CP on the declared DP × CP gather (#41432) 2026-09-27 16:52:10 -07:00
Cheng Wan f0ecf15c84 [Refactor] Move CuTe DSL-fused layers onto the declared boundaries and FFN-exit kernel entries (#41431) 2026-09-27 16:51:48 -07:00
Cheng Wan 8c43c667cb [Refactor] Nemotron-H: build each layer's boundaries from its stage and the previous one (#41430) 2026-09-27 16:51:27 -07:00
Cheng Wan 6f370c3bcd [Refactor] Build each decoder boundary from the declarations of its two sides (#41429) 2026-09-27 16:51:05 -07:00
Cheng Wan 2e7f0bb28c [Refactor] Publish the LoRA token layout from the batch's FFN input rows (#41428) 2026-09-27 16:50:43 -07:00
Cheng Wan c289656434 [Refactor] Let the MoE declare whether its skipped reduction is one TP all-reduce (#41427) 2026-09-27 16:50:23 -07:00
Cheng Wan deaa7f1463 [Refactor] Run two-batch-overlap layers on the declared boundaries (#41426) 2026-09-27 16:50:03 -07:00
Cheng Wan 6f2d2437b6 [Refactor] Run MHC layers on the shared boundary steps with MHC's residual operations (#41425) 2026-09-27 16:49:42 -07:00
Cheng Wan f5ae989a0e [Refactor] Choose fully-DP dense and DSA / MLA prefill CP layers' steps from declarations (#41424) 2026-09-27 16:49:21 -07:00
Cheng Wan cad9eaf27d [Fix] Gather a dense FFN's input across attention DP and CP in one DP sum (#41423) 2026-09-27 16:48:59 -07:00
Cheng Wan 1c919e401d [Fix] Keep one copy of CP-replicated rows in the DP gather (#41422) 2026-09-27 16:48:39 -07:00
Cheng Wan 7a833ad0a2 [Refactor] Choose a GQA prefill CP extend's steps from declarations (#41421) 2026-09-27 16:48:17 -07:00
Cheng Wan 380a4d0332 [Refactor] Run every batch of a layer from one BoundarySteps (#41420) 2026-09-27 16:47:55 -07:00
Cheng Wan 9f87d28db6 [Refactor] Choose the LayerNorm SP region's and input-scattered batches' steps from declarations (#41419) 2026-09-27 16:47:32 -07:00
Cheng Wan 846181f1c4 [Refactor] Give the fused prepare_mlp kernels an explicit contract (#41418) 2026-09-27 16:47:11 -07:00
Cheng Wan 446d181e23 [Refactor] Choose the boundaries of plain-TP dense layers and MoE layers from declarations (#41417) 2026-09-27 16:46:50 -07:00
Yoni GozlanandYoni Gozlan 583fada954 Port chat_parsing core (#40477)
Co-authored-by: Yoni Gozlan <yonigozlan@users.noreply.github.com>
2026-09-27 16:14:24 -07:00
78eee88113 [Spec] Add LiLiCorr: a candidate-lattice reranker for DFlash drafts (#37462)
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
Co-authored-by: kpham-sgl <khoa.pham@radixark.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 15:36:48 -07:00
Shangming CaiandClaude Fable 5 14ab74ef95 [PD] Fan drain abort ACKs out to every decode peer of the room (#41402)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-09-28 05:16:23 +08:00
e581520c67 [HiCache] Add the page-unified KV load-back JIT kernel (#39726)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: huangtingwei <141888744+huangtingwei9988@users.noreply.github.com>
Co-authored-by: Zhangheng <hzh0425@apache.org>
2026-09-28 01:45:02 +08:00
Xinyuan Tong d10a09ecf3 Add MiniMax arch fallback to auto parser resolution (#40930) 2026-09-28 01:33:27 +08:00
CuzMi b2800461e5 [Diffusion] Reuse bit-exact packed SwiGLU for Ming-Image (#41266) 2026-09-27 23:57:43 +08:00
1ba84a4b59 [bench] Take each request's prompt length from the server (#39889)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
2026-09-27 23:57:18 +08:00
Yuan Luoandluoyuan.luo c311bc9611 [VLM] Introduce FA4 into ViT for SM100/SM103 (#41344)
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
2026-09-27 23:04:21 +08:00
588049fe2e [diffusion] fix: run an all-valid attention mask on the backend's unmasked kernel (#41309)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 22:54:46 +08:00
Kangrui Du 84622ce9d5 [diffusion] fix: lora-wrapped linears crash qwen-image and minimax-h3 inference (#41272)
Signed-off-by: rockdu <kangrdu@gmail.com>
2026-09-27 22:54:10 +08:00
a0ba1968b1 [Diffusion] Apply latent-ids and packing to caller-provided initial latents (#34418)
Co-authored-by: aimicahchen <aimicahchen@tencent.com>
Co-authored-by: WenhaoZhang <42087078+niehen6174@users.noreply.github.com>
2026-09-27 18:32:20 +08:00
WenhaoZhang f1e62e3a2e [diffusion] feat: add MiniMax-H3 to ComfyUI integrated mode (#35990) 2026-09-27 17:58:54 +08:00
WenhaoZhang ae47bcd4da [diffusion] feat: minimax-h3 spectrum skip-step + fused RMSNorm/AdaLN (#35684) 2026-09-27 17:56:13 +08:00
WenhaoZhang 2a14f65fd7 [diffusion] feat: support in-place lora merge/unmerge under layerwise offload (#36192) 2026-09-27 17:54:40 +08:00
Liangsheng Yin 37556c1b91 [mem_cache] Replace cache_finished_req with insert_req; release_kv_cache frees and unpins (#41281) 2026-09-27 02:53:16 -07:00
Xiaoyu Zhangandkernel-design-agents 0bc074db83 [KDA+Kimi K3] Speed up SANA-Video residual gate add on H200 (#41305)
Co-authored-by: kernel-design-agents <334018530+kernel-design-agents@users.noreply.github.com>
2026-09-27 16:57:12 +08:00