Commit Graph
13245 Commits
Author SHA1 Message Date
Cheng Wan 19eb56fa57 [Fix] Falcon-H1: count the Mamba mixer's output once under tensor parallelism (#41433) 2026-09-27 16:52:31 -07:00
Cheng Wan 31329c222c [Fix] Run MoE layers under attention DP and GQA prefill CP on the declared DP × CP gather (#41432) 2026-09-27 16:52:10 -07:00
Cheng Wan f0ecf15c84 [Refactor] Move CuTe DSL-fused layers onto the declared boundaries and FFN-exit kernel entries (#41431) 2026-09-27 16:51:48 -07:00
Cheng Wan 8c43c667cb [Refactor] Nemotron-H: build each layer's boundaries from its stage and the previous one (#41430) 2026-09-27 16:51:27 -07:00
Cheng Wan 6f370c3bcd [Refactor] Build each decoder boundary from the declarations of its two sides (#41429) 2026-09-27 16:51:05 -07:00
Cheng Wan 2e7f0bb28c [Refactor] Publish the LoRA token layout from the batch's FFN input rows (#41428) 2026-09-27 16:50:43 -07:00
Cheng Wan c289656434 [Refactor] Let the MoE declare whether its skipped reduction is one TP all-reduce (#41427) 2026-09-27 16:50:23 -07:00
Cheng Wan deaa7f1463 [Refactor] Run two-batch-overlap layers on the declared boundaries (#41426) 2026-09-27 16:50:03 -07:00
Cheng Wan 6f2d2437b6 [Refactor] Run MHC layers on the shared boundary steps with MHC's residual operations (#41425) 2026-09-27 16:49:42 -07:00
Cheng Wan f5ae989a0e [Refactor] Choose fully-DP dense and DSA / MLA prefill CP layers' steps from declarations (#41424) 2026-09-27 16:49:21 -07:00
Cheng Wan cad9eaf27d [Fix] Gather a dense FFN's input across attention DP and CP in one DP sum (#41423) 2026-09-27 16:48:59 -07:00
Cheng Wan 1c919e401d [Fix] Keep one copy of CP-replicated rows in the DP gather (#41422) 2026-09-27 16:48:39 -07:00
Cheng Wan 7a833ad0a2 [Refactor] Choose a GQA prefill CP extend's steps from declarations (#41421) 2026-09-27 16:48:17 -07:00
Cheng Wan 380a4d0332 [Refactor] Run every batch of a layer from one BoundarySteps (#41420) 2026-09-27 16:47:55 -07:00
Cheng Wan 9f87d28db6 [Refactor] Choose the LayerNorm SP region's and input-scattered batches' steps from declarations (#41419) 2026-09-27 16:47:32 -07:00
Cheng Wan 846181f1c4 [Refactor] Give the fused prepare_mlp kernels an explicit contract (#41418) 2026-09-27 16:47:11 -07:00
Cheng Wan 446d181e23 [Refactor] Choose the boundaries of plain-TP dense layers and MoE layers from declarations (#41417) 2026-09-27 16:46:50 -07:00
Yoni GozlanandYoni Gozlan 583fada954 Port chat_parsing core (#40477)
Co-authored-by: Yoni Gozlan <yonigozlan@users.noreply.github.com>
2026-09-27 16:14:24 -07:00
78eee88113 [Spec] Add LiLiCorr: a candidate-lattice reranker for DFlash drafts (#37462)
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
Co-authored-by: kpham-sgl <khoa.pham@radixark.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 15:36:48 -07:00
Shangming CaiandClaude Fable 5 14ab74ef95 [PD] Fan drain abort ACKs out to every decode peer of the room (#41402)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-09-28 05:16:23 +08:00
e581520c67 [HiCache] Add the page-unified KV load-back JIT kernel (#39726)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: huangtingwei <141888744+huangtingwei9988@users.noreply.github.com>
Co-authored-by: Zhangheng <hzh0425@apache.org>
2026-09-28 01:45:02 +08:00
Xinyuan Tong d10a09ecf3 Add MiniMax arch fallback to auto parser resolution (#40930) 2026-09-28 01:33:27 +08:00
CuzMi b2800461e5 [Diffusion] Reuse bit-exact packed SwiGLU for Ming-Image (#41266) 2026-09-27 23:57:43 +08:00
1ba84a4b59 [bench] Take each request's prompt length from the server (#39889)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
2026-09-27 23:57:18 +08:00
Yuan Luoandluoyuan.luo c311bc9611 [VLM] Introduce FA4 into ViT for SM100/SM103 (#41344)
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
2026-09-27 23:04:21 +08:00
588049fe2e [diffusion] fix: run an all-valid attention mask on the backend's unmasked kernel (#41309)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 22:54:46 +08:00
Kangrui Du 84622ce9d5 [diffusion] fix: lora-wrapped linears crash qwen-image and minimax-h3 inference (#41272)
Signed-off-by: rockdu <kangrdu@gmail.com>
2026-09-27 22:54:10 +08:00
a0ba1968b1 [Diffusion] Apply latent-ids and packing to caller-provided initial latents (#34418)
Co-authored-by: aimicahchen <aimicahchen@tencent.com>
Co-authored-by: WenhaoZhang <42087078+niehen6174@users.noreply.github.com>
2026-09-27 18:32:20 +08:00
WenhaoZhang f1e62e3a2e [diffusion] feat: add MiniMax-H3 to ComfyUI integrated mode (#35990) 2026-09-27 17:58:54 +08:00
WenhaoZhang ae47bcd4da [diffusion] feat: minimax-h3 spectrum skip-step + fused RMSNorm/AdaLN (#35684) 2026-09-27 17:56:13 +08:00
WenhaoZhang 2a14f65fd7 [diffusion] feat: support in-place lora merge/unmerge under layerwise offload (#36192) 2026-09-27 17:54:40 +08:00
Liangsheng Yin 37556c1b91 [mem_cache] Replace cache_finished_req with insert_req; release_kv_cache frees and unpins (#41281) 2026-09-27 02:53:16 -07:00
Xiaoyu Zhangandkernel-design-agents 0bc074db83 [KDA+Kimi K3] Speed up SANA-Video residual gate add on H200 (#41305)
Co-authored-by: kernel-design-agents <334018530+kernel-design-agents@users.noreply.github.com>
2026-09-27 16:57:12 +08:00
Qiaolin Yu aa7a976807 [qwen 3.8 next] Fuse small CUDA graph input buffer copies (#41166) 2026-09-27 01:27:56 -07:00
ShunkangzandZhangheng df1fc6ab64 [kv-shard 3/4] Enable Control Plane B (#39964)
Co-authored-by: Zhangheng <hzh0425@apache.org>
2026-09-27 14:19:08 +08:00
Jianghaiandaimicahchen acc15c193b [Diffusion] Preserve per-sample rollout trajectories across multi-output merge (#34416)
Co-authored-by: aimicahchen <aimicahchen@tencent.com>
2026-09-27 13:08:45 +08:00
Shuwen Wang 38d865489a [DSV4.1][HiCache] fix: wait for the layer transfer before reading low-ratio index-K (#41345) 2026-09-26 22:04:03 -07:00
Liangsheng Yin d27efca353 [mem_cache] Never free the protected prefix on request release (#41312) 2026-09-26 22:00:02 -07:00
Kevin Miandx effb752188 dsv4.1-amd: KV cache layouts, FP4 indexer, compressor and router kernels (#41019)
Co-authored-by: x <x>
2026-09-26 19:44:47 -07:00
Ankith Averineni 425a1f8f24 [AMD] Honor an explicit triton moe_runner_backend for mxfp8 on ROCm (#41377) 2026-09-26 17:18:08 -07:00
Lianmin ZhengandLu Fang fc9e1c8d29 [MemCache] Fix LMCache component cursors and per-cache backend selection (#41328)
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com>
2026-09-26 11:24:39 -07:00
c10fa03fc4 Make sliding-window caching and speculative batch padding extensible (#41325)
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com>
Co-authored-by: jasonjk-park <jasonjk@fb.com>
2026-09-26 11:24:02 -07:00
Thomas Wang c73f7077eb [AMD] Fix jit broken on rocm env (#41356) 2026-09-26 09:59:59 -07:00
ArielandClaude Fable 5.1 e1e97e8b49 Pin triton_kernels num_warps for MXFP4 MoE below Hopper (6x gpt-oss decode on RTX 4090) (#41292)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-27 00:43:25 +08:00
Mohammad Miadh AngkadandMohammad Angkad cdbea5dccf [CI] Move DeepSelect into the attention kernel group to fix the namespace test (#41349)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-09-26 21:39:00 +08:00
Cheng Wan bd37e7f513 [Refactor] Choose a dense layer's boundaries under attention DP from both sides' declarations (#41257) 2026-09-26 02:41:43 -07:00
Cheng Wan 9cd6616c24 [Refactor] Pick the two-batch-overlap split's layout moves once and remove execute (#41256) 2026-09-26 02:41:09 -07:00
Cheng Wan d3fcf8997c [Refactor] Run the LayerNorm SP region's boundary steps in the communicator itself (#41255) 2026-09-26 02:40:34 -07:00
Cheng Wan 35b369ebba [Refactor] Declare at construction the layers whose FFN completes its own reduction (#41254) 2026-09-26 02:39:58 -07:00
Cheng Wan c081d4aacc [Refactor] Drive an FFN exit's flags and its completion from one selection (#41253) 2026-09-26 02:39:25 -07:00