Commit Graph
13071 Commits
Author SHA1 Message Date
Yueming Yuan 542a043f1b [RL] Release the weight-checker snapshot once compare passes (#37284) 2026-09-23 16:06:02 -07:00
James LiuandClaude Opus 5.5 208f6f7501 [Spec] Support DFLASH for Kimi K3 (#40794)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 15:58:39 -07:00
6fe4b66f5c [RL] Keep pause_generation and weight updates from deadlocking each other (#40779)
Co-authored-by: maocheng23 <maocheng@berkeley.edu>
Co-authored-by: Zhichenzzz <zczeng@uw.edu>
2026-09-23 15:13:42 -07:00
Cheng Wan 79fec592a8 [Refactor] Read parallel placement in consumers (#40638) 2026-09-23 14:57:25 -07:00
f2a1366584 MiniMax-M3: wave64 histogram-select decode top-k, and raise kMaxNumBlocks for CUDA graphs (#36560)
Co-authored-by: Kevin Mi <mikevin920@yahoo.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Kevin Mi <mikevin920@gmail.com>
Co-authored-by: Kevin Mi <45493463+kevin-mii@users.noreply.github.com>
Co-authored-by: YC Yen-Ching Tseng <yctseng@amd.com>
2026-09-23 14:50:13 -07:00
4fa2c9c1de MiniMax-M3: run the sparse prefill main attention through AITER Gluon paged attention (#36546)
Co-authored-by: Kevin Mi <mikevin920@yahoo.com>
Co-authored-by: Alex Sun <alex.s@amd.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Kevin Mi <45493463+kevin-mii@users.noreply.github.com>
Co-authored-by: Kevin Mi <mikevin920@gmail.com>
Co-authored-by: YC Yen-Ching Tseng <yctseng@amd.com>
Co-authored-by: Kevin Mi <kevin.mi@radixark.ai>
2026-09-23 14:49:10 -07:00
Cheng Wan f766397f2e [Refactor] Trim server configuration and runtime context comments (#40971) 2026-09-23 14:47:13 -07:00
Liangsheng Yin c39a1c0663 [mem_cache] Free the rows below the SWA evict floor on all-SWA request release (#40798) 2026-09-23 14:46:28 -07:00
Cheng Wan ab0c31bd94 [Refactor] Move eleven more MoE models to LayerCommunicator.ffn_exit (#40871) 2026-09-23 14:43:41 -07:00
Cheng Wan e1048a215c [Refactor] Let LayerCommunicator own the FFN exit in Qwen3-MoE, DeepSeek-V2 and GLM4-MoE (#40870) 2026-09-23 14:42:58 -07:00
Cheng Wan 7dd9640752 [Refactor] Compare token layouts instead of group sizes when selecting communicator paths (#40869) 2026-09-23 14:41:55 -07:00
Cheng Wan e31cbc01a2 [Fix] Stop deferring the last layer's FFN all-reduce in five models (#40868) 2026-09-23 14:41:14 -07:00
Cheng Wan 701cf7e47e [Refactor] Run Nemotron-H DP attention through the standard layer communicator (#40867) 2026-09-23 14:40:33 -07:00
Cheng Wan 379e8f916b [Fix] Capture complete Nemotron auxiliary hidden states (#40801) 2026-09-23 14:39:50 -07:00
Cheng Wan 9ebe42238b [Fix] Reduce Nemotron MTP attention outputs once (#40800) 2026-09-23 14:39:04 -07:00
Cheng Wan 0e80c73c92 [Fix] Avoid duplicate residual in LongCat MoE shortcut (#40799) 2026-09-23 14:37:39 -07:00
Liangsheng Yin 954458567e [mem_cache] Remove unreachable RadixCache paths in KV canary and HiCache accessors (#40807) 2026-09-23 13:22:58 -07:00
2909f84e3b [JIT] Add an occupancy-preserving L1 carveout preference (#40767)
Co-authored-by: metamergebot <324680979+metamergebot@users.noreply.github.com>
Co-authored-by: Oguz Ulgen <oulgen7@gmail.com>
2026-09-23 11:49:34 -07:00
890a9605ce [Metrics] Log forward and forward+idle occupancy over total wall time (#40802)
Co-authored-by: metamergebot <324680979+metamergebot@users.noreply.github.com>
Co-authored-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
2026-09-23 11:48:31 -07:00
4bb5611dba [Fix] Give the full prefill CUDA graph replay view the captured bucket's input_ids (#40851)
Co-authored-by: metamergebot <324680979+metamergebot@users.noreply.github.com>
Co-authored-by: fei-xx <135589532+fei-xx@users.noreply.github.com>
2026-09-23 11:48:03 -07:00
f2f223e697 Allow attention layers to opt out of the prefill wrapper (#40683)
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com>
Co-authored-by: metamergebot <324680979+metamergebot@users.noreply.github.com>
2026-09-23 11:39:22 -07:00
Byron Hsu f2eebd5533 [RL] Fix Kimi K3 expert-count lookup for routed-expert capture (#40700) 2026-09-23 11:18:04 -07:00
7fef014d51 feat(kv-hints): add kv hint envelope to request transport (#38891)
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
Co-authored-by: Ishan Dhanani <ishandhanani@gmail.com>
2026-09-23 10:31:13 -07:00
Even Zhou 5c154c214d Revert " [NPU] Enable piecewise CUDA graph support on NPU" (#40895) 2026-09-23 23:38:14 +08:00
Xiaoyu Zhang 8993f790be [Diffusion] Fuse lossless SenseNova RoPE for 5% faster H200 inference (#40374) 2026-09-23 22:37:22 +08:00
Xiaoyu Zhang 4e60d70ba4 [Diffusion] Fuse Joy Image Edit QKV concatenation and avoid QK copies (#40494) 2026-09-23 22:20:14 +08:00
Xiaoyu Zhang 172b1b4825 [Diffusion] Accelerate Cosmos3 Edge on Hopper with lossless fusions (#40386) 2026-09-23 21:16:58 +08:00
iridiumine 16e353d93e [NPU] Skip fused gmm1+swiglu for swiglu_limit (SiLU-with-clamp) checkpoints (#40438) 2026-09-23 21:02:54 +08:00
iridiumine 3fdd63a562 [NPU] Fuse FIA KV-cache K/V writes into one npu_scatter_pa_kv_cache call (#40445) 2026-09-23 21:02:39 +08:00
Xiaoyu Zhang abef3efb64 [Diffusion] Fuse rounded SwiGLU for quantized MiniMax-H3 MLPs (#40378) 2026-09-23 19:55:48 +08:00
hanwlax 2d25767759 [NPU] Enable piecewise CUDA graph support on NPU (#28417) 2026-09-23 17:30:30 +08:00
inkcherryandDuyi-Wang 58f622ebc4 Reduce decode bootstrap latency with request-owned speculative KV (#38978)
Signed-off-by: inkcherry <mingzhi.liu@amd.com>
Co-authored-by: Duyi-Wang <duyi.wang@amd.com>
2026-09-23 02:24:53 -07:00
TianDi101andDuyi-Wang 4cd63da996 [Unified Cache] Dedup replicated MLA/DSA KV in the UMBP direct linker (#38778)
Co-authored-by: Duyi-Wang <duyi.wang@amd.com>
2026-09-23 02:15:01 -07:00
a89f849158 [ROCm] Fuse the MLA q absorb into the RoPE + KV-write kernel on gfx950 (#38340)
Co-authored-by: JohnQinAMD <zhengsheng1997@gmail.com>
Co-authored-by: RuibinCheung <ruibzhan@amd.com>
2026-09-23 02:07:54 -07:00
siliangchen-amd 401d5aedf3 [AMD] Drop the unreachable vLLM fallback from ROCm FP8 activation quant (#40879) 2026-09-23 01:59:15 -07:00
Yoray Zack de123f38bb [3/N] elastic-ep: Recapture decode CUDA graphs after scale-up (#33723) 2026-09-23 01:51:30 -07:00
aa0b65c7c7 [AMD] Fix DeepSeek-V4 accuracy by not passing num_token_non_padded to MoE topk (#39804)
Co-authored-by: Thomas Wang <thomawan@amd.com>
Co-authored-by: HAI <hixiao@gmail.com>
2026-09-23 01:35:56 -07:00
RuibinCheungandxiaobochen-amd 48c3854620 [ROCm] feat: enable aiter allreduce fusion for GLM models (#39790)
Co-authored-by: xiaobochen-amd <xiaobo.chen@amd.com>
2026-09-23 01:24:08 -07:00
MatsueYu d4dcce12d4 [npu] decoding procedure optimization on qwen3.5/3.6 (#35958) 2026-09-23 15:23:51 +08:00
ZiruandNiu Ziru 0b0f947265 [CPU] Add fused_sigmod_mul_cpu operators to the Meta Muse Glimmer model. (#39538)
Co-authored-by: Niu Ziru <niuziru@a4bf018d3341.jf.intel.com>
2026-09-23 15:12:43 +08:00
ANSHUMAN TRIPATHY eb5e8c84be Speculative Decoding with NGRAM support for XPU (#31362) 2026-09-23 14:04:20 +08:00
a3d11f9d95 [diffusion] feat: support permanent lifetime for layerwise resident layers (#40599)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-23 13:49:09 +08:00
28be39f72e [deepep_v2] support GLM-5.3-Flash (Glm5NextForConditionalGeneration) (#40466)
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-22 22:32:34 -07:00
jacky.chengandEddie A. Tsai 58988bee2f [AMD] Add diffusion (Wan2.2) extras to gfx1151 Docker image (#40844)
Co-authored-by: Eddie A. Tsai <56776714+vvvdwbvvv@users.noreply.github.com>
2026-09-22 22:18:55 -07:00
Jeff HansonandJeff Hanson 5b8d8b2f88 [Fix] Keep Inkling automatic tool grammar active across the response (#40468)
Co-authored-by: Jeff Hanson <2469458+jshanson7@users.noreply.github.com>
2026-09-23 13:14:36 +08:00
Shuwen Wang ced0a2c98c [Hisparse] fix: account for MiniMax HiSparse full-pool memory (#40743) 2026-09-23 13:13:48 +08:00
Eric Zhang 9d267655e3 Fix NIXL transfer of MXFP8 KV block scales (#40792) 2026-09-23 13:12:04 +08:00
973fb44471 [Diffusion] Support MiniMax-H3 PDD(Parallel Decoding Distillation) inference (#40568)
Co-authored-by: Forrest Lin <forrestl@nvidia.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-23 12:57:26 +08:00
c2bc456b64 [chore] point agents at the cookbook before test configs (#40818)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-23 10:54:15 +08:00
Shuwen Wang 2bc435d32e [HiCache] fix: bound the controller reset join so a stalled storage thread cannot hang the scheduler (#40312) 2026-09-23 10:52:03 +08:00