Commit Graph
6055 Commits
Author SHA1 Message Date
bookinstockandMick Qian 32f34e898b [diffusion] fix: honor requested inference step count for MiniMax H3(#38671)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-10-10 09:02:57 +08:00
6acaf3374a [kernel] SwiGLU-OAI MXFP8 producer; move swizzled MXFP8 producers to ops/quantization (#42810)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Denji-kk <51020465+Denji-kk@users.noreply.github.com>
2026-10-10 08:51:57 +08:00
Yuwei AnandYuwei An 2d38934493 [DSv4.1] mxfp8 dispatch cache (#42156)
Co-authored-by: Yuwei An <yuwei.an@MacBook-Pro-yuweian.local>
2026-10-09 16:36:28 -07:00
Liangsheng Yin 46498bc8c5 [MLX] Sync decode KV to the pool once at release, clamped to the owned prefix (#43292) 2026-10-09 16:20:45 -07:00
Liangsheng Yin 114cb44920 [MLX] Read radix prefix slots from the model runner's req_to_token pool (#43291) 2026-10-09 16:18:58 -07:00
Liangsheng Yin 8020ea2ee9 [MLX] Drop a retracted request's runner state at its re-prefill (#43284) 2026-10-09 16:16:59 -07:00
Liangsheng Yin f7e601b936 [MLX] Drain in-flight jobs before a retract-mode pause_generation (#43282) 2026-10-09 16:14:23 -07:00
f6c848c882 [PD] Use 1 to disable decode host receive and 0 for always-on (#43407)
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: cctry <cctry@users.noreply.github.com>
2026-10-09 15:50:52 -07:00
Cheng Wan 58f6976ce7 [Fix] Unblock CUDA graph startup for hybrid parallelism (#43211) 2026-10-09 15:39:59 -07:00
cb9bd1d4d4 [Perf] Reduce multi-detokenizer router IPC sends (#42238)
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
2026-10-09 15:32:14 -07:00
66d840a8ef [cuda_graph] Size dp-local decode graphs by the runner's own gather requirement (#43303)
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: charlotte12l <charlotte12l@users.noreply.github.com>
2026-10-09 15:21:23 -07:00
Kevin MiandClaude Opus 5.5 6d8ddaf1d7 [AMD] MiniMax-M3 indexer CP: packed scoring for EAGLE verify rows (#42614)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 15:05:50 -07:00
fbf8d730fd [Fix] MiniMax-M3: default to the breakable prefill CUDA graph (#41845)
Co-authored-by: Hao Phan <htphan@nvidia.com>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
Co-authored-by: Kevin Mi <45493463+kevin-mii@users.noreply.github.com>
Co-authored-by: Po-Han Huang <pohanh@nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 15:05:05 -07:00
Kevin MiandClaude Opus 5.5 70e82f7b73 [AMD] Run the MiniMax indexer CP test on MI35X, not MI300 (#43259)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 15:04:25 -07:00
weireweireandweireweire 46c2585faa [DSpark] Fold draft sampling for DeepSeek-V4.1 heads under AUTO (#42987)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
2026-10-09 14:40:28 -07:00
Huabin 2f1acef3d8 [Model] Add LLaDA2.2 Block Routing MoE support (#31768) 2026-10-09 12:14:44 -07:00
Mohammad Miadh AngkadandMohammad Angkad 72eab59bf6 [GLM-5.3-Flash] Enable breakable prefill CUDA graph by default (#42845)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-10-10 02:41:02 +08:00
87845cda2b [Multimodal] Chunk and cache audio items whose offsets span several placeholder runs (#43301)
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: YavorGIvanov <YavorGIvanov@users.noreply.github.com>
2026-10-09 10:57:01 -07:00
3bffe69b74 [moe] Fill num_token_non_padded for the customized A2A backend at EP 1 (#43245)
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: cctry <cctry@users.noreply.github.com>
2026-10-09 10:55:02 -07:00
3c612132fd Overlap scheduler startup with the data parallel controller (#43177)
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: Yongji Wu <30348494+libertyeagle@users.noreply.github.com>
2026-10-09 10:54:22 -07:00
121aa8dcc7 [diffusion] perf: fuse MiniMax-H3's RMSNorm and indexed AdaLN under quality lossless (#43327)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 22:39:11 +08:00
xiaofei-zheng fba3ad4b1f [Frontend] Parallelize long chat prompt encoding (#41259) 2026-10-09 21:30:22 +08:00
xiaofei-zheng db32673b78 [AMD] Let the GLM DSA NextN draft declare its own shared-expert fusion architecture (#41258) 2026-10-09 21:28:45 +08:00
xiaofei-zheng d5c3ae9752 [AMD] Eliminate remaining gfx95 FP8 scale relayout copies (#41030) 2026-10-09 21:27:38 +08:00
78996e3f02 [Performance] Batch Mooncake hybrid-pool metadata and puts (#39696)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
Co-authored-by: Po-Han Huang <pohanh@nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 21:12:42 +08:00
348689da40 [PP+PD] Fix #38206 Reduce bootstrap consensus latency (#38959)
Co-authored-by: inkcherry <mingzhi.liu@amd.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 20:53:48 +08:00
Shuwen Wang 0955169ed2 [mem_cache] Bound FlexKV checkpoint stores and restore DSPARK cache checks (#35694) 2026-10-09 20:25:33 +08:00
Shuwen Wang f48ed2127e test: allow bounded checkpoint lag in Mamba HiCache tests (#42523) 2026-10-09 19:24:39 +08:00
Xiaoyu Zhang 0eb24e7d5b [Performance] Optimize Qwen3.8-Flash-Next BF16 decode on Blackwell (#42175) 2026-10-09 18:46:12 +08:00
516fd1c1b5 [diffusion] perf: tile MiniMax-H3's indexed modulation kernels by 2048 columns (#43267)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 18:23:52 +08:00
vOvandXiaoyu Zhang 7cfecbef56 [diffusion][model] Add native SANA-Video 2.0 support (#41492)
Signed-off-by: cr-gao <gaochenrui@sjtu.edu.cn>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
2026-10-09 17:28:03 +08:00
DevashishLal-CB 4debae7534 Add blackwell dual gemm fusion support for MLP (#41887)
Signed-off-by: Devashish Lal <devcode@meta.com>
2026-10-09 17:26:33 +08:00
Mohammad Miadh AngkadandMohammad Angkad 7578432c21 [LoRA][Test] Use the 0.9 ROUGE-L tolerance for the CUDA multi-LoRA batch test (#43297)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-10-09 01:51:01 -07:00
Cheng Wan dc0b394471 [Refactor] Remove Kimi K3's own layer communication and let the consumer set an FFN's exit (#42936) 2026-10-09 01:27:46 -07:00
Cheng Wan 305ad5503d [Refactor] Run Kimi K3's fused collectives through stage boundaries (#42898) 2026-10-09 01:27:23 -07:00
Cheng Wan 2b469040d2 [Refactor] Support attention-TP dense FFNs and unpadded batches on stage boundaries (#42897) 2026-10-09 01:27:00 -07:00
Cheng Wan b9ae3d1a65 [Fix] Run attention on a padded extend batch's real rows (#42896) 2026-10-09 01:26:38 -07:00
Cheng Wan 7203747cc8 [Fix] Shard Kimi K3's shared experts over the TP group they sum over (#42895) 2026-10-09 01:26:15 -07:00
Cheng Wan 125f88cb75 [Refactor] Run Kimi K3 through stage boundaries (#42894) 2026-10-09 01:25:51 -07:00
Cheng Wan 48a6c0a5d0 [Fix] Return an FFN's output to the attention rows at a pipeline handoff (#42892) 2026-10-09 01:25:05 -07:00
Cheng Wan b2402ccad6 [Fix] Merge DeepSeek-V4 two-batch-overlap microbatches without a residual (#42891) 2026-10-09 01:24:42 -07:00
b130f8af02 [sgl-router] Serve /v1/completions through /generate, with the shared OpenAI layer (#42997)
Co-authored-by: Kan Wu <kan.wu@radixark.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 16:04:04 +08:00
Gabriel Wu 217980566a fix(fa4): isolate sheared-bias workspaces by CUDA stream (#33000) 2026-10-08 23:37:02 -07:00
areneau-cfandJimmy Shong bd2d73daa5 [model] support Clef and Clef-Flash decision models on /v1/systemone (#42721)
Co-authored-by: Jimmy Shong <69131491+Jiminator@users.noreply.github.com>
2026-10-09 14:25:52 +08:00
08fc290f48 [AMD] gfx950 small-batch MoE: expert-count gate for the small sort and FP8 block-scale small-M kernel (#41982)
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: jacky.cheng <yichiche@amd.com>
2026-10-08 23:05:50 -07:00
cba1d07a94 [Memory] Share one full prefill graph output buffer across shapes for tuple and list outputs (#43176)
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: 842974287 <842974287@users.noreply.github.com>
2026-10-08 23:05:29 -07:00
e14ef4b658 [Scheduler] Skip the decode for a request whose last token is already in flight (#42720)
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
2026-10-08 23:00:46 -07:00
Colin Z 3c1703d2ff [AMD] Support Quark DeepSeek-V4-MXFP4 checkpoints on the DSV4 stack, enabling shared-expert fusion and MTP (#31054) 2026-10-08 22:54:49 -07:00
Yuan Luoandluoyuan.luo 9a1aeacc3b [GLM-5.3-Flash] Capture the KDA extend in breakable prefill CUDA graphs (#41105)
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
2026-10-09 13:37:20 +08:00
saatwiknagpal 6e7beace14 [diffusion] fix: fix int32 offset overflow in the Ulysses packed-QKV pack kernel (#43140) 2026-10-09 11:51:46 +08:00