Commit Graph
10841 Commits
Author SHA1 Message Date
Xiaoyu Zhang c5bd3d7dce [diffusion][benchmark] Add reproducible request-manifest offline benchmark (#32917) 2026-07-30 22:11:18 +08:00
Xiaoyu Zhang 7784ac8f91 [diffusion][docs] Fix Cosmos3 model sizes (#32916) 2026-07-30 22:10:23 +08:00
Xiaoyu Zhang 2e9c82b359 [Kernel] Remove unreachable AOT headers (#32842) 2026-07-30 22:08:57 +08:00
cen121212 b78d3999b5 【NPU】fix decode MTP + eagle shape error (#32791) 2026-07-30 21:34:16 +08:00
Mick db3da62333 [diffusion] feat: unify encoder folding and batch data-parallel encoding (#30211) 2026-07-30 20:15:22 +08:00
Yi (Vincent) Zhong 1f04eaab6a Fix LFM 2 tool parser. (#27614) 2026-07-30 10:15:08 +00:00
YC Yen-Ching Tseng fd86795107 [AMD] MiniMax-M3: opt-in custom/quick all-reduce on ROCm (#32230) 2026-07-30 02:56:07 -07:00
Liangsheng Yin 9f56553408 [Perf] Fast-path chain-style draft token organization in multi-layer EAGLE (#32887) 2026-07-30 02:55:21 -07:00
YC Yen-Ching Tseng 04d6fb4d6c [AMD] Minimax-M3 : unblock mxfp8 block convert on gfx950 (#32036) 2026-07-30 02:43:27 -07:00
Zheng Wengang 4ba7d5ad93 [BugFix][EPD] Early-release mooncake GPU embeddings; fix gpu_id via scheduler.ps (#31591) 2026-07-30 17:36:55 +08:00
Liangsheng YinandKaixi 6ab3231b97 [Perf] Skip the target-verify tree mask fill when the backend never reads it (#32886)
Co-authored-by: Kaixi <kaiximatteoc@nvidia.com>
2026-07-30 02:32:38 -07:00
kangwangamd 4b52758c76 [AMD] Skip test_update_weights_from_disk on ROCm pending reload fix (#31924) (#31925) 2026-07-30 02:32:09 -07:00
Ding Yinandyinding fc007e1f00 Add SM90 FP8 MegaMoE support for DeepSeek-V4 (#29016)
Co-authored-by: yinding <yinding@bytedance.com>
2026-07-30 01:48:10 -07:00
f46d5f25b4 [4/N][CP] Support interleave strategy for cp v2 (#30482)
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
2026-07-30 01:45:32 -07:00
Liangsheng Yin c192145830 [Kernel] Fuse KV-cache writes for asymmetric K/V (head_dim != v_head_dim) (#32813) 2026-07-30 00:26:10 -07:00
Liangsheng Yin 2625fdfe6b [Fix] Count multi-layer draft-extend replays in the fwd-occupancy device timer (#32867) 2026-07-30 00:21:34 -07:00
Yanbin Jiang 92b3a51ba6 [LoRA] Fix Marlin MoE kernel import (#32884) 2026-07-30 15:17:29 +08:00
Ho-Ren (Jack) ChuangandClaude Fable 5 e4a40a71f8 [DSA] Q8KV8 FP8 Sparse Prefill on GLM-5.2 & DeepSeek-V3.2: Q8-Path & Shared-Path Optimizations (#31888)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-30 15:15:11 +08:00
Jinwoo Jeong 4f51dad1da fix: prevent ReqTimeStats from being dropped during IPC serialization (#31339) 2026-07-30 14:42:38 +08:00
Shangming Cai f6ff5e8bb0 [PD] Handle abort requests in PP mode (#32797) 2026-07-30 14:39:44 +08:00
wenxuewuhdandronnie_zheng 36afd442c7 [DLLM] vectorized joint/low-confidence decoding and skip redundant attn init (#21094)
Co-authored-by: ronnie_zheng <zl19940307@163.com>
2026-07-30 09:13:14 +03:00
Ke Bao 07a087bf45 Fix Inkling tool-call parsing recovery, content handling, and streaming (#32861) 2026-07-30 14:11:09 +08:00
Liangsheng Yin f4e0ac382e [misc] Remove unused multi_layer_draft_forward_cg module (#32881) 2026-07-29 21:17:16 -07:00
Jimmy Shong ed361ae7f0 Fix attention backends for models with per-layer head counts (num_attention_heads_per_layer) (#32625) 2026-07-29 20:03:00 -07:00
Mick 22faf9fef8 embedding: centralize capabilities and complete OpenAI compatibility (#32481) 2026-07-30 10:28:52 +08:00
Liangsheng Yin 313a518bee [Spec] Emit step trace span for multi-layer draft-extend graph replays (#32850) 2026-07-29 19:22:58 -07:00
Sam ShleiferandClaude Fable 5 62dfaaa0e0 [Nemotron] Fix decode track-save reading the stale tail of the CUDA-graph track buffer (#32555)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-29 19:03:44 -07:00
Xuanyi LiandR0CKSTAR 8fbf960980 [MLX] Size request capacity by attention DP (#32115)
Co-authored-by: R0CKSTAR <yeahdongcn@gmail.com>
2026-07-29 18:18:22 -07:00
Xiaoyu Zhang 1d9c292547 [Kernel] Add inventory guards and clean benchmark layout (#32788) 2026-07-30 09:03:24 +08:00
Mohammad Miadh Angkad a55e1764a2 Enable GPT-OSS FlashInfer MXFP4 on SM120 (#32668) 2026-07-30 00:04:23 +00:00
Liangsheng Yin e5c46ff07d [Fix] Route asymmetric-KV models to fa4 on SM100 and pin MiMoV2 FP8 MoE to flashinfer_trtllm (#32818) 2026-07-29 16:37:43 -07:00
cctry 3c1717d9b6 Follow up on #30157 post-merge review (#32672) 2026-07-29 15:03:59 -07:00
Sam (Kesen Li) 8fc54d46ef Fix MoE reduce-scatterv eligibility check (#32663) 2026-07-29 14:58:55 -07:00
Willow LopezandJiminator ffd4705baa fix(reasoning): honor Poolside template thinking defaults (#32540)
Co-authored-by: Jiminator <69131491+Jiminator@users.noreply.github.com>
2026-07-29 21:02:19 +00:00
Sam Shleifer d0e69d3881 [feat] Optional base64 encoding for the flat prompt top logprob arrays (#31960) 2026-07-29 12:15:56 -07:00
e4f7f7b380 fix(qwen3.5): restrict MoE weights to local PP layers (#32022)
Co-authored-by: chunjiang.hc <chunjiang.hc@alibaba-inc.com>
Co-authored-by: Yihao Wang <42559837+AgainstEntropy@users.noreply.github.com>
2026-07-29 12:01:09 -07:00
ziang663andChao Shi eefb434d17 [PD+PP] Honor PP consensus for bootstrap and prealloc (#31869)
Co-authored-by: Chao Shi <chao.shi@alibaba-inc.com>
2026-07-30 01:19:17 +08:00
Yoray Zack 62d0f81f16 [2/N] elastic-ep: Enable EPLB after scale-up (#30553) 2026-07-30 01:06:56 +08:00
Piotr Mazurek d19999b755 [LFM2] Wire Lfm2MoeForCausalLM into the LFM2 serving override tables (#30780) 2026-07-29 17:02:14 +00:00
Hert4andMohammad Miadh Angkad f69af7b7ad [Bugfix] compressed-tensors: mixed-precision checkpoints silently load unquantized (#32736)
Co-authored-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>
2026-07-29 09:55:33 -07:00
YAMY fddfc1fb5e [GDN] Support FlashInfer GDN prefill with extra-buffer radix cache (#29735) 2026-07-30 00:47:35 +08:00
huangtingweiandalphabetc1 50029f05a3 [HiCache] Merge HiCache event checks to reduce decode overhead (#30511)
Co-authored-by: alphabetc1 <47200617+alphabetc1@users.noreply.github.com>
2026-07-29 23:48:48 +08:00
Nicolas Castet e1f2f9d1fa Disable extra NCCL CUDA event synchronization with symm mem (#27089) 2026-07-29 22:39:36 +08:00
Wu Jiangming 1c6a0e91e1 fix mqa preshuffle layout issue for deepseek v4 (#31563) 2026-07-29 07:36:48 -07:00
Mick 22151edca1 [diffusion] optimization: accelerate CUDA video output finalization (#32784) 2026-07-29 22:04:20 +08:00
Xiaoyu Zhang 0ebbe43dbb fix(diffusion): size VSA top-k from padded blocks (#32695) 2026-07-29 21:58:41 +08:00
Xiaoyu Zhang 4f5b50c576 perf(diffusion): decode Wan VAE in BF16 (#32697) 2026-07-29 21:57:50 +08:00
Xiaoyu Zhang 917e900d4d feat(diffusion): add regional torch compile (#32696) 2026-07-29 21:57:05 +08:00
Ke Bao 50b029257f Skip mamba lock during decoding (#32228) 2026-07-29 21:53:28 +08:00
pllimax d004a15a3e Fix GLM4-7B-Flash accuracy test configuration, tune Qwen3.6-27B/35B performance test parameters, and harden Ascend NPU multi-node E2E test utilities against pod name format errors. (#32371) 2026-07-29 21:44:17 +08:00