Commit Graph
14070 Commits
Author SHA1 Message Date
YAMY b3be2e7402 [dsv4] Pad MLA decode q-heads to 64 (not full n_heads) for FlashMLA head64 kernel (#27954) 2026-06-15 17:18:10 -07:00
Liangsheng Yin 14f6348524 [Fix] Demote OpenAIServingResponses init failure log to one-line WARNING (#28349) 2026-06-15 16:58:33 -07:00
Hanming Lu 81dcb00673 [Spec v2] Use decode kernel for TRT-LLM MHA draft extend (#28241) 2026-06-15 16:57:08 -07:00
Kangyan-ZhouandClaude Fable 5 cad43d3212 [CI] Reclaim leaked /dev/shm segments on server startup (#28089)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-06-15 16:14:08 -07:00
zijiexiaandClaude Opus 4.8 7221be2cec feat(cookbook): MTP --max-running-requests callout + skill sync (#28340)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 15:53:01 -07:00
Baizhou Zhang 4c0457f440 [misc] Update codeowner (#28342) 2026-06-15 14:42:01 -07:00
Yi Zhong 3a0dd69f8e Minor refactorings to the LFM2.5 cookbook for accuracy (#28072) 2026-06-15 14:39:10 -07:00
jinhaosong-source 30d8ee87b0 Add OrcaRouter usage example (#28004) 2026-06-15 21:15:26 +00:00
Jia Guo 4ed698a491 fix(fa3): no NaN embeddings with fa_skip_kv_cache under piecewise CUDA graph (#27343) 2026-06-15 13:46:22 -07:00
YAMY f870bf1ed0 [dsv4] Prewarm MHC prenorm kernel at startup (#27986) 2026-06-15 13:26:26 -07:00
Lianmin ZhengandIan O'Connell 7e629a2f8c Allow overriding tokenizer path in benchmark harness (#28280)
Co-authored-by: Ian O'Connell <ianoc@meta.com>
2026-06-15 13:07:50 -07:00
33719cfb31 [PD] Optimize SWA allocation (#28085)
Co-authored-by: cctry <cctry@fb.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
2026-06-15 11:01:55 -07:00
YC Yen-Ching Tsengandbingxche 19e85868f6 [AMD] Point AITER scout at amd/aiter-ci (#28313)
Co-authored-by: bingxche <bingxche@amd.com>
2026-06-15 22:50:16 +08:00
Shangming Cai 378e66d248 [PD] Remove outdated backend whitelist for decode radix cache (#28238)
Signed-off-by: Shangming Cai <csmthu@gmail.com>
2026-06-15 22:36:05 +08:00
Trevor Morris 20f4272109 fix: Fix DSR1 perf regression due to unnecessarily falling back to triton gemm (#28073) 2026-06-15 09:45:09 -04:00
YAMY d5899b95c4 fix(qwen3.5): keep CUDA dual-stream overlap (regressed by #25885) (#27868) 2026-06-15 09:44:21 -04:00
littleyellowbicycle 09e9c4fde3 【bugfix】The NPU's forward_dsa_prepare_npu also needs special handling for is_nextn (#28118) 2026-06-15 21:31:39 +08:00
Kurkur e985422b2b [Fix][MTP][MM] Fix EAGLE v2 chunked-prefill next-token chain crash on multimodal models due to placeholder tokens (#27863) 2026-06-15 21:30:45 +08:00
qinsir5522 81166f382d Fix inaccuracies and add NPU constraints in ascend_npu_profiling.mdx. (#28283) 2026-06-15 21:12:18 +08:00
jianzhao-xu f8d1d397b6 [NPU] fix ascend_docs (#28279) 2026-06-15 21:11:55 +08:00
McZyWu 8bdb007e58 [NPU] Docs op performance optimize (#28277) 2026-06-15 20:28:18 +08:00
Mick 818808d152 [diffusion] optimize: optimize causal conv3d vae padding (#28204) 2026-06-15 20:18:08 +08:00
loading66 f768344b1a [DOCS][NPU]Supplementary Notes (#28295) 2026-06-15 20:06:05 +08:00
longxin9715 7bd1a9d163 Update documentation for Ascend NPU Guide (#28284) 2026-06-15 20:02:50 +08:00
amote-i edd5eff519 [NPU] [DOC] fix issues in ascend_npu_support_new_models (#28296) 2026-06-15 19:58:33 +08:00
iridiumine 3df6e2f968 [NPU] Add MiMo-V2-Flash manual testcases (#28223) 2026-06-15 19:57:49 +08:00
eb349efb14 [EPD][BugFix] Fix encode_with_global_cache_mooncake (#28031)
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
Co-authored-by: Michael Qiu <qiudayu.qdy@antgroup.com>
2026-06-15 19:47:52 +08:00
c4ec39a785 [AMD] refactor sparse MLA decode kernel for Deepseek V4 triton backend (#28265)
Co-authored-by: Raiden-Makoto <Raiden-Makoto@users.noreply.github.com>
Co-authored-by: yichiche@amd.com <jacky.cheng>
2026-06-15 03:26:59 -07:00
Thomas Wang da12f36629 [AMD] Refactor unified_kv attention metadata to data class and fuse c4/128 out_loc (#28275) 2026-06-15 02:59:01 -07:00
Lianmin Zhengandgemini-code-assist[bot] 3b419f66da [JIT] Track angle-bracket includes in source hash (#28273)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-06-15 02:58:14 -07:00
Zhonghua Deng 2a33724c9b [perf] Reuse a pooled HTTP session for multimodal URL downloads (#28056)
Signed-off-by: Abatom <abzhonghua@gmail.com>
2026-06-15 17:47:47 +08:00
McZyWu bf186cf8fc bugfix revise interface get cpu copy for npu mem pool to align with gpu (#27802) 2026-06-15 17:24:10 +08:00
monkeyLoveding 1180b70440 triton-ascend update (#27624) 2026-06-15 17:20:19 +08:00
Bingxu Chen 9864059e2b [AMD] Update AITER commit (#28249) 2026-06-15 02:04:34 -07:00
Bingxu ChenandCursor 19c78552dc [AMD] Restrict CI image fallback to versioned tags (#28263)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-15 16:08:48 +08:00
Xinyuan Tongandzijiexia 33f99831f8 docs(minimax-m3): refresh B200 benchmarks (tp8, piecewise) + add GPQA (#28207)
Co-authored-by: zijiexia <37504505+zijiexia@users.noreply.github.com>
2026-06-15 00:15:38 -07:00
Jia-Wei Jiang a88ba6cc0b [Fix] Reduce power of two to constant time (#28228)
Signed-off-by: JiangJiaWei1103 <waynechuang97@gmail.com>
2026-06-14 23:25:22 -07:00
Duyi-Wang 63df86f5e7 [AMD] Skip eplb bookkeeping and topk remap when EPLB is not in use on mori-ep / HIP (#22985) (#28188) 2026-06-14 23:00:23 -07:00
Mick 578e936d8d [diffusion] feat: persist torch.compile inductor/triton cache across restarts (#28205) 2026-06-15 13:34:19 +08:00
07b9108348 [Diffusion] FLUX: fuse FeedForward GELU into up-proj GEMM (cublasLt epilogue) (#28166)
Co-authored-by: BBuf <xiaoyu.zhang@radixark.net>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 13:00:26 +08:00
Prajjandprajjwal1 441b75ee69 [quantization] NVFP4 MoE: split fused w13 gate/up global scales (#27588)
Co-authored-by: prajjwal1 <prajjwal1@protonmail.com>
2026-06-14 21:18:36 -07:00
Ryan Zzzandzhujunyu ce9fad7196 [Bugfix][DeepSeek-V4] Fix Spec V2 Draft Input ID Dtype for DP Collectives (#28043)
Co-authored-by: zhujunyu <zhujunyu.666@bytedance.com>
2026-06-14 21:11:53 -07:00
weireweire bf38a0b03d Fix disaggregated decode load token accounting (#25736) 2026-06-15 11:37:41 +08:00
0417951a86 [Bug Fix] Validate tokenizer-dependent features with skip_tokenizer_init (#27882)
Co-authored-by: Randall <randall@iterationlab.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-14 20:07:52 -07:00
Zyann 37505eca27 feat: report multimodal (image/audio/video) token counts in usage.prompt_tokens_details (#27122) 2026-06-15 11:04:10 +08:00
Kangyan-ZhouandClaude Opus 4.8 c0dfe4c8ec [router] Reconcile workers that registered without resolving model_ids (#27980)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 18:47:50 -07:00
Mohammad Miadh Angkad 69b02ea68a [Distributed] Guard torch symm mem all-reduce sizes (#24548) 2026-06-14 18:28:57 -07:00
Xinyuan Tong 1a66059c4e [Spec] Restore index_share_for_mtp_iteration in EAGLE V2 draft worker (#28192) 2026-06-14 18:01:11 -07:00
Michael c127ba6483 [AMD] ci: fix scheduled AMD runs startup failure when calling extra-a suite (#28214) 2026-06-14 17:43:47 -07:00
Xinyuan Tong 000fc975c7 ci(docker): support layered overlay images in release-docker-dev (#28206) 2026-06-14 16:34:02 -07:00