Commit Graph
15502 Commits
Author SHA1 Message Date
Hanming Lu d03c8cee80 Negotiate PrefillDelayer only after KV-budget admission checks (#31835) 2026-07-21 12:07:30 -07:00
Hanming Lu 927979e127 Fix pad-row top-k masking with custom_routing_function under DP attention (#31838) 2026-07-21 11:49:17 -07:00
9a6c96083f [Cookbook] Add Laguna-S-2.1 (#31918)
Co-authored-by: Zijie Xia <zijie.xia@radixark.ai>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: zijiexia <37504505+zijiexia@users.noreply.github.com>
2026-07-21 17:01:50 +00:00
Yanbin Jiang 57e5846b90 [LoRA] Guard TMA down path for LoRA hooks (#31608) 2026-07-21 09:47:17 -07:00
Ke Bao 604b1507d2 Add Inkling to nightly test (#31939) 2026-07-21 23:06:11 +08:00
Zheng Wengangandsiyu 6f55de0468 [EPD] Make encoder register/unregister health-check robust (#31576)
Co-authored-by: siyu <liusy58@linux.alibaba.com>
2026-07-21 22:36:11 +08:00
Артем Савкин 8260ade61b [Bugfix] Fix CUDA import on non-CUDA platforms (#31919) 2026-07-21 21:47:46 +08:00
Xiaoyu Zhang 075bd97952 [Benchmark] Remove obsolete auto-benchmark remnants (#31941) 2026-07-21 20:44:52 +08:00
Ke Bao 0a06cc5317 Fix extra_buffer_lazy guard bypass (#31942) 2026-07-21 20:41:35 +08:00
e4eea7ce2f Optimize LongCat-Flash router GEMM with the HPC-Ops bf16xfp32 kernel (#30247)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Halcyon <56064364+VAthree@users.noreply.github.com>
2026-07-21 20:05:17 +08:00
qyb233 303896a475 [Bugfix] Place empty Qwen encoder-DP embeddings on the communication device (#31663) 2026-07-21 19:54:36 +08:00
YAMY 7c9257529f [Mamba] Support speculative decoding with extra_buffer_lazy (#30437) 2026-07-21 19:38:03 +08:00
1a53f231d6 [HiCache]: Optimize hybrid/DSA L3 prefetch result sync and usable-prefix clamping (#31443)
Co-authored-by: AlejandroParedesLT <alejandroparedeslatorre@gmail.com>
Co-authored-by: Kevin Flansburg <kevin.flansburg@gmail.com>
Co-authored-by: Chao Shi <chao.shi@alibaba-inc.com>
2026-07-21 18:59:51 +08:00
df39a7b0b6 [CI][PD] Add NIXL disaggregation functional tests (#27894)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
2026-07-21 18:17:06 +08:00
Ke Bao 2cf2e7377f Fix dropped tool calls when a stream delta carries several (#31860) 2026-07-21 18:07:43 +08:00
Chao ShiandZhangheng f369a820d4 [HiCache] Remove redundant parameters of build_xxx_stack and others (#31308)
Co-authored-by: Zhangheng <hzh0425@apache.org>
2026-07-21 17:54:06 +08:00
dcd9014f15 [AMD][MXFP4] Reland "Online MXFP4 quantization 2/N - FP8 to MXFP4 requantization on AMD GPUs" (#28291)
Co-authored-by: Bowen Bao <bowenbao@amd.com>
Co-authored-by: HAI <hixiao@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-07-21 02:50:23 -07:00
Alison Shao 696e8f80d1 [CI] Graceful teardown in kl_mamba hicache tests to release pinned host pool (#31871) 2026-07-21 02:09:26 -07:00
Yuan Luoandluoyuan.luo 66ea79dc2e Update FLA directory in CODEOWNERS (#31916)
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
2026-07-21 01:57:10 -07:00
wenxuewuhd c0ed009f5b [NPU] Fix LLaDA2 MoE OOM after the FRACTAL_NZ cast, re-enabling the NZ speedup (#31772) 2026-07-21 16:09:34 +08:00
Lianmin Zheng 429f6b6d15 Turn on breakable prefill cuda graph for dp attention by default (#31682) 2026-07-21 00:03:43 -07:00
Yuhao Yang 37a830b667 [Feature] Add DWDP (Distributed Weight Data Parallelism) for MoE prefill (#29778) 2026-07-20 23:59:54 -07:00
JINO ROHITandgithub-actions[bot] 01f558d905 Sm120 scatter fallback (#31669)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-07-20 23:58:27 -07:00
Michael c4c405a46b [AMD] batch 3: register newly-added JIT kernel benchmarks for jit-kernel-benchmark-test-amd (#31615) 2026-07-20 23:51:51 -07:00
iridiumine d6ef68881e [NPU] Adapt MiMo-V2.5-W8A8 (#29131) 2026-07-21 09:16:48 +08:00
Rahul Vijayaraghavan fa0ced195e [XPU] Enable breakable prefill CUDA graph on XPU (#30273) 2026-07-21 09:09:40 +08:00
Zaili Wang bfefdc52d7 [CPU] Fix mxfp4 padding size (#31334) 2026-07-21 09:03:24 +08:00
paulzhang-tm d093c6a4bb [FullCG] Preserve attention LSE through the custom-op boundary (#31050) 2026-07-21 09:01:40 +08:00
Yang Liu 9462c303a5 Fix stop boundaries for grammar-constrained speculative decoding (#31738) 2026-07-20 17:55:27 -07:00
Mick 4682ded472 vlm: parallelize multimodal preprocessing with customized worker num (#31438) 2026-07-21 08:44:58 +08:00
Lianmin ZhengandJason Park e7e8aaa73c Overlap grammar (constrained decoding) with speculative decode verify (#31488)
Co-authored-by: Jason Park <jasonjk@fb.com>
2026-07-20 17:36:05 -07:00
zijiexia 8905cbd42f Fix MiniMax-M3 crash on ROCm by making its override fields resolvable (#31837) 2026-07-20 16:14:12 -07:00
zijiexiaandClaude Opus 4.8 0a2d3ca071 [cookbook] Inkling: add measured accuracy numbers to benchmark cards (#31823)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-20 22:43:35 +00:00
Mohammad Miadh Angkad a82ead53bd Make Q contiguous before TRT-LLM MHA decode (#31667) 2026-07-20 15:21:00 -07:00
Liangsheng Yin 7fe9ad25ac [Spec] Extract DFlash compact draft-cache rebuild helpers (#31677) 2026-07-20 15:07:55 -07:00
Kaixi 91b210f7b0 [GLM5][MoE] perf: Write FlashInfer TRT-LLM MoE output directly (#28416) 2026-07-20 13:45:23 -07:00
e856eae921 use sgl_kernel_npu rmsrope accelerate llada2 (#27127)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: ronnie_zheng <zl19940307@163.com>
2026-07-20 22:46:17 +03:00
Liangsheng Yin ff6c755952 [Refactor] Unify logprob results into a single LogprobResult and rename chunk env vars (#31733) 2026-07-20 12:44:27 -07:00
Douglas YangandClaude Opus 4.8 e149cdb337 docs(cookbook): revert MiniMax-M3 to dev image (model not yet in a release) (#31819)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 12:16:47 -07:00
Lianmin Zheng 54aaedd76d Clean up prefill CUDA graph runner (#31654) 2026-07-20 12:14:21 -07:00
RockduandClaude Opus 4.8 4e8eb1457b [Diffusion] msgpack raw-bytes transport (drop base64/JSON) (#31565)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-20 11:32:02 -07:00
Ke Bao 5ab3d90b81 Fix dropped Inkling reasoning at stream end (#31787) 2026-07-20 23:23:15 +08:00
Ke Bao 7fc545b649 Align reasoning_effort schema across chat, tokenize, and responses (#31784) 2026-07-20 23:22:43 +08:00
YC Yen-Ching Tsengandbingxche b6be150798 [AMD] Split ROCm 7.2 Stage-B large 1-GPU tests into three partitions (#31792)
Co-authored-by: bingxche <bingxche@amd.com>
2026-07-20 23:16:10 +08:00
c41c573ce9 [GDN] Support ReplaySSM Ring Spec-Verify (#28695)
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: vincentzed <207368749+vincentzed@users.noreply.github.com>
2026-07-20 22:06:30 +08:00
monkeyLoveding d1c2a1de08 [NPU] memfabric-zbal update (#31777) 2026-07-20 22:01:28 +08:00
heziiop 370f454e3d [NPU] fix modelslim quant tensor name (#31456) 2026-07-20 20:32:57 +08:00
Mohammad Miadh Angkad 3d82dacd58 Bump CuTe DSL to 4.6.0 (#31714) 2026-07-20 02:11:59 -07:00
amote-i 97e0647bdc [NPU] [DOC] Fix issues about npu docs found by aidd (#31302) 2026-07-20 16:31:27 +08:00
Jinyan Yi fce5c75a30 [NPU] Fix vit graph tnd cu seqlens (#31701) 2026-07-20 16:20:33 +08:00