Commit Graph
6055 Commits
Author SHA1 Message Date
YAMY dd67452b4f [Cleanup] Move mamba-max-states-per-path validation into _handle_mamba_backend (#32502) 2026-07-28 14:21:21 -07:00
paulzhang-tmandClaude Fable 5 4e5a05148a [FullCG] Support chunked cached-prefix prefill (#30825)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 11:20:25 -07:00
IvanShan177andClaude Opus 4.8 1eee8fbdcc [PD] Drain NIXL completion notifications before enforcing the WaitingForInput timeout (#32267)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-29 00:20:03 +08:00
Xinyuan TongandFAN YUCHEN ee236086db Fix invalid escape warnings in tool parsers (#28370)
Co-authored-by: FAN YUCHEN <2994114386@qq.com>
2026-07-28 23:31:47 +08:00
32c30c0f96 Return 400 instead of 500 for unfetchable or unparseable multimodal inputs (#31417)
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
2026-07-28 05:23:42 -07:00
Xiaoyu Zhang 9cffc2ba52 [Kernel] Remove unused implementations and stale registry entries (#32636) 2026-07-28 18:12:10 +08:00
Zhangheng 60d6914f17 [UnifiedTree]: move /mem_cache/unifed_cache_component dir to /mem_cache/unified_cache (#32484) 2026-07-28 16:28:21 +08:00
Yuan Luoandluoyuan.luo d9cf7b0a8b [MTP] Cut spec-v2 host-seam overhead in hybrid-linear MTP decode (#32219)
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
2026-07-28 11:38:33 +08:00
longxin9715 356c11d5d9 [Fix] --mm-process-config crash when video config contains (#30260) 2026-07-28 09:24:40 +08:00
8d6549bc40 [Attention Backend] Extend hpc_ops dynamic-scheduled decode to bf16 (#32304)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Halcyon <56064364+VAthree@users.noreply.github.com>
2026-07-27 21:31:04 +08:00
1d350aaad3 fix(reasoning): let --enable-strict-thinking works for DeepSeek-V4 (#32400)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
2026-07-27 18:15:47 +08:00
Jackey HuaandClaude Opus 5 9a0bd24bed model: serve bare Qwen3Model backbone natively as an embedding model (#32457)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 15:49:58 +08:00
Sam Shleifer 5cc273a780 [feat] Opt-in flat response format for prompt top logprobs (#32078) 2026-07-26 23:44:34 -07:00
Yiqi Yangandhzh0425 4ea17169b0 [UnifiedTree] fix: drop prefetched host refill under an un-backed-up parent (#31902)
Co-authored-by: hzh0425 <hzh0425@apache.org>
2026-07-27 13:53:02 +08:00
Ethan (Yusheng) Su ee1736f39a [LoRA] Support LoRA under the breakable/full prefill CUDA graph (#30988) 2026-07-26 22:10:03 -07:00
DAI0818andybyang 2abb1d2c37 fix(hisparse): correct DSA KV memory budget (#31992)
Co-authored-by: ybyang <ybyang7@iflytek.com>
2026-07-27 11:19:32 +08:00
Mick abb8f4b5e3 model: support EmbeddingGemma (#32375) 2026-07-27 10:40:47 +08:00
xdtbynd 78d7928296 [UT][NPU] add NPU attention unit tests for ascend_backend and ascend_dsv4_backend (#32294) 2026-07-26 19:59:44 +08:00
YC Yen-Ching Tseng 1d0cd2e473 [AMD] Nightly Test Coverage - Minimax-M3-MXFP8 Accuracy Test (#30613) 2026-07-25 23:52:22 -07:00
ormandjandMohammad Miadh Angkad 2cbddb842d [DSV4/SM120] Allow fused MHC opt-in with standalone TileLang pre disabled (#30954)
Signed-off-by: David Orman <ormandj@corenode.com>
Co-authored-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>
2026-07-26 09:02:54 +08:00
55c4853487 [comm] Enable multi-node custom-AR v2 on a single NVLink clique (#32339)
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
2026-07-25 17:25:18 -07:00
2c63a2f12b Fix --hicache-size allocating ~2x host memory on hybrid SWA (#32373)
Co-authored-by: cctry <cctry@fb.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
2026-07-25 17:19:44 -07:00
Lianmin ZhengandAlec S 9989077f24 Use native batched llguidance mask generation (#32412)
Co-authored-by: Alec S <10566873+alecsolder@users.noreply.github.com>
2026-07-25 16:36:32 -07:00
Lianmin Zheng 5f330004bd Fix flaky test_sampling_mask: mask length can legitimately be top_k + 1 (#32410) 2026-07-25 16:05:57 -07:00
Liangsheng Yin 3da1071d56 [Spec] Hold the grammar bitmask in one GrammarMask type across all decode paths (#32409) 2026-07-25 15:27:43 -07:00
Lianmin Zheng d3cf4dfbaa Update audio container test time estimate (#32408)
Signed-off-by: Lianmin Zheng <lianminzheng@gmail.com>
2026-07-25 14:45:08 -07:00
Jialin Ouyang cd145f840f Radix Cache Split: Spin off TreeCore (#29901) 2026-07-25 14:31:59 -07:00
YAMY 91f386a5b2 fix(disagg): support pipeline-parallel hybrid-linear transfer (#32270) 2026-07-25 13:34:38 -07:00
Cheng WanandShu Wang 659d349b61 [core/loader] Add presharded load format (#24256)
Co-authored-by: Shu Wang <shuwanguc@google.com>
2026-07-25 13:03:39 -07:00
Ke Bao 69a3c54c70 Fix SWA admission livelock on cached-prefix resumes (#32379) 2026-07-25 22:37:35 +08:00
e943e609dc [DSPARK] Grammar-constrained decoding, incl. tool_choice=auto (#31753)
Co-authored-by: shanemort1982 <shanemort1982@users.noreply.github.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-07-25 05:20:25 -07:00
a678a42033 [KDA] Add target_verify support for speculative decoding (#26888)
Co-authored-by: yuyanqi <yuyanqi@meituan.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
2026-07-25 19:52:44 +08:00
7c4b22fae5 [Hicache][1/2]Support Mamba branching in Unified Radix Cache with HiCache (#31181)
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
Co-authored-by: Zhangheng <hzh0425@apache.org>
2026-07-25 19:44:40 +08:00
Mick 1054060ef1 perf: speed up marlin moe with occupancy-aware launch specialization (#31552) 2026-07-25 19:38:11 +08:00
Hồ Sỹ Thếandhnyls2002 d021990bf5 [DFLASH] Support grammar-constrained decoding in speculative verify (#30096)
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-07-25 04:36:46 -07:00
Liangsheng Yin 3c5bf1f6d2 [Spec] Derive NGRAM grammar tree links on the host instead of reading back retrive_next_token (#32380) 2026-07-25 02:25:16 -07:00
Xun SunandShangming Cai 9eb2dccbb7 [Elastic EP] Fix recovery lifecycle and add manual coverage (#31744)
Co-authored-by: Shangming Cai <csmthu@gmail.com>
2026-07-25 14:32:47 +08:00
f9c14e6bd4 [FEAT] Support fast engine recovery through weight cache (#27139)
Signed-off-by: Michael Qiu <qiudayu.qdy@antgroup.com>
Co-authored-by: liusy58 <liusy58@linux.alibaba.com>
Co-authored-by: Alex Nails <alex.nails@radixark.ai>
2026-07-25 14:31:21 +08:00
ebcb74abd4 feat(hicache): Add shared memory allocator for host KV cache (#29326)
Co-authored-by: Jimmy Shong <69131491+Jiminator@users.noreply.github.com>
Co-authored-by: Zhangheng <hzh0425@apache.org>
2026-07-25 12:05:37 +08:00
cctryandYinghai Lu ce705bb6dc Report accelerator type in /v1/loads (#32348)
Co-authored-by: Yinghai Lu <yinghai@meta.com>
2026-07-24 17:31:28 -07:00
DarkSharpnessandClaude Fable 5 9402012f0f [Perf] Halve the non-finite sanitization overhead in per_token_group_quant (#32296)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-25 08:17:12 +08:00
Pranjal Shankhdharandpranjalssh 9c483cccfe Support a same-size mixed q dtype in the fused RoPE kernels (#31834)
Co-authored-by: pranjalssh <pranjalssh@fb.com>
2026-07-24 16:19:08 -07:00
962c076934 Decode input_audio media containers with PyAV & Update memory profiler (#31832)
Signed-off-by: Shiyan Deng <dsy842974287@meta.com>
Signed-off-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: Lucia Fang <116399278+luccafong@users.noreply.github.com>
2026-07-24 16:18:45 -07:00
cctry 8727d105db Add prefill and decode load counters to LoadSnapshot (#32245) 2026-07-24 14:04:54 -07:00
mosya415andmosya415 71015f3fea fix(dsa): fail fast on fp8_e4m3 KV with tilelang DSA backend on CUDA (#31346)
Co-authored-by: mosya415 <263250241+mosya415@users.noreply.github.com>
2026-07-24 12:10:38 -07:00
Jinyan Chen 1e69765bae Add FP4 Indexer for DeepSeek V4 on SM120 (#27059) 2026-07-24 11:37:23 -07:00
YAMY 2428f56145 [Bugfix] Fix Kimi-Linear state transfer across heterogeneous TP (#32262) 2026-07-24 10:31:17 -07:00
Zhiqiang Xie 5da0b6ec39 Write-back policy fix for unified tree (#31845) 2026-07-24 09:58:18 -07:00
Lu Fang 448662e85e [mm] Accept per-item embedding lists from DataEmbeddingFunc (#31826) 2026-07-24 08:27:24 -07:00
Shangming Cai fa243fee40 [CI] Skip flaky test in CI for disaggregation group (#32324) 2026-07-24 21:59:20 +08:00