Commit Graph
6055 Commits
Author SHA1 Message Date
Liangsheng Yin 64e2a73c80 [Fix] Serialize FanOutCommunicator queueing calls with a FIFO-fair asyncio.Lock (#30606) 2026-07-09 00:04:43 -07:00
weireweireandweireweire 1c9eb6bb0b [DSV4] Fix draft SWA transfer for disaggregated MTP (#30461)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
2026-07-09 13:18:34 +08:00
Haotong ZouandValentine233 3b43df5b6d Support speculative decoding on CPU (#27862)
Co-authored-by: Valentine233 <xuan.liao@intel.com>
2026-07-09 10:27:09 +08:00
jianan-guandMa Mingfei 177c048c68 [Diffusion][CPU] Adding AMX optimizations for CPU platform (#28527)
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
2026-07-09 10:26:22 +08:00
Michael cf8f1df6e8 [AMD] add dedicated jit-kernel-benchmark-test-amd stage + register portable JIT benches (#30307) 2026-07-08 18:09:05 -07:00
Xinyuan Tong 074bb928f0 Move template manager files under parser; update CODEOWNERS (#26052) 2026-07-08 16:47:36 -07:00
Cheng WanandClaude Opus 4.8 cc13e2eae7 [CI] Move piecewise CUDA graph (pcg) tests to nightly (#30563)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 16:07:08 -07:00
cctry 096551eed6 Make CUDA graph disabling PD-role-aware (prefill/decode) (#30409) 2026-07-08 15:08:22 -07:00
Lianmin Zheng ca8f15cd70 Fix FlashInfer A2A IMA by DP-synchronizing the decode graph bucket (#30242) (#30450) 2026-07-08 14:34:49 -07:00
Michael c55190a638 [AMD] Register 2 CPU-bound 1-GPU tests (phase_checker, scripted_runtime_core) for AMD PR CI (#30446) 2026-07-08 14:04:23 -07:00
Michael 47a6dfd708 [AMD] Register 3 ROCm-portable JIT kernel tests for AMD CI (#30212) 2026-07-08 13:38:17 -07:00
Xinyuan TongandEazyReal 45019b56ce [Bugfix] Map reasoning_effort=low to Nemotron-3 Super low_effort + warn on unsupported levels (#30463)
Co-authored-by: EazyReal <8047065+EazyReal@users.noreply.github.com>
2026-07-08 12:19:44 -07:00
sonle5andsunxxuns 8d2b66fd90 Fix gfx95 bpreshuffle FP8 activation scale layout (#29275)
Co-authored-by: sunxxuns <126995791+sunxxuns@users.noreply.github.com>
2026-07-08 10:55:09 -07:00
Xiaoyu Zhang b8ca06fdad Fix zero expert routed ids for MoE backends (#30387) 2026-07-08 21:23:25 +08:00
shuwenn 108a183f6b [mem_cache][6/N] refactor: move MHA host-pool into pool_host/mha.py (#30249) 2026-07-08 20:12:52 +08:00
amote-i c9303a08da Fix scheduler crash on prefill-unreachable decode abort (#29834) 2026-07-08 14:06:37 +08:00
Cheng Wan 7709a1f358 [refactor] ctx.resources: named slots, stream leases, and workspace buffer leases (#30348) 2026-07-07 21:30:10 -07:00
Cheng Wan b7cca0bf8f [refactor] Collect MoE and DP-attention runtime state into typed flag groups (#30347) 2026-07-07 21:29:27 -07:00
Cheng Wan be32c57598 [refactor] Read resolved config from server_args fields; retire the flags mirror tier (#30346) 2026-07-07 21:28:34 -07:00
Cheng Wan b14f7b4f75 [refactor] Move model-capability adjustments into the resolution pipeline (#30299) 2026-07-07 21:26:55 -07:00
7bc343470f [Spec] DFlash: support pure-MLA targets with an fp8 KV cache (Kimi-K2.x-NVFP4) (#29218)
Co-authored-by: Hao Phan <htphan@nvidia.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 19:52:14 -07:00
Yuchen Tian 9f5948c391 feat(mem_cache): add client-side metadata cache for HiCacheFile storage (#29716) 2026-07-07 18:21:46 -07:00
d7dcdf3efd [DSV4] perf: Make FP8 quant output tensor contiguous (#27926)
Co-authored-by: liqichao <liqichao@baidu.com>
Co-authored-by: chenbong <bhchen@stu.xmu.edu.cn>
2026-07-07 17:40:59 -07:00
YAMY 48ad6a83cf [DeepSeek-V4] Enable non-paged indexer by default for large prefill chunks (#30140) 2026-07-07 15:51:24 -07:00
Alex NailsandClaude Opus 4.7 3d2e7cc601 [gRPC] Native server: launcher + HTTP + server args wiring (3/4) (#23508)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-07-07 14:57:25 -07:00
Michael 60f502a4fd [AMD] Register 2 hardware-agnostic 1-GPU PR tests for AMD CI (#30207) 2026-07-07 14:46:22 -07:00
Michael 090efa27a2 [AMD] Register 5 CI-verified 1-GPU kernel/attention unit tests for AMD PR CI (#30290) 2026-07-07 14:44:45 -07:00
bbc537035a [DSA] Re-enable fused top-k v2 for MTP: clamp padded-row seq_lens to >= 0 (#30378)
Co-authored-by: ziyi.xu <ziyi.xu@radixark.ai>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 13:44:01 -07:00
cctry 2ad9a243f5 Size KV pool after CUDA graph capture (opt-in) (#30157) 2026-07-07 12:05:01 -07:00
e339c83f82 [Model] Support LongCat 2.0 FP8 (#30275)
Co-authored-by: sunjiaqi11 <sunjiaqi11@meituan.com>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
2026-07-07 19:51:12 +08:00
Jialin Ouyang 7fdc1cef17 [fix] Fix two trunk test regressions due to flexkv change (#29701) (#30372) 2026-07-07 16:58:45 +08:00
fefc1743a9 Cute-DSL FP8 MQA logits (#25220)
Co-authored-by: Mindy Li <11663212+limin2021@users.noreply.github.com>
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
2026-07-06 23:34:07 -07:00
Xiaoyu ZhangandClaude Opus 4.8 1da7d3a50b [MoE] Retire the AOT moe_fused_gate / kimi_k2_moe_fused_gate gate kernels (#26771) (#29997)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 13:53:17 +08:00
36b449af19 [EPD] Optimize multimodal global cache with paged embedding pool (#28441)
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
Co-authored-by: liusy58 <liusy58@linux.alibaba.com>
2026-07-07 12:05:52 +08:00
Zyann 541f9221da feat(metrics): add Prometheus metrics for the EPD encoder server (#27564) 2026-07-07 12:04:51 +08:00
Jialin Ouyang abafe0022e [Unified Radix Cache] Rename tree variables to cache in unittest (#30281) 2026-07-07 11:48:37 +08:00
16372b4c5f [Spec] Anchor GLM-5.2 MTP IndexShare topk on the draft-extend step (#29787)
Co-authored-by: kpham-sgl <264503018+kpham-sgl@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 20:36:48 -07:00
Siming Deng 4145e595cf [MLX] Add correctness tests for qwen2_moe and qwen3_moe (#29440) 2026-07-06 20:32:12 -07:00
Siming Dengandsiming-deng df06e03662 [MLX] Size the attention KV pool at the compute dtype for quantized models (#30097)
Co-authored-by: siming-deng <deng_siming@apple.com>
2026-07-06 20:31:59 -07:00
NOOB c3da0a2582 [MLX] Fix single-token chunked-prefill continuation misrouted as decode (#30181) 2026-07-06 20:31:33 -07:00
3a679459e5 [bench] Add agentic-trace multi-turn dataset to bench_serving (#29215)
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 19:45:44 -07:00
e85ef54877 Support Cutedsl BF16 GEMM JIT kernel (#30117)
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
2026-07-06 19:16:16 -07:00
Brayden ZhongandBrayden Zhong 267ff1b5f9 Fix LTX2 RoPE JIT kernel CI (#30278)
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
2026-07-07 10:15:21 +08:00
laixinandPeng Zhang 6279805962 [DSv4] Loading Time Weight Dequant (#27867)
Co-authored-by: Peng Zhang <aniz1905@gmail.com>
2026-07-07 09:54:34 +08:00
Yuwei An 3cbb7568bd [Experimental] Full Cuda Graph Support for Prefill (#27988) 2026-07-06 18:13:03 -07:00
Cheng Wan c861896721 [refactor] Resolve config declarations onto server_args at the end of __post_init__ (#30297) 2026-07-06 18:04:44 -07:00
Rahul VijayaraghavanandMa Mingfei 4b5c612257 Skip redundant moe_sum_reduce for single-expert routing on XPU (#22660)
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
2026-07-07 09:03:16 +08:00
Ma Mingfei 30fb0dd851 [CPU] add fused_qk_gemma_norm and refactor norm kernel implementation (#30216) 2026-07-07 08:52:59 +08:00
Lijuan Tang 1579a82d17 [MLX] Fix FakeOverlapScheduler test stub broken by forward_ct accounting (#30125) 2026-07-06 13:43:40 -07:00
hhhh1252023 1b481deade feat: sync npu nightly test improvements from Ascend testcases (#29403) 2026-07-06 22:41:15 +08:00