Commit Graph
5847 Commits
Author SHA1 Message Date
Khoa Phamandvincentzed 8bf145d119 [DCP] Decode context parallelism for DSA (GLM-5.2 / GLM-5.3) with MTP (#43735)
Co-authored-by: vincentzed <207368749+vincentzed@users.noreply.github.com>
2026-10-13 11:35:34 -07:00
6ade23696d [Core] Keep request priority within the signed 64-bit range in radix caches (#43920)
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: qandrew <qandrew@users.noreply.github.com>
2026-10-13 11:33:26 -07:00
7f9f742535 [Rust] Refuse an unsupported multimodal model before the Rust listener binds (#43925)
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: shodoco <shodoco@users.noreply.github.com>
2026-10-13 11:30:43 -07:00
2a8e9db3b0 [lora] Support LoRA for the NemotronH_Nano_VL_V2 model family (#39798)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Po-Han Huang <pohanh@nvidia.com>
Co-authored-by: Yanbin Jiang <jybsuper@gmail.com>
2026-10-13 11:29:48 -07:00
ff3f8c348c [Observability] Report per-request storage prefetch and staging waits in meta_info (#43946)
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: xiezhq-hermann <xiezhq-hermann@users.noreply.github.com>
2026-10-13 11:13:01 -07:00
Kailash RanganathanandYanbin Jiang c0584cf9af [perf] [multi-lora] remove empty CTAs from fused LoRA expand and add split-K to shrink (#41951)
Co-authored-by: Yanbin Jiang <jybsuper@gmail.com>
2026-10-13 10:16:14 -07:00
11cd2701a8 fix(openai): preserve zero cached tokens when cache reporting is enabled (#43005)
Co-authored-by: ai-jz <ai-jz@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
2026-10-13 23:29:19 +08:00
3f1e185f6e fix(openai): keep chat stream created timestamp stable (#43026)
Co-authored-by: ai-jz <ai-jz@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
2026-10-13 22:43:33 +08:00
jiaqiang-dot-liuandCursor 4c5ed0597e Skip fully-masked KV tiles in sliding-window extend attention (#39769)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-13 22:40:53 +08:00
27b68303d0 fix(openai): omit absent tool_calls from chat responses (#43027)
Co-authored-by: ai-jz <ai-jz@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
2026-10-13 20:56:58 +08:00
Po-Han Huang (NVIDIA)andClaude Opus 5.5 8e37c6cf37 [CI] Run test_pi05_modelopt_fp8_cuda in the kernel-unit lane (#44055)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 20:39:02 +08:00
Ho-Ren (Jack) ChuangandXiaoyu Zhang 5100f8d11d [DSA] Q8KV8 FP8 Sparse Prefill on GLM-5.3-Flash with BF16 KV Cache (#41790)
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
2026-10-13 20:30:13 +08:00
jlqibmandClaude Opus 5 74e0a47863 Fix/compressed tensors nvfp4 hopper (#40022)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-10-13 20:10:31 +08:00
Yunhao Zhouand周云昊 991a29d0ad fix(multimodal): convert grayscale tensors to RGB in PIL backend (#42127)
Co-authored-by: 周云昊 <zhouyunhao@zhouyunhaodeMacBook-Pro.local>
2026-10-13 19:03:48 +08:00
4b5b727145 Fix EVS for NemotronH_Omni_Reasoning_V3 (#42933)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
2026-10-13 10:59:00 +00:00
MichaelandClaude Opus 5.5 aa15b06abe [AMD][CI] Run test_tracing.py in the extra tier, matching CUDA (#43928)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-13 02:45:58 -07:00
Xiaoyu Zhang 4e0fa52530 [Perf] Compact GDN causal-conv grids for skewed prefill batches (#43570) 2026-10-13 17:45:06 +08:00
Ziang Li 01f6840693 [FlashInfer] Add startup autotuning for MegaMoE (#43789) 2026-10-13 02:30:31 -07:00
Mohammad Miadh AngkadandMohammad Angkad 195cb7afb4 [CI] Use sgl-eval for GLM-5.3-Flash GSM8K tests (#43975)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-10-13 02:26:50 -07:00
Shu WangandAnish Shanbhag e0c957e222 [HiCache][PD] Allow DSA index-K elision under HiCache and disaggregation (#41769)
Co-authored-by: Anish Shanbhag <ashanbhag@nvidia.com>
2026-10-13 02:07:54 -07:00
Shu WangandPo-Han Huang 27be99ae73 [NVIDIA][MegaMoE] Fix the default spec decoding moe backend when the target model uses flashinfer megamoe (#40772)
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
2026-10-13 02:00:24 -07:00
50a144057c [AMD][PD] enable mori transfer for PD decode DCP (#37358)
Signed-off-by: inkcherry <mingzhi.liu@amd.com>
Co-authored-by: Duyi-Wang <duyi.wang@amd.com>
Co-authored-by: HAI <hixiao@gmail.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
2026-10-13 14:35:09 +08:00
Xiaoyu Zhang 46b7357779 [Diffusion] Speed up LTX-2 guidance and video output (#43667) 2026-10-13 14:23:33 +08:00
Jimmy Shong 3eebf987ed [PD] Support heterogeneous TP for hybrid SWA KV transfer (#40909) 2026-10-12 23:21:54 -07:00
603a05e6c0 [ROCm][Spec] Route NGRAM tree reconstruction to the JIT kernel on HIP (#43328)
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Thomas Wang <thomawan@amd.com>
2026-10-12 23:17:49 -07:00
Xinyi Song a69e893b1a [AMD] Fuse prefill index-Q RoPE fake-quant and fp4 quantize kernels in DSV4.1 (#42911) 2026-10-12 22:33:53 -07:00
6a89181614 tokenizer perf optimization (#39771)
Co-authored-by: BBuf <1182563586@qq.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
2026-10-13 12:27:21 +08:00
4d5692b6cd fix(openai): honor tool_choice=none in DeepSeek encoders (#43006)
Co-authored-by: ai-jz <ai-jz@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
2026-10-13 11:41:50 +08:00
Mohammad Miadh AngkadandMohammad Angkad 7bd41aec12 [CI] Remove Mistral Small 4 e2e test (#43973)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-10-12 20:14:22 -07:00
Xiaoyu Zhang 03416e8c0f [Fix] Account for captured inputs in DeepGEMM masked workspace budget (#43563) 2026-10-13 10:53:10 +08:00
Liangsheng Yin 43e4e84731 [PD] Recompute retracted DeepSeek-V4 requests on prefill instead of crashing decode (#43936) 2026-10-12 19:32:03 -07:00
Yuwei An 07566cbf0d [DSv4.1] Temporarily disable B300 PD/CP accuracy test (#43948) 2026-10-12 19:31:10 -07:00
75a0d91c1f [Fix] Preserve HTTP errors for non-streaming /v1/messages (#43796)
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
2026-10-13 10:10:45 +08:00
45418d7dc6 [HiCache] Avoid full-store scans for uncached file prefix lookups (#42077)
Signed-off-by: apex-mochen <2756823972@qq.com>
Co-authored-by: apex-mochen <265981728+apex-mochen@users.noreply.github.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
2026-10-12 18:52:50 -07:00
Yuwei An 23150f7ae0 [DSv4.1] Reduce sparse-prefill workspace padding (#43921) 2026-10-12 18:51:36 -07:00
e775fa7253 [Build][CI] Drop Python 3.10: require Python 3.11+ and run CUDA CI on Python 3.12 (#43021)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
Co-authored-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>
2026-10-12 18:40:16 -07:00
Michael 7224fce1b5 [AMD] Recalibrate test_quark_mxfp4 GSM8K thresholds to observed MI355X noise (#41980) 2026-10-12 16:46:03 -07:00
Michael 6c2a682b59 [AMD] Keep the FP4 varctx schedule test off AITER's 2048-row compile (#41793) 2026-10-12 14:39:08 -07:00
Zhang, JiejingandJohnQinAMD 6966c5d027 [ROCm] Eliminate HiCache migration stalls and use exact-size host allocations (#42553)
Signed-off-by: JohnQinAMD <yanyuan.qin@amd.com>
Co-authored-by: JohnQinAMD <yanyuan.qin@amd.com>
2026-10-12 14:05:51 -07:00
Liangsheng Yin 0efbb3d2b9 [DFlash] Decide the Domino candidate pool in one place and apply simulated acceptance inside the accept (#43791) 2026-10-12 11:35:57 -07:00
Liangsheng Yin 392039d752 [DSpark] Share draft params, logits dtype, confidence and the accept tail between graph and eager (#43790) 2026-10-12 11:35:41 -07:00
Shuwen Wang f92199b569 [Unified Cache] Abort requests whose external linker load fails instead of crashing the scheduler (#40896) 2026-10-13 00:03:21 +08:00
Yuan Luoandluoyuan.luo 23a1f53fc3 [QSA] Bound the decode indexer's work by the context length (#43463)
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
2026-10-12 22:47:45 +08:00
279f1fad90 [diffusion] chore: bind worker CPUs to the GPU's NUMA node when memory policy is not permitted (#43756)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 21:09:41 +08:00
Артем Савкин d1357bf87b [NPU] Diffusion CI fixes (#43210) 2026-10-12 14:53:51 +03:00
Sasha SidorovandShangming Cai 7953751bfa [PD][Spec] Give a PD decode's first EAGLE draft step a proposal distribution (#41783)
Co-authored-by: Shangming Cai <csmthu@gmail.com>
2026-10-12 18:49:52 +08:00
CandCheng Wan 28621739fc [Fix] Make tokenizer request-state initialization atomic (#41911)
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
2026-10-12 03:10:57 -07:00
Jinyan Yi f006fb49ec [NPU] Stabilize HiCache and registered CI tests (#43308) 2026-10-12 17:10:26 +08:00
BingjiaWang 62ff466ac0 [DSV4.1][PD][1/N] Support prefill node use cp parallel strategy (#40323) 2026-10-12 01:05:21 -07:00
BingjiaWang 54409c2fa0 [BCG][DSV4-Flash][CP] Support DeepSeek V4 prefill with interleave CP and breakable CUDA graphs (#38603) 2026-10-12 01:03:10 -07:00