Commit Graph
6087 Commits
Author SHA1 Message Date
Anihcandsunguozhong ebc71e650e [Perf] [NPU] mem_cache: Add optional ScatterPa KV cache writes (#40535)
Co-authored-by: sunguozhong <sun_gz2772@163.com>
2026-10-20 09:33:34 +08:00
71ee2dc950 [Fix] inkling: Return 400 instead of 500 for unfetchable/undecodable multimodal input (#40899)
Co-authored-by: apex-mochen <265981728+apex-mochen@users.noreply.github.com>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
Co-authored-by: apex-mochen <2756823972@qq.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Po-Han Huang <pohanh@nvidia.com>
2026-10-20 09:18:18 +08:00
Xiaoyu ZhangandClaude Opus 5.5 693d333306 [Refactor] kernels: Regroup operators by computation and dissolve ops/muse_spark (#45112)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 08:47:57 +08:00
Liangsheng Yin ab580b2e06 [CI] radix-cache: Split unified radix cache unit tests into topic files and a shared kit (#45236) 2026-10-19 17:00:42 -07:00
Cheng Wan 6b90e5716d [Refactor] moe: Scope shared-expert configuration to each model (#44913) 2026-10-19 16:59:37 -07:00
Cheng Wan 964ab2b280 [Refactor] moe: Split the two meanings of num_fused_shared_experts (#44912) 2026-10-19 16:57:15 -07:00
Cheng Wan 48ea486928 [Fix] moe: Renormalize per-rank shared-slot routing over routed experts only (#44904) 2026-10-19 16:56:21 -07:00
Cheng Wan 3a059de94c [Fix] moe: Count routed_scaling_factor once when top-k folds it (#44901) 2026-10-19 16:54:27 -07:00
ab258b6453 [Fix] symm-mem: Name the window registration in the scheduler watchdog dump (#44681)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-10-19 15:29:35 -07:00
Jiuqiang Tang 49766a74f9 [Fix] attn_residual: int64 program ids in the score/combine kernels (int32 overflow at 37,451 tokens) (#43438) 2026-10-19 15:12:12 -07:00
0z5aandRichard Wang 80ea5ee638 [Fix] dLLM: Preserve prompt tails and literal mask tokens across prefix reuse (#39592)
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Co-authored-by: Richard Wang <wangrichard08@gmail.com>
2026-10-19 14:48:37 -07:00
Liu Keandhnyls2002 81ac15391c [Fix] metrics: record HTTP request metrics in tokenizer workers (#44663)
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-10-19 14:36:16 -07:00
Mohammad Miadh AngkadandMohammad Angkad 7656ea0e74 [CI] python: Move the remaining CUDA build paths off Python 3.10 (#44393)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-10-19 14:29:00 -07:00
Liangsheng Yin b20cb3665d [CI] radix-cache: Drop redundant unified radix cache unit tests (#45232) 2026-10-19 14:20:32 -07:00
Baizhou Zhang 642a671c30 [CI] tests: Re-enable muse spark B300 PD coverage (#44975) 2026-10-19 13:40:02 -07:00
Rain Jiang aaa9388c91 [Fix] kda: Keep FlashInfer prefill off its grouped kernel's bf16/flat-dt_bias contract (#45018) 2026-10-19 13:16:54 -07:00
Qiaolin Yu 5385866108 [Fix] moe: gated NVFP4 MoE intermediate alignment for FlashInfer TRTLLM (#44138) 2026-10-19 12:52:36 -07:00
0ce41c5a06 [Fix] PD: retraction restores racing the in-flight decode forward (#42702)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
2026-10-19 11:55:04 -07:00
YAMY 722669e606 [Feat] [NVIDIA] megakernels: Add --enable-nvidia-megakernel and a layer for NVIDIA megakernel plugins (#44945) 2026-10-19 11:54:44 -07:00
chenyang08056032 716c9151f2 [CI] [NPU] npu-coverage: run only coverage-recommended tests on PRs (precise test selection) (#44019) 2026-10-20 00:57:27 +08:00
Jan SchalkampandXiaoyu Zhang 8e7601ae4a [Feat] paged-experts: Serve MoE models whose weights do not fit VRAM (#42557)
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
2026-10-19 23:10:04 +08:00
Monishver 806e7423af [Perf] deepseek_v4: size the masked silu-mul-quant grid to the GPU, not the padded capacity (#44618)
Signed-off-by: Monishver Chandrasekaran <monishverchandrasekaran@gmail.com>
2026-10-19 23:07:12 +08:00
Siliang ChenandCursor 7c4cc5a51c [Perf] [AMD] attention: Spread uneven Triton decode batches across XCDs (#41229)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-19 23:03:27 +08:00
Sam Shleifer 96ba108072 [Fix] dsa-eagle-topk: Copy token keys and scales when relocating index cache (#44564) 2026-10-19 22:54:08 +08:00
Sam Shleifer 417606527d [Fix] nvfp4: Own streamed tensors while waiting for paired weights (#44500) 2026-10-19 22:41:53 +08:00
pllimax 8c4680cb65 [CI] npu: Move llada2-mini FDFO test to full test (#45153) 2026-10-19 22:29:30 +08:00
Niko Maandinkcherry a90e9c5dd8 [Fix] [AMD] mhc: Bound the Triton fused post+pre buffer cache (#44443)
Co-authored-by: inkcherry <mingzhi.liu@amd.com>
2026-10-19 01:37:50 -07:00
8111841103 [Perf] deepseek-v4.1: Replay the breakable prefill CUDA graph at any context length (#43733)
Co-authored-by: Oasis-Git <ayw.sirius19@gmail.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-10-19 01:29:14 -07:00
33dd8f6a2e [Fix] hybrid-SSM: DP attention failures at low concurrency (#34535)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Jimmy Shong <69131491+Jiminator@users.noreply.github.com>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
2026-10-19 16:24:41 +08:00
Bingxu Chen 3d588779e1 [CI] [AMD] test: Run UMBP HiCache E2E through standalone DRAM server (#40674) 2026-10-19 00:23:52 -07:00
desmond-intel dc81f17b90 [Fix] [XPU] gemma3: Enable bidirectional image attention (#44225) 2026-10-19 14:08:24 +08:00
Liangsheng Yin a3330237db [Perf] deepseek-v4.1: Run decoder replay tail layers from CUDA graphs on eager prefill steps (#45059) 2026-10-18 23:07:50 -07:00
7c9a78efbe [Feat] nemotron: Port Nemotron 3.5 Super serving changes (#44518)
Co-authored-by: BBuf <1182563586@qq.com>
Co-authored-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Co-authored-by: Jimmy Shong <jimmy.shong@mac.attlocal.net>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 13:27:22 +08:00
Yuwei An 834bfe4ef0 [Fix] attention: Opt FlashMLA scheduler into dynamic shared memory (#44580) 2026-10-18 20:31:39 -07:00
Duyi-Wang 8d3eee46a7 [Perf] [AMD] deepseek-v4: Skip top-k refinement for proven positive-zero ties (#44927) 2026-10-19 11:27:47 +08:00
Kotthagattu Meher Sai 7f96dc6874 [Perf] [XPU] server_args: Enable fused QK-norm + RoPE by default (#43345) 2026-10-19 10:19:52 +08:00
Khoa Pham a7fd2e1158 [Feat] dsv4.1: Return prompt logprobs under encoder SWA bounded replay (#45050) 2026-10-18 19:08:20 -07:00
Wuhen DuanandEven Zhou 69076b9f9e [CI] [NPU] tests: Reduce HiCache sizes and enable expandable pinned memory in focused tests (#41675)
Co-authored-by: Even Zhou <even.y.zhou@outlook.com>
2026-10-19 10:06:35 +08:00
fea48754de [Feat] rust/grpc: Add ListModels, Detokenize, and Abort to api.v1 (#42118)
Signed-off-by: jain-ria <riajain@NVIDIA.com>
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Ishan Dhanani <ishandhanani@gmail.com>
2026-10-19 09:53:56 +08:00
Mohammad Miadh AngkadandMohammad Angkad 377cd867fd [Perf] dsa: Route the k-pool indexer top-k through the top-k v2 kernel (#43775)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-10-19 09:49:39 +08:00
Yuwei An df078cc729 [Fix] DSv4.1: Support CUDA image requests with encoder bounded replay and CI Test (#45048) 2026-10-18 18:47:17 -07:00
A&W cff88d6eac [Fix] diffusion: Invalidate merged LoRA cache after adapter rewrites (#44204) 2026-10-19 01:08:57 +00:00
Xiaoyu ZhangandClaude Opus 5.5 329072d99a [Refactor] kernels/diffusion/tests: Prune dead code and fix tests that never ran (#44690)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 08:55:47 +08:00
Liangsheng Yin 59fec011dd [Fix] deepseek-v4.1: Reject encoder replay on a chunk too small for a hit or on other models (#45051) 2026-10-18 17:55:35 -07:00
Liangsheng Yin 3c560089e1 [Refactor] scheduler: Bill encoder replay rows in _update_prefill_budget (#45047) 2026-10-18 17:17:38 -07:00
Liangsheng Yin 1637a4e305 [Fix] scheduler: Clear stale encoder_swa_reset in convert_decode_to_extend (#45042) 2026-10-18 17:09:19 -07:00
Liangsheng Yin 6708426728 [Fix] spec: Remove --speculative-skip-dp-mlp-sync (#45043) 2026-10-18 16:41:21 -07:00
Liangsheng Yin 1461152cb0 [Fix] deepseek-v4.1: Leave room for encoder replay rows when sizing a prefill chunk (#45040) 2026-10-18 15:43:08 -07:00
Liangsheng Yin fe89a7c890 [Fix] deepseek-v4.1: Start the PD decode draft SWA window at the prefill draft valid-from (#45037) 2026-10-18 15:06:20 -07:00
Liangsheng Yin da414adda7 [Fix] engram: Clamp multimodal pad ids in host-built history rows (#45038) 2026-10-18 12:24:05 -07:00