Commit Graph
14112 Commits
Author SHA1 Message Date
Liangsheng Yin a8b54f138e [Session] Run streaming sessions only on UnifiedRadixCache; reject unverified tree caches (#42295) 2026-10-02 18:15:59 -07:00
8ff02f5820 [QSA] Enable breakable prefill CUDA graphs for text-only Qwen3.8 Flash-Next (capture-safe metadata, MTP side-channel padding) (#41729)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Qiaolin Yu <liin1211@outlook.com>
2026-10-02 17:57:42 -07:00
Kevin MiandClaude Opus 5 65f759144d [HiCache] Let the MiniMax-M3 K-only host pool join a host pool group (#42166)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-10-02 17:00:13 -07:00
Baizhou Zhang 3bde8eb2d8 [CP 1/5] Remove the CP adapter and separate interleave transport from boundary reduction (#42036) 2026-10-02 16:50:20 -07:00
b69a5f296a [AMD] Add opt-in MiniMax-M3 TP4 indexer context partitioning (#41488)
Co-authored-by: ThomasNing <thomas.ning@amd.com>
Co-authored-by: Kevin Mi <mikevin920@yahoo.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Kevin Mi <45493463+kevin-mii@users.noreply.github.com>
2026-10-02 15:34:29 -07:00
fa7784d140 [unified-memory] Copy page envelopes in place during compaction (#42252)
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com>
Co-authored-by: ZYHowell <ZYHowell@users.noreply.github.com>
2026-10-02 15:12:14 -07:00
e8fab85a02 [KDA] Add an opt-in decode-parity mode to the Triton multi-token recurrence (#42251)
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com>
Co-authored-by: jiayisuse <jiayisuse@users.noreply.github.com>
2026-10-02 14:59:56 -07:00
Liangsheng Yin 8a328e867b [mem_cache] Rename the tree's request-level insert_req to checkpoint (#42202) 2026-10-02 14:42:35 -07:00
2ec9e3b11d [eplb] Warm default-group NCCL P2P transports before KV-cache sizing (#42026)
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com>
Co-authored-by: luccafong <luccafong@users.noreply.github.com>
2026-10-02 14:34:55 -07:00
Yuhao Wang 0098b9d29d [Spec] Fix Qwen3.5 text model EAGLE3/DFLASH aux-layer capture (#42002) 2026-10-02 14:05:01 -07:00
raghotham a64725d407 [HiCache] Fall back to host memory when no cgroup fs is mounted (#42120) 2026-10-02 13:59:40 -07:00
7dd93b01a5 [PD] Make the prebuilt last-token H2D copy non-blocking under overlap (#42113)
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com>
Co-authored-by: bbus <bbus@users.noreply.github.com>
2026-10-02 13:48:39 -07:00
Liangsheng Yin 07e1a4f082 [mem_cache] Skip the release-time insert on optimistic prefill requeue; make refresh_fill_ids public (#42194) 2026-10-02 13:46:32 -07:00
354ed2a7b6 Size the VMM graph-input exchange by the widest input across ranks (#42125)
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com>
Co-authored-by: oulgen <oulgen@users.noreply.github.com>
2026-10-02 13:45:32 -07:00
4a4dad1cf1 [Metrics] Label scheduler stage wall time by sampled forward overlap (#42025)
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com>
Co-authored-by: Jialin <Jialin@users.noreply.github.com>
2026-10-02 13:43:53 -07:00
ChangLiu0709andNikolai Protasov f6fcda8827 [AMD][ROCm] Keep cos_sin_cache fp32 on HIP for fused QSA indexer kernel (#41282)
Co-authored-by: Nikolai Protasov <289648307+amd-nprotaso@users.noreply.github.com>
2026-10-02 11:57:03 -07:00
Shuwen Wang 89f21671bb fix: preserve QSA indexer state through HiCache (#39893) 2026-10-02 23:49:38 +08:00
kkandwunhuang a74f259f38 [AMD][V4.1][*/N] Fuse MXFP8 activation quant into producer kernels on gfx950 (#42055)
Co-authored-by: wunhuang <wunhuang@amd.com>
2026-10-02 08:34:44 -07:00
Alex O. P.andPo-Han Huang e667ab10d5 [diffusion] quantization: ConvRot INT8 online W8A8 for DiTs (sgl-kernel + convrot_int8_customkernel) (#38040)
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
2026-10-02 23:26:28 +08:00
Xiaoyu Zhang 1a2eb6c426 [Kernel] Use Cake softmax for qualified SM103 sampling workloads (#42004) 2026-10-02 23:06:43 +08:00
bb9a820f09 [diffusion] chore: make perf dump one writer per replica with no first-request NCCL setup (#41711)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 21:44:10 +08:00
MickandMick Qian 0f71ec8656 [diffusion] UX: deduplicate SM120 FP8 fallback warnings across layers (#41542)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-10-02 19:25:15 +08:00
YAMYandPo-Han Huang 895525f062 [Mamba] Warm cache COW kernel before serving (#36266)
Signed-off-by: Yangmin Li <yangminl@nvidia.com>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
2026-10-02 18:52:37 +08:00
Alex 1db33bc367 [PD][NIXL] Fix prepared dlists for KV entries of different lengths (DeepSeek-V4) (#42005) 2026-10-02 17:59:31 +08:00
Liangsheng Yin 50be533d09 [mem_cache] Rename cache_unfinished_req to checkpoint_req and count cache hits only when a request finishes (#41520) 2026-10-01 21:41:45 -07:00
saatwiknagpal bf8adf9602 [diffusion] fix: fix int32 offset overflow in SubBlock router kernels (#41965) 2026-10-02 11:22:53 +08:00
Khoa Pham 6be84e1b72 [Fix] Select the Marlin MoE runner for MiMo-V2 packed MXFP4 experts on SM90 (#42172) 2026-10-01 20:01:03 -07:00
Yuwei An ef867fa40d [Fix][DSV4.1] SWA page size with bounded replay (#42128) 2026-10-01 18:50:43 -07:00
jiaryangandThomas Wang 58f0d250ec [ROCm] Use the fused MLA absorb + RoPE + KV-write kernel for decode-sized forward modes only (#41533)
Co-authored-by: Thomas Wang <thomawan@amd.com>
2026-10-01 18:45:29 -07:00
kkandwunhuang 29f6d408c0 [AMD][V4.1][*/N] Fix shared-expert fusion accuracy and speed up MoE routing on ROCm (#42011)
Co-authored-by: wunhuang <wunhuang@amd.com>
2026-10-01 18:19:06 -07:00
Qiaolin Yu 67eab57057 [qwen 3.8 next] Fuse NEXTN verify and draft graph input preparation (#41175) 2026-10-01 17:17:34 -07:00
caihuali95andCheng Wan 2a3655fea1 [unified-memory] Name the fused KV translate for what it computes (7/7) (#40331)
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
2026-10-01 17:14:03 -07:00
df530479c6 [unified-memory] Stride the KV translate kernel and route every translate through it (6/7) (#40330)
Co-authored-by: Caihua Li <caihua.li@bytedance.com>
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
2026-10-01 17:12:14 -07:00
9855e9c6f8 [unified-memory] Remove the kernel-page multiplier plumbing (5/7) (#40329)
Co-authored-by: Caihua Li <caihua.li@bytedance.com>
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
2026-10-01 17:11:25 -07:00
4c4e352a4b [unified-memory] Mark the write loc physical and check it at every write door (4/7) (#40328)
Co-authored-by: Caihua Li <caihua.li@bytedance.com>
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
2026-10-01 17:10:37 -07:00
cdadb1c79a [unified-memory] Token-major dense views for the unified memory pool (3/7) (#38592)
Co-authored-by: Caihua Li <caihua.li@bytedance.com>
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
2026-10-01 17:09:38 -07:00
caihuali95andCheng Wan 5df52e6382 [unified-memory] Build paged KV views through one helper (2/7) (#40327)
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
2026-10-01 17:08:42 -07:00
caihuali95andCheng Wan d3dbb71e5a [unified-memory] Derive KV row addresses from strides, not shapes (1/7) (#40326)
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
2026-10-01 17:07:47 -07:00
shyeh25andBrayden Zhong 5423a4d885 [MegaMoE] Preserve W13 layout for ModelOpt NVFP4 experts (#39388)
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
2026-10-01 16:12:34 -07:00
f17f7705a5 [AMD] [GLM-5.3-Flash] Enable FP8 and MXFP4 serving on gfx950 (#39273)
Co-authored-by: Byron Hsu <byron+per@periodiclabs.ai>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: long10024070 <long10024070@users.noreply.github.com>
Co-authored-by: Kevin Mi <45493463+kevin-mii@users.noreply.github.com>
Co-authored-by: Kevin Mi <mikevin920@yahoo.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: sunxxuns <126995791+sunxxuns@users.noreply.github.com>
Co-authored-by: Long Luong <long.luong2+personal@onemount.com>
2026-10-01 15:59:44 -07:00
Jan Bernlöhrandjbernloehr 80bb3fb651 [Fix] Select the MXFP4 MoE runner for MiMo-V2 packed experts on SM100 (#41668)
Co-authored-by: jbernloehr <janbernloehr@users.noreply.github.com>
2026-10-01 15:33:30 -07:00
Martin Hua 52d5faed5b [Bugfix] Llama4 local attention: read page ids from the graph tables in CUDA-graph capture/replay (page_size > 1) (#38187) 2026-10-01 14:42:22 -07:00
bf2d686a82 Add triton autotune on the Mamba2 SSD kernels (#39130)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
2026-10-01 14:25:37 -07:00
Jiajun LiandClaude Fable 5.1 dcf4e18283 fix(nccl): disable graph buffer registration for pausable graph pools (#40648)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-01 12:34:23 -07:00
3f4bfda13c [Bugfix] Fix DeepSeek V4 Pro TP24 vocabulary-padding failure on Hopper (#31801)
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
Co-authored-by: Po-Han Huang <pohanh@nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-01 11:11:18 -07:00
a0fcebb3bb NIXL: Use stride desc API (#37827)
Signed-off-by: Ilia Yastrebov <iyastrebov@nvidia.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
2026-10-01 10:53:00 -07:00
jain-ria fb92ba1127 [Rust] Wire the gRPC server into the frontend lifecycle (#41964)
Signed-off-by: jain-ria <riajain@NVIDIA.com>
2026-10-01 10:44:56 -07:00
JoeandAlex Nails f03a183719 Fix Kimi-K3 MLA output gate dispatch on non-CUDA devices (#36406)
Co-authored-by: Alex Nails <alex.nails@radixark.ai>
2026-10-01 08:53:31 -07:00
JoeandAlex Nails 3031091c36 [Kimi-K3] Guard optimized paths by platform (#40269)
Co-authored-by: Alex Nails <alex.nails@radixark.ai>
2026-10-01 08:53:05 -07:00
Артем Савкин 1bbce57d9e [NPU] [Diffusion] Fix NPU multimodal-gen CI (#41528) 2026-10-01 18:02:37 +03:00