Commit Graph
6055 Commits
Author SHA1 Message Date
Zhanghengand晟海 c9eb475a88 [Unified Cache][1/N]: Support cache contract for external linker (#37091)
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
2026-08-30 22:24:13 +08:00
Mick e6a6492057 [vlm] fix: preserve per-request vit graph metadata for qwen-vl (#37043) 2026-08-30 21:02:14 +08:00
Mick 26c754e06e [vlm] fix: recover multimodal decode and processor failures (#36983) 2026-08-30 20:50:19 +08:00
Cheng WanandClaude Opus 5 e51a3ae65e [Config] Round 5.2: the per-model declarations get their own modules (#37087)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-30 02:24:35 -07:00
Cheng Wan 7e751153eb [Config] Round 5.1: the published-side readers ask the bags, and a platform fact gets one address (#37086) 2026-08-30 02:18:33 -07:00
Jason Wiemels e635577431 [rotary] Fix the fused Qwen3.5 RoPE kernel discarding mrope height and width (#34446) 2026-08-30 15:37:40 +08:00
Oguz Ulgen d249672ad3 Scatter mm embeddings with row index_copy_ instead of masked_scatter_ to cut transient GPU memory (#37070) 2026-08-30 00:15:39 -07:00
Liangsheng Yin 5ec959965b [mem_cache] Settle extend kv_committed_len inside alloc_for_extend (#37085) 2026-08-29 23:14:31 -07:00
Liangsheng Yin 0438b16154 [mem_cache] Move kv_committed_len into ReqKvInfo (#37078) 2026-08-29 22:42:25 -07:00
EchO 5b7c62d5d6 fix(frontend): bound stop strings and regex patterns (#37029) 2026-08-29 22:25:26 -07:00
Alex Nails 78fa921189 [ROCm] Fix QuickReduce fp16 saturation corrupting bf16 all-reduces (106M non-finite -> 0, +0.3%) (#34484) 2026-08-29 22:08:17 -07:00
Liangsheng Yin 6be767c2d2 [mem_cache] Move cache_protected_len and swa_evict_floor into ReqKvInfo (#36982) 2026-08-29 19:37:53 -07:00
ed39568e79 [MoE] Add extension points for custom runner backends (#32665)
Co-authored-by: Alex Nails <alex.nails@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-29 19:03:41 -07:00
EchOandGuanyi Chen ca8ff035c3 fix(hicache): isolate decode offload state per request (#37026)
Co-authored-by: Guanyi Chen <939416532@qq.com>
2026-08-29 18:15:05 -07:00
Liangsheng Yin 9a489f8d2f [Test] Move gpqa and aime25 onto sgl-eval, drop unused eval paths (#36979) 2026-08-29 17:36:13 -07:00
Alex NailsandClaude Opus 5 00fbb6e8ac [Perf] Tune the W4AFP8 DeepEP low-latency requant launch geometry (#35760)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-29 16:03:01 -07:00
3a0f1a1344 [NPU] fix: reach torch>=2.8 CUDA memory-pool APIs lazily via torch._C (#29100)
Co-authored-by: Alex Nails <alex.nails@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-29 15:55:14 -07:00
6afb5e1771 Gate the idle-loop tree-cache sanity check behind a default-off env (#36205)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: hzh0425 <hzh0425@apache.org>
2026-08-30 00:10:28 +08:00
HZYandZhangheng cdbfe90b4a [HiCache] Align chunked CUDA host registrations (#36798)
Co-authored-by: Zhangheng <hzh0425@apache.org>
2026-08-29 20:36:12 +08:00
Cheng WanandClaude Opus 5 4d53767b09 config: the lazy imports that buy nothing become eager (#36975)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-29 04:21:54 -07:00
Cheng WanandClaude Opus 5 f0d621cfa6 config: the dead record parameters go (#36974)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-29 04:20:55 -07:00
Cheng WanandClaude Opus 5 1a3e152f03 config: six more runtime readers ask the bags (#36973)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-29 04:19:14 -07:00
Cheng WanandClaude Opus 5 b65e677e48 config: the resolution callbacks into the record go to zero (#36972)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-29 04:18:05 -07:00
Cheng WanandClaude Opus 5 48b88e1256 config: the resolution pipeline's dispatcher leaves the record (#36896)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-29 04:16:25 -07:00
Joe RowellandJimmy Shong 46ccd7ce3e Add Laguna-XS-2.1 / S-2.1 NVFP4 nightly gsm8k tests (#35547)
Signed-off-by: Joe Rowell <joe@poolside.ai>
Co-authored-by: Jimmy Shong <69131491+Jiminator@users.noreply.github.com>
2026-08-29 03:34:46 -07:00
Zhiqiang Xie a328c19c81 [HiCache] buffer mode: decide staged-fetch fate against the live tree (#36834) 2026-08-29 01:51:47 -07:00
Joel Gustafsonandmmangkad 0a585d5bb1 [BugFix] Allow model_loader_extra_config with remote_instance + modelexpress backend (#34639)
Signed-off-by: joeltg <joel@reflection.ai>
Co-authored-by: mmangkad <176301910+mmangkad@users.noreply.github.com>
2026-08-29 00:43:10 -07:00
Liangsheng Yin 24a3a63e6b [misc] Keep req.kv non-optional and key KV ownership on req_pool_idx (#36958) 2026-08-28 23:47:42 -07:00
24c9251ac5 [AMD][Spec][PD] Enable the PD DSA fused-TopK seed remap on ROCm (#36714)
Co-authored-by: Tianxiao Jiang <tianxiaojiang@Tianxiaos-MacBook-Pro.local>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 22:34:23 -07:00
Eric Zhang 3904b309db Add configurable HTTP/2 connection window (#36920) 2026-08-29 13:18:42 +08:00
3760296be8 [PD] Pack DCP1→DCP-N PD KV transfers into dest-contiguous RDMA blocks (#35762)
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 21:41:30 -07:00
+8 5f216fc33f qwen 3.8 rebase (#35758)
Co-authored-by: cherichy <cherichy@outlook.com>
Co-authored-by: guangyunh-nv <guangyunh@nvidia.com>
Co-authored-by: jiahanc <jiahanc@nvidia.com>
Co-authored-by: jinyangyuan-nvidia <joyuan@nvidia.com>
Co-authored-by: Cheng Hang <chang@nvidia.com>
Co-authored-by: Yicheng Qiang <yqiang@nvidia.com>
Co-authored-by: Sam Li <lsam@nvidia.com>
Co-authored-by: Tom-Zheng <tizheng@nvidia.com>
Co-authored-by: Yangmin Li <yangminl@nvidia.com>
Co-authored-by: xiaoweiw-nv <xiaoweiw@nvidia.com>
Co-authored-by: Zheng Li <lizheng.cs@zju.edu.cn>
Co-authored-by: yizhang2077 <1109276519@qq.com>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: Zijie Xia <zijie.xia@radixark.ai>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-08-28 20:41:34 -07:00
Chunyuan WU 92b0d447b0 [CPU] Fix wrongly causal-masked bidirectional attention (#35434) 2026-08-28 19:49:43 -07:00
Aurick Qiao 7276a30c45 [Fix] Support LSE on the RadixAttention extra-kwargs graph path (#35453) 2026-08-28 18:04:00 -07:00
Olga Miroshnichenko 36afd70f9c [ROCm][Bugfix] Use token-level KV indices in the aiter ASM context-prefill gather (#36852) 2026-08-28 17:17:18 -07:00
valechenandHAI 4944e50e2c [AMD] Add Work-Centric (Lean) Attention: a persistent-CTA decode kernel for long-context serving (#33576)
Signed-off-by: Valerie Chen <valechen@amd.com>
Co-authored-by: HAI <hixiao@gmail.com>
2026-08-28 17:00:44 -07:00
Xiaoyu Zhang db6f0a9d53 Refactor JIT kernel and expert-pack directory layout (#36704) 2026-08-29 07:41:25 +08:00
Gregory Leleytner e1b3bba3cc [Kernel] Route large SM90 row/column-scaled FP8 GEMMs to Torch (#34318) 2026-08-29 07:35:37 +08:00
Xiaoyu Zhang 96a4dcdde8 [CI] Slim JIT kernel unit tests (#36887) 2026-08-29 07:26:49 +08:00
Liangsheng Yin 60f6d77a98 [mem_cache] Carry swa_evicted_seqlen into SWARadixCache.cache_unfinished_req (#36909) 2026-08-28 14:29:27 -07:00
Jonny Kongandjonnykong 4d78d59e51 [MoE] Make simulated expert routing support DP>1, and fuse into one triton kernel (#29718)
Co-authored-by: jonnykong <jonnykong@fb.com>
2026-08-28 14:22:45 -07:00
karverma-amd 2a96ebf648 [AMD][DSV4] perf: retune decode split-K heuristic for MI355X (#36094) 2026-08-28 13:48:41 -07:00
Mohammad Miadh Angkad 70088aa5db Fix KeyError on batch requests whose state is freed before it is read (#36638) 2026-08-28 12:57:07 -07:00
Zhiqiang Xie f611e0c073 [HiCache] Fence load-back behind the forward stream (#36738) 2026-08-28 11:12:51 -07:00
Zhiqiang Xie c9bba091f8 [HiCache] Key storage prefetch by the request namespace (#36382) 2026-08-28 11:12:10 -07:00
Cheng WanandClaude Opus 5 ef20fab38a config: the forwarding slots go; the dispatcher calls the family directly (#36792)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-28 10:26:29 -07:00
Cheng WanandClaude Opus 5 7bc3204117 config: three cache and pool readers take the bags (#36791)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-28 10:21:32 -07:00
Cheng WanandClaude Opus 5 43c63a22ff config: the derived parallel widths are computed from the leaves (#36790)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-28 10:18:46 -07:00
Cheng WanandClaude Opus 5 c2928e86d7 config: the resolution pipeline moves out of the record (#36789)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-28 10:17:24 -07:00
Chao Shi 8e04f66a70 HiCache: avoid unnecessary all-reduce in check_prefetch_progress (#36425) 2026-08-29 00:52:53 +08:00