Commit Graph
621 Commits
Author SHA1 Message Date
e83f585ae6 [Perf] dsv41: Serve token-interleaved V4.1 KV caches with sgl-flash-mla (#45893)
Co-authored-by: kpham-sgl <khoa.pham@radixark.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 22:02:49 -07:00
metamergebotandJialin 94e4611eac [Refactor] unified-cache: Factor match validators and match completion into helpers (#45714)
Co-authored-by: Jialin <Jialin@users.noreply.github.com>
2026-10-22 18:58:15 -07:00
Liangsheng Yin 9c7dd3c45f [Perf] dsv4: Dequantize each shared sparse-prefill KV source once per forward (#45909) 2026-10-22 17:08:40 -07:00
3b0003dac1 [Fix] HiCache: stale host KV from write-through copies racing the in-flight forward (#42701)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
2026-10-22 14:46:51 -07:00
aae2145eb2 [Feat] radix-tree: Report the last visited node and a shorter FULL device match in match results (#45703)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: Jialin <Jialin@users.noreply.github.com>
2026-10-22 13:11:33 -07:00
Liangsheng Yin 4f96807ea8 [CI] radix-cache: Build unified radix cache test states through production paths (#45334) 2026-10-22 13:04:23 -07:00
ShunkangzandClaude Opus 5.5 cc811ff0c3 [Feat] KVShard: Add attention and indexer support for page-sharded DSA caches (#44280)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 00:16:35 +08:00
Shuwen Wang 8cd3c494a3 [Fix] mem_cache: Skip linker L3 lookup on PD decode (#45538) 2026-10-22 22:36:32 +08:00
81841d6bdc [Fix] hicache: Fail fast on a null pointer from the Mooncake host allocator (#40565)
Co-authored-by: huangtingwei <141888744+huangtingwei9988@users.noreply.github.com>
Co-authored-by: Teng Ma <sima.mt@alibaba-inc.com>
2026-10-22 20:12:25 +08:00
Tamir BaydasovandClaude Opus 5.5 b057d5b3f1 [Fix] [NPU] mla-pool: DCP allocator page and index-K bound, short-pool warning (#42976)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 05:24:14 +03:00
Baizhou Zhang 59149553b9 [Refactor] attention: centralize linear and draft TP group selection (#45684) 2026-10-21 18:25:40 -07:00
Liangsheng Yin 3438ed660d [Test] cpu: prune mirror and tautology cases from the per-commit CPU suite (#45672) 2026-10-21 16:02:47 -07:00
991b846227 [Perf] speculative: Parallelize EAGLE spec-decode KV index building over token blocks (flashinfer draft decode; verify/draft-extend for flashinfer, flashinfer_mla, aiter) (#42803)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: YAMY1234 <74099316+YAMY1234@users.noreply.github.com>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
Co-authored-by: Po-Han Huang <pohanh@nvidia.com>
2026-10-21 15:26:35 -07:00
Liangsheng Yin f7d4f06242 [Perf] dsv4: Compute request-window history rows once per batch (#45579) 2026-10-21 14:01:19 -07:00
Liangsheng Yinandlixiufei-leo 2c7a9e3b50 [Fix] radix-cache: Order linker offloads behind the forward that writes the pages (#45559)
Co-authored-by: lixiufei-leo <294029388+lixiufei-leo@users.noreply.github.com>
2026-10-21 13:53:37 -07:00
740d5f9f9e [Perf] scheduler: Plan hybrid Mamba extend batches without host-device syncs (#45472)
Co-authored-by: Lu Fang <116399278+luccafong@users.noreply.github.com>
Co-authored-by: Shiyan Deng <20773414+842974287@users.noreply.github.com>
2026-10-21 11:56:50 -07:00
5e79333c95 [Refactor] radix-tree: Reuse the insert walk, component hooks and bindings for custom tree cores (#45439)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: Jialin <Jialin@users.noreply.github.com>
2026-10-21 10:53:26 -07:00
Khoa PhamandAMD-yanfeiwang da7e3f1c12 [Fix] cache: Accept packed draft layer mappings in the UMBP linker (#44929)
Co-authored-by: AMD-yanfeiwang <yanfei.wang@amd.com>
2026-10-20 23:27:22 -07:00
eb91a41f38 [Feat] unified_cache: Support FULL + MAMBA MLA models (Kimi-K3) in the direct external linker (#44977)
Co-authored-by: Yutong Wu <yutong.wu@amd.com>
Co-authored-by: Zhangheng <hzh0425@apache.org>
2026-10-21 10:47:17 +08:00
Chunan Zeng 93088901fb [Chore] attention: Remove the tokenspeed_mla backend and dependency (#44803) 2026-10-20 18:50:05 -07:00
cc7ae372ac [Refactor] radix-cache: Route pending-backup split bookkeeping through one component action (#45242)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: Jialin <Jialin@users.noreply.github.com>
2026-10-20 13:24:49 -07:00
0b536ca112 [Fix] radix-cache: Separate logical and reclaimable value lengths in tree components (#45238)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: Jialin <Jialin@users.noreply.github.com>
2026-10-20 13:24:20 -07:00
bab4a808d8 [Perf] unified-memory: Allocate free pages from each sub-pool's own end first (#45222)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: ZYHowell <ZYHowell@users.noreply.github.com>
2026-10-20 13:24:09 -07:00
0d7638b035 [Fix] scheduler: Exclude accepted storage-tier hits from the waiting prefill load (#45158)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: Jialin <Jialin@users.noreply.github.com>
2026-10-20 13:24:04 -07:00
dc62c3effd [Feat] radix-tree: Report each path node's child count in collect_path_info (#45241)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: Jialin <Jialin@users.noreply.github.com>
2026-10-20 13:23:59 -07:00
eb4b167d27 [Feat] KVShard: PD transfer and e2e test (#38473)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Zhangheng <hzh0425@apache.org>
2026-10-20 22:48:43 +08:00
Jensen 3bf4cbc6cf [Perf] [NPU] mem_cache: Increase GLM-5.2-W4A8C8 KV cache capacity with compact DSA indexer allocation (#44436) 2026-10-20 16:30:01 +03:00
1ec902e00f [Fix] cache: Support DeepSeek-V4.1 external linker pools (#44623)
Co-authored-by: Zhangheng <hzh0425@apache.org>
Co-authored-by: Shuwen Wang <47200617+alphabetc1@users.noreply.github.com>
2026-10-19 23:40:25 -07:00
lpdinkandZhangheng 4a651146de [Feat] hicache: Notify storage backends when the initial host-pool registration burst completes (#44030)
Co-authored-by: Zhangheng <hzh0425@apache.org>
2026-10-20 14:09:13 +08:00
huangtingwei 3eb4bd598f [Perf] hicache: Unify PP prefetch hit queries and buffer allocation (#43344) 2026-10-20 11:05:00 +08:00
kpjeeja 8683f8ad4a [Feat] [Intel] hicache: Enable L1<->L2 support on XPU (#32503) 2026-10-20 10:42:21 +08:00
Xiaoyu ZhangandClaude Opus 5.5 693d333306 [Refactor] kernels: Regroup operators by computation and dissolve ops/muse_spark (#45112)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 08:47:57 +08:00
Liangsheng Yin ab580b2e06 [CI] radix-cache: Split unified radix cache unit tests into topic files and a shared kit (#45236) 2026-10-19 17:00:42 -07:00
0z5aandRichard Wang 80ea5ee638 [Fix] dLLM: Preserve prompt tails and literal mask tokens across prefix reuse (#39592)
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Co-authored-by: Richard Wang <wangrichard08@gmail.com>
2026-10-19 14:48:37 -07:00
Liangsheng Yin b20cb3665d [CI] radix-cache: Drop redundant unified radix cache unit tests (#45232) 2026-10-19 14:20:32 -07:00
0ce41c5a06 [Fix] PD: retraction restores racing the in-flight decode forward (#42702)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
2026-10-19 11:55:04 -07:00
Sam Shleifer 96ba108072 [Fix] dsa-eagle-topk: Copy token keys and scales when relocating index cache (#44564) 2026-10-19 22:54:08 +08:00
Liangsheng Yin fe89a7c890 [Fix] deepseek-v4.1: Start the PD decode draft SWA window at the prefill draft valid-from (#45037) 2026-10-18 15:06:20 -07:00
Ke Bao 4ec19ca27f [Refactor] memory-pool: Build KV host pools from buffer info (#40917) 2026-10-18 23:21:16 +08:00
Ke Bao bb1d4cddd9 [Refactor] hicache: Route Mamba host transfers from pool info (#44968) 2026-10-18 23:02:46 +08:00
Cheng Wan 57db5089c2 [Perf] attention: Capture translated FA4 verify metadata for DSPARK (#44973) 2026-10-18 05:42:16 -07:00
Cheng Wan 2e9b2cacda [Perf] speculative: Fuse draft write-window gathering and KV translation (#44972) 2026-10-18 05:41:34 -07:00
Cheng Wan d318a546da [Perf] unified-memory: Reduce allocation and SWA retirement overhead (#44970) 2026-10-18 05:40:25 -07:00
3edb156dec [Feat] unified-memory: Fuse drafts with sliding-window and recurrent-state layers (#43326)
Co-authored-by: Caihua Li <caihua.li@bytedance.com>
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
2026-10-18 05:39:09 -07:00
caihuali95 19cb24f1e1 [Feat] unified-memory: Support asymmetric KV rows (#43325) 2026-10-18 05:37:50 -07:00
caihuali95andCaihua Li ba50ea3d77 [Perf] unified-memory: Free SWA segments without a host sync (#43170)
Co-authored-by: Caihua Li <caihua.li@bytedance.com>
2026-10-18 05:36:56 -07:00
caihuali95andCaihua Li 47eeddcd18 [Fix] swa: Retain a whole window behind insertion pages (#43168)
Co-authored-by: Caihua Li <caihua.li@bytedance.com>
2026-10-18 05:36:02 -07:00
3b932cde26 [Fix] VLM: harden Kimi image cache identity across preprocessing and radix hashing (#43043)
Co-authored-by: Rahul Chalamala <22563365+rchalamala@users.noreply.github.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
2026-10-18 17:39:37 +08:00
Shuwen Wang ba3fc9441d [Fix] mamba: Release consumed lazy Mamba prefix locks before donation retry (#43559) 2026-10-18 15:16:31 +08:00
Shuwen Wang a11fa06be7 [Fix] hicache: Key MLA Mamba state per TP rank in the file backend (#41907) 2026-10-18 15:10:50 +08:00