Commit Graph
36 Commits
Author SHA1 Message Date
3b932cde26 [Fix] VLM: harden Kimi image cache identity across preprocessing and radix hashing (#43043)
Co-authored-by: Rahul Chalamala <22563365+rchalamala@users.noreply.github.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
2026-10-18 17:39:37 +08:00
05c1539098 [Feat] unified_cache: Support hybrid SWA + Mamba models in the Mooncake direct linker (#41343)
Co-authored-by: Zhangheng <hzh0425@apache.org>
Co-authored-by: Teng Ma <sima.mt@alibaba-inc.com>
2026-10-17 13:24:08 -07:00
f75232cb42 [Fix] hicache: Order write-back host copies after the forward that produced the KV (#44800)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: xiezhq-hermann <xiezhq-hermann@users.noreply.github.com>
2026-10-17 01:31:40 -07:00
e7d6a88bfc [HiCache] Buffer-only L3 for an auxiliary SWA window; load back only missing window rows (#43932)
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: houseroad <houseroad@users.noreply.github.com>
Co-authored-by: xiezhq-hermann <xiezhq-hermann@users.noreply.github.com>
2026-10-14 16:43:24 -07:00
010776e94f [KV events] Publish opaque LoRA block IDs and adapter names (#42817)
Co-authored-by: Kan Wu <kan.wu@radixark.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
2026-10-14 22:56:28 +08:00
77a15cf4e2 [MemCache] Add shared component size and path APIs to the tree core interface (#43722)
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: Jialin <Jialin@users.noreply.github.com>
2026-10-11 20:14:07 -07:00
Banruo LiuandShangming Cai d62b9c2480 [KV Events] Publish LoRA adapter names and namespaced block hashes (#42455)
Signed-off-by: Banruo Liu <lichangxinlivingshade@gmail.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
2026-10-11 23:23:30 +08:00
Shuwen Wang 99d6f4e158 [MemCache] refactor: require an anchored lock receipt for non-root releases (#41452) 2026-10-10 22:43:49 +08:00
49e629d239 [Rust] Upgrade sglang-radix-tree to PyO3 0.29 (#42807)
Co-authored-by: colleen-tml <296620403+colleen-tml@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 20:56:50 +08:00
f2e9a2bed1 [unified memory pool] Allocate without transfer waits, pin floating SWA reservations, drop the HiCache startup asserts (#43206)
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: ZYHowell <ZYHowell@users.noreply.github.com>
2026-10-09 20:19:30 -07:00
Liangsheng Yin 5a4d01a3ae [MemCache] Return only the matched length from match_prefix; read KV indices off the node path (#42923) 2026-10-08 01:10:14 -07:00
Mohammad Miadh AngkadandMohammad Angkad 0bdd0cc8dc [Deps] Upgrade the CUDA PyTorch stack to 2.14 (#38641)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-10-04 00:48:02 -07:00
937c0a6cfe Fix unified HiCache physical transfers (#39479)
Co-authored-by: yhzhuang <yhzhuang@fb.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: Yonghao Zhuang <yhzhuang@users.noreply.github.com>
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
2026-10-04 00:38:56 -07:00
4611ea365e [HiCache]: Fix host lock ownership across radix splits (#38480)
Signed-off-by: Lukas Humbel <lhumbel@nvidia.com>
Co-authored-by: Jialin Ouyang <jialino@meta.com>
Co-authored-by: Shuwen Wang <47200617+alphabetc1@users.noreply.github.com>
2026-10-04 11:16:04 +08:00
2bae12b9b3 [HiCache] Fix write-back SWA insert backups tripping the write-through pending-ack assert (#42264)
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com>
Co-authored-by: 842974287 <842974287@users.noreply.github.com>
2026-10-03 09:45:56 -07:00
Liangsheng Yin 50be533d09 [mem_cache] Rename cache_unfinished_req to checkpoint_req and count cache hits only when a request finishes (#41520) 2026-10-01 21:41:45 -07:00
77091cea68 [Radix Cache] Sync Rust TreeCore and make it the default (#39627)
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
Co-authored-by: Zhangheng <hzh0425@apache.org>
Co-authored-by: alphabetc1 <alphabetc1@users.noreply.github.com>
2026-09-29 10:18:57 +08:00
Lianmin ZhengandLu Fang 3ecb6cd729 [MemCache] Unify component eviction cursors and lock receipts (#41276)
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com>
2026-09-25 20:53:35 -07:00
Lianmin ZhengandJialin Ouyang f5a1434700 [HiCache] Document transfer arguments (#40239)
Clarify the legacy host_indices argument and label Mamba test arguments,
including the current staging_tokens parameter. Executable code is unchanged.

Co-authored-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
2026-09-18 14:08:19 -07:00
Shuwen Wang 191172fa74 [Unified Tree] fix: exempt host-locked aux nodes from the sanity_check host-LRU check (#39980) 2026-09-18 15:29:29 +00:00
Thomas Wang 6c73368c32 [AMD][DSV4] Enable hicache on deepseek-v4 fp8 unified attn (#37778) 2026-09-17 11:29:23 -07:00
Liangsheng Yin 2f7d04da5d dsv4.1: Rust extension modules for image preprocessing, KV pool names, and PD bootstrap (#39677) 2026-09-15 18:11:48 -07:00
Zhiqiang Xie 7f5dd19256 [HiCache] Rework the buffer-mode storage prefetch pipeline and retry bookkeeping (#39283) 2026-09-15 10:48:49 -07:00
ishandhanani 3f871a246c feat(agent sessions): attribute stored KV cache blocks to sessions (#37482)
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
2026-09-14 21:34:49 -07:00
Shuwen WangandClaude Opus 5 7f1f8c706a [mem_cache][10/N] refactor: drop the redundant _component suffix in unified_cache/components (#35644)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-13 21:38:20 +08:00
Jundong LiuandShuwen Wang 7078e5ffbc [HiCache] Publish a host store event for storage-prefetch refills (#38486)
Co-authored-by: Shuwen Wang <47200617+alphabetc1@users.noreply.github.com>
2026-09-13 10:29:14 +00:00
Shuwen Wang bd45cd50ca [Unified Tree] Port SWA Branching-Point Caching to the Rust TreeCore (#37584) 2026-09-12 18:37:57 +08:00
Yanbin JiangandShuwen Wang 0b415fa573 [HiCache][LoRA] Isolate storage pages by extra key (#38577)
Co-authored-by: Shuwen Wang <47200617+alphabetc1@users.noreply.github.com>
2026-09-12 18:18:14 +08:00
Shuwen Wang b9cb96496d [Unified Tree] Preserve aux LRU recency when splitting nodes (#38482) 2026-09-12 18:17:38 +08:00
Jialin Ouyang 908226fea2 [Rust TreeCore] Support external cache linker (#37306) 2026-09-10 19:22:24 +08:00
Zhiqiang Xie beaf3d9252 [HiCache] Replace skip_lock_node_ids with a segment lock protocol (#36848) 2026-09-09 14:55:51 -07:00
Jialin Ouyang 7a464a7014 [Rust TreeCore] Harden runtime and CI parity (#37303) 2026-09-09 10:40:22 +08:00
Shuwen Wang 19b46863f3 fix: align write-through pending across tree cores (#37278) 2026-09-04 22:08:25 +08:00
f8cbf000f4 [AMD] Enable FP4 indexer for Deepseek V4 (#37353)
Co-authored-by: 1am9trash <1am9trash@gmail.com>
Co-authored-by: AMD-yanfeiwang <256076023+AMD-yanfeiwang@users.noreply.github.com>
Co-authored-by: Thomas Wang <thomawan@amd.com>
2026-09-02 09:45:08 -07:00
Liangsheng Yin 832d029870 [mem_cache] Split duplicate insert frees at the SWA eviction floor (#37481) 2026-09-01 23:01:40 -07:00
Jialin Ouyang a77283fb02 [Rust] Rename mem-cache to sglang-radix-tree (#37290) 2026-09-01 03:03:44 -07:00