Commit Graph
436 Commits
Author SHA1 Message Date
77091cea68 [Radix Cache] Sync Rust TreeCore and make it the default (#39627)
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
Co-authored-by: Zhangheng <hzh0425@apache.org>
Co-authored-by: alphabetc1 <alphabetc1@users.noreply.github.com>
2026-09-29 10:18:57 +08:00
Shuwen Wang bddeb61d94 [Unified Memory] fix: preserve FP8 dtype in unified MHA pool (#38133) 2026-09-28 05:19:03 -07:00
Liangsheng Yin 37556c1b91 [mem_cache] Replace cache_finished_req with insert_req; release_kv_cache frees and unpins (#41281) 2026-09-27 02:53:16 -07:00
Shuwen Wang 38d865489a [DSV4.1][HiCache] fix: wait for the layer transfer before reading low-ratio index-K (#41345) 2026-09-26 22:04:03 -07:00
Lianmin ZhengandLu Fang fc9e1c8d29 [MemCache] Fix LMCache component cursors and per-cache backend selection (#41328)
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com>
2026-09-26 11:24:39 -07:00
c10fa03fc4 Make sliding-window caching and speculative batch padding extensible (#41325)
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com>
Co-authored-by: jasonjk-park <jasonjk@fb.com>
2026-09-26 11:24:02 -07:00
Xiaoyu Zhang 38ec649048 [Refactor] Restore logical kernel groups and test organization (#41243) 2026-09-26 12:30:44 +08:00
Lianmin ZhengandLu Fang 3ecb6cd729 [MemCache] Unify component eviction cursors and lock receipts (#41276)
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com>
2026-09-25 20:53:35 -07:00
Liangsheng Yin f35ec18d8b [Test] Remove more unit tests that mirror implementation or never run in CI (#41297) 2026-09-25 19:22:14 -07:00
Shuwen Wang 4273e7de77 [PD] fix: cache resumed decode-radix requests from root instead of an unlocked re-match (#41261) 2026-09-26 10:21:41 +08:00
cf5df82680 [KVCache] Support lmcache unified radix cache (#38652)
Signed-off-by: chunxiaozheng <1179548172@qq.com>
Co-authored-by: Yuwei An <ayw.sirius19@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-25 14:08:45 -07:00
metamergebotandYonghao Zhuang 671000aee6 [unified-memory] Honor move gates in float relocation and size auto HiCache from host capacity (#41248)
Co-authored-by: Yonghao Zhuang <yhzhuang@meta.com>
2026-09-25 12:17:28 -07:00
Shuwen Wang 434c2e3adc [HiCache] fix: Drain pending backups before internal Mamba write-back (#41092) 2026-09-25 15:06:24 +08:00
Liangsheng Yin 8ca82118e0 [mem_cache] Drop is_insert from cache_finished_req; release rows from release_kv_cache (#40988) 2026-09-24 17:23:26 -07:00
466e985f4b [HiCache] Batch buffer-only KV backups within each flush (#40960)
Co-authored-by: oss-sync bot (pranjalssh) <pranjalssh@users.noreply.github.com>
Co-authored-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
2026-09-24 16:13:02 -07:00
Liangsheng Yin 0c578d97fe [DSV4] Size compressed pools from one per-ratio table in DSV4PoolConfigurator (#41049) 2026-09-24 15:19:02 -07:00
78b382b1ad Support unified memory decode host pools (#39478)
Co-authored-by: yhzhuang <yhzhuang@fb.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: Yonghao Zhuang <yhzhuang@users.noreply.github.com>
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
2026-09-24 11:02:15 -07:00
Shangming CaiandClaude Fable 5 77173ffb08 [HiCache] Demote SWA KV to host on write_back eviction instead of dropping it (#40712)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-09-24 23:56:17 +08:00
paulzhang-tm 28ec6700da [HiCache] Make host reclamation independent of transfer order (#40512) 2026-09-24 21:03:25 +08:00
Liangsheng Yin cae4927fbf [DSV4] Budget the ratio-2 pair state pool in DSV4PoolConfigurator (#41048) 2026-09-23 23:53:46 -07:00
Michael a164c6d9f1 [AMD] Register mem-cache unit tests in PR CI (#40812) 2026-09-23 23:35:51 -07:00
cctryandcctry efff836911 [PD] Add decode host receive for custom transfer backends (#40238)
Co-authored-by: cctry <cctry@fb.com>
2026-09-23 21:57:33 -07:00
Nan Jiang c3685dff42 [Sampling] Add selected/support sampling logprob modes (#40932) 2026-09-23 21:38:19 -07:00
Liangsheng Yin 621136e094 [mem_cache] Remove unused helpers in mem_cache, storage backends, and metrics (#40963) 2026-09-23 18:16:44 -07:00
Liangsheng Yin c39a1c0663 [mem_cache] Free the rows below the SWA evict floor on all-SWA request release (#40798) 2026-09-23 14:46:28 -07:00
Liangsheng Yin 954458567e [mem_cache] Remove unreachable RadixCache paths in KV canary and HiCache accessors (#40807) 2026-09-23 13:22:58 -07:00
TianDi101andDuyi-Wang 4cd63da996 [Unified Cache] Dedup replicated MLA/DSA KV in the UMBP direct linker (#38778)
Co-authored-by: Duyi-Wang <duyi.wang@amd.com>
2026-09-23 02:15:01 -07:00
Liangsheng Yin 4ce23542bf [HiCache] Remove the unused HiRadixCache (#40787) 2026-09-22 16:17:12 -07:00
Liangsheng Yin d34f7b2326 [mem_cache] Clean up SWA/Mamba radix cache leftovers and drop SGLANG_ENABLE_UNIFIED_RADIX_TREE (#40780) 2026-09-22 15:40:57 -07:00
Liangsheng Yin 9a53f75124 [mem_cache] Remove the experimental C++ radix tree (#40775) 2026-09-22 14:19:45 -07:00
Shangming Cai 6f4c2b9b91 [HiCache] Demote internal-node mamba states on write_back eviction (#40680) 2026-09-23 02:18:58 +08:00
Shunkangz ddebc52f23 [kv-shard 3/4] Enable Control plane (#38468) 2026-09-22 19:57:44 +08:00
Michael 3f00fb7e2e [AMD] Register unified KV page-zeroing test in PR CI (#40123) 2026-09-22 02:16:31 -07:00
Guangda LiuandGuangda Liu 04c0913434 [HiSparse] Add MHA hisparse support for MiniMax M3 (#31446)
Co-authored-by: Guangda Liu <bingps@users.noreply.github.com>
2026-09-22 13:28:03 +08:00
Cheng Wan 506698761d [unified-memory] Hierarchical cache for every unified pool shape (#37507) 2026-09-21 16:50:37 -07:00
Cheng Wan acac4dd9d9 [Refactor] Clean up parallel runtime comments (#40632) 2026-09-21 14:32:22 -07:00
Cheng Wan 73f071db52 Deprecate the parallel getters the context answers, and ratchet them shut (#40342) 2026-09-21 12:25:32 -07:00
Cheng Wan 2d0e94e3a3 Check the topology identities where the layout is written, and build at the published widths (#40340) 2026-09-21 12:22:59 -07:00
cctry 7a6191c4b9 Preallocate HiCache MHA staging before post-capture KV sizing (#40256) 2026-09-21 10:44:29 -07:00
Shuwen WangandSeokhoon Kang 9f3d275940 [HiCache] Fix sparse hybrid transfer layer IDs (#37870)
Co-authored-by: Seokhoon Kang <sh.kang@postech.ac.kr>
2026-09-20 16:24:24 +08:00
amd-danli103andHAI e54009240a [AMD][DSV4] feat: enable DSpark with fp8 unified_kv on gfx950 (#38901)
Co-authored-by: HAI <hixiao@gmail.com>
2026-09-20 01:16:39 -07:00
Ke Bao a8a4d86be9 Remove swa and mamba radix cache (#40313) 2026-09-20 16:16:27 +08:00
Sasha Sidorov 59dd2fc734 [2/N] [Kernel] Fuse padding-preserving HiSparse slot translation (#39837) 2026-09-20 12:04:03 +08:00
Mohammad Miadh AngkadandMohammad Angkad df0dc44931 [Fix] Forward SWA prealloc reclaim through the DSV4 HiSparse allocator (#40354)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-09-19 20:27:51 -07:00
huangtingweiandChao Shi 020703923d [PP + HiCache] Add PP Prefetch Tickets for eager cross-stage storage prefetch (#36700)
Co-authored-by: Chao Shi <stepinto@live.com>
2026-09-20 11:19:31 +08:00
Zhiqiang Xie 7a6c652c77 [HiCache] Auto-size the host pool to fit available host memory (#40135) 2026-09-19 12:50:43 -07:00
993d1fccba [ROCm] Widen the HiCache JIT copy rounds and enable the K-only host pool (#37152)
Co-authored-by: Xiaobo Chen <xiaobche@smci355-ccs-aus-n05-33.prov.aus.ccs.cpe.ice.amd.com>
Co-authored-by: HAI <hixiao@gmail.com>
2026-09-19 08:58:10 -07:00
Lianmin Zheng 5d703de9e4 [HiCache] Size MHA host pools from device row width (#40304) 2026-09-18 23:37:58 -07:00
5e9342d16f [PP][DeepSeek V4] Overlap communication and optimize SM120 prefill (#38792)
Co-authored-by: Yangmin Li <yangminl@nvidia.com>
Co-authored-by: YAMY <74099316+YAMY1234@users.noreply.github.com>
2026-09-18 21:18:04 -07:00
Cheng Wan afe71f4b9e Read process groups through the runtime context (#40068) 2026-09-18 17:40:32 -07:00