Commit Graph
476 Commits
Author SHA1 Message Date
61ba98443d [DSA] k-pool: 256-token logical page so index page id = logical page id (#42178)
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
2026-10-05 15:02:44 -07:00
cafb37a10d [JIT] Fuse FP8 KV-cache quantization into the prefix-valid commit kernel (#40785)
Co-authored-by: Jayden Yu <jaydenyu@fb.com>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
2026-10-05 14:47:54 +08:00
Liangsheng Yin 503b898540 [Session] Count streaming session KV by owner in pool accounting (#42539) 2026-10-04 22:15:57 -07:00
SuperSongandCheng Wan bab04cd791 fix(unified-memory): propagate unified-memory lazy checkpoint policy and handle unallocated session slots (#42072)
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
2026-10-04 12:51:22 -07:00
Shuwen Wang 137c084000 [HiCache] refactor: retire in-flight storage prefetches through one helper (#41453) 2026-10-05 00:13:07 +08:00
SuperSongandCheng Wan 6cec8f98ce [Bugfix] Fix unified-memory compaction gates and pending page reuse (#39982)
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
2026-10-04 02:48:08 -07:00
Mohammad Miadh AngkadandMohammad Angkad 0bdd0cc8dc [Deps] Upgrade the CUDA PyTorch stack to 2.14 (#38641)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-10-04 00:48:02 -07:00
937c0a6cfe Fix unified HiCache physical transfers (#39479)
Co-authored-by: yhzhuang <yhzhuang@fb.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: Yonghao Zhuang <yhzhuang@users.noreply.github.com>
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
2026-10-04 00:38:56 -07:00
Liangsheng Yin 048b2a07f3 [Session] Release aborted streaming turns through the normal release path (#42330) 2026-10-03 22:30:25 -07:00
4611ea365e [HiCache]: Fix host lock ownership across radix splits (#38480)
Signed-off-by: Lukas Humbel <lhumbel@nvidia.com>
Co-authored-by: Jialin Ouyang <jialino@meta.com>
Co-authored-by: Shuwen Wang <47200617+alphabetc1@users.noreply.github.com>
2026-10-04 11:16:04 +08:00
5606fb8592 [Fix] HiCache: carry registered Mamba slot side states (Qwen4-Exp PLE) through the host tier (#39862)
Co-authored-by: Shuwen Wang <47200617+alphabetc1@users.noreply.github.com>
Co-authored-by: alphabetc1 <2508695655@qq.com>
Co-authored-by: ispobock <ispobaoke@gmail.com>
2026-10-04 10:15:59 +08:00
4ab720e655 [HiCache] Fix cgroup page-cache accounting and the sizing fallback when cgroup discovery fails (#42420)
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com>
Co-authored-by: xiezhq-hermann <xiezhq-hermann@users.noreply.github.com>
Co-authored-by: nvjullin <nvjullin@users.noreply.github.com>
2026-10-03 16:01:32 -07:00
Cheng Wan 10edb05e72 [Refactor] Let the remaining placement consumers read the parallel context (#42348) 2026-10-03 14:59:32 -07:00
Liangsheng Yin df53c978f1 [mem_cache] Run mamba models on UnifiedRadixCache when the radix cache is disabled (#42354) 2026-10-03 14:11:13 -07:00
b016ca406f Support post-capture KV sizing for the unified hybrid-SWA pool (#41961)
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com>
Co-authored-by: ZYHowell <ZYHowell@users.noreply.github.com>
2026-10-03 09:54:20 -07:00
2bae12b9b3 [HiCache] Fix write-back SWA insert backups tripping the write-through pending-ack assert (#42264)
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com>
Co-authored-by: 842974287 <842974287@users.noreply.github.com>
2026-10-03 09:45:56 -07:00
Liangsheng Yin 122b59152c [mem_cache] Replace is_chunk_cache / is_tree_cache with supports_prefix_sharing (#42362) 2026-10-03 02:07:55 -07:00
Mohammad Miadh AngkadandMohammad Angkad 00bcc25f6c [HiCache] Fix DeepSeek-V4 storage backend crash from missing storage_format_tag (#42215)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-10-02 22:27:57 -07:00
Liangsheng Yin a8b54f138e [Session] Run streaming sessions only on UnifiedRadixCache; reject unverified tree caches (#42295) 2026-10-02 18:15:59 -07:00
Kevin MiandClaude Opus 5 65f759144d [HiCache] Let the MiniMax-M3 K-only host pool join a host pool group (#42166)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-10-02 17:00:13 -07:00
Liangsheng Yin 8a328e867b [mem_cache] Rename the tree's request-level insert_req to checkpoint (#42202) 2026-10-02 14:42:35 -07:00
raghotham a64725d407 [HiCache] Fall back to host memory when no cgroup fs is mounted (#42120) 2026-10-02 13:59:40 -07:00
Liangsheng Yin 07e1a4f082 [mem_cache] Skip the release-time insert on optimistic prefill requeue; make refresh_fill_ids public (#42194) 2026-10-02 13:46:32 -07:00
Liangsheng Yin 50be533d09 [mem_cache] Rename cache_unfinished_req to checkpoint_req and count cache hits only when a request finishes (#41520) 2026-10-01 21:41:45 -07:00
Yuwei An ef867fa40d [Fix][DSV4.1] SWA page size with bounded replay (#42128) 2026-10-01 18:50:43 -07:00
caihuali95andCheng Wan 2a3655fea1 [unified-memory] Name the fused KV translate for what it computes (7/7) (#40331)
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
2026-10-01 17:14:03 -07:00
df530479c6 [unified-memory] Stride the KV translate kernel and route every translate through it (6/7) (#40330)
Co-authored-by: Caihua Li <caihua.li@bytedance.com>
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
2026-10-01 17:12:14 -07:00
9855e9c6f8 [unified-memory] Remove the kernel-page multiplier plumbing (5/7) (#40329)
Co-authored-by: Caihua Li <caihua.li@bytedance.com>
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
2026-10-01 17:11:25 -07:00
4c4e352a4b [unified-memory] Mark the write loc physical and check it at every write door (4/7) (#40328)
Co-authored-by: Caihua Li <caihua.li@bytedance.com>
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
2026-10-01 17:10:37 -07:00
cdadb1c79a [unified-memory] Token-major dense views for the unified memory pool (3/7) (#38592)
Co-authored-by: Caihua Li <caihua.li@bytedance.com>
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
2026-10-01 17:09:38 -07:00
caihuali95andCheng Wan 5df52e6382 [unified-memory] Build paged KV views through one helper (2/7) (#40327)
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
2026-10-01 17:08:42 -07:00
caihuali95andCheng Wan d3dbb71e5a [unified-memory] Derive KV row addresses from strides, not shapes (1/7) (#40326)
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
2026-10-01 17:07:47 -07:00
f17f7705a5 [AMD] [GLM-5.3-Flash] Enable FP8 and MXFP4 serving on gfx950 (#39273)
Co-authored-by: Byron Hsu <byron+per@periodiclabs.ai>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: long10024070 <long10024070@users.noreply.github.com>
Co-authored-by: Kevin Mi <45493463+kevin-mii@users.noreply.github.com>
Co-authored-by: Kevin Mi <mikevin920@yahoo.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: sunxxuns <126995791+sunxxuns@users.noreply.github.com>
Co-authored-by: Long Luong <long.luong2+personal@onemount.com>
2026-10-01 15:59:44 -07:00
amd-danli103 9c3262d855 [AMD][DSV4] feat: enable PD-disagg with fp8 unified_kv on gfx950 (#39166) 2026-09-30 23:42:31 -07:00
Peng Wu 11500701f1 [DSV4] Reserve the FULL logical page in the c4 indexer pool like its KV pool (#41163) 2026-09-30 23:19:27 -07:00
Ke Bao 5733bcaad3 Declare HiCache host pools for DSA indexer (#40913) 2026-10-01 13:08:26 +08:00
f02828ea39 fix(dsv4): ship the fp8 unified_kv rope pool through the direct external linker (#40471)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: HaiShaw <hixiao@gmail.com>
Co-authored-by: billishyahao <bill.he@amd.com>
2026-09-30 21:46:56 -07:00
Shuwen Wang bb62f125aa [HiSparse] fix: return HiSparse slots on PD decode waiting abort and DeepSeek V4 free (#41346) 2026-10-01 10:24:51 +08:00
9f9f9e702a [HiCache] Attribute buffer-mode storage hits against the joint device match (#41758)
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com>
Co-authored-by: xiezhq-hermann <xiezhq-hermann@users.noreply.github.com>
2026-09-30 11:46:30 -07:00
paulzhang-tmandKe Bao 8055ccd2cd [Linear Attention] Expose GDN/KDA prefill hooks and auxiliary cache accounting (#40227)
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
2026-09-30 17:19:02 +08:00
77091cea68 [Radix Cache] Sync Rust TreeCore and make it the default (#39627)
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
Co-authored-by: Zhangheng <hzh0425@apache.org>
Co-authored-by: alphabetc1 <alphabetc1@users.noreply.github.com>
2026-09-29 10:18:57 +08:00
Shuwen Wang bddeb61d94 [Unified Memory] fix: preserve FP8 dtype in unified MHA pool (#38133) 2026-09-28 05:19:03 -07:00
Liangsheng Yin 37556c1b91 [mem_cache] Replace cache_finished_req with insert_req; release_kv_cache frees and unpins (#41281) 2026-09-27 02:53:16 -07:00
Shuwen Wang 38d865489a [DSV4.1][HiCache] fix: wait for the layer transfer before reading low-ratio index-K (#41345) 2026-09-26 22:04:03 -07:00
Lianmin ZhengandLu Fang fc9e1c8d29 [MemCache] Fix LMCache component cursors and per-cache backend selection (#41328)
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com>
2026-09-26 11:24:39 -07:00
c10fa03fc4 Make sliding-window caching and speculative batch padding extensible (#41325)
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com>
Co-authored-by: jasonjk-park <jasonjk@fb.com>
2026-09-26 11:24:02 -07:00
Xiaoyu Zhang 38ec649048 [Refactor] Restore logical kernel groups and test organization (#41243) 2026-09-26 12:30:44 +08:00
Lianmin ZhengandLu Fang 3ecb6cd729 [MemCache] Unify component eviction cursors and lock receipts (#41276)
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com>
2026-09-25 20:53:35 -07:00
Liangsheng Yin f35ec18d8b [Test] Remove more unit tests that mirror implementation or never run in CI (#41297) 2026-09-25 19:22:14 -07:00
Shuwen Wang 4273e7de77 [PD] fix: cache resumed decode-radix requests from root instead of an unlocked re-match (#41261) 2026-09-26 10:21:41 +08:00