Commit Graph
28 Commits
Author SHA1 Message Date
Liangsheng Yin 37556c1b91 [mem_cache] Replace cache_finished_req with insert_req; release_kv_cache frees and unpins (#41281) 2026-09-27 02:53:16 -07:00
Lianmin ZhengandLu Fang fc9e1c8d29 [MemCache] Fix LMCache component cursors and per-cache backend selection (#41328)
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com>
2026-09-26 11:24:39 -07:00
Lianmin ZhengandLu Fang 3ecb6cd729 [MemCache] Unify component eviction cursors and lock receipts (#41276)
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com>
2026-09-25 20:53:35 -07:00
Liangsheng Yin 8ca82118e0 [mem_cache] Drop is_insert from cache_finished_req; release rows from release_kv_cache (#40988) 2026-09-24 17:23:26 -07:00
Zhiqiang Xie beaf3d9252 [HiCache] Replace skip_lock_node_ids with a segment lock protocol (#36848) 2026-09-09 14:55:51 -07:00
3a770da756 [Unified Tree] Support Branching-Point Caching for the SWA Component (#34565)
Co-authored-by: alphabetc1 <2508695655@qq.com>
Co-authored-by: Shuwen Wang <47200617+alphabetc1@users.noreply.github.com>
2026-09-05 12:29:16 +08:00
Liangsheng Yin 3484f7f836 [mem_cache] Add free_kv_row to release a request's kv row by row range (#36721) 2026-09-01 01:14:43 -07:00
Liangsheng Yin 5d12ad4fd7 [mem_cache] Move mamba state and retraction_backup into ReqKvInfo (#37164) 2026-08-30 22:03:01 -07:00
Shuwen Wang 62f86ce470 [CI] Fix unreachable FakeReq field initialization (#37182) 2026-08-30 20:51:40 -07:00
Liangsheng Yin 4bb8de34cc [mem_cache] Share one ReqKvInfo between a streaming session slot and its request (#37108) 2026-08-30 16:36:48 -07:00
Liangsheng Yin 007ef5e23a [mem_cache] Move req_pool_idx into ReqKvInfo (#37094) 2026-08-30 14:46:21 -07:00
Liangsheng Yin 0438b16154 [mem_cache] Move kv_committed_len into ReqKvInfo (#37078) 2026-08-29 22:42:25 -07:00
Liangsheng Yin 6be767c2d2 [mem_cache] Move cache_protected_len and swa_evict_floor into ReqKvInfo (#36982) 2026-08-29 19:37:53 -07:00
092d85eb87 [Feature] Add MiniCPM-SALA support (#30360)
Co-authored-by: Alex Nails <alex.nails@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 02:25:16 -07:00
jthomson04 385903b0ac feat: add cache salt support to KV cache events (#30827)
Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
2026-08-12 16:14:04 -07:00
Ke Bao 50b029257f Skip mamba lock during decoding (#32228) 2026-07-29 21:53:28 +08:00
fzyzcjy 27256aee5b Let cache backend do not couple with owned committed kv details and avoid kv_committed_freed/kv_overallocated_freed fields (#29428) 2026-07-15 14:43:10 +08:00
fzyzcjy d8d76c4d12 Introduce req.kv container for coupled owned kv field lifecycle (#29427) 2026-07-15 14:40:38 +08:00
Liangsheng Yin b7d62bd724 [CI] Rename basic CI stage-a/b/c -> base-a/b/c for symmetry with extra CI (#25420) 2026-05-15 18:26:55 -07:00
Ke Bao e4378ff37f [UnifiedRadixCache] Fix HiCache load back start node (#25088) 2026-05-14 13:18:12 +08:00
sglang-botandsglang-bot da175b964d chore: update CI test est_time values (#23785)
Co-authored-by: sglang-bot <sglang-bot@users.noreply.github.com>
2026-04-26 20:17:50 -07:00
Liangsheng Yin 6cc2eee50d [misc] CI hygiene: enforce __main__ entry, drop silent-skipped tests, fix rerun-test protoc (#23305) 2026-04-20 21:16:24 -07:00
Liangsheng Yin eb76aaba88 [core] Always-on StreamingSession in UnifiedRadixCache (#23202) 2026-04-19 21:19:43 -07:00
Liangsheng Yin d3ce664612 move session to python/sglang/srt/session (#23144) 2026-04-19 17:34:19 -07:00
Liangsheng Yin f9792166c3 trim_overshoot: cap swa_evicted_seqlen + unit test (#22900) 2026-04-15 15:05:35 -07:00
Liangsheng Yin aa78564e1a Refactor streaming session abort handling (#22790) 2026-04-15 00:13:05 -07:00
Liangsheng Yin 33a3ba256f Delete dead rematch path in SessionAwareCache.release_session (#22735) 2026-04-13 17:02:40 -07:00
c1ab68b45e fix: streaming session race condition + some metrics (#21875)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
2026-04-12 18:05:23 -07:00