Liangsheng Yin
|
37556c1b91
|
[mem_cache] Replace cache_finished_req with insert_req; release_kv_cache frees and unpins (#41281)
|
2026-09-27 02:53:16 -07:00 |
|
Liangsheng Yin
|
8ca82118e0
|
[mem_cache] Drop is_insert from cache_finished_req; release rows from release_kv_cache (#40988)
|
2026-09-24 17:23:26 -07:00 |
|
Liangsheng Yin
|
954458567e
|
[mem_cache] Remove unreachable RadixCache paths in KV canary and HiCache accessors (#40807)
|
2026-09-23 13:22:58 -07:00 |
|
Liangsheng Yin
|
4ce23542bf
|
[HiCache] Remove the unused HiRadixCache (#40787)
|
2026-09-22 16:17:12 -07:00 |
|
Liangsheng Yin
|
6cc9090d1f
|
[mem_cache] Release up to owned_kv_len on radix cache insert (#40075)
|
2026-09-18 15:37:52 -07:00 |
|
Nan Jiang
|
20518d8518
|
[Scheduler] Align RadixCache no-insert cleanup with kv_len_to_handle (#35204)
|
2026-09-17 17:00:59 -07:00 |
|
 Yanbin JiangandShuwen Wang
|
0b415fa573
|
[HiCache][LoRA] Isolate storage pages by extra key (#38577)
Co-authored-by: Shuwen Wang <47200617+alphabetc1@users.noreply.github.com>
|
2026-09-12 18:18:14 +08:00 |
|
 Xiaoyu ZhangandCursor
|
0a57403468
|
[Perf] Optimize Qwen3-VL unique-image serving on H100 (#36411)
Co-authored-by: Cursor <cursoragent@cursor.com>
|
2026-09-12 15:16:59 +08:00 |
|
Shuwen Wang
|
6e312af8c2
|
fix: collect prefix hash values iteratively (#38204)
|
2026-09-07 11:10:32 +08:00 |
|
 Shuwen WangandClaude Opus 5
|
4b44a1cde2
|
[Refactor] Let eviction policies take construction parameters (#37795)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-05 13:48:30 +08:00 |
|
 Khoa PhamandClaude Opus 5
|
cf3173aeb9
|
[Perf] Walk the radix tree by offset instead of re-slicing token storage (ported from #36507) (#37324)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-03 11:13:03 +08:00 |
|
 Alex NailsandAlison Shao
|
28262c20df
|
[CI][RFC] Replace black-jupyter with ruff-format (#37210)
Co-authored-by: Alison Shao <a.shao@wustl.edu>
|
2026-09-02 19:46:08 -07:00 |
|
Liangsheng Yin
|
007ef5e23a
|
[mem_cache] Move req_pool_idx into ReqKvInfo (#37094)
|
2026-08-30 14:46:21 -07:00 |
|
Liangsheng Yin
|
6be767c2d2
|
[mem_cache] Move cache_protected_len and swa_evict_floor into ReqKvInfo (#36982)
|
2026-08-29 19:37:53 -07:00 |
|
Ke Bao
|
480033def0
|
Refactor kv cache event mixin into a recorder (#35164)
|
2026-08-19 01:20:17 +08:00 |
|
jthomson04
|
385903b0ac
|
feat: add cache salt support to KV cache events (#30827)
Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
|
2026-08-12 16:14:04 -07:00 |
|
Leon Gao
|
cfb354bcfc
|
[Bugfix] Fix batched KV free aliasing (#34067)
|
2026-08-08 03:34:44 -07:00 |
|
Leon Gao
|
4d4f8023c4
|
[srt] Batch scheduler cache frees (#33475)
|
2026-08-06 21:49:59 -07:00 |
|
  
|
056474cdb0
|
feat: Session-reference-aware Unified Radix Cache for agentic multi-turn workloads (#29173)
Co-authored-by: Ishan Dhanani <ishandhanani@gmail.com>
Co-authored-by: hzh0425 <hzh0425@apache.org>
Co-authored-by: ispobock <ispobaoke@gmail.com>
|
2026-08-02 13:43:55 +08:00 |
|
Liangsheng Yin
|
c4fc241fd3
|
[Perf] Free KV pages by segment in the paged allocator without a device sync (#32701)
|
2026-07-28 22:24:16 -07:00 |
|
Liangsheng Yin
|
10908a6793
|
[Fix] Respect cache_protected_len in ChunkCache and disabled-radix release paths (#31662)
|
2026-07-18 12:04:25 -07:00 |
|
fzyzcjy
|
27256aee5b
|
Let cache backend do not couple with owned committed kv details and avoid kv_committed_freed/kv_overallocated_freed fields (#29428)
|
2026-07-15 14:43:10 +08:00 |
|
huangtingwei
|
5e1ccd9320
|
[HiCache] Optimize HiCache hash generation with bulk token byte conversion (#28287)
|
2026-07-01 15:44:23 +08:00 |
|
ishandhanani
|
aaa31eb0a1
|
feat: first-class session identity in SGLang (#29436)
|
2026-06-28 07:16:38 -07:00 |
|
ishandhanani
|
e67b228d4c
|
feat: session radix cache (#27058)
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
|
2026-06-23 01:51:41 -07:00 |
|
Qiaolin Yu
|
f293ddf3ce
|
[perf] reduce overhead of fill_ids list reconstruction and decref (#27965)
|
2026-06-14 00:41:11 -07:00 |
|
Cheng Wan
|
97a0031799
|
[lint] Enable Ruff UP037 to drop redundant quoted annotations (#27984)
|
2026-06-11 17:38:05 -07:00 |
|
fzyzcjy
|
259a2da3e0
|
Refactor Req.fill_ids into full_untruncated_fill_ids + fill_len with equivalence (#26637)
|
2026-06-08 14:52:18 +08:00 |
|
Qiaolin Yu
|
4b0f629082
|
[perf] reduce radix cache match overhead by changing the match algorithm (#27364)
|
2026-06-06 15:40:28 -07:00 |
|
ishandhanani
|
978fb6ed1a
|
hicache kv events: publish split write-through fragments (#27072)
|
2026-06-03 14:52:25 -07:00 |
|
Zhangheng
|
7662210406
|
[UnifiedTree]: Support eviction priority (#26549)
|
2026-05-30 15:19:43 +08:00 |
|
 Ethan ZHUandZhangheng
|
e86fdf3a3c
|
[Bug Fix][HiCache] TreeNode.get_prefix_hash_values @lru_cache can return mutated list (#26177)
Co-authored-by: Zhangheng <hzh0425@apache.org>
|
2026-05-25 11:15:10 +08:00 |
|
 Jialin Ouyangandjialino
|
06c23d55b5
|
perf: migrate Req token-id storage to array.array('q') in Scheduler (#25098)
Co-authored-by: jialino <jialino@fb.com>
|
2026-05-22 10:51:07 -07:00 |
|
Ke Bao
|
e4378ff37f
|
[UnifiedRadixCache] Fix HiCache load back start node (#25088)
|
2026-05-14 13:18:12 +08:00 |
|
 zhongdaor-nvandzhongdaor-nv
|
2cf1a4ab38
|
feat: Add KV events for Mamba radix cache (#23678)
Signed-off-by: zhongdaor-nv <220807034+zhongdaor-nv@users.noreply.github.com>
Co-authored-by: zhongdaor-nv <220807034+zhongdaor-nv@users.noreply.github.com>
|
2026-05-08 11:53:36 -07:00 |
|
Lianmin Zheng
|
3c3f0bd55e
|
Cache empty MatchResult in RadixCache (#24470)
|
2026-05-07 17:13:20 -07:00 |
|
ishandhanani
|
5b7ce417d0
|
[P/D disagg] - support decode side radix cache (#19746)
|
2026-05-01 21:55:34 +08:00 |
|
Liangsheng Yin
|
2b2cad70d6
|
[Refactor] Move radix-cache utils onto RadixKey as methods (#23209)
|
2026-04-20 23:11:58 -07:00 |
|
 ishandhananiandjthomson04
|
3c007ee5d4
|
fix(hicache): emit KV events for L2 host cache insertions (#22894)
Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Co-authored-by: jthomson04 <jwillthomson19@gmail.com>
|
2026-04-20 19:07:03 -07:00 |
|
Liangsheng Yin
|
c7a4ebf3c8
|
[Refactor] Replace page_align_keys helper with RadixKey.page_aligned method (#23107)
|
2026-04-20 18:10:42 -07:00 |
|
Liangsheng Yin
|
8cb957ccff
|
[Perf] Make EAGLE bigram key an O(1) view on RadixKey (#23106)
|
2026-04-20 12:01:11 -07:00 |
|
Liangsheng Yin
|
49cb7d546e
|
Move hash utils out of hicache_storage to break CUDA import chain (#22214)
|
2026-04-06 18:16:40 -07:00 |
|
ishandhanani
|
1081a25983
|
revert: remove TTL-based hard pin from HiRadixCache (#21884)
|
2026-04-01 16:51:15 -07:00 |
|
pansicheng
|
97b2a89334
|
[RadixTree][8/N Refactor]: unify lock interface (#20330)
|
2026-03-16 11:49:51 +08:00 |
|
 liubiyonggeandzhangheng
|
cc73355a1f
|
[Feature] Add SLRU eviction policy & fix RadixCache hit_count bug (#18843)
Co-authored-by: zhangheng <hzh0425@apache.org>
|
2026-03-08 21:30:55 +08:00 |
|
 ishandhananiandClaude
|
f7da379b61
|
feat: TTL-based prefix pinning with refresh-on-hit for HiRadixCache (#18941)
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-03-01 23:27:22 -08:00 |
|
zhangheng
|
a0d8a7ae6d
|
[RadixTree][6/N Refactor]: Refactor SWARadixTree to simplify the computation and alignment of bigram keys. (#19427)
|
2026-02-28 20:01:39 +08:00 |
|
SoluMilken
|
07a24f1a38
|
update pre-commit config (#18860)
|
2026-02-16 00:18:31 +08:00 |
|
![copilot-swe-agent[bot]](/assets/img/avatar_default.png) Yi-Chia Chenandcopilot-swe-agent[bot]
|
2bfab1bb67
|
Fix radix cache key to include generated tokens in multi-turn (regression) (#16521)
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
|
2026-02-10 14:08:34 -08:00 |
|
ishandhanani
|
01e3f4682e
|
feat(kv-events): Add medium field to KV event types for storage tier tracking (#18205)
|
2026-02-09 12:39:15 -08:00 |
|