Commit Graph
34 Commits
Author SHA1 Message Date
Liangsheng Yin 37556c1b91 [mem_cache] Replace cache_finished_req with insert_req; release_kv_cache frees and unpins (#41281) 2026-09-27 02:53:16 -07:00
Liangsheng Yin 8ca82118e0 [mem_cache] Drop is_insert from cache_finished_req; release rows from release_kv_cache (#40988) 2026-09-24 17:23:26 -07:00
Liangsheng Yin 954458567e [mem_cache] Remove unreachable RadixCache paths in KV canary and HiCache accessors (#40807) 2026-09-23 13:22:58 -07:00
Liangsheng Yin 4ce23542bf [HiCache] Remove the unused HiRadixCache (#40787) 2026-09-22 16:17:12 -07:00
Ke Bao a8a4d86be9 Remove swa and mamba radix cache (#40313) 2026-09-20 16:16:27 +08:00
Liangsheng Yin 6cc9090d1f [mem_cache] Release up to owned_kv_len on radix cache insert (#40075) 2026-09-18 15:37:52 -07:00
ishandhanani 3f871a246c feat(agent sessions): attribute stored KV cache blocks to sessions (#37482)
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
2026-09-14 21:34:49 -07:00
Yanbin JiangandShuwen Wang 0b415fa573 [HiCache][LoRA] Isolate storage pages by extra key (#38577)
Co-authored-by: Shuwen Wang <47200617+alphabetc1@users.noreply.github.com>
2026-09-12 18:18:14 +08:00
Xiaoyu ZhangandCursor 0a57403468 [Perf] Optimize Qwen3-VL unique-image serving on H100 (#36411)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-12 15:16:59 +08:00
sglang-botandsglang-bot 6252993afe chore: update CI test est_time values (#38238)
Co-authored-by: sglang-bot <sglang-bot@users.noreply.github.com>
2026-09-06 17:49:41 -07:00
Liangsheng Yin 3b14f37b74 [Fix] Use real ReqKvInfo in unit-test req mocks (#37339) 2026-08-31 20:25:51 -07:00
Liangsheng Yin 007ef5e23a [mem_cache] Move req_pool_idx into ReqKvInfo (#37094) 2026-08-30 14:46:21 -07:00
Liangsheng Yin 6be767c2d2 [mem_cache] Move cache_protected_len and swa_evict_floor into ReqKvInfo (#36982) 2026-08-29 19:37:53 -07:00
Ke Bao 480033def0 Refactor kv cache event mixin into a recorder (#35164) 2026-08-19 01:20:17 +08:00
jthomson04 903439044a perf(kv-events): coalesce cache events (#31479) 2026-08-13 13:36:53 -07:00
jthomson04 385903b0ac feat: add cache salt support to KV cache events (#30827)
Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
2026-08-12 16:14:04 -07:00
Leon Gao cfb354bcfc [Bugfix] Fix batched KV free aliasing (#34067) 2026-08-08 03:34:44 -07:00
Leon Gao 4d4f8023c4 [srt] Batch scheduler cache frees (#33475) 2026-08-06 21:49:59 -07:00
Liangsheng Yin c4fc241fd3 [Perf] Free KV pages by segment in the paged allocator without a device sync (#32701) 2026-07-28 22:24:16 -07:00
ANSHUMAN TRIPATHY 23f2b77d82 Make UTs compatible for XPU (#27106) 2026-07-15 12:35:56 +08:00
Liangsheng Yin c53559ba10 [misc] Remove unit test cases that fail the admission criteria (#30690) 2026-07-09 15:31:28 -07:00
Yan Ru Pei c0d4d1897c fix: batch BlockRemoved events per radix node (#29265) 2026-06-26 16:13:40 +08:00
Liangsheng Yin 0ce3db3c0a [Bug] Fix out-of-range token id crashing tp=1 VocabParallelEmbedding (#27482) 2026-06-06 23:00:17 -07:00
Qiaolin Yu 4b0f629082 [perf] reduce radix cache match overhead by changing the match algorithm (#27364) 2026-06-06 15:40:28 -07:00
Ethan ZHUandZhangheng e86fdf3a3c [Bug Fix][HiCache] TreeNode.get_prefix_hash_values @lru_cache can return mutated list (#26177)
Co-authored-by: Zhangheng <hzh0425@apache.org>
2026-05-25 11:15:10 +08:00
Jialin Ouyangandjialino 06c23d55b5 perf: migrate Req token-id storage to array.array('q') in Scheduler (#25098)
Co-authored-by: jialino <jialino@fb.com>
2026-05-22 10:51:07 -07:00
Liangsheng Yin b7d62bd724 [CI] Rename basic CI stage-a/b/c -> base-a/b/c for symmetry with extra CI (#25420) 2026-05-15 18:26:55 -07:00
Liangsheng Yin 22d3f3996c ci: decouple stage and runner for cuda registry (#25197) 2026-05-13 17:28:21 -07:00
sglang-botandsglang-bot da175b964d chore: update CI test est_time values (#23785)
Co-authored-by: sglang-bot <sglang-bot@users.noreply.github.com>
2026-04-26 20:17:50 -07:00
Liangsheng Yin c7a4ebf3c8 [Refactor] Replace page_align_keys helper with RadixKey.page_aligned method (#23107) 2026-04-20 18:10:42 -07:00
265696b176 chore: update CI test est_time values (#22565)
Co-authored-by: Cheng Wan <chwan@rice.edu>
Co-authored-by: sglang-bot <sglang-bot@users.noreply.github.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
2026-04-10 18:15:02 -07:00
Liangsheng YinandClaude Opus 4.6 875a615993 fix(ci): update est_time for 57 tests based on runtime analysis (#21896)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 20:16:13 -07:00
Lianmin Zheng 814202704b ci: unify PR test suite naming (#21187) 2026-03-23 00:18:45 -07:00
Ke Bao c3483e8e97 [CI] Move existing unit tests into unit directory (#20631) 2026-03-15 23:25:18 +08:00