 Zhiqiang Xie
|
7a6c652c77
|
[HiCache] Auto-size the host pool to fit available host memory (#40135)
|
2026-09-19 12:50:43 -07:00 |
|
|
Zhiqiang Xieandcctry
|
ceb1d2e580
|
[PD] Enable optimistic prefill with buffer-only L3 write-through HiCache (#40043)
Co-authored-by: cctry <csycfl@gmail.com>
|
2026-09-18 16:04:15 -07:00 |
|
|
Zhiqiang Xie
|
a0534f8cca
|
[HiCache] Stop arming a prefetch retry for a too-short storage span (#40042)
|
2026-09-18 15:33:58 -07:00 |
|
|
Zhiqiang Xie
|
4e9e407d37
|
[HiCache] Label radix-cache metrics per rank and split the "shrunk" prefetch reason (#39280)
|
2026-09-15 17:54:04 -07:00 |
|
|
Zhiqiang Xie
|
49f540f3c0
|
[HiCache] Optimize buffer-mode storage existence bookkeeping (#39480)
|
2026-09-15 10:49:58 -07:00 |
|
|
Zhiqiang Xie
|
7f5dd19256
|
[HiCache] Rework the buffer-mode storage prefetch pipeline and retry bookkeeping (#39283)
|
2026-09-15 10:48:49 -07:00 |
|
|
Zhiqiang Xie
|
beaf3d9252
|
[HiCache] Replace skip_lock_node_ids with a segment lock protocol (#36848)
|
2026-09-09 14:55:51 -07:00 |
|
|
Zhiqiang Xie
|
b44496389c
|
[HiCache] Count hit allocations and in-flight backups in the buffer pipeline idle check (#37883)
|
2026-09-03 16:08:50 -07:00 |
|
|
Zhiqiang Xie
|
a480f388b2
|
[HiCache] L3 storage prefetch lifecycle metrics and cross-tier attribution fixes (#37503)
|
2026-09-03 16:00:04 -07:00 |
|
|
Zhiqiang Xie
|
d0c95f6c91
|
[HiCache] buffer mode: anchor-lock staged prefetches by default (#37464)
|
2026-09-03 12:08:22 -07:00 |
|
|
Zhiqiang Xie
|
68978f8d52
|
[Scheduler] Count the parked chunked-prefill request in the busy mem check (#37502)
|
2026-09-03 12:08:17 -07:00 |
|
|
Zhiqiang Xie
|
a328c19c81
|
[HiCache] buffer mode: decide staged-fetch fate against the live tree (#36834)
|
2026-08-29 01:51:47 -07:00 |
|
|
Zhiqiang Xie
|
f611e0c073
|
[HiCache] Fence load-back behind the forward stream (#36738)
|
2026-08-28 11:12:51 -07:00 |
|
|
Zhiqiang Xie
|
c9bba091f8
|
[HiCache] Key storage prefetch by the request namespace (#36382)
|
2026-08-28 11:12:10 -07:00 |
|
|
Zhiqiang Xie
|
3785b2d20f
|
[HiCache] Reject load-back specs that claim nodes pinned by an in-flight load-back (#35931)
|
2026-08-28 01:59:06 -07:00 |
|
|
Zhiqiang Xie
|
1061e34785
|
[HiCache] Retry L3 storage prefetch after a missed attempt (#36227)
|
2026-08-28 01:35:21 -07:00 |
|
|
Zhiqiang Xie
|
3f1031d697
|
[HiCache] Heal the storage existence cache on a hybrid prefetch discard (#36386)
|
2026-08-28 01:33:40 -07:00 |
|
|
Zhiqiang Xie
|
c35683fda0
|
[HiCache] Clamp tombstoned SWA locs in UnifiedSWAKVPool translation (#35933)
|
2026-08-21 23:08:56 -07:00 |
|
|
Zhiqiang Xie
|
0db2bdfec5
|
Fix buffer-mode HiCache load-back ownership races; add optional prefetch anchor lock (#35769)
Signed-off-by: Zhiqiang Xie <zqx@meta.com>
|
2026-08-21 20:20:36 -07:00 |
|
|
Zhiqiang Xie
|
01814e110d
|
[HiCache] Simple style change for buffer mode (#35574)
|
2026-08-19 15:23:21 -07:00 |
|
|
Zhiqiang Xie
|
977412ae61
|
[HiCache] Buffer-only mode for HiCache host memory layer (#34798)
|
2026-08-18 19:21:24 -07:00 |
|
|
Zhiqiang XieandTingwei Huang
|
5469faec45
|
HiSparse: shared-index (IndexShare) plan-then-IO swap-in prefetch (#34329)
Co-authored-by: Tingwei Huang <huangtingwei9988@gmail.com>
|
2026-08-11 01:58:28 -07:00 |
|
|
Zhiqiang Xie
|
3dc91366ac
|
[HiCache] write_back: reclaim duplicated host copy first under host pressure (#33777)
|
2026-08-07 15:59:59 -07:00 |
|
|
Zhiqiang Xie
|
f6de147b8d
|
Remove revoke queue after hit-then-alloc refactoring (#33613)
|
2026-08-06 03:10:10 -07:00 |
|
|
Zhiqiang Xie
|
106bcc1293
|
Observability enhancement for HiCache (#32388)
|
2026-08-05 14:13:06 -07:00 |
|
|
Zhiqiang Xie
|
5da0b6ec39
|
Write-back policy fix for unified tree (#31845)
|
2026-07-24 09:58:18 -07:00 |
|
|
Zhiqiang Xie
|
f19246e59a
|
[HiCache] write_back policy refinement (#29817)
|
2026-07-02 12:16:03 -07:00 |
|
|
|
da0f4f6f92
|
[HiCache] remove large host mem constraint (#28614)
Co-authored-by: Teng Ma <stmatengss@gmail.com>
Co-authored-by: Vladislav Nosivskoy <vladnosiv@gmail.com>
|
2026-06-26 16:20:48 -07:00 |
|
|
|
41c7c97ff3
|
fix hisparse LRU policy (#22170)
Co-authored-by: huangtingwei9988 <huangtingwei9988@users.noreply.github.com>
Co-authored-by: hzh0425 <hzh0425@users.noreply.github.com>
|
2026-04-05 18:47:58 -07:00 |
|
|
|
30ba1f78b0
|
Hisparse Minor Fix (#22131)
Co-authored-by: huangtingwei9988 <141888744+huangtingwei9988@users.noreply.github.com>
Co-authored-by: hzh0425 <58988019+hzh0425@users.noreply.github.com>
|
2026-04-05 16:15:47 -07:00 |
|
|
Zhiqiang Xie
|
9eb75211b1
|
style refinement for hisparse (#21198)
|
2026-04-01 01:03:17 -07:00 |
|
|
Zhiqiang Xie
|
13f4f010d8
|
HiSparse for Sparse Attention (#20343)
|
2026-03-22 23:09:31 -07:00 |
|
|
Zhiqiang Xie
|
d9ed80b9f1
|
fix AMD CI failure of NUMA binding (#17184)
|
2026-01-16 00:21:04 -08:00 |
|
|
Zhiqiang Xieandhxie
|
675acecec6
|
Attention backend selection bug fix for hicache (#16779)
Co-authored-by: hxie <haiyong.xie@ustc.edu>
|
2026-01-10 00:22:41 -08:00 |
|
|
Zhiqiang XieandHao Chen
|
cfcf2758aa
|
[HiCache] Critical fix to host memory double free (#13501)
Co-authored-by: Hao Chen <cighao@gmail.com>
|
2025-11-18 01:34:23 -08:00 |
|
|
Zhiqiang Xie
|
3d40794fcf
|
[HiCache] Cleaning the deprecated host memory state (#10778)
|
2025-09-25 14:43:53 +08:00 |
|
|
Zhiqiang Xie
|
3b99f23c44
|
[Bugfix] Retract not releasing enough memory when page size > 1 (#9989)
|
2025-09-07 21:41:50 -07:00 |
|
|
Zhiqiang Xie
|
0b8c5721f1
|
[HiStorage] Remove delete and clear as necessary methods (#10039)
|
2025-09-06 10:27:26 +08:00 |
|
|
Zhiqiang Xie
|
369b143366
|
[HiCache] Minor fix on file storage backend (#9869)
|
2025-09-02 15:52:37 -07:00 |
|
|
Zhiqiang Xieandpansicheng
|
8b6966d020
|
[HiCache] Storage Refactoring (#9797)
Co-authored-by: pansicheng <27603155+pansicheng@users.noreply.github.com>
|
2025-08-31 22:58:21 +08:00 |
|
|
Zhiqiang Xie
|
f9076a5a2c
|
hot fix for mooncake batch set api (#9836)
|
2025-08-30 21:01:51 -07:00 |
|
|
Zhiqiang Xie
|
54e872d343
|
[HiCache] resolve conflict between chunked-prefill and hicache hit count (#9776)
|
2025-08-30 01:30:54 +08:00 |
|
|
Zhiqiang Xie
|
001f51940a
|
[HiCache] change the default policy to write through (#9772)
|
2025-08-28 18:28:39 -07:00 |
|
|
Zhiqiang Xie
|
43de1d7304
|
HiCache Storage fix host memory leak (#9648)
|
2025-08-26 10:49:40 -07:00 |
|
|
Zhiqiang Xieandgemini-code-assist[bot]
|
0eec4cb6cc
|
HiCache, add bench long context plus minor fixs (#9086)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2025-08-11 16:54:52 -07:00 |
|
|
Zhiqiang Xie
|
9f78f391ae
|
HiCache Storage: generate hash when inserting new nodes (#9053)
|
2025-08-11 14:18:59 -07:00 |
|
|
Zhiqiang Xie
|
b32792516a
|
REVIEWERS.md typo fix (#9048)
|
2025-08-10 22:33:37 -07:00 |
|
|
Zhiqiang Xie
|
6e0b646832
|
HiCache Storage tp fix (#8878)
|
2025-08-09 01:16:51 -07:00 |
|
|
Zhiqiang Xie
|
b0add2da00
|
HiCache storage, style change and bug fix (#8719)
|
2025-08-03 15:05:04 +08:00 |
|
|
Zhiqiang Xie
|
dd7ca00601
|
Interface change for kvcache io to support page first layout (#8318)
|
2025-08-01 11:37:49 +08:00 |
|
|
Zhiqiang Xie
|
9305ea6c2d
|
HiCache, fixing hash value indexing (#8636)
|
2025-08-01 11:29:51 +08:00 |
|
|
Zhiqiang Xie
|
528bd1ed85
|
HiCache, check before terminate prefetching (#8372)
|
2025-07-26 23:13:16 -07:00 |
|
|
Zhiqiang Xie
|
d40846d456
|
breakdown kernel update (#8334)
|
2025-07-25 08:33:17 +08:00 |
|
|
Zhiqiang Xieandpansicheng
|
145482f422
|
HiCache Storage TP Refinement (#8307)
Co-authored-by: pansicheng <sicheng.pan.chn@gmail.com>
|
2025-07-25 08:31:47 +08:00 |
|
|
Zhiqiang Xieandpansicheng
|
f39037fffb
|
HiCache Fix (#8288)
Co-authored-by: pansicheng <sicheng.pan.chn@gmail.com>
|
2025-07-23 16:51:32 +08:00 |
|
|
Zhiqiang Xie
|
ce86e201df
|
bug fix and tag (#8282)
|
2025-07-23 16:50:31 +08:00 |
|
|
Zhiqiang Xie
|
b43263307f
|
Hicache IO kernel refactoring (#8264)
|
2025-07-23 16:49:03 +08:00 |
|
|
Zhiqiang Xie
|
9d33fcfb8e
|
Hicache Storage Layer Prototype (#7704)
|
2025-07-18 15:20:19 +08:00 |
|
|
Zhiqiang Xie
|
2fc824b84c
|
Kernels for efficient KV cache IO (#7313)
|
2025-07-06 22:53:36 -07:00 |
|
|
Zhiqiang Xie
|
f9eb04ddb2
|
upgrade sgl kernel to 0.2.1 for main (#7676)
|
2025-07-01 00:00:13 -07:00 |
|
|
Zhiqiang Xie
|
34c3f9b2d3
|
kvcache io kernels and test case (#7382)
|
2025-06-23 11:58:59 -07:00 |
|
|
Zhiqiang Xie
|
e56685ac1b
|
Upstreaming hicache bug fixes (#7267)
|
2025-06-17 17:44:57 -07:00 |
|
|
Zhiqiang Xie
|
b26cb1c55a
|
Fix problem of large page size with chunked prefill (#6046)
|
2025-05-06 15:19:47 +08:00 |
|
|
Zhiqiang Xie
|
f8e460930a
|
Fix prefill OOM error in the case of large page size (#5081)
|
2025-05-05 16:02:55 -07:00 |
|
|
Zhiqiang Xie
|
70645f4d7d
|
upstream hicache fixes (#5570)
|
2025-04-20 23:08:30 -07:00 |
|
|
Zhiqiang Xie
|
e2574ee986
|
fix hicache write back (#5543)
|
2025-04-19 21:56:22 -07:00 |
|
|
Zhiqiang Xie
|
9d0b36c47a
|
fix deepgemm as well (#5030)
|
2025-04-03 02:41:37 -07:00 |
|
|
Zhiqiang Xie
|
a2aea59b6e
|
update cutlass tag (#5011)
|
2025-04-02 18:30:30 -07:00 |
|
|
Zhiqiang Xie
|
3fadc64793
|
bug fix for hicache host eviction (#4989)
|
2025-04-02 00:33:50 -07:00 |
|
|
Zhiqiang Xie
|
e119f04215
|
Large page size aligned hierarchical caching (#4581)
|
2025-04-01 22:38:15 -07:00 |
|
|
Zhiqiang Xie
|
12047f5e94
|
Prevent memory leak of retract_decode when page_size > 1 (#4977)
|
2025-04-01 15:30:45 -07:00 |
|
|
Zhiqiang XieandLianmin Zheng
|
a169b9f813
|
Fix oom error for large page size (#4913)
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
|
2025-03-30 21:34:21 -07:00 |
|
|
Zhiqiang Xie
|
4d25305700
|
Move mem_state update into debug mode (#4525)
|
2025-03-23 00:52:27 -07:00 |
|
|
Zhiqiang Xie
|
ecbfe58bb0
|
Bug fix for metrics counter (#4660)
|
2025-03-22 13:39:21 -07:00 |
|
|
Zhiqiang Xie
|
a98290aea3
|
Unit test for Hierarchical Caching (#4486)
|
2025-03-17 17:45:00 -07:00 |
|
|
Zhiqiang Xie
|
f5bbf6037d
|
Fix: Complete int32 to int64 conversion (#4465)
|
2025-03-16 18:14:27 -07:00 |
|
|
Zhiqiang Xie
|
fbdb50501f
|
Hot fix for hicache with new page aligned radixtree (#4397)
|
2025-03-13 15:50:49 -07:00 |
|
|
Zhiqiang Xie
|
10b544ae9b
|
Hierarchical Caching Refactoring and Fixing TP issue (#4082)
|
2025-03-12 11:22:35 -07:00 |
|
|
Zhiqiang Xie
|
9376ac361d
|
Memory pool fix for upstream change about eagle (#4170)
|
2025-03-07 00:58:20 -08:00 |
|
|
Zhiqiang Xie
|
aee30630d8
|
Add a pointer to the real KV cache pool (#4113)
|
2025-03-05 21:39:07 -08:00 |
|
|
|
6c7a152c5a
|
Hierarchical Caching for SGLang (#2693)
Co-authored-by: Wenxuan Tan <wenxuan.tan@wisc.edu>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
|
2025-02-23 21:56:30 -08:00 |
|
|
Zhiqiang XieandLianmin Zheng
|
08104b56de
|
Sanity check to prevent performance regression (#3171)
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
|
2025-01-27 12:28:17 -08:00 |
|
|
Zhiqiang Xie
|
8af7048dcf
|
Query remaining memory dynamically for PrefillAdder (#2941)
|
2025-01-17 20:20:26 -08:00 |
|
|
Zhiqiang Xie
|
13387e6b7a
|
Multi-turn benchmark for hierarchical caching (#2942)
|
2025-01-17 16:17:24 -08:00 |
|
|
Zhiqiang Xie
|
5d6e9467d4
|
Cache controller for hierarchical caching (#2804)
|
2025-01-10 20:22:01 -08:00 |
|
|
Zhiqiang Xie
|
51caee740f
|
Host memory pool for hierarchical caching (#2771)
|
2025-01-07 21:38:37 +00:00 |
|