minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
13,208 Commits 2 Branches 0 Tags
bed20249f16903853e472e181e6a6358bf5df802
Commit Graph
77 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Zhangheng 5f8911183b [UnifiedTree]: Update Unified Radix Cache README (#26485) 2026-05-27 21:50:53 +08:00
Zhangheng 38f32c38ab [UnifiedRadixTree]: Support L3 HiStorage framework (#26062) 2026-05-26 22:38:00 +08:00
Zhanghengand晟海 a4db563c87 [hisparse]: update user guide (#26249) 
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
 2026-05-25 17:54:55 +08:00
Zhangheng 0be539024f [BugFix]: Fix DeepSeek V4 HiCache layer count logic (#25477) 2026-05-16 23:50:01 +08:00
Zhangheng 21b3ac52b4 [UnifiedTree]: Fix UnifiedRadixCache device match semantics with HiCache (#25277) 2026-05-16 00:40:55 +08:00
Zhangheng fe7a5dd3bf [UnifiedTree]: Add nightly hicache ci for dsa model (#25348) 2026-05-15 16:54:50 +08:00
Zhangheng c7e879e43f Add hicache feature in dsv4 cookbook (#25369) 2026-05-15 00:06:44 -07:00
Zhanghengispobock晟海
 		d9fa84b25b [UnifiedTree]: Support HiCache For DeepSeek_V4 (#24691) 
Co-authored-by: ispobock <ispobaoke@gmail.com>
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
 2026-05-14 20:20:52 -07:00
ZhanghengandJINZ 6140e45ef3 [UnifiedTree]: Fix the leaf determination logic in _cascade_evict. (#25068) 
Co-authored-by: JINZ <1023553676@qq.com>
 2026-05-13 10:57:32 +08:00
Zhangheng 91907b7b93 [UnifiedTree]: Fix Unified HiCache tombstone lock release replay (#24972) 2026-05-12 13:16:06 +08:00
Zhangheng e299ec1bff [UnifiedRadixTree]: Fix flaky ci (#24421) 2026-05-05 20:22:19 +08:00
Zhangheng 05aed5e1d5 [UnifiedRadixTree]: Add KL accuracy CI for UnifiedTree with HiCache (#24346) 2026-05-04 20:18:10 +08:00
ZhanghengandShangming Cai 9a5450ad73 [PD]: Support incremental transfer for mooncake transfer engine (#24257) 
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2026-05-04 00:57:59 +08:00
ZhanghengJINZdiemchai
 		c0f5950636 [UnifiedRadixTree]: Support HiCache Framework for UnifiedRadixTree (#23316) 
Co-authored-by: JINZ <1023553676@qq.com>
Co-authored-by: diemchai <diemchai@tencent.com>
 2026-05-03 22:13:22 +08:00
ZhanghengandShangming Cai 44ca2d01fc [pd]: (Bug Fix) Incorrect out_cache_loc slicing in prepare_for_prebuilt (#24230) 
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2026-05-03 18:35:16 +08:00
Zhangheng ab3ce02de9 [Hybrid-Cache]: Refactor hybrid_pool_assembler.py (#23243) 2026-04-21 10:45:23 +08:00
Zhangheng 7d47f40a96 [UnifiedRadixTree]: Add HiCache hook interface for TreeComponent (#22924) 2026-04-17 12:09:41 +08:00
Zhangheng 14bcdfca21 [HiSparse]: Adding e2e ut for hisparse (#22979) 2026-04-16 23:20:07 +08:00
Zhangheng 0a5c9728a1 [HiSparse][BugFix]: Fix the memory leak issue during health checks. (#22882) 2026-04-15 19:49:54 +08:00
Zhangheng 5549d910c6 [hisparse]: Adding ci for hisparse kvcache-swap-in jit-kernel (#22155) 2026-04-13 12:50:29 +08:00
Zhangheng 305b42935a [HiSparse]: Add benchmark for hisparse kernel (#22187) 2026-04-13 12:49:18 +08:00
Zhanghengispobockpansichengyizhang2077xiezhq-hermann
 		bc59cc0f96 [RaidxTree Refactor]: Support Unified HybridRadixTree V2 (#21206) 
Co-authored-by: ispobock <ispobaoke@gmail.com>
Co-authored-by: pansicheng <sicheng.pan.chn@gmail.com>
Co-authored-by: yizhang2077 <1109276519@qq.com>
Co-authored-by: xiezhq-hermann <xiezhq@stanford.edu>
 2026-04-13 10:28:22 +08:00
Zhangheng f2af00d05a [HiSparse-pd] Add device-buffer budget and fix logical pool admission in decode side (#22453) 2026-04-11 12:30:38 +08:00
ZhanghengZhiqiang Xie晟海
 		5ba7d4e523 [HiSparse]: Update HiSparse's user-guide (#22499) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
 2026-04-10 15:06:43 +08:00
Zhangheng晟海Zhiqiang Xie
 		19bbaeb3ee [HiSparse]: Add HiSpares-DSA Model's nightly CI (#22425) 
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2026-04-09 01:00:55 -07:00
Zhangheng 3d3a32c0b9 [HiSparse]: Add readme docs for HiSparse Feature (#22238) 2026-04-07 00:39:24 -07:00
Zhangheng d72f58d1c1 [Qwen3-Specv2]: Fix flaky ci (#22194) 2026-04-07 00:40:44 +08:00
Zhangheng f6c9072e42 [SpecV2]: Reopen kl accuracy test for qwen3 + SpecV2 (#22104) 2026-04-05 23:26:40 +08:00
Zhangheng晟海
 		51b276de74 [BugFix][RadixTree]: Fix backup invariant violation in Hi-MambaRadixTree (#22062) 
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
Co-authored-by: linjianyu77@foxmail.com
 2026-04-05 23:19:50 +08:00
Zhangheng ed3435e37f [HiSparse]: Optimize server args checking-HiSparse is temporarily only available for DSA models. (#22065) 2026-04-04 02:23:56 +08:00
ZhanghengTingwei HuangShangming CaiZhiqiang Xie
 		4d097047f2 [PD]: Add support for HiSparse to directly transfer the cache from Prefill to Decode DRAM. (#21591) 
Co-authored-by: Tingwei Huang <huangtingwei9988@gmail.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2026-04-02 23:06:12 -07:00
Zhangheng fae66b4050 Support PP key for file backend (#21901) 2026-04-02 12:23:58 +08:00
Zhangheng 91048b2a8e [HiMambaTree]: Optimize mamba host lock mechanism (#21750) 2026-03-31 21:52:24 +08:00
hzh0425pansicheng晟海ispobock
 		0986bed8e2 [HiCache][HybridModel]: Support mamba state offloading & HybridCacheController (#20457) 
Co-authored-by: pansicheng <sicheng.pan.chn@gmail.com>
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
Co-authored-by: ispobock <ispobaoke@gmail.com>
 2026-03-23 20:02:50 -07:00
hzh0425 c43d495dd5 [RadixTree][9/N Refactor]: Support unified init_load_back params (#20590) 2026-03-18 11:19:52 +08:00
hzh0425andispobock 7142a594f9 [CI]: Add CI For HiMambaRadixTree and qwen3.5 (#20540) 
Co-authored-by: ispobock <ispobaoke@gmail.com>
 2026-03-15 20:16:12 +08:00
hzh0425 197f807134 [RadixTree][7/N Refactor]: Refactor mamba radix tree, release dup kvcache in insert func (#19429) 2026-03-13 12:28:32 +08:00
hzh0425 0ac6c63ae4 [SpecV2-Mamba]: Refactor additional_ratio calculation when init mamba pool (#19660) 2026-03-09 00:39:26 +08:00
zhangheng a0d8a7ae6d [RadixTree][6/N Refactor]: Refactor SWARadixTree to simplify the computation and alignment of bigram keys. (#19427) 2026-02-28 20:01:39 +08:00
zhanghengYi Zhongyizhang2077Hanming Lu
 		e4b708d3e9 [Spec V2] Support specV2 for mamba hybrid attention (#18808) 
Co-authored-by: Yi Zhong <207368749+vincentzed@users.noreply.github.com>
Co-authored-by: yizhang2077 <1109276519@qq.com>
Co-authored-by: Hanming Lu <hanming@x.ai>
 2026-02-27 00:36:01 +08:00
zhangheng 079fc8f3c5 [piecewise graph]: support MiniMax-M2 (#18217) 2026-02-04 23:24:38 -08:00
zhangheng be557cbc5f [RadixTree][5/N Refactor]: Introduce pre and post-processing methods for key matching (#18147) 2026-02-04 17:10:46 +08:00
zhanghengand晟海 180594358b [HiCache]: Support DeepSeek v32 cpu offloading (#17415) 
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
 2026-02-02 18:07:37 -08:00
zhangheng f33022d039 [RadixTree][3/N Refactor]：Support unified insert/evict params (#17401) 2026-01-22 17:36:31 +08:00
zhanghengyizhang2077pansicheng
 		20b0523eca [RadixTree][1/N Refactor]: Support unified match_prefix params (#17142) 
Co-authored-by: yizhang2077 <1109276519@qq.com>
Co-authored-by: pansicheng <sicheng.pan.chn@gmail.com>
 2026-01-19 22:39:40 +08:00
zhangheng晟海Zhiqiang Xie
 		2d02c150dc [3/N][Sparse With Hicache]: Init sparse coordinator (#16086) 
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2026-01-05 17:51:04 -08:00
zhangheng晟海huangtingweiMagicYang1573
 		a89e85e739 [1/N][Sparse With Hicache]: Add Sparse Interface (#14741) 
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
Co-authored-by: huangtingwei <141888744+huangtingwei9988@users.noreply.github.com>
Co-authored-by: MagicYang1573 <1328657938@qq.com>
 2025-12-25 00:12:47 -08:00
zhanghengandZhiqiang Xie ea7c69ce28 [hotfix]: Add missing args for 3FS bench_client.py (#14791) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-12-14 11:13:19 -10:00
hzh0425 ed5e905c89 [HotFix]: Add missing SGLANG_EPLB_HEATMAP_COLLECTION_INTERVAL env var (#12776) 2025-11-06 18:58:44 -08:00
hzh0425 d9c812d881 [hotfix]: Resolve ModuleNotFoundError in PD deployment for is_in_ci() (#12772) 2025-11-06 18:48:03 -08:00
hzh0425 6b634493c3 [HICache / PD]: Support offloading incremental KV cache in decode side. (#11966) 2025-11-01 14:59:37 -07:00
hzh0425 318424e2c8 [HICache]: Support 3FS-Store with page_first_direct layout (#11460) 2025-10-13 15:47:22 +08:00
hzh0425andyizhang2077 f5b34a510c Bugfix: Fix Type consistency for KV indices in SWARadixCache (#11452) 
Co-authored-by: yizhang2077 <1109276519@qq.com>
 2025-10-12 23:19:44 +08:00
hzh0425pansichengZhiqiang Xie
 		ee3bd8a1c8 feat(hicache): Support passing prefix keys for l3 store. (#9045) 
Co-authored-by: pansicheng <sicheng.pan.chn@gmail.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-10-10 00:22:05 -07:00
hzh0425晟海huangtingweiZhiqiang Xie
 		df08bf9b9f [Doc]: Best Practice for HICache (#11001) 
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
Co-authored-by: huangtingwei <141888744+huangtingwei9988@users.noreply.github.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-10-08 00:59:21 +08:00
hzh0425pansichengZhiqiang Xie
 		c70e58e837 [HICache]: Refactor HiCache CI (#11011) 
Co-authored-by: pansicheng <sicheng.pan.chn@gmail.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-10-03 20:51:56 -04:00
hzh0425andTeng Ma c8a5d12abe [HiCache]: Support dynamic loading backends for hicache (#10551) 
Co-authored-by: Teng Ma <sima.mt@alibaba-inc.com>
 2025-09-26 18:34:11 -07:00
hzh0425andShangming Cai 7ec5b4e89c [PD-HiCache]: Support Async Offloading KVCache In Decode Side (#10192) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2025-09-25 23:20:49 -07:00
hzh0425 2a37b24d23 [HotFix]: Hot fix import path in 3fs_bench_client.py (#10463) 2025-09-14 23:45:46 -07:00
hzh0425 ec99668ab7 [Hicache]: Add E2E CI For 3FS-KVStore (#10131) 2025-09-08 01:54:50 -07:00
hzh0425 1a3d6f31da Modify ci workflow for auto-partitioning in 2-GPU backend tests (#10029) 2025-09-06 10:28:42 +08:00
hzh0425andZhiqiang Xie 106c2b31fb feat(hicache): Add generic hicache ci e2e test and benchmark test (#9846) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-09-04 20:43:46 +08:00
hzh0425 4d89389c4f Fix the key passing issue in page first layout. (#9929) 2025-09-02 11:30:11 -07:00
hzh0425 58d06fdc95 [HiCacheStorage]: Improve 3fs kvstore‘s performance and resolve mla issues (#9876) 2025-09-01 19:01:48 -07:00
hzh0425andZhiqiang Xie 8c2ffaaf0f fix(hicahce-long-bench): adjust context workload generator to use full query set (#9847) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-08-31 14:51:18 -07:00
hzh0425andyizhang2077 c2a26e725c feature(eplb): add min-rebalancing-utilization-threshold for eplb (#8345) 
Co-authored-by: yizhang2077 <1109276519@qq.com>
 2025-08-30 11:24:29 +08:00
hzh0425 161e9dc51e feat(hicache-3fs): 3FS-Store Backup Optimizations For MLA Model. (#9692) 2025-08-29 10:48:51 -07:00
hzh0425 38cd5fb1e0 bugfix(hicache): Move exists check before key suffixing (#9749) 2025-08-28 18:29:47 -07:00
hzh0425andTeng Ma c04c17edfa refactor(hicache): Introduce generic HiCacheStorageConfig for improved configuration management (#9555) 
Co-authored-by: Teng Ma <805522925@qq.com>
 2025-08-26 17:55:20 -07:00
hzh0425andZhiqiang Xie 79ce3688bb BugFix(hicache): Fix host indices out of bound error (#9637) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-08-26 10:42:23 -07:00
hzh0425 83871aa12d feat(hicache): Supports 3fs-hicache compatibility with dp-attention (#9372) 2025-08-23 02:08:32 -07:00
hzh0425 f7dd651dbd feat(hicache-3fs): 3FS-SGLang Hierarchical Cache Deployment Guide​ (#9213) 2025-08-14 23:54:31 -07:00
hzh0425andZhiqiang Xie 1b7afad0dd feature(hicache): Support hf3fs-hicache reusing kvcache across different instances (#8673) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-08-09 01:03:00 -07:00
hzh0425 d1c4d51c08 bugfix(hicache): Fix 'MooncakeStore' not defined error. (#8668) 2025-08-01 15:58:17 -07:00
hzh0425Shangming CaiZhiqiang Xie
 		2fbb754e1d feature(pd-hicache): Prefill instances support reusing the RemoteStorage Cache via HiCache. (#8516) 
Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-07-29 21:19:25 -07:00
hzh0425 a85ebf50b8 feat(hicache): support file backend reading directory config form env. (#8498) 2025-07-29 21:18:46 -07:00
hzh0425 5c08a36cbf [Fix] ensure DeepGEMM is only enabled for FP8_W8A8 models (#8110) 2025-07-16 21:33:29 -07:00