minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 10 Packages Projects Releases Wiki Activity
18,691 Commits 2 Branches 0 Tags
dsv41-pd
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Zhangheng e3305b3b87 Add code owner for sglang-simulator (#37922) 2026-09-04 12:20:48 +08:00
Zhanghengand晟海 abed680320 [Unified Cache][5/N]: Integrate external linker mode end to end (#37381) 
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
 2026-09-04 02:02:58 +08:00
Zhanghengand晟海 5d92e60783 [Unified Cache Linker][3/N]: Add backend-independent linker core (#37151) 
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
 2026-08-31 14:07:39 +08:00
Zhanghengand晟海 6a9366f036 [Unified Cache Linker][2/N]: Add device pool assembly for external linkers (#37098) 
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
 2026-08-30 23:54:00 +08:00
Zhanghengand晟海 c9eb475a88 [Unified Cache][1/N]: Support cache contract for external linker (#37091) 
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
 2026-08-30 22:24:13 +08:00
Zhangheng bec6248272 [Unified Cache]: add glm5.2 per commit ci (#36281) 2026-08-26 10:04:19 +08:00
Zhangheng 44806dc507 Using unified radix tree by default for all case (#35081) 2026-08-21 10:45:46 +08:00
Zhangheng 1480687cff [CP]: Support CP V2 Strategy for dsv4 (#33532) 2026-08-07 14:03:27 -07:00
Zhangheng 12de7fb1f6 Remove the HiMambaRadixTree that is no longer in use (#33468) 2026-08-07 23:01:40 +08:00
Zhangheng f1b41a5b3d [CP]: FIx some issue for glm5.2 cp v2 (#33100) 2026-08-01 13:50:10 -07:00
ZhanghengXinyuan TongBaizhou Zhang
 		f46d5f25b4 [4/N][CP] Support interleave strategy for cp v2 (#30482) 
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
 2026-07-30 01:45:32 -07:00
Zhangheng 60d6914f17 [UnifiedTree]: move /mem_cache/unifed_cache_component dir to /mem_cache/unified_cache (#32484) 2026-07-28 16:28:21 +08:00
ZhanghengAlejandroParedesLTKevin FlansburgChao Shi
 		1a53f231d6 [HiCache]: Optimize hybrid/DSA L3 prefetch result sync and usable-prefix clamping (#31443) 
Co-authored-by: AlejandroParedesLT <alejandroparedeslatorre@gmail.com>
Co-authored-by: Kevin Flansburg <kevin.flansburg@gmail.com>
Co-authored-by: Chao Shi <chao.shi@alibaba-inc.com>
 2026-07-21 18:59:51 +08:00
Zhanghengandispobock afa3c06d1f Using UnifiedRadixTree by default for SWA, Mamba, and DSA models (#30468) 
Co-authored-by: ispobock <ispobaoke@gmail.com>
 2026-07-14 15:17:08 +08:00
Zhangheng 0299393758 [UnifiedTree]: Sync mamba int8 checkpoint (#30626) 2026-07-10 23:46:38 +08:00
Zhangheng 2286e25a21 [UnifiedTree]: Sync Replay SSM (#30636) 2026-07-10 21:54:59 +08:00
Zhangheng dad0120c57 [HiSparse]: Skip flaky hisparse-nixl ci (#29710) 2026-06-30 14:04:34 +08:00
Zhangheng 70883cb1b0 [UnifiedTree]: Rollback mamba hicache test to direct io backend (#28904) 2026-06-22 23:48:48 +08:00
Zhanghengandispobock 867707f1f2 [UnifiedTree]: move some kl test from base to extra stage (#28627) 
Co-authored-by: ispobock <ispobaoke@gmail.com>
 2026-06-18 01:39:46 -07:00
Zhangheng 78b6a4fabf [UnifiedTree]: Clean up some unused dead code. (#28389) 2026-06-16 21:53:35 +08:00
Zhangheng 6c908b3a3a [UnifiedTree]: Replace anonymous tuples with NamedTuples in UnifiedRadixCache (#28375) 2026-06-16 14:54:04 +08:00
Zhangheng 8ce05e8a20 [UnifiedTree]: Pin host buffers across async H→D in UnifiedRadixCache.load_back (#27444) 2026-06-13 15:16:56 +08:00
Zhangheng 5e0271536a [UnifiedTree]: HybridModel launches HiCache via UnifiedTree by default. (#27759) 2026-06-11 12:03:36 +08:00
Zhangheng 1ff7c627cd [UnifiedTree]: Support hicache metrics (#27554) 2026-06-08 20:18:02 +08:00
Zhangheng a39c428d3f [UnifiedTree][CI]: Reduce HiCache PP KL test concurrency to avoid decode OOM (#27483) 2026-06-07 17:12:42 +08:00
Zhangheng fe548f36b0 [UnifiedTree]: Fix SWA admission budget under-counts HiCache load-back consumption (#27391) 2026-06-07 10:47:08 +08:00
Zhangheng 86b9bf5812 [CI]: Fix CI Stage for 3fs backend test (#27388) 2026-06-06 00:38:03 +08:00
Zhangheng e4a7388187 feat(agentic router, 1/N): Add LoadBasedPolicy (#26480) 2026-06-06 00:13:27 +08:00
Zhangheng 6cfdc18585 HiCache: Fix Flaky CI For 3FS Backend (#27358) 2026-06-05 22:00:19 +08:00
Zhangheng faa6286946 [BugFix]: Fix HiMamba HiCache prefetch hang after L3 sidecar transfer (#27366) 2026-06-05 20:37:40 +08:00
Zhangheng 4df1ccdadc [UnifiedTree]: Fix CP Reduce For L3 HiCache (#27330) 2026-06-05 14:03:54 +08:00
Zhangheng 631db6c757 [UnifiedTree]: Sync sidecar component hits across TP ranks and make SWA prefetch all-or-nothing (#27264) 2026-06-05 09:23:54 +08:00
Zhanghengand晟海 736263f3dc [UnifiedTree]: Support l3 storage for swa and deepseek v4 (#26881) 
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
 2026-06-04 10:17:34 +08:00
Zhangheng ee4bf0a9d3 [UnifiedTree]: Add HiCache Nightly CI For GLM5 (#26927) 2026-06-02 19:02:50 +08:00
ZhanghenghuangtingweiShangming Cai晟海
 		3e993f6140 [PD]: Support HiCache prefetching and pd-incremental transfer on decode side (#26227) 
Co-authored-by: huangtingwei <141888744+huangtingwei9988@users.noreply.github.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
 2026-06-02 15:40:10 +08:00
Zhangheng 7662210406 [UnifiedTree]: Support eviction priority (#26549) 2026-05-30 15:19:43 +08:00
Zhangheng c048ebd10d [Hicache]: skip flaky test (#26764) 2026-05-30 15:16:12 +08:00
Zhangheng eb5d4827e8 [UnifiedTree]: Split unified tree kl ci into multiple files to reduce GPU usage. (#26666) 2026-05-29 02:46:34 -07:00
Zhangheng 5f8911183b [UnifiedTree]: Update Unified Radix Cache README (#26485) 2026-05-27 21:50:53 +08:00
Zhangheng 38f32c38ab [UnifiedRadixTree]: Support L3 HiStorage framework (#26062) 2026-05-26 22:38:00 +08:00
Zhanghengand晟海 a4db563c87 [hisparse]: update user guide (#26249) 
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
 2026-05-25 17:54:55 +08:00
Zhangheng 0be539024f [BugFix]: Fix DeepSeek V4 HiCache layer count logic (#25477) 2026-05-16 23:50:01 +08:00
Zhangheng 21b3ac52b4 [UnifiedTree]: Fix UnifiedRadixCache device match semantics with HiCache (#25277) 2026-05-16 00:40:55 +08:00
Zhangheng fe7a5dd3bf [UnifiedTree]: Add nightly hicache ci for dsa model (#25348) 2026-05-15 16:54:50 +08:00
Zhangheng c7e879e43f Add hicache feature in dsv4 cookbook (#25369) 2026-05-15 00:06:44 -07:00
Zhanghengispobock晟海
 		d9fa84b25b [UnifiedTree]: Support HiCache For DeepSeek_V4 (#24691) 
Co-authored-by: ispobock <ispobaoke@gmail.com>
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
 2026-05-14 20:20:52 -07:00
ZhanghengandJINZ 6140e45ef3 [UnifiedTree]: Fix the leaf determination logic in _cascade_evict. (#25068) 
Co-authored-by: JINZ <1023553676@qq.com>
 2026-05-13 10:57:32 +08:00
Zhangheng 91907b7b93 [UnifiedTree]: Fix Unified HiCache tombstone lock release replay (#24972) 2026-05-12 13:16:06 +08:00
Zhangheng e299ec1bff [UnifiedRadixTree]: Fix flaky ci (#24421) 2026-05-05 20:22:19 +08:00
Zhangheng 05aed5e1d5 [UnifiedRadixTree]: Add KL accuracy CI for UnifiedTree with HiCache (#24346) 2026-05-04 20:18:10 +08:00
ZhanghengandShangming Cai 9a5450ad73 [PD]: Support incremental transfer for mooncake transfer engine (#24257) 
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2026-05-04 00:57:59 +08:00
ZhanghengJINZdiemchai
 		c0f5950636 [UnifiedRadixTree]: Support HiCache Framework for UnifiedRadixTree (#23316) 
Co-authored-by: JINZ <1023553676@qq.com>
Co-authored-by: diemchai <diemchai@tencent.com>
 2026-05-03 22:13:22 +08:00
ZhanghengandShangming Cai 44ca2d01fc [pd]: (Bug Fix) Incorrect out_cache_loc slicing in prepare_for_prebuilt (#24230) 
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2026-05-03 18:35:16 +08:00
Zhangheng ab3ce02de9 [Hybrid-Cache]: Refactor hybrid_pool_assembler.py (#23243) 2026-04-21 10:45:23 +08:00
Zhangheng 7d47f40a96 [UnifiedRadixTree]: Add HiCache hook interface for TreeComponent (#22924) 2026-04-17 12:09:41 +08:00
Zhangheng 14bcdfca21 [HiSparse]: Adding e2e ut for hisparse (#22979) 2026-04-16 23:20:07 +08:00
Zhangheng 0a5c9728a1 [HiSparse][BugFix]: Fix the memory leak issue during health checks. (#22882) 2026-04-15 19:49:54 +08:00
Zhangheng 5549d910c6 [hisparse]: Adding ci for hisparse kvcache-swap-in jit-kernel (#22155) 2026-04-13 12:50:29 +08:00
Zhangheng 305b42935a [HiSparse]: Add benchmark for hisparse kernel (#22187) 2026-04-13 12:49:18 +08:00
Zhanghengispobockpansichengyizhang2077xiezhq-hermann
 		bc59cc0f96 [RaidxTree Refactor]: Support Unified HybridRadixTree V2 (#21206) 
Co-authored-by: ispobock <ispobaoke@gmail.com>
Co-authored-by: pansicheng <sicheng.pan.chn@gmail.com>
Co-authored-by: yizhang2077 <1109276519@qq.com>
Co-authored-by: xiezhq-hermann <xiezhq@stanford.edu>
 2026-04-13 10:28:22 +08:00
Zhangheng f2af00d05a [HiSparse-pd] Add device-buffer budget and fix logical pool admission in decode side (#22453) 2026-04-11 12:30:38 +08:00
ZhanghengZhiqiang Xie晟海
 		5ba7d4e523 [HiSparse]: Update HiSparse's user-guide (#22499) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
 2026-04-10 15:06:43 +08:00
Zhangheng晟海Zhiqiang Xie
 		19bbaeb3ee [HiSparse]: Add HiSpares-DSA Model's nightly CI (#22425) 
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2026-04-09 01:00:55 -07:00
Zhangheng 3d3a32c0b9 [HiSparse]: Add readme docs for HiSparse Feature (#22238) 2026-04-07 00:39:24 -07:00
Zhangheng d72f58d1c1 [Qwen3-Specv2]: Fix flaky ci (#22194) 2026-04-07 00:40:44 +08:00
Zhangheng f6c9072e42 [SpecV2]: Reopen kl accuracy test for qwen3 + SpecV2 (#22104) 2026-04-05 23:26:40 +08:00
Zhangheng晟海
 		51b276de74 [BugFix][RadixTree]: Fix backup invariant violation in Hi-MambaRadixTree (#22062) 
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
Co-authored-by: linjianyu77@foxmail.com
 2026-04-05 23:19:50 +08:00
Zhangheng ed3435e37f [HiSparse]: Optimize server args checking-HiSparse is temporarily only available for DSA models. (#22065) 2026-04-04 02:23:56 +08:00
ZhanghengTingwei HuangShangming CaiZhiqiang Xie
 		4d097047f2 [PD]: Add support for HiSparse to directly transfer the cache from Prefill to Decode DRAM. (#21591) 
Co-authored-by: Tingwei Huang <huangtingwei9988@gmail.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2026-04-02 23:06:12 -07:00
Zhangheng fae66b4050 Support PP key for file backend (#21901) 2026-04-02 12:23:58 +08:00
Zhangheng 91048b2a8e [HiMambaTree]: Optimize mamba host lock mechanism (#21750) 2026-03-31 21:52:24 +08:00
hzh0425pansicheng晟海ispobock
 		0986bed8e2 [HiCache][HybridModel]: Support mamba state offloading & HybridCacheController (#20457) 
Co-authored-by: pansicheng <sicheng.pan.chn@gmail.com>
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
Co-authored-by: ispobock <ispobaoke@gmail.com>
 2026-03-23 20:02:50 -07:00
hzh0425 c43d495dd5 [RadixTree][9/N Refactor]: Support unified init_load_back params (#20590) 2026-03-18 11:19:52 +08:00
hzh0425andispobock 7142a594f9 [CI]: Add CI For HiMambaRadixTree and qwen3.5 (#20540) 
Co-authored-by: ispobock <ispobaoke@gmail.com>
 2026-03-15 20:16:12 +08:00
hzh0425 197f807134 [RadixTree][7/N Refactor]: Refactor mamba radix tree, release dup kvcache in insert func (#19429) 2026-03-13 12:28:32 +08:00
hzh0425 0ac6c63ae4 [SpecV2-Mamba]: Refactor additional_ratio calculation when init mamba pool (#19660) 2026-03-09 00:39:26 +08:00
zhangheng a0d8a7ae6d [RadixTree][6/N Refactor]: Refactor SWARadixTree to simplify the computation and alignment of bigram keys. (#19427) 2026-02-28 20:01:39 +08:00
zhanghengYi Zhongyizhang2077Hanming Lu
 		e4b708d3e9 [Spec V2] Support specV2 for mamba hybrid attention (#18808) 
Co-authored-by: Yi Zhong <207368749+vincentzed@users.noreply.github.com>
Co-authored-by: yizhang2077 <1109276519@qq.com>
Co-authored-by: Hanming Lu <hanming@x.ai>
 2026-02-27 00:36:01 +08:00
zhangheng 079fc8f3c5 [piecewise graph]: support MiniMax-M2 (#18217) 2026-02-04 23:24:38 -08:00
zhangheng be557cbc5f [RadixTree][5/N Refactor]: Introduce pre and post-processing methods for key matching (#18147) 2026-02-04 17:10:46 +08:00
zhanghengand晟海 180594358b [HiCache]: Support DeepSeek v32 cpu offloading (#17415) 
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
 2026-02-02 18:07:37 -08:00
zhangheng f33022d039 [RadixTree][3/N Refactor]：Support unified insert/evict params (#17401) 2026-01-22 17:36:31 +08:00
zhanghengyizhang2077pansicheng
 		20b0523eca [RadixTree][1/N Refactor]: Support unified match_prefix params (#17142) 
Co-authored-by: yizhang2077 <1109276519@qq.com>
Co-authored-by: pansicheng <sicheng.pan.chn@gmail.com>
 2026-01-19 22:39:40 +08:00
zhangheng晟海Zhiqiang Xie
 		2d02c150dc [3/N][Sparse With Hicache]: Init sparse coordinator (#16086) 
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2026-01-05 17:51:04 -08:00
zhangheng晟海huangtingweiMagicYang1573
 		a89e85e739 [1/N][Sparse With Hicache]: Add Sparse Interface (#14741) 
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
Co-authored-by: huangtingwei <141888744+huangtingwei9988@users.noreply.github.com>
Co-authored-by: MagicYang1573 <1328657938@qq.com>
 2025-12-25 00:12:47 -08:00
zhanghengandZhiqiang Xie ea7c69ce28 [hotfix]: Add missing args for 3FS bench_client.py (#14791) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-12-14 11:13:19 -10:00
hzh0425 ed5e905c89 [HotFix]: Add missing SGLANG_EPLB_HEATMAP_COLLECTION_INTERVAL env var (#12776) 2025-11-06 18:58:44 -08:00
hzh0425 d9c812d881 [hotfix]: Resolve ModuleNotFoundError in PD deployment for is_in_ci() (#12772) 2025-11-06 18:48:03 -08:00
hzh0425 6b634493c3 [HICache / PD]: Support offloading incremental KV cache in decode side. (#11966) 2025-11-01 14:59:37 -07:00
hzh0425 318424e2c8 [HICache]: Support 3FS-Store with page_first_direct layout (#11460) 2025-10-13 15:47:22 +08:00
hzh0425andyizhang2077 f5b34a510c Bugfix: Fix Type consistency for KV indices in SWARadixCache (#11452) 
Co-authored-by: yizhang2077 <1109276519@qq.com>
 2025-10-12 23:19:44 +08:00
hzh0425pansichengZhiqiang Xie
 		ee3bd8a1c8 feat(hicache): Support passing prefix keys for l3 store. (#9045) 
Co-authored-by: pansicheng <sicheng.pan.chn@gmail.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-10-10 00:22:05 -07:00
hzh0425晟海huangtingweiZhiqiang Xie
 		df08bf9b9f [Doc]: Best Practice for HICache (#11001) 
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
Co-authored-by: huangtingwei <141888744+huangtingwei9988@users.noreply.github.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-10-08 00:59:21 +08:00
hzh0425pansichengZhiqiang Xie
 		c70e58e837 [HICache]: Refactor HiCache CI (#11011) 
Co-authored-by: pansicheng <sicheng.pan.chn@gmail.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-10-03 20:51:56 -04:00
hzh0425andTeng Ma c8a5d12abe [HiCache]: Support dynamic loading backends for hicache (#10551) 
Co-authored-by: Teng Ma <sima.mt@alibaba-inc.com>
 2025-09-26 18:34:11 -07:00
hzh0425andShangming Cai 7ec5b4e89c [PD-HiCache]: Support Async Offloading KVCache In Decode Side (#10192) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2025-09-25 23:20:49 -07:00
hzh0425 2a37b24d23 [HotFix]: Hot fix import path in 3fs_bench_client.py (#10463) 2025-09-14 23:45:46 -07:00
hzh0425 ec99668ab7 [Hicache]: Add E2E CI For 3FS-KVStore (#10131) 2025-09-08 01:54:50 -07:00
hzh0425 1a3d6f31da Modify ci workflow for auto-partitioning in 2-GPU backend tests (#10029) 2025-09-06 10:28:42 +08:00
hzh0425andZhiqiang Xie 106c2b31fb feat(hicache): Add generic hicache ci e2e test and benchmark test (#9846) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-09-04 20:43:46 +08:00