minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
18,691 Commits 2 Branches 0 Tags
dsv41-pd
Commit Graph
68 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
huangtingweiandChao Shi 020703923d [PP + HiCache] Add PP Prefetch Tickets for eager cross-stage storage prefetch (#36700) 
Co-authored-by: Chao Shi <stepinto@live.com>
 2026-09-20 11:19:31 +08:00
huangtingweiandZhangheng e9300f643e [Unified Cache][9/N] add opt-in MLA load deduplication for Mooncake Linker (#39565) 
Co-authored-by: Zhangheng <hzh0425@apache.org>
 2026-09-20 09:56:56 +08:00
huangtingweiandhzh0425 a7cf4a6fbc [Unified Cache][7/N] Support MTP, EAGLE, and DSpark draft KV caches in the external linker (#37914) 
Co-authored-by: hzh0425 <hzh0425@apache.org>
 2026-09-13 18:22:06 +08:00
huangtingwei 73c4cdb795 update codeowners (#38394) 2026-09-08 10:54:26 +08:00
huangtingweiandZhiqiang Xie 0d0e2f92be [HiCache] Buffer mode support sidecar pool (#37424) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2026-09-04 23:24:18 +08:00
huangtingweiandhzh0425 b21000aef1 [Unified Cache][4/N]: Add Mooncake backend for external linker (#37205) 
Co-authored-by: hzh0425 <hzh0425@apache.org>
 2026-09-01 11:40:06 +08:00
huangtingwei 0f7ba3d115 [HiSparse] Support hisparse multi-step swap io kernel (#32162) 2026-08-24 17:29:44 -07:00
huangtingwei cef8a32b9d update codeowner (#34866) 2026-08-16 00:37:18 +08:00
huangtingweiandHanming Lu 8f3d3a31f4 [HiCache] Fix Mamba track-boundary bookkeeping under overlap scheduling (#29792) 
Co-authored-by: Hanming Lu <hanminglu@meta.com>
 2026-08-12 00:36:48 +08:00
huangtingweiandjackyYang6 3953788596 [HiSparse]Fix DeepSeek V4 HiSparse PD Transfers with Separate Host and Device KV Indices (#31901) 
Co-authored-by: jackyYang6 <82102811+jackyYang6@users.noreply.github.com>
 2026-08-03 20:35:04 +08:00
huangtingweiandalphabetc1 50029f05a3 [HiCache] Merge HiCache event checks to reduce decode overhead (#30511) 
Co-authored-by: alphabetc1 <47200617+alphabetc1@users.noreply.github.com>
 2026-07-29 23:48:48 +08:00
huangtingwei 86e1bb584d [HiCache] Add model-aware key isolation to Mooncake Store (#31920) 2026-07-23 10:20:13 +08:00
huangtingweiandZhangheng 44e3dd2713 [HiCache] Optimize HiCache host pool free-list release (#30658) 
Co-authored-by: Zhangheng <hzh0425@apache.org>
 2026-07-17 14:52:08 +08:00
huangtingwei 1e80d938ff [HiCache]fix draft host pool allocator type (#29823) 2026-07-01 19:31:16 +08:00
huangtingwei 5e1ccd9320 [HiCache] Optimize HiCache hash generation with bulk token byte conversion (#28287) 2026-07-01 15:44:23 +08:00
huangtingweiandhzh0425 6b7ecca663 [HiCache]Support hybrid pool staged H2D kernel (#28434) 
Co-authored-by: hzh0425 <hzh0425@apache.org>
 2026-06-19 09:48:03 +08:00
huangtingwei 9b4432fe18 [HiCache]Asymmetric pool support direct backend (#28446) 2026-06-16 13:17:57 -07:00
huangtingwei b5bcd76a41 [HiCache & JIT Kernel] Refactoring HiCache Write-Back Kernel (#21631) 2026-06-15 19:44:27 -07:00
huangtingwei f101b287ef [Unified Tree]fix compatibility with eagle key and l3 hicache (#27655) 2026-06-10 10:54:45 +08:00
huangtingwei 857ecb2dbc [HiSparse & HiCache]Support mooncake store layer first layout (#27454) 2026-06-07 12:46:42 +08:00
huangtingwei 00fefef16b [PD & HiSparse] Add DeepSeek V4 support for HiSparse direct Prefill-to-Decode DRAM (#24880) 2026-06-05 15:39:48 +08:00
huangtingwei b97a3dbb46 [HiSparse PD & PP]Fix HiSparse compatibility with PP decode (#27258) 2026-06-04 21:37:42 +08:00
huangtingwei a5c7e9d236 [HiCache] fix PD L3 cache hit details from decode responses (#27046) 2026-06-04 18:01:21 +08:00
huangtingwei 373cadc92e [bugfix] mooncake store double-tag bug fix (#26569) 2026-05-31 15:15:38 -07:00
huangtingwei c93a559e5a Fix MoE LoRA wrapper exposing moe_runner_config (#26710) 2026-05-30 01:59:15 -07:00
huangtingweiandZhiqiang Xie f9f82d238c fix deepseek v4 hisparse (#25646) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2026-05-20 17:28:38 -07:00
huangtingweihzh0425Zhiqiang Xie
 		67fd005b97 [HiSparse & PD] Support hisparse memory pool host page > 1 (#23606) 
Co-authored-by: hzh0425 <hzh0425@apache.org>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2026-05-19 01:29:35 -07:00
huangtingweiandhzh0425 c2a212bfe2 [UnifiedTree] Support DeepSeek V4 host pool with multiple layouts. (#25282) 
Co-authored-by: hzh0425 <hzh0425@apache.org>
 2026-05-19 09:36:00 +08:00
huangtingwei 27445f9836 Add ChatCompletionRequest-style support to /v1/tokenize (#23981) 2026-05-06 18:35:20 -07:00
huangtingweihzh0425flyerming
 		efa71ce5ab [HiCache]Fix hybrid model move_indices (#22940) 
Co-authored-by: hzh0425 <hzh0425@apache.org>
Co-authored-by: flyerming <flyerming@163.com>
 2026-04-21 00:15:39 -07:00
huangtingweiandhzh0425 7d7fdc1309 [HiCache]Fix CP support for hybrid model (#22782) 
Co-authored-by: hzh0425 <hzh0425@apache.org>
 2026-04-15 23:50:29 +08:00
huangtingweiandhzh0425 3511c2deb4 [HiCache] Fix memory host free logic when share_indices_with_anchor enabled (#22767) 
Co-authored-by: hzh0425 <hzh0425@apache.org>
 2026-04-15 16:31:18 +08:00
huangtingweihzh0425pansichengispobockVladislav Nosivskoy
 		e9d6b9eb2d [HiCache & HybridModel] mooncake backend support DSA & mamba model (#21259) 
Signed-off-by: Vladislav Nosivskoy <vladnosiv@gmail.com>
Co-authored-by: hzh0425 <hzh0425@apache.org>
Co-authored-by: pansicheng <sicheng.pan.chn@gmail.com>
Co-authored-by: ispobock <ispobaoke@gmail.com>
Co-authored-by: Vladislav Nosivskoy <vladnosiv@gmail.com>
 2026-04-13 18:47:36 -07:00
huangtingwei 945d73824f [HiSparse] Clarify decode token usage logs (#22331) 2026-04-13 18:03:25 -07:00
huangtingweihzh0425Zhiqiang Xie
 		0c204fbd57 [HiSparse] Optimize the scheduling of decode backup. (#21932) 
Co-authored-by: hzh0425 <hzh0425@apache.org>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2026-04-07 10:34:58 -07:00
huangtingwei 6dd2f774de [HiCache & PD]Fixed detailed cache hit breakdown in PD scenarios. (#21764) 2026-04-01 17:44:55 -07:00
huangtingwei d864622a68 [Hicache & JIT_kernel] Support page first layout & mla jit kernel (#18311) 2026-03-27 08:54:36 -07:00
huangtingwei 8787cf4566 Fix the scope of io_backend in NSATokenToKVPoolHost (#20327) 2026-03-12 10:33:11 +08:00
huangtingwei 0a43229be1 Fix mooncake store write/read bandwidth logs (#18294) 2026-03-09 19:54:37 -07:00
huangtingweiandhzh0425 254d3cee0b [HiCache] Supports Indexer layout for NSATokenToKVPoolHost (#19912) 
Co-authored-by: hzh0425 <hzh0425@apache.org>
 2026-03-09 18:24:06 -07:00
huangtingweiandmaoyuhan 8dfb6e1684 [HiCache] fix compatibility bugs with eagle and HiCacheStorage (#19570) 
Co-authored-by: maoyuhan <susanmao1997@outlook.com>
 2026-03-02 18:29:51 -08:00
huangtingweiandhzh0425 36dc973cbf [HiCache] refactor page_first_direct io kernel (#18113) 
Co-authored-by: hzh0425 <hzh0425@apache.org>
 2026-02-27 11:43:14 -08:00
huangtingweiandhzh0425 d40cb2f725 [HiCache] Support heterogeneous tp for hicache storage (#18541) 
Co-authored-by: hzh0425 <hzh0425@apache.org>
 2026-02-25 00:13:57 -08:00
huangtingweiZhiqiang XieLei Huang
 		77fc4c4a53 Add mooncake store read/write bandwidth logs (#10598) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
Co-authored-by: Lei Huang <lei.huang@novita.ai>
 2026-01-15 12:15:41 -08:00
huangtingwei 1dcde53928 [HiCache] support memory_pool_host page head layout (#11644) 2025-11-17 13:45:17 +08:00
huangtingwei 4ac65e3c7d [PD / HiCache]fix deocde kvcache offload manager memory leak (#12774) 2025-11-14 03:31:53 +08:00
huangtingwei a5e5088dfb Fix errors of page head kernels in sgl-kernel for ROCm (#12604) 2025-11-10 15:15:50 +08:00
huangtingwei 5538e05cb1 fix default env var for mooncake store (#12429) 2025-11-01 01:25:33 +08:00
huangtingwei 3e6281d0aa [HiCache]Page head layout IO kernel (#11615) 2025-10-26 15:53:50 +08:00
huangtingwei cae3956585 check master server for mooncake store (#10510) 2025-10-20 09:37:09 +08:00
huangtingwei e05555fad8 [HiCacheStorage] mooncake store support page_first_direct layout (#10591) 2025-09-28 20:45:48 -07:00
huangtingwei 7f399e4bce [HiCacheStorage]support page_first_direct layout for generic set&get (#10522) 2025-09-19 05:47:16 -07:00
huangtingweiandZhiqiang Xie b4c2c421e9 support memory_pool_host page first direct layout (#10031) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-09-11 23:19:44 -07:00
huangtingweiandZhiqiang Xie 5be8c2f7f7 Page first direct IO kernel (#10060) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-09-10 13:35:34 +08:00
huangtingweiandZhiqiang Xie cb9e0e4180 [HiCacheStorage] fix abort request host memory leaks (#9874) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-09-01 18:59:29 -07:00
huangtingwei b361750a4a Mooncake store get zero copy meta optimization (#9857) 2025-09-01 03:27:56 -07:00
huangtingwei 55349e361d support mooncake store dp attention (#9684) 2025-08-28 12:31:31 +08:00
huangtingwei ae7428a8a7 fix mooncake store mla zero copy meta (#9678) 2025-08-27 15:43:16 +08:00
huangtingweiandZhiqiang Xie 6078d5fcc0 [HiCacheStorage] backup optimization for MLA model (#8865) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-08-22 18:03:51 +08:00
huangtingweiandZhiqiang Xie 0edda32001 Support page first layout zero copy for mooncake store (#8651) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-08-12 15:59:26 -07:00
huangtingweiandZhiqiang Xie 86497d99f2 fix page first per layer pf2lf kernel (#8915) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-08-09 17:16:11 -07:00
huangtingwei 76ba5bbe12 fix args typo in memory_pool_host (#8662) 2025-08-03 13:47:29 -07:00
huangtingweiZhiqiang XieAniZpZzuoyuan@wangyueneng.wynJinYan Su
 		d904959233 Support l3 cache (mooncake store) for hiradix cache (#7211) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
Co-authored-by: AniZpZ <zhuangsen.zp@antgroup.com>
Co-authored-by: zuoyuan <zhangzuo21@mails.tsinghua.edu.cn>
Co-authored-by: @wangyueneng.wyn <wangyueneng.wyn@antgroup.com>
Co-authored-by: JinYan Su <jinyansu792@gmail.com>
 2025-07-30 23:15:51 -07:00
huangtingweiandZhiqiang Xie 26c8a310bd fix incorrect increase of hit count (#8533) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-07-31 06:02:42 +00:00
huangtingweiandChang Su 7732bbe458 bugfix: Prevent global mutation of conv.stop_str across requests (#7347) 
Co-authored-by: Chang Su <chang.s.su@oracle.com>
 2025-06-23 19:36:23 -07:00
huangtingweiandZhiqiang Xie d2cb3024f2 fix bug that gpu0 occupies more memory when hicache is turned on (#5778) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-05-09 15:36:08 -07:00
huangtingweiandAniZpZ 5fbafbb8f8 fix MLATokenToKVPoolHost get_size_per_token bug (#5161) 
Co-authored-by: AniZpZ <zhuangsen.zp@antgroup.com>
 2025-04-13 12:37:26 -07:00
huangtingwei 27f8e6b9c1 fix multimodal hash feature (#5083) 2025-04-07 22:43:23 -07:00