minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 10 Packages Projects Releases Wiki Activity
18,148 Commits 2 Branches 0 Tags
bd45cd50ca900dd821f829ca9adfbf9aa3336bda
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Shuwen Wang bd45cd50ca [Unified Tree] Port SWA Branching-Point Caching to the Rust TreeCore (#37584) 2026-09-12 18:37:57 +08:00
Shuwen Wang b9cb96496d [Unified Tree] Preserve aux LRU recency when splitting nodes (#38482) 2026-09-12 18:17:38 +08:00
Shuwen Wang a713349faf [HiCache] fix: preserve SWA host lock boundaries across splits (#38835) 2026-09-11 14:58:26 +08:00
Shuwen Wang 19b83b4b9d [Radix Cache] Fix PureSWA tail release without insertion (#38349) 2026-09-10 23:55:48 +08:00
Shuwen Wang cc7e43ad22 [HiCache] Account for newly pinned ancestors in load-back quota (#38481) 2026-09-10 23:54:08 +08:00
Shuwen Wang 058cca0192 [HiCache] Fix side pools to use resolved host allocator (#38350) 2026-09-10 14:28:20 +08:00
Shuwen Wang da1c017ae8 [CI] test: initialize disable_radix_cache in scheduler stub (#38418) 2026-09-08 13:15:47 +08:00
Shuwen Wang 61d501427d fix: make dfs weight ordering iterative (#38313) 2026-09-08 05:10:22 +00:00
Shuwen Wang 62bca081a3 [CI] Fix request receiver EP scale joiner test patch (#38342) 2026-09-07 23:52:32 +08:00
Shuwen Wangandhuangtingwei df623d3cbd fix: keep queued Mooncake linker loads after abort (#38195) 
Co-authored-by: huangtingwei <141888744+huangtingwei9988@users.noreply.github.com>
 2026-09-07 20:25:13 +08:00
Shuwen Wang 755f97c622 [CI] Fix the DSpark dp-tier unit test fixture after #34919 (#38314) 2026-09-07 03:16:00 -07:00
Shuwen Wang 6e312af8c2 fix: collect prefix hash values iteratively (#38204) 2026-09-07 11:10:32 +08:00
Shuwen Wang f25848913d fix: preserve SWA host lock on node split (#38138) 2026-09-07 11:04:22 +08:00
Shuwen WangandClaude Opus 5 4b44a1cde2 [Refactor] Let eviction policies take construction parameters (#37795) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-05 13:48:30 +08:00
Shuwen WangClaude Opus 5Alison Shao
 		09f542b23a [CI] Add /rerun-test --changed to rerun every test file a PR modifies (#37618) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Alison Shao <a.shao@wustl.edu>
 2026-09-04 22:38:11 -07:00
Shuwen WangandClaude Fable 5.1 01e19ddf55 chore: add code owners for the Rust radix tree core (#38021) 
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-04 23:40:34 +08:00
Shuwen WangandClaude Fable 5.1 5cd2a7661d fix: reformat cosmos3_edge.py to satisfy ruff-format (#38023) 
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-04 23:39:36 +08:00
Shuwen Wang 19b46863f3 fix: align write-through pending across tree cores (#37278) 2026-09-04 22:08:25 +08:00
Shuwen WangandClaude Opus 5 e4adf63275 [Fix] Vacuous marker writes in the cache tests, and an undebited Mamba admission slot (#36415) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-04 19:43:17 +08:00
Shuwen WangandClaude Opus 5 fe3d4b9bbb fix: restore missing get_component_forced_attn_backend import in minimax_h3 (#37566) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-02 15:58:31 +08:00
Shuwen Wang a2b8681d1d [CI][MLX] Restore the mamba_branching_seqlen attribute the MLX runner reads off a request (#37453) 2026-09-01 11:16:45 -07:00
Shuwen Wang 9e9d26a4af Fix Mooncake serving benchmark trace rows (#37201) 2026-08-31 11:43:59 -07:00
Shuwen Wang 62f86ce470 [CI] Fix unreachable FakeReq field initialization (#37182) 2026-08-30 20:51:40 -07:00
Shuwen WangandClaude Opus 5 000c636342 docs: state that HiCache L2 is instance-private and only L3 is shared (#37050) 
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-29 23:47:50 +08:00
Shuwen Wang ade4f4ba8b fix: report the real backend for non-CUDA CI registrations in /rerun-test (#36778) 2026-08-28 13:45:59 -07:00
Shuwen WangDellCurryClaude Opus 5
 		2380121e9b [HiCache] Stop populating host-pool mmaps twice (-13% allocation time) (#36705) 
Co-authored-by: DellCurry <30748980+DellCurry@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-28 11:54:11 +08:00
Shuwen WangandClaude Opus 5 303227951a fix: use a bf16-relative tolerance in the DSA indexer K kernel test (#35795) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-25 19:50:29 -07:00
Shuwen WangClaude Opus 5Alex Nails
 		effe0d14d2 [Fix] Keep the MiniCPM-SALA config reads visible to the resolution ratchets (#36178) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Alex Nails <alex.nails@radixark.ai>
 2026-08-24 16:05:09 -07:00
Shuwen Wangandispobock 0bdd28d487 [mem_cache] docs: add a layer map and placement rules (#35643) 
Co-authored-by: ispobock <ispobaoke@gmail.com>
 2026-08-22 00:35:07 +08:00
Shuwen Wangandispobock a7ec6b97f7 Restructure mem_cache auto-labels by layer (#25122) 
Co-authored-by: ispobock <ispobaoke@gmail.com>
 2026-08-21 23:29:59 +08:00
Shuwen Wang 8ff9c2b227 [mem_cache][9/N] refactor: move DSAIndexerPoolHost to pool_host.dsa (#35306) 2026-08-21 15:59:15 +08:00
Shuwen Wang 9b249a25a1 test: switch the Inkling-Small NVFP4 deterministic suite to DSPARK (#35293) 2026-08-20 21:54:52 +08:00
Shuwen Wangandhzh0425 41c018a9ec [UnifiedTree] feat: support runtime attach/detach (#35269) 
Co-authored-by: hzh0425 <hzh0425@apache.org>
 2026-08-19 22:49:08 +08:00
Shuwen Wang a72d8d29d3 Fix HiCache PP sync test fixture (#35446) 2026-08-19 12:25:17 +08:00
Shuwen Wang 0077f84d37 [mem_cache][8/N] refactor: move MambaPoolHost to pool_host.mamba (#31180) 2026-08-18 05:22:45 +00:00
shuwenn 9dd57ef8c4 [mem_cache][7/N] refactor: move MLATokenToKVPoolHost to pool_host.mla (#30616) 2026-07-13 14:23:38 +08:00
shuwenn 108a183f6b [mem_cache][6/N] refactor: move MHA host-pool into pool_host/mha.py (#30249) 2026-07-08 20:12:52 +08:00
shuwenn ff1fc1fbdf [mem_cache][5/N] refactor: extract host KV cache base layer into pool_host package (#27273) 2026-06-20 20:44:08 +08:00
shuwenn 941a11ada3 [HiCache] refactor: remove unused transfer buffer (#26003) 2026-06-19 09:46:04 +00:00
shuwenn b23477af44 bench: infer tokenizer from serving model info (#28195) 2026-06-15 20:03:12 -07:00
shuwenn f4029d0fc0 [HiCache] fix: clear storage reset state (#26009) 2026-06-12 23:51:26 -07:00
shuwenn 9d37e710b7 [Bench] Add consistent p90/p95/p99 percentiles for all latency metrics (#27662) 2026-06-12 13:44:38 -07:00
shuwennandClaude Opus 4.8 66076f2409 [mem_cache][3/N] refactor: move HiSparse allocators to allocator/hisparse.py (#26678) 
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-11 19:50:24 +08:00
shuwennandQiaolin Yu 3600a9ac5f [SPEC] feat: init adaptive spec params from config (#27493) 
Co-authored-by: Qiaolin Yu <liin1211@outlook.com>
 2026-06-10 18:55:25 +00:00
shuwenn e57323cae9 [mem_cache][4/N] refactor: extract MambaTokenToKVPoolAllocator into allocator/ (#27256) 2026-06-07 10:46:29 +08:00
shuwenn 280280ace9 [SPEC] fix: import copy module for eagle sampling info clone (#27390) 2026-06-06 12:12:46 +00:00
shuwenn efe24704b7 [HiCache] feat: truncate mamba prefetch length to available host KV size (#26945) 2026-06-05 03:01:37 +08:00
shuwenn 1af53f67b2 [mem_cache][2/N] refactor: move SWATokenToKVPoolAllocator to allocator/swa.py (#26676) 2026-06-04 15:10:37 +08:00
shuwenn 1a57145975 [codex] Fix adaptive metrics test flake (#27135) 2026-06-03 19:07:17 -07:00
shuwenn f65aae8493 [HiCache] fix: truncate prefetch key on degraded allocation (#25991) 2026-06-03 22:06:18 +08:00
shuwennhuangtingwei9988stmatengss
 		f4e7a98fe5 [HiCache] feat: support draft offload for mooncake (#24984) 
Co-authored-by: huangtingwei9988 <141888744+huangtingwei9988@users.noreply.github.com>
Co-authored-by: stmatengss <11641725+stmatengss@users.noreply.github.com>
 2026-06-02 21:42:04 -07:00
shuwenn f40b2ca9d3 chore(CODEOWNERS): add allocator/ owners and @alphabetc1 to mem_cache (#27003) 2026-06-01 22:07:15 -07:00
shuwennClaude Opus 4.7Jarrod Barnes
 		a0670b5ba3 [SPEC] feat: add adaptive speculative decoding metrics (#25940) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Jarrod Barnes <jbarnes850@gmail.com>
 2026-06-01 13:53:30 -07:00
shuwenn c06220159b [mem_cache][1/N] refactor: split allocator.py into allocator/ subpackage (#26675) 2026-05-31 20:31:29 +08:00
shuwennandmaodoudou168 68706e615a [SPEC] fix: use effective max draft tokens for adaptive spec initiali… (#26354) 
Co-authored-by: maodoudou168 <maodoudou168@users.noreply.github.com>
 2026-05-28 13:33:11 -07:00
shuwenn 36eb72bf12 [UnifiedTree] fix: backup SWA-split parent before child under write-through (#25065) 2026-05-25 00:21:57 +08:00
shuwennandClaude Opus 4.7 66ef97c00f [Lint] Fix Optional[X] = (None,) typo defaults in two dataclasses (#25252) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-15 00:32:40 -07:00
shuwennandClaude Opus 4.7 d6b28b4a69 [Refactor] Remove dead key_convert_fn / convert_to_bigram_key (#25161) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-13 13:54:36 -07:00
shuwenn 5ed9a494d0 [UnifiedTree] fix: allow partial match on evicted+backuped nodes (#24943) 2026-05-13 11:43:12 +08:00
shuwenn 5495026a3b [HiCache] feat: default storage prefetch timeout (#23309) 2026-05-11 18:49:35 -07:00
shuwenn b4d347e86e [SPEC V2] fix: skip stale state updates in spec-v2 overlap (#23456) 2026-05-09 23:56:24 -07:00
shuwenn d5564c2a96 fix(fa3): translate page table to SWA loc in EAGLE3 topk>1 spec metadata (#24617) 2026-05-09 18:22:45 +08:00
shuwennandClaude Opus 4.7 9d12f9e6fa [HiCache] ci: lower est_time for test_hicache_spec_file_storage (#24713) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-09 00:33:18 -07:00
shuwennandQiaolin-Yu d9dddd4d7d [SPEC V2][2/N] feat: adaptive spec support spec v2 (#23336) 
Co-authored-by: Qiaolin-Yu <liin1211@outlook.com>
 2026-05-07 18:33:47 -07:00
shuwennandClaude Opus 4.7 03147f66b8 ci: add /rerun-group to rerun all registered tests in a group (#24023) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-29 10:24:16 -07:00
shuwenn 2c41ef4c93 [HiCache] feat: add draft KV cache backing for L2/L3 (#21125) 2026-04-28 23:47:31 -07:00
shuwenn 233048212a [HiCache][SPEC] fix: normalize storage prefetch key (#23631) 2026-04-28 15:53:20 -07:00
shuwennandClaude Opus 4.6 3e1c5e1b74 [HiCache][SPEC] fix: empty key after page alignment in match_prefix (#23387) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-04-28 14:06:55 -07:00
shuwenn f30a6f4d7e [DOC] Add DFLASH speculative decoding documentation (#23553) 2026-04-24 17:18:46 -07:00
shuwennandClaude Opus 4.6 4befc31408 fix: pass v_head_dim to MHA KV pools and validate MiMo HiCache geometry (#23173) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-04-21 19:48:45 -07:00
shuwennLiangsheng Yinhnyls2002
 		e3782d04d2 fix: fallback to triton for attention-sink models (flashinfer unsupported) (#23139) 
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
 2026-04-21 13:48:50 -07:00
shuwennandQiaolin-Yu b65799cf83 [SPEC][1/N] feat: add adaptive speculative_num_steps for EAGLE topk=1 (#21599) 
Co-authored-by: Qiaolin-Yu <liin1211@outlook.com>
 2026-04-20 14:25:04 -07:00
shuwenn dbcf7459b5 fix: reset empty prefill batch fullness (#23138) 2026-04-20 14:14:00 -07:00
shuwennandZhangheng 90c76d665e [HiCache] fix: HiCacheFile component key suffixing (#22891) 
Co-authored-by: Zhangheng <hzh0425@apache.org>
 2026-04-17 13:06:28 -07:00
shuwenn ec5742f4ab fix: Auto-correct page_size for Mamba no_buffer radix cache mode (#20538) 2026-04-08 00:19:31 +08:00
shuwenn 4f84ce5807 [CI] ci: add test_http_server_auth.py to CI (#21866) 2026-04-03 16:32:18 +08:00
shuwennandZhiqiang Xie 9cb362f70e [HiCache] fix: Clone host indices to avoid memory leak (#21624) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2026-04-01 17:42:07 -07:00
shuwenn a1c725bdc5 fix: pre-init tokenizer_manager to avoid AttributeError in shutdown (#21824) 2026-04-01 10:54:53 -07:00
shuwenn 6098c51bc2 fix(MiMo-V2-Flash): add mimo reasoning parser (#21414) 2026-04-02 00:47:27 +08:00
shuwenn 8e84f846cc [Diffusion] Add --uvicorn-access-log-exclude-prefixes to suppress noisy access logs (#20379) 2026-04-01 09:31:44 +08:00
shuwenn c34593f951 [HiCache] fix: graceful shutdown of pending async tasks in bench_mix.py (#20276) 2026-03-29 00:46:32 -07:00
shuwenn 18074e25dc fix: scheduler launch hang when non-current rank dies (#20287) 2026-03-29 00:28:45 -07:00
shuwenn 646573e4e8 fix: use get_rope_config() to support models without rope_parameters (#21135) 2026-03-26 11:22:12 -07:00
shuwenn 2fba2bdad1 refactor: Remove dead code from utils/common.py (#20668) 2026-03-21 21:54:17 -07:00
shuwenn 6c91590e1b [HiCache] refactor: hicache normalization flow and compatibility checks (#19669) 2026-03-20 18:38:44 -07:00
shuwenn 515b3a323d feat: support human-readable suffixes (25.6k, 1M, 1Mi) for token CLI (#20577) 2026-03-16 20:05:33 -07:00
shuwenn 42f18fe560 [HiCache] fix: release write-through lock_ref during decode (#20049) 2026-03-16 14:49:31 +08:00
shuwenn 1ac6a26464 fix: Nemotron chunk size alias (#20458) 2026-03-14 23:23:39 -07:00
shuwenn 538acb4c46 fix: Add .text property to HttpResponse to prevent AttributeError (#20518) 2026-03-14 22:59:32 -07:00
shuwenn acab24a76a fix: gracefully abort last request in retract_decode on OOM (#19881) 2026-03-11 15:13:03 -07:00
shuwenn 5a11ae19c1 [CI] fix: notebook ci often OOM (#20199) 2026-03-09 22:32:41 -07:00
shuwenn 72f6dfcc31 fix: add ModelScope cache lookup and speculative path support (#20098) 2026-03-07 22:23:16 -08:00
shuwenn 7bd3dd9270 fix: image URL in notebook to use raw.githubusercontent.com (#20100) 2026-03-07 13:28:20 -08:00
shuwenn bdffb027a8 [CI] fix: handle missing repo in lora notebook (#19700) 2026-03-02 10:27:32 -08:00
shuwenn e3e71f275a docs: refactor speculative decoding doc (#19186) 2026-03-01 22:03:20 -05:00
shuwenn 0c224c3c62 docs: Embed release lookup tool into Sphinx documentation site (#19264) 2026-02-24 11:11:27 -08:00
shuwenn 4cf4f0859f [Doc] Convert the speculative decoding notebook to markdow (#18395) 2026-02-14 18:18:56 -08:00
shuwenn bd39de7d5e [Env] centralize hicache vars in environ.py (#17204) 2026-02-13 18:02:31 -08:00
shuwenn 3299c4f9c1 [CI] feat: add early exit to wait_for_server when process dies (#18602) 2026-02-13 16:46:09 -08:00
shuwenn bc2405e6c1 feat: support release lookup (#18450) 2026-02-13 10:47:02 +08:00