 shuwenn
|
ff1fc1fbdf
|
[mem_cache][5/N] refactor: extract host KV cache base layer into pool_host package (#27273)
|
2026-06-20 20:44:08 +08:00 |
|
|
shuwenn
|
941a11ada3
|
[HiCache] refactor: remove unused transfer buffer (#26003)
|
2026-06-19 09:46:04 +00:00 |
|
|
shuwenn
|
b23477af44
|
bench: infer tokenizer from serving model info (#28195)
|
2026-06-15 20:03:12 -07:00 |
|
|
shuwenn
|
f4029d0fc0
|
[HiCache] fix: clear storage reset state (#26009)
|
2026-06-12 23:51:26 -07:00 |
|
|
shuwenn
|
9d37e710b7
|
[Bench] Add consistent p90/p95/p99 percentiles for all latency metrics (#27662)
|
2026-06-12 13:44:38 -07:00 |
|
|
shuwennandClaude Opus 4.8
|
66076f2409
|
[mem_cache][3/N] refactor: move HiSparse allocators to allocator/hisparse.py (#26678)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-06-11 19:50:24 +08:00 |
|
|
shuwennandQiaolin Yu
|
3600a9ac5f
|
[SPEC] feat: init adaptive spec params from config (#27493)
Co-authored-by: Qiaolin Yu <liin1211@outlook.com>
|
2026-06-10 18:55:25 +00:00 |
|
|
shuwenn
|
e57323cae9
|
[mem_cache][4/N] refactor: extract MambaTokenToKVPoolAllocator into allocator/ (#27256)
|
2026-06-07 10:46:29 +08:00 |
|
|
shuwenn
|
280280ace9
|
[SPEC] fix: import copy module for eagle sampling info clone (#27390)
|
2026-06-06 12:12:46 +00:00 |
|
|
shuwenn
|
efe24704b7
|
[HiCache] feat: truncate mamba prefetch length to available host KV size (#26945)
|
2026-06-05 03:01:37 +08:00 |
|
|
shuwenn
|
1af53f67b2
|
[mem_cache][2/N] refactor: move SWATokenToKVPoolAllocator to allocator/swa.py (#26676)
|
2026-06-04 15:10:37 +08:00 |
|
|
shuwenn
|
1a57145975
|
[codex] Fix adaptive metrics test flake (#27135)
|
2026-06-03 19:07:17 -07:00 |
|
|
shuwenn
|
f65aae8493
|
[HiCache] fix: truncate prefetch key on degraded allocation (#25991)
|
2026-06-03 22:06:18 +08:00 |
|
|
|
f4e7a98fe5
|
[HiCache] feat: support draft offload for mooncake (#24984)
Co-authored-by: huangtingwei9988 <141888744+huangtingwei9988@users.noreply.github.com>
Co-authored-by: stmatengss <11641725+stmatengss@users.noreply.github.com>
|
2026-06-02 21:42:04 -07:00 |
|
|
shuwenn
|
f40b2ca9d3
|
chore(CODEOWNERS): add allocator/ owners and @alphabetc1 to mem_cache (#27003)
|
2026-06-01 22:07:15 -07:00 |
|
|
|
a0670b5ba3
|
[SPEC] feat: add adaptive speculative decoding metrics (#25940)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Jarrod Barnes <jbarnes850@gmail.com>
|
2026-06-01 13:53:30 -07:00 |
|
|
shuwenn
|
c06220159b
|
[mem_cache][1/N] refactor: split allocator.py into allocator/ subpackage (#26675)
|
2026-05-31 20:31:29 +08:00 |
|
|
shuwennandmaodoudou168
|
68706e615a
|
[SPEC] fix: use effective max draft tokens for adaptive spec initiali… (#26354)
Co-authored-by: maodoudou168 <maodoudou168@users.noreply.github.com>
|
2026-05-28 13:33:11 -07:00 |
|
|
shuwenn
|
36eb72bf12
|
[UnifiedTree] fix: backup SWA-split parent before child under write-through (#25065)
|
2026-05-25 00:21:57 +08:00 |
|
|
shuwennandClaude Opus 4.7
|
66ef97c00f
|
[Lint] Fix Optional[X] = (None,) typo defaults in two dataclasses (#25252)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
2026-05-15 00:32:40 -07:00 |
|
|
shuwennandClaude Opus 4.7
|
d6b28b4a69
|
[Refactor] Remove dead key_convert_fn / convert_to_bigram_key (#25161)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
2026-05-13 13:54:36 -07:00 |
|
|
shuwenn
|
5ed9a494d0
|
[UnifiedTree] fix: allow partial match on evicted+backuped nodes (#24943)
|
2026-05-13 11:43:12 +08:00 |
|
|
shuwenn
|
5495026a3b
|
[HiCache] feat: default storage prefetch timeout (#23309)
|
2026-05-11 18:49:35 -07:00 |
|
|
shuwenn
|
b4d347e86e
|
[SPEC V2] fix: skip stale state updates in spec-v2 overlap (#23456)
|
2026-05-09 23:56:24 -07:00 |
|
|
shuwenn
|
d5564c2a96
|
fix(fa3): translate page table to SWA loc in EAGLE3 topk>1 spec metadata (#24617)
|
2026-05-09 18:22:45 +08:00 |
|
|
shuwennandClaude Opus 4.7
|
9d12f9e6fa
|
[HiCache] ci: lower est_time for test_hicache_spec_file_storage (#24713)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
2026-05-09 00:33:18 -07:00 |
|
|
shuwennandQiaolin-Yu
|
d9dddd4d7d
|
[SPEC V2][2/N] feat: adaptive spec support spec v2 (#23336)
Co-authored-by: Qiaolin-Yu <liin1211@outlook.com>
|
2026-05-07 18:33:47 -07:00 |
|
|
shuwennandClaude Opus 4.7
|
03147f66b8
|
ci: add /rerun-group to rerun all registered tests in a group (#24023)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
2026-04-29 10:24:16 -07:00 |
|
|
shuwenn
|
2c41ef4c93
|
[HiCache] feat: add draft KV cache backing for L2/L3 (#21125)
|
2026-04-28 23:47:31 -07:00 |
|
|
shuwenn
|
233048212a
|
[HiCache][SPEC] fix: normalize storage prefetch key (#23631)
|
2026-04-28 15:53:20 -07:00 |
|
|
shuwennandClaude Opus 4.6
|
3e1c5e1b74
|
[HiCache][SPEC] fix: empty key after page alignment in match_prefix (#23387)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
|
2026-04-28 14:06:55 -07:00 |
|
|
shuwenn
|
f30a6f4d7e
|
[DOC] Add DFLASH speculative decoding documentation (#23553)
|
2026-04-24 17:18:46 -07:00 |
|
|
shuwennandClaude Opus 4.6
|
4befc31408
|
fix: pass v_head_dim to MHA KV pools and validate MiMo HiCache geometry (#23173)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
|
2026-04-21 19:48:45 -07:00 |
|
|
|
e3782d04d2
|
fix: fallback to triton for attention-sink models (flashinfer unsupported) (#23139)
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
|
2026-04-21 13:48:50 -07:00 |
|
|
shuwennandQiaolin-Yu
|
b65799cf83
|
[SPEC][1/N] feat: add adaptive speculative_num_steps for EAGLE topk=1 (#21599)
Co-authored-by: Qiaolin-Yu <liin1211@outlook.com>
|
2026-04-20 14:25:04 -07:00 |
|
|
shuwenn
|
dbcf7459b5
|
fix: reset empty prefill batch fullness (#23138)
|
2026-04-20 14:14:00 -07:00 |
|
|
shuwennandZhangheng
|
90c76d665e
|
[HiCache] fix: HiCacheFile component key suffixing (#22891)
Co-authored-by: Zhangheng <hzh0425@apache.org>
|
2026-04-17 13:06:28 -07:00 |
|
|
shuwenn
|
ec5742f4ab
|
fix: Auto-correct page_size for Mamba no_buffer radix cache mode (#20538)
|
2026-04-08 00:19:31 +08:00 |
|
|
shuwenn
|
4f84ce5807
|
[CI] ci: add test_http_server_auth.py to CI (#21866)
|
2026-04-03 16:32:18 +08:00 |
|
|
shuwennandZhiqiang Xie
|
9cb362f70e
|
[HiCache] fix: Clone host indices to avoid memory leak (#21624)
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
|
2026-04-01 17:42:07 -07:00 |
|
|
shuwenn
|
a1c725bdc5
|
fix: pre-init tokenizer_manager to avoid AttributeError in shutdown (#21824)
|
2026-04-01 10:54:53 -07:00 |
|
|
shuwenn
|
6098c51bc2
|
fix(MiMo-V2-Flash): add mimo reasoning parser (#21414)
|
2026-04-02 00:47:27 +08:00 |
|
|
shuwenn
|
8e84f846cc
|
[Diffusion] Add --uvicorn-access-log-exclude-prefixes to suppress noisy access logs (#20379)
|
2026-04-01 09:31:44 +08:00 |
|
|
shuwenn
|
c34593f951
|
[HiCache] fix: graceful shutdown of pending async tasks in bench_mix.py (#20276)
|
2026-03-29 00:46:32 -07:00 |
|
|
shuwenn
|
18074e25dc
|
fix: scheduler launch hang when non-current rank dies (#20287)
|
2026-03-29 00:28:45 -07:00 |
|
|
shuwenn
|
646573e4e8
|
fix: use get_rope_config() to support models without rope_parameters (#21135)
|
2026-03-26 11:22:12 -07:00 |
|
|
shuwenn
|
2fba2bdad1
|
refactor: Remove dead code from utils/common.py (#20668)
|
2026-03-21 21:54:17 -07:00 |
|
|
shuwenn
|
6c91590e1b
|
[HiCache] refactor: hicache normalization flow and compatibility checks (#19669)
|
2026-03-20 18:38:44 -07:00 |
|
|
shuwenn
|
515b3a323d
|
feat: support human-readable suffixes (25.6k, 1M, 1Mi) for token CLI (#20577)
|
2026-03-16 20:05:33 -07:00 |
|
|
shuwenn
|
42f18fe560
|
[HiCache] fix: release write-through lock_ref during decode (#20049)
|
2026-03-16 14:49:31 +08:00 |
|
|
shuwenn
|
1ac6a26464
|
fix: Nemotron chunk size alias (#20458)
|
2026-03-14 23:23:39 -07:00 |
|
|
shuwenn
|
538acb4c46
|
fix: Add .text property to HttpResponse to prevent AttributeError (#20518)
|
2026-03-14 22:59:32 -07:00 |
|
|
shuwenn
|
acab24a76a
|
fix: gracefully abort last request in retract_decode on OOM (#19881)
|
2026-03-11 15:13:03 -07:00 |
|
|
shuwenn
|
5a11ae19c1
|
[CI] fix: notebook ci often OOM (#20199)
|
2026-03-09 22:32:41 -07:00 |
|
|
shuwenn
|
72f6dfcc31
|
fix: add ModelScope cache lookup and speculative path support (#20098)
|
2026-03-07 22:23:16 -08:00 |
|
|
shuwenn
|
7bd3dd9270
|
fix: image URL in notebook to use raw.githubusercontent.com (#20100)
|
2026-03-07 13:28:20 -08:00 |
|
|
shuwenn
|
bdffb027a8
|
[CI] fix: handle missing repo in lora notebook (#19700)
|
2026-03-02 10:27:32 -08:00 |
|
|
shuwenn
|
e3e71f275a
|
docs: refactor speculative decoding doc (#19186)
|
2026-03-01 22:03:20 -05:00 |
|
|
shuwenn
|
0c224c3c62
|
docs: Embed release lookup tool into Sphinx documentation site (#19264)
|
2026-02-24 11:11:27 -08:00 |
|
|
shuwenn
|
4cf4f0859f
|
[Doc] Convert the speculative decoding notebook to markdow (#18395)
|
2026-02-14 18:18:56 -08:00 |
|
|
shuwenn
|
bd39de7d5e
|
[Env] centralize hicache vars in environ.py (#17204)
|
2026-02-13 18:02:31 -08:00 |
|
|
shuwenn
|
3299c4f9c1
|
[CI] feat: add early exit to wait_for_server when process dies (#18602)
|
2026-02-13 16:46:09 -08:00 |
|
|
shuwenn
|
bc2405e6c1
|
feat: support release lookup (#18450)
|
2026-02-13 10:47:02 +08:00 |
|
|
shuwenn
|
8da14aea88
|
[HiCache] fix: StorageMetricsCollector was initialized twice (#18354)
|
2026-02-10 00:53:00 -08:00 |
|
|
shuwenn
|
f600965b0e
|
[CI] fix: notebook ci may not working (#18417)
|
2026-02-07 22:26:35 -08:00 |
|
|
shuwenn
|
ef1d0ea885
|
[Doc] add a summary section for spec decode document (#18323)
|
2026-02-05 16:34:31 -05:00 |
|
|
shuwenn
|
8b21dd4b77
|
[Doc] refine spec decode docs for SpecV2/STANDALONE/NGRAM (#18321)
|
2026-02-05 15:12:33 -05:00 |
|
|
shuwenn
|
a723d1c5ef
|
[model-gateway] ignore error for embeddings/classify in PD router (#15931)
|
2026-01-26 22:49:14 -05:00 |
|
|
shuwenn
|
57e432d951
|
fix: preserve disconnect events in api key middleware (#17253)
|
2026-01-26 22:48:24 -05:00 |
|
|
shuwenn
|
fd3b179ffd
|
[HiCache][HA 1/N] Support HiCache storage runtime attach/detach (#15892)
|
2026-01-26 19:33:19 -08:00 |
|
|
shuwenn
|
8fb45523f3
|
feat: support bitsandbytes quantization algorithm (#15325)
|
2026-01-19 18:36:56 -08:00 |
|
|
shuwenn
|
71279e31f7
|
[CI] fix test_vlm_models.py (#17049)
|
2026-01-19 10:00:00 -08:00 |
|
|
shuwenn
|
8ec160ed46
|
feature: support uvicorn access log filter(disable logging /metrics) (#15513)
|
2026-01-15 20:00:06 -08:00 |
|
|
shuwenn
|
9227d9f60c
|
[Docs] sort and update server_arguments.md (#17163)
|
2026-01-15 12:07:18 -05:00 |
|
|
shuwennandXinyuan Tong
|
de94d793ad
|
feat: support qwen3(-VL) rerank scoring&chat template (#16403)
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
|
2026-01-15 00:45:46 +08:00 |
|
|
shuwenn
|
48c2aca9ba
|
[Env] centralize pd vars in environ.py (#16264)
|
2026-01-14 23:06:01 +08:00 |
|
|
shuwennandSimo Lin
|
cd33694585
|
feat: add --admin-api-key for finer-grained endpoint auth (#15908)
Co-authored-by: Simo Lin <linsimo.mark@gmail.com>
|
2026-01-13 20:21:55 -08:00 |
|
|
shuwenn
|
ac11e6a7c5
|
Add alphabetc1 into CI_PERMISSION (#16198)
|
2025-12-31 11:31:47 +08:00 |
|
|
shuwenn
|
c0fc7a89e7
|
[sgl-kernel] fix: make sgl-kernel build respect MAX_JOBS (#15575)
|
2025-12-31 10:44:45 +08:00 |
|
|
shuwenn
|
7380ec9d55
|
[CI] fix test_mla_deepseek_v3.py (#16096)
|
2025-12-29 10:18:02 -08:00 |
|
|
shuwenn
|
fe3d47fc9d
|
fix: warn once per env var key (#15846)
|
2025-12-26 18:41:15 +08:00 |
|
|
shuwenn
|
53f974b973
|
fix: potential crash for missing stream attribute (#15644)
|
2025-12-23 23:24:31 +08:00 |
|
|
shuwenn
|
758b9067a0
|
[CI] fix UT assert error in test_tokenizer_manager.py (#15646)
|
2025-12-23 22:24:33 +08:00 |
|
|
shuwenn
|
1d9ba2ce4d
|
[model-gateway] Use UUIDs for router-managed worker resources (#15540)
|
2025-12-21 07:40:03 -10:00 |
|
|
shuwenn
|
1d90b194b2
|
[model-gateway] bugfix: backward compatibility for GET endpoints (#15413)
|
2025-12-20 10:37:53 -10:00 |
|
|
shuwenn
|
5a0ad7310e
|
fix: update model name after weights update (#15416)
|
2025-12-19 21:53:14 +08:00 |
|
|
shuwennandLiangsheng Yin
|
fb17845723
|
fix: unreachable error check in retraction (#15433)
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
|
2025-12-19 16:33:34 +08:00 |
|
|
shuwenn
|
621061f017
|
[Bugfix] input prompt was not logged (#13936)
|
2025-11-28 16:00:51 +08:00 |
|