100 Commits
Author SHA1 Message Date
Shuwen Wang 77bc8cab18 [Fix] hicache/pd: Promise prefill only the L3 span the decode prefetch takes (#43548) 2026-10-23 16:32:41 +08:00
Shuwen Wang 8cd3c494a3 [Fix] mem_cache: Skip linker L3 lookup on PD decode (#45538) 2026-10-22 22:36:32 +08:00
Shuwen Wang 9d5386fd13 [Perf] dsv41: Fuse request-window layout construction (#45726) 2026-10-22 00:13:33 -07:00
Shuwen Wang 80263b5580 [Fix] sgl-router: Do not abort forwarded requests on reqwest's default 30 s TCP_USER_TIMEOUT (#45696) 2026-10-22 12:43:54 +08:00
Shuwen Wang 6929afb528 [Fix] hisparse: Keep PD destination host pages until the decode KV release (#45108) 2026-10-22 11:19:21 +08:00
eb91a41f38 [Feat] unified_cache: Support FULL + MAMBA MLA models (Kimi-K3) in the direct external linker (#44977)
Co-authored-by: Yutong Wu <yutong.wu@amd.com>
Co-authored-by: Zhangheng <hzh0425@apache.org>
2026-10-21 10:47:17 +08:00
Shuwen Wang 15e8e687db [Chore] environ: Remove retired unified radix tree environment variable (#45090) 2026-10-20 07:02:43 +00:00
Shuwen Wang c53b98d1fe [Fix] test-utils: Allow expected server exits in canary tests (#45172) 2026-10-19 21:07:03 -07:00
Shuwen Wang 8dee3f6bdb [Fix] [H200] ci: Split GLM-5.3-Flash serving recipes into separate tests (#44892) 2026-10-18 19:26:40 +08:00
Shuwen Wang f8e32706fc [CI] radix-cache: Re-enable Inkling unified cache linker KL test (#44938) 2026-10-18 19:26:22 +08:00
Shuwen Wang ba3fc9441d [Fix] mamba: Release consumed lazy Mamba prefix locks before donation retry (#43559) 2026-10-18 15:16:31 +08:00
Shuwen Wang a11fa06be7 [Fix] hicache: Key MLA Mamba state per TP rank in the file backend (#41907) 2026-10-18 15:10:50 +08:00
Shuwen Wang ffc4573c98 [Fix] hicache: Key MLA Mamba state per TP rank in the SeaweedFS backend (#43816) 2026-10-18 15:09:58 +08:00
Shuwen Wang e6c2cd47a9 [Fix] hicache: Keep unified memory off the direct page_first_direct copies (#43333) 2026-10-18 15:09:26 +08:00
05c1539098 [Feat] unified_cache: Support hybrid SWA + Mamba models in the Mooncake direct linker (#41343)
Co-authored-by: Zhangheng <hzh0425@apache.org>
Co-authored-by: Teng Ma <sima.mt@alibaba-inc.com>
2026-10-17 13:24:08 -07:00
Shuwen Wang 86e67f7d93 [Fix] unified_cache: Prevent misaligned Mamba checkpoints after speculative truncation (#43387) 2026-10-17 20:20:49 +08:00
Shuwen Wang ae28306a74 [Fix] cuda_graph: Wait for in-flight layer-wise cache loads before prefill graph replay (#44220) 2026-10-17 19:13:57 +08:00
Shuwen WangandZhangheng 5fcb07da34 [Fix] hicache: Key MLA Mamba state per TP rank in the HF3FS backend (#43817)
Co-authored-by: Zhangheng <hzh0425@apache.org>
2026-10-17 11:08:17 +00:00
Shuwen Wang 6a39ddf1f2 [Fix] unified_cache: Resize test allocator before allocating backed pages (#44843) 2026-10-16 20:21:49 -07:00
Shuwen Wang 33b5d26fa6 [CI] lora: Use the canonical Llama model ID in eviction tests (#44771) 2026-10-16 18:13:37 -07:00
Shuwen Wang f9df33494a [Fix] scheduler: Keep PD decode non-idle while deferred KV releases are held (#44386) 2026-10-15 19:17:51 -07:00
Shuwen Wang 3c28614f12 [Fix] hicache: Defer query beliefs until the rank-synchronized hit drain (#43108) 2026-10-14 17:25:08 -07:00
Shuwen Wang b67785153a [Fix] glm5.2: Align HiSparse warmup backends and move test to nightly (#44303) 2026-10-15 01:28:03 +08:00
Shuwen Wang 7b53a13401 [Fix] glm5.2: Align PD HiSparse prefill KV layout (#44256) 2026-10-14 20:40:28 +08:00
Shuwen Wang f92199b569 [Unified Cache] Abort requests whose external linker load fails instead of crashing the scheduler (#40896) 2026-10-13 00:03:21 +08:00
Shuwen Wang f5f5003323 [HiCache] Support Mamba in buffer-only host memory mode (#36345) 2026-10-12 00:35:16 -07:00
Shuwen Wang ef3e5c83c5 [HiCache] Keep storage workers alive on backend exceptions (#43550) 2026-10-11 22:26:56 +08:00
Shuwen Wang 0923bcd258 [Unified Radix] Invalidate lazy Mamba checkpoints before in-place tracking (#43567) 2026-10-11 21:12:51 +08:00
Shuwen Wang d26b6a5a9c fix: default Inkling to fa4 on Hopper (#35292) 2026-10-11 20:57:42 +08:00
Shuwen Wang 58cb1a2be6 [Scheduler] Keep waiting-prefix LRU refresh consistent across TP ranks (#43479) 2026-10-10 21:18:41 -07:00
Shuwen WangandKe Bao f9cee8d2b2 [HiCache] Keep protected cgroup page cache charged in host budgets (#42788)
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
2026-10-10 17:03:40 +00:00
Shuwen Wang 99d6f4e158 [MemCache] refactor: require an anchored lock receipt for non-root releases (#41452) 2026-10-10 22:43:49 +08:00
Shuwen Wang 44bc478863 [HiSparse] Avoid host synchronization in eager backup (#41446) 2026-10-10 21:49:55 +08:00
Shuwen Wang 416ebe6090 [CI] Fix EAGLE3 CUDA parity prefill chunk alignment (#43538) 2026-10-10 21:11:17 +08:00
Shuwen Wang 0955169ed2 [mem_cache] Bound FlexKV checkpoint stores and restore DSPARK cache checks (#35694) 2026-10-09 20:25:33 +08:00
Shuwen Wang f48ed2127e test: allow bounded checkpoint lag in Mamba HiCache tests (#42523) 2026-10-09 19:24:39 +08:00
Shuwen WangandClaude Opus 5 2889158792 [MemCache] test: require every ComponentData field to be classified by each split hook (#42518)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-10-07 14:53:34 +08:00
Shuwen Wang 137c084000 [HiCache] refactor: retire in-flight storage prefetches through one helper (#41453) 2026-10-05 00:13:07 +08:00
Shuwen Wang 35f3c96ff4 [CI] Fix bootstrap sender registration test on main (#42487) 2026-10-04 18:45:31 +08:00
Shuwen Wang 89f21671bb fix: preserve QSA indexer state through HiCache (#39893) 2026-10-02 23:49:38 +08:00
Shuwen Wang bb62f125aa [HiSparse] fix: return HiSparse slots on PD decode waiting abort and DeepSeek V4 free (#41346) 2026-10-01 10:24:51 +08:00
Shuwen Wang c9d6c966d0 [HiSparse] fix: reject speculative decoding and mixed chunk at startup (#41347) 2026-10-01 10:24:15 +08:00
Shuwen Wang b62b21e7f4 [HiCache][PD] fix: keep decode load-back polling out of rank-local collectives (#41393) 2026-10-01 10:22:30 +08:00
Shuwen WangandShangming Cai aec4b799e6 [PD][Mamba] fix: free the COW mamba slot of decode requests dropped before preallocation (#41451)
Co-authored-by: Shangming Cai <csmthu@gmail.com>
2026-09-30 17:50:44 +00:00
Shuwen Wang 51cae5f303 [HiCache][PD] fix: clamp the decode restore to the prefix promised to prefill (#41450) 2026-09-30 15:26:15 +08:00
Shuwen Wang 89e1316eae [mem cache] refactor: remove the index-K continuous getters orphaned by the CP v1 removal (#41469) 2026-09-28 16:41:07 -07:00
Shuwen Wang da3eb6db32 [Unified Memory] Fix Inkling conv-checkpoint track ids written to virtual slot numbers (#41144) 2026-09-28 05:19:19 -07:00
Shuwen Wang bddeb61d94 [Unified Memory] fix: preserve FP8 dtype in unified MHA pool (#38133) 2026-09-28 05:19:03 -07:00
Shuwen Wang 38d865489a [DSV4.1][HiCache] fix: wait for the layer transfer before reading low-ratio index-K (#41345) 2026-09-26 22:04:03 -07:00
Shuwen Wang 4273e7de77 [PD] fix: cache resumed decode-radix requests from root instead of an unlocked re-match (#41261) 2026-09-26 10:21:41 +08:00
Shuwen Wang 434c2e3adc [HiCache] fix: Drain pending backups before internal Mamba write-back (#41092) 2026-09-25 15:06:24 +08:00
Shuwen Wang cbe137727e [HiCache] Give trailing sidecar storage transfers a contiguous prefix_keys chain (#40456) 2026-09-25 15:06:05 +08:00
Shuwen Wangandhnyls2002 a2025b8c4d [DSV4] Account for FlashMLA physical KV page padding in memory budgets (#41091)
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-09-24 17:14:08 -07:00
Shuwen Wang 961404b010 [DSV4] Fix TRTLLM uniform FP8 KV memory budgeting (#41090) 2026-09-24 13:25:00 -07:00
Shuwen Wang 90663ccb41 [DSV4] fix: size the C4 state ring by the page it is addressed by (#40337) 2026-09-23 22:05:32 -07:00
Shuwen Wang 66ce8c55cc [HiCache] ci: add HiCache and unified radix rerun group (#40831) 2026-09-23 13:56:24 +08:00
Shuwen Wang ced0a2c98c [Hisparse] fix: account for MiniMax HiSparse full-pool memory (#40743) 2026-09-23 13:13:48 +08:00
Shuwen Wang 1d59ce7c90 fix: partition selective CI reruns into matrix jobs (#40760) 2026-09-22 20:37:02 -07:00
Shuwen Wang 224a247d5b [HiSparse] ci: add cross-directory rerun test group (#40749) 2026-09-23 11:07:03 +08:00
Shuwen Wang 2bc435d32e [HiCache] fix: bound the controller reset join so a stalled storage thread cannot hang the scheduler (#40312) 2026-09-23 10:52:03 +08:00
Shuwen Wang 80da4432d0 [Simulator] Fix meta host memory budgets on constrained runners (#40440) 2026-09-21 00:42:55 +08:00
Shuwen WangandSeokhoon Kang 9f3d275940 [HiCache] Fix sparse hybrid transfer layer IDs (#37870)
Co-authored-by: Seokhoon Kang <sh.kang@postech.ac.kr>
2026-09-20 16:24:24 +08:00
Shuwen Wang 191172fa74 [Unified Tree] fix: exempt host-locked aux nodes from the sanity_check host-LRU check (#39980) 2026-09-18 15:29:29 +00:00
Shuwen Wang 0be8a0af0e [DSV4] fix: keep the TileLang JIT cache under SGLANG_CACHE_DIR (#39364) 2026-09-18 00:06:35 -07:00
Shuwen Wang 6ca866ea29 [HiCache][Perf] fix: batch HiCache D2H submits per step for hybrid pools (#39050) 2026-09-17 17:53:58 +08:00
Shuwen Wang 33d46376a6 [HiCache] Yield idle scheduler so storage workers can drain (#38504) 2026-09-17 10:56:11 +08:00
Shuwen Wang 4c85172f3a [HiCache] Back up MXFP8 KV scales in the host pool (#39089) 2026-09-17 09:40:26 +08:00
Shuwen WangandZhangheng 6b33338242 [HiCache] Forward prefix metadata to v2 storage calls (#39567)
Co-authored-by: Zhangheng <hzh0425@apache.org>
2026-09-16 14:05:42 +08:00
Shuwen Wang 5298d85218 fix: stop shadowing the DSpark shared-experts fusion guard (#39366) 2026-09-14 22:24:49 -07:00
Shuwen Wang 0d95a9c1ff [HiCache] Keep the file backend temp file name within NAME_MAX (#38925) 2026-09-14 05:12:28 +00:00
Shuwen Wang 7b4972a9cb chore: add hzh0425 and xiezhq-hermann as Rust tree and HiSparse code owners (#39337) 2026-09-14 10:34:55 +08:00
Shuwen Wang ff228d11fe [HiCache] Release buffer prefetch anchor locks during storage cleanup (#38483) 2026-09-13 20:47:21 +00:00
Shuwen WangandClaude Opus 5 7f1f8c706a [mem_cache][10/N] refactor: drop the redundant _component suffix in unified_cache/components (#35644)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-13 21:38:20 +08:00
Shuwen Wang 14b647cf27 chore: add HiSparse coordinator and allocator code owners (#38682) 2026-09-13 18:42:33 +08:00
Shuwen Wang bd45cd50ca [Unified Tree] Port SWA Branching-Point Caching to the Rust TreeCore (#37584) 2026-09-12 18:37:57 +08:00
Shuwen Wang b9cb96496d [Unified Tree] Preserve aux LRU recency when splitting nodes (#38482) 2026-09-12 18:17:38 +08:00
Shuwen Wang a713349faf [HiCache] fix: preserve SWA host lock boundaries across splits (#38835) 2026-09-11 14:58:26 +08:00
Shuwen Wang 19b83b4b9d [Radix Cache] Fix PureSWA tail release without insertion (#38349) 2026-09-10 23:55:48 +08:00
Shuwen Wang cc7e43ad22 [HiCache] Account for newly pinned ancestors in load-back quota (#38481) 2026-09-10 23:54:08 +08:00
Shuwen Wang 058cca0192 [HiCache] Fix side pools to use resolved host allocator (#38350) 2026-09-10 14:28:20 +08:00
Shuwen Wang da1c017ae8 [CI] test: initialize disable_radix_cache in scheduler stub (#38418) 2026-09-08 13:15:47 +08:00
Shuwen Wang 61d501427d fix: make dfs weight ordering iterative (#38313) 2026-09-08 05:10:22 +00:00
Shuwen Wang 62bca081a3 [CI] Fix request receiver EP scale joiner test patch (#38342) 2026-09-07 23:52:32 +08:00
Shuwen Wangandhuangtingwei df623d3cbd fix: keep queued Mooncake linker loads after abort (#38195)
Co-authored-by: huangtingwei <141888744+huangtingwei9988@users.noreply.github.com>
2026-09-07 20:25:13 +08:00
Shuwen Wang 755f97c622 [CI] Fix the DSpark dp-tier unit test fixture after #34919 (#38314) 2026-09-07 03:16:00 -07:00
Shuwen Wang 6e312af8c2 fix: collect prefix hash values iteratively (#38204) 2026-09-07 11:10:32 +08:00
Shuwen Wang f25848913d fix: preserve SWA host lock on node split (#38138) 2026-09-07 11:04:22 +08:00
Shuwen WangandClaude Opus 5 4b44a1cde2 [Refactor] Let eviction policies take construction parameters (#37795)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-05 13:48:30 +08:00
09f542b23a [CI] Add /rerun-test --changed to rerun every test file a PR modifies (#37618)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Alison Shao <a.shao@wustl.edu>
2026-09-04 22:38:11 -07:00
Shuwen WangandClaude Fable 5.1 01e19ddf55 chore: add code owners for the Rust radix tree core (#38021)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-04 23:40:34 +08:00
Shuwen WangandClaude Fable 5.1 5cd2a7661d fix: reformat cosmos3_edge.py to satisfy ruff-format (#38023)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-04 23:39:36 +08:00
Shuwen Wang 19b46863f3 fix: align write-through pending across tree cores (#37278) 2026-09-04 22:08:25 +08:00
Shuwen WangandClaude Opus 5 e4adf63275 [Fix] Vacuous marker writes in the cache tests, and an undebited Mamba admission slot (#36415)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-04 19:43:17 +08:00
Shuwen WangandClaude Opus 5 fe3d4b9bbb fix: restore missing get_component_forced_attn_backend import in minimax_h3 (#37566)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-02 15:58:31 +08:00
Shuwen Wang a2b8681d1d [CI][MLX] Restore the mamba_branching_seqlen attribute the MLX runner reads off a request (#37453) 2026-09-01 11:16:45 -07:00
Shuwen Wang 9e9d26a4af Fix Mooncake serving benchmark trace rows (#37201) 2026-08-31 11:43:59 -07:00
Shuwen Wang 62f86ce470 [CI] Fix unreachable FakeReq field initialization (#37182) 2026-08-30 20:51:40 -07:00
Shuwen WangandClaude Opus 5 000c636342 docs: state that HiCache L2 is instance-private and only L3 is shared (#37050)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-29 23:47:50 +08:00
Shuwen Wang ade4f4ba8b fix: report the real backend for non-CUDA CI registrations in /rerun-test (#36778) 2026-08-28 13:45:59 -07:00
2380121e9b [HiCache] Stop populating host-pool mmaps twice (-13% allocation time) (#36705)
Co-authored-by: DellCurry <30748980+DellCurry@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-28 11:54:11 +08:00