Shangming Cai
|
1e894eb92d
|
[CI] disaggregation: Run the SWA decode radix cache test on Mooncake (#46126)
|
2026-10-24 01:41:39 +08:00 |
|
Shangming Cai
|
9569a36561
|
[Feat] scheduler: Gate --prefill-decode-interval on running batch size (#44001)
|
2026-10-22 14:30:12 +08:00 |
|
Shangming Cai
|
a6a6beffac
|
[Refactor] disaggregation: Share the flag all-reduce and host-destination predicate in the decode transfer queue (#45641)
|
2026-10-22 13:42:55 +08:00 |
|
Shangming Cai
|
a175e39b84
|
[CI] cuda: Rebuild workspace crates when the shared Cargo target dir came from another rust/ tree (#45557)
|
2026-10-21 19:32:22 +08:00 |
|
Shangming Cai
|
43a3f956cb
|
[Fix] sgl-router: Wait for the detached PD prefill in routing tests (#45502)
|
2026-10-21 19:04:26 +08:00 |
|
Shangming Cai
|
7554be5a69
|
[CI] codeowners: Add owners for the top-level sgl-router (#45564)
|
2026-10-21 17:28:21 +08:00 |
|
Shangming Cai
|
be65391abd
|
[Feat] mooncake: Enable the failed-session probe by default with backoff and idle pruning (#45338)
|
2026-10-21 17:16:19 +08:00 |
|
Shangming Cai
|
cd61db49c7
|
[Fix] sgl-router: Bump rustls-webpki and webpki-roots for emSign advisories (#45526)
|
2026-10-21 15:24:11 +08:00 |
|
Shangming Cai
|
574f9cdb2e
|
[CI] slash-command: Add /run-cpu-ci to run only the CPU unit suite (#45501)
|
2026-10-21 13:59:16 +08:00 |
|
Shangming Cai
|
952c63b1bb
|
[CI] sgl-router: Gate on live run-ci label so /tag-and-rerun-ci can start the tiers (#45383)
|
2026-10-20 21:33:42 +08:00 |
|
Shangming Cai
|
0025f93944
|
[Fix] sgl-router: Refuse prefill workers that cannot compute before bootstrap under --decode-dispatch-limit (#45120)
|
2026-10-19 18:24:17 +08:00 |
|
Shangming Cai
|
80914b7c9b
|
[Refactor] sgl-router: Rename --max-in-flight to --max-inflight-requests (#44950)
|
2026-10-18 15:33:58 +08:00 |
|
 Shangming CaiandShangming Cai
|
1ff2c1d71e
|
[Fix] sgl-router: Fill the media fields when converting TextGenerate to Generate (#44951)
Co-authored-by: Shangming Cai <shangming.cai@radixark.ai>
|
2026-10-18 13:05:45 +08:00 |
|
Shangming Cai
|
c7fb57d990
|
[Fix] sgl-router: Bump rustls to 0.23.46 for RUSTSEC-2026-0346..0349 (#44509)
|
2026-10-16 01:28:57 +08:00 |
|
 Shangming CaiandClaude Fable 5.1
|
4195980f5a
|
[PD] Poison the deferred abort ack when a Mooncake send fails (#43719)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-10-12 15:33:09 +08:00 |
|
 Shangming CaiandKevin Flansburg
|
e1f43c9a61
|
[PD][HiCache] Fold the decode restore outcome into the MIN-reduced KV poll (#43562)
Co-authored-by: Kevin Flansburg <kflansburg@cloudflare.com>
|
2026-10-11 10:12:34 +08:00 |
|
Shangming Cai
|
1814ece57f
|
[PP+PD] Harden PPConsensusStore: msgspec wire format, drop malformed messages (#43329)
|
2026-10-10 12:38:38 +08:00 |
|
Shangming Cai
|
9578cb1c63
|
[CI] Don't run the full GPU suite for renderer-only Cargo.lock changes (#43025)
|
2026-10-08 18:25:06 +08:00 |
|
Shangming Cai
|
1291f3175d
|
[rust-processor] Fix chat-template precedence wording in the processor README (#42756)
|
2026-10-06 19:10:26 +08:00 |
|
 Shangming CaiandClaude Fable 5.1
|
28525b43c4
|
[sgl-router] Document and reject a zero request timeout (#42751)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-10-06 19:07:37 +08:00 |
|
 Shangming CaiandClaude Opus 5.5
|
ab4f8a44f4
|
[sgl-router] Point the design doc at the shared engine ranking (#42572)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-05 18:16:42 +08:00 |
|
 Shangming CaiandClaude Opus 5
|
148da8e77d
|
[PD] Validate decode state layout once at registration (#42051)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-10-04 18:32:30 +08:00 |
|
 Shangming CaiandClaude Fable 5
|
a71c7d19e7
|
[Router] Document peer bootstrap: flags, RBAC, downward API, probe implications (#42049)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-10-02 02:20:36 +08:00 |
|
 Shangming CaiandClaude Fable 5
|
c78fa09290
|
[Router] Drop the snapshot client's redirect-following fallback (#42010)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-10-01 14:31:53 +08:00 |
|
 Shangming CaiandClaude Fable 5
|
047be72a8c
|
[Router] Harden peer-bootstrap edges: refuse redirects and metric docs (#41999)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-10-01 14:15:42 +08:00 |
|
Shangming Cai
|
8c75ad9e62
|
chore: bump mooncake version to 0.3.13.post1 (#41678)
|
2026-09-30 15:16:54 +08:00 |
|
 Shangming CaiandClaude Fable 5
|
2a1c477648
|
[PD] Defer decode KV release on every transfer failure, not only decode-initiated aborts (#41404)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-09-29 02:19:51 +08:00 |
|
 Shangming CaiandClaude Fable 5
|
14ab74ef95
|
[PD] Fan drain abort ACKs out to every decode peer of the room (#41402)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-09-28 05:16:23 +08:00 |
|
 Shangming CaiandClaude Fable 5
|
710a41ba57
|
[sgl-router] Book the input_ids forwarding outcome only for built bodies (#41342)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-09-26 19:39:01 +08:00 |
|
 Shangming CaiandClaude Fable 5
|
77173ffb08
|
[HiCache] Demote SWA KV to host on write_back eviction instead of dropping it (#40712)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-09-24 23:56:17 +08:00 |
|
 Shangming CaiandClaude Opus 5
|
cd11037009
|
[PD] Enable deferred decode-side KV release by default (#41023)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-24 22:35:11 +08:00 |
|
Shangming Cai
|
6f4c2b9b91
|
[HiCache] Demote internal-node mamba states on write_back eviction (#40680)
|
2026-09-23 02:18:58 +08:00 |
|
Shangming Cai
|
861b11f087
|
[PD] Simplify late-abort quiescent ack branch to else (#40711)
|
2026-09-22 19:08:47 +08:00 |
|
 
|
630b1ef322
|
[sgl-router] Fix reorg admission proxy test build after BucketResolver::new (#40537)
Co-authored-by: Kan Wu <wukanustc@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-21 16:42:29 +08:00 |
|
Shangming Cai
|
70b5b03e78
|
[NPU][CI] Fix paths-filter negation that makes every PR run the NPU tier (#40549)
|
2026-09-21 16:07:27 +08:00 |
|
Shangming Cai
|
745de73ba3
|
Add CODEOWNERS entry for sglang-renderer (#40483)
|
2026-09-21 01:37:50 +08:00 |
|
 Shangming CaiandClaude Opus 5
|
3f8eb35ead
|
[PD] Do not admit intake-rejected requests to a PD handoff (#38935)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 12:44:49 +08:00 |
|
 Shangming CaiandClaude Opus 5
|
47a157f257
|
[CI][Disaggregation] Fix EADDRINUSE flake in test_disaggregation_dwdp_gpt_oss (#39611)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-15 23:05:05 +08:00 |
|
Shangming Cai
|
4b186cfea5
|
[CI][PD] Skip the flaky decode HiCache file-backend disaggregation test (#39368)
|
2026-09-15 11:19:03 +08:00 |
|
 Shangming CaiandXinyuan Tong
|
14a131ad5b
|
[PD][OpenAI] Gate /v1/responses persistence behind --enable-response-store, default off (#39122)
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
|
2026-09-13 21:20:47 +08:00 |
|
 Shangming CaiandClaude Fable 5
|
981b947568
|
[Fix] Seed raw tokenizer_path for smg-grpc-servicer in gRPC mode (#39105)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-09-12 14:41:15 +08:00 |
|
Shangming Cai
|
2c10f87991
|
[PD] Read nixl TransferInfo.is_dummy as a field in unit tests (#39100)
|
2026-09-11 23:20:02 +08:00 |
|
Shangming Cai
|
a4ff5634b8
|
[CI] Add CI permissions for PP contributor stepinto (#39077)
|
2026-09-11 18:04:57 +08:00 |
|
 
|
97ba99067d
|
Publish per-scheduler load on a dedicated socket for load-aware routers (#34608)
Co-authored-by: Kangyan Zhou <zky314343421@gmail.com>
Co-authored-by: Zhangheng <hzh0425@apache.org>
|
2026-08-27 19:33:18 +08:00 |
|
Shangming Cai
|
c8b56b1f44
|
chore: bump mooncake version to 0.3.13 (#36493)
|
2026-08-26 22:21:15 +08:00 |
|
Shangming Cai
|
edff717ef0
|
fix(disagg): snapshot affected rooms before iterating outside the lock (#36351)
|
2026-08-26 03:31:10 +08:00 |
|
Shangming Cai
|
f464e77d17
|
fix(mini-lb): forward the flush_cache timeout param to workers (#36150)
|
2026-08-24 15:32:57 +08:00 |
|
Shangming Cai
|
852b04b358
|
fix(xpu): read enable_deterministic_inference from the config bag (#36149)
|
2026-08-24 13:56:41 +08:00 |
|
Shangming Cai
|
8270ac4621
|
refactor(disagg): move _is_watermark_ready into StagingManagerMixin (#36030)
|
2026-08-23 14:18:41 +08:00 |
|
Shangming Cai
|
849ce71976
|
refactor(disagg): dedupe mooncake failure_exception into a mixin (#36031)
|
2026-08-23 14:17:44 +08:00 |
|
Shangming Cai
|
bd72190728
|
refactor(disagg): register SGLANG_ENCODER_MM_LOAD_WORKERS in Envs (#36006)
|
2026-08-23 12:47:38 +08:00 |
|
Shangming Cai
|
cce0a1244b
|
refactor(disagg): hoist staging helper imports out of the bootstrap loops (#35980)
|
2026-08-23 01:24:58 +08:00 |
|
Shangming Cai
|
3c69a4c744
|
fix(test): unbreak test_kv_transfer_replica_metric after #35950 (#35974)
|
2026-08-22 19:23:46 +08:00 |
|
Shangming Cai
|
5c03069d4b
|
refactor(disagg): drop dead placeholder overrides in Common KV sender/receiver (#35950)
|
2026-08-22 15:45:18 +08:00 |
|
Shangming Cai
|
15a4398320
|
refactor(disagg): hoist duplicated _handle_staging_req into a mixin (#35948)
|
2026-08-22 15:35:55 +08:00 |
|
Shangming Cai
|
ac179eec11
|
fix(disagg): PD transfer-failure injection was silently inert (#35890)
|
2026-08-22 11:51:16 +08:00 |
|
Shangming Cai
|
729a050ea3
|
refactor(disagg): extract _all_reduce_polls helper (#35886)
|
2026-08-22 02:30:33 +08:00 |
|
Shangming Cai
|
a41da991c8
|
refactor(disagg): collapse duplicated branches in get_kv_class (#35847)
|
2026-08-21 22:11:31 +08:00 |
|
Shangming Cai
|
4f343abc13
|
refactor(disagg): remove unreferenced dead code (#35838)
|
2026-08-21 22:09:25 +08:00 |
|
 Shangming CaiandClaude Opus 5
|
dad6fd0f04
|
refactor(disagg): remove dead get_embedding_port (#35844)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-08-21 18:12:23 +08:00 |
|
Shangming Cai
|
e7a37c8550
|
refactor(disagg): remove dead build_and_send_encode_request (#35843)
|
2026-08-21 17:58:35 +08:00 |
|
Shangming Cai
|
23cb04093c
|
fix(multimodal): keep LLaVA image fetch off the CPU-preprocess timeout budget (flaky test_mixed_batch) (#35700)
|
2026-08-21 01:26:30 +08:00 |
|
Shangming Cai
|
628674a5c7
|
Remove unused MOONCAKE_COMPILE_ARG argument from Dockerfile (#35649)
|
2026-08-20 14:19:52 +08:00 |
|
Shangming Cai
|
adca19c497
|
[PD] Deferred decode-side KV release for the NIXL backend (#35360)
|
2026-08-19 17:29:45 +08:00 |
|
 Shangming CaiandClaude Opus 4.8
|
97dedd1ce9
|
[PD] Deferred decode-side KV release for aborts mid-transfer (#35049)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-08-18 23:56:33 +08:00 |
|
Shangming Cai
|
0772e79ee7
|
[CI][PD] Pin nccl rendezvous port per side to fix flaky disaggregation tests (#34755)
|
2026-08-14 01:44:20 +08:00 |
|
Shangming Cai
|
f6ff5e8bb0
|
[PD] Handle abort requests in PP mode (#32797)
|
2026-07-30 14:39:44 +08:00 |
|
 
|
1d350aaad3
|
fix(reasoning): let --enable-strict-thinking works for DeepSeek-V4 (#32400)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
|
2026-07-27 18:15:47 +08:00 |
|
Shangming Cai
|
bdd0698541
|
chore: bump mooncake version to 0.3.12.post1 (#32302)
|
2026-07-25 14:12:46 +08:00 |
|
Shangming Cai
|
fa243fee40
|
[CI] Skip flaky test in CI for disaggregation group (#32324)
|
2026-07-24 21:59:20 +08:00 |
|
Shangming Cai
|
34d02bae47
|
Add CI permissions for Elastic EP contributor UNIDY2002 (#32306)
|
2026-07-24 17:20:15 +08:00 |
|
Shangming Cai
|
93926b9a95
|
Add Qwen3 MoE tests for PP compatibility with CP and DP (#29640)
Signed-off-by: Shangming Cai <csmthu@gmail.com>
|
2026-06-29 18:52:03 +08:00 |
|
Shangming Cai
|
0460f277b7
|
Fix manual chunked-prefill test to use req.fill_len after fill_ids refactor (#29032)
|
2026-06-23 17:52:26 +08:00 |
|
Shangming Cai
|
a65c68c1fb
|
Revert "[CI] Fix flaky optimistic test by adding contention handling" (#29023)
|
2026-06-23 16:53:32 +08:00 |
|
Shangming Cai
|
43b1fb95f6
|
[CI] Optimize nixl dependency installation script (#28950)
Signed-off-by: Shangming Cai <csmthu@gmail.com>
|
2026-06-23 01:58:22 +08:00 |
|
Shangming Cai
|
ab21dc984a
|
[CI] Fix flaky optimistic test by adding contention handling (#28947)
|
2026-06-23 00:35:13 +08:00 |
|
Shangming Cai
|
06e001347a
|
[CI] Update nixl installation to include nixl-cu13 for h20 runner (#28930)
|
2026-06-22 20:35:14 +08:00 |
|
Shangming Cai
|
b8e64fb56d
|
[CI] Modify nixl installation to force reinstall (#28927)
|
2026-06-22 19:26:26 +08:00 |
|
Shangming Cai
|
378e66d248
|
[PD] Remove outdated backend whitelist for decode radix cache (#28238)
Signed-off-by: Shangming Cai <csmthu@gmail.com>
|
2026-06-15 22:36:05 +08:00 |
|
 Shangming CaiandAbatom
|
6a012fbb2d
|
[PD] Fix ZMQ stale socket reconnection in PD disaggregation (#27796)
Signed-off-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Abatom <abzhonghua@gmail.com>
|
2026-06-11 19:51:12 +08:00 |
|
Shangming Cai
|
6394a8b381
|
[EPD] Dynamic encoder registration cleanup (#27542)
|
2026-06-08 19:08:29 +08:00 |
|
Shangming Cai
|
8ff0c9fef9
|
[PD] Downgrade propagated rank failure logs from error to debug (#27534)
Signed-off-by: Shangming Cai <csmthu@gmail.com>
|
2026-06-08 16:56:46 +08:00 |
|
Shangming Cai
|
57909f731b
|
[PD] Fix KV cache corruption on abort by notifying ongoing prefill (#27372)
Signed-off-by: Shangming Cai <csmthu@gmail.com>
|
2026-06-06 00:56:13 +08:00 |
|
Shangming Cai
|
f3acb6d4de
|
[CI] Fix multimodal-gen path filter for shared trace code (#27222)
Signed-off-by: Shangming Cai <csmthu@gmail.com>
|
2026-06-04 14:20:27 +08:00 |
|
Shangming Cai
|
afd2d0b2f4
|
[PP][Bugfix] Handle input_ids assignment in prepare_for_extend (#26883)
|
2026-06-01 14:43:50 +08:00 |
|
Shangming Cai
|
fba083c80f
|
[CI] Split PP tests into base and extra suites (#26648)
Signed-off-by: Shangming Cai <csmthu@gmail.com>
|
2026-05-29 12:15:27 +08:00 |
|
Shangming Cai
|
c8c1aed5e9
|
[PD] Fix cross-rank queue divergence by gating metadata readiness before all-reduce (#26394)
Signed-off-by: Shangming Cai <csmthu@gmail.com>
|
2026-05-26 21:59:42 +08:00 |
|
Shangming Cai
|
2aa6995308
|
[CI] Enable EPD CI for EPD architecture enhancements (#26281)
|
2026-05-25 23:52:58 +08:00 |
|
Shangming Cai
|
bc8d64bf36
|
[CI] Align score threshold in dsv4 disaggregation test (#26268)
|
2026-05-25 17:48:45 +08:00 |
|
Shangming Cai
|
7f2829af39
|
chore: bump mooncake version to 0.3.11.post1 (#25989)
|
2026-05-25 16:51:03 +08:00 |
|
Shangming Cai
|
a241659d18
|
[PD] Consolidate shared logic into common backend (#25979)
Signed-off-by: Shangming Cai <csmthu@gmail.com>
|
2026-05-23 10:41:20 +08:00 |
|
Shangming Cai
|
fbebdd5105
|
[CI] Enable nixl disaggregation test for decode radix cache (#25990)
|
2026-05-21 19:29:24 +08:00 |
|
Shangming Cai
|
1fbee74fb6
|
[PD] Clean early abort logic in PD module (#25677)
Signed-off-by: Shangming Cai <csmthu@gmail.com>
|
2026-05-20 11:17:42 +08:00 |
|
Shangming Cai
|
86c6c77f2f
|
[Bugfix] Fix missing group arg in get dp buffer (#25585)
|
2026-05-19 02:08:10 +08:00 |
|
Shangming Cai
|
f04c522534
|
[PD] Add conclude_state to fake KV backend (#25599)
Signed-off-by: Shangming Cai <csmthu@gmail.com>
|
2026-05-18 19:56:42 +08:00 |
|
 Shangming CaiandHanHan009527
|
4984552cc9
|
Fix tests for decode radix cache (#25145)
Co-authored-by: HanHan009527 <hanhan7630@outlook.com>
|
2026-05-13 20:31:44 +08:00 |
|
Shangming Cai
|
5227b07669
|
[CI] Add DSV4 Flash disaggregation test (#24973)
Signed-off-by: Shangming Cai <csmthu@gmail.com>
|
2026-05-13 18:42:42 +08:00 |
|
Shangming Cai
|
d102b0cbc3
|
Add CODEOWNERS for EPD module (#24942)
|
2026-05-11 14:42:48 +08:00 |
|
Shangming Cai
|
e264b5785d
|
[PD] Centralize per-room cleanup in common backend (#24601)
Signed-off-by: Shangming Cai <csmthu@gmail.com>
|
2026-05-07 18:47:55 +08:00 |
|
Shangming Cai
|
32d9998b9d
|
[PD] Prevent update_status to Failed from cleared entries (#24539)
Signed-off-by: Shangming Cai <csmthu@gmail.com>
|
2026-05-06 23:32:04 +08:00 |
|