minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
17,853 Commits 2 Branches 0 Tags
39a80354aa1996d881704bcbdbbd9489956efe21
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Shangming CaiKangyan ZhouZhangheng
 		97ba99067d Publish per-scheduler load on a dedicated socket for load-aware routers (#34608) 
Co-authored-by: Kangyan Zhou <zky314343421@gmail.com>
Co-authored-by: Zhangheng <hzh0425@apache.org>
 2026-08-27 19:33:18 +08:00
Shangming Cai c8b56b1f44 chore: bump mooncake version to 0.3.13 (#36493) 2026-08-26 22:21:15 +08:00
Shangming Cai edff717ef0 fix(disagg): snapshot affected rooms before iterating outside the lock (#36351) 2026-08-26 03:31:10 +08:00
Shangming Cai f464e77d17 fix(mini-lb): forward the flush_cache timeout param to workers (#36150) 2026-08-24 15:32:57 +08:00
Shangming Cai 852b04b358 fix(xpu): read enable_deterministic_inference from the config bag (#36149) 2026-08-24 13:56:41 +08:00
Shangming Cai 8270ac4621 refactor(disagg): move _is_watermark_ready into StagingManagerMixin (#36030) 2026-08-23 14:18:41 +08:00
Shangming Cai 849ce71976 refactor(disagg): dedupe mooncake failure_exception into a mixin (#36031) 2026-08-23 14:17:44 +08:00
Shangming Cai bd72190728 refactor(disagg): register SGLANG_ENCODER_MM_LOAD_WORKERS in Envs (#36006) 2026-08-23 12:47:38 +08:00
Shangming Cai cce0a1244b refactor(disagg): hoist staging helper imports out of the bootstrap loops (#35980) 2026-08-23 01:24:58 +08:00
Shangming Cai 3c69a4c744 fix(test): unbreak test_kv_transfer_replica_metric after #35950 (#35974) 2026-08-22 19:23:46 +08:00
Shangming Cai 5c03069d4b refactor(disagg): drop dead placeholder overrides in Common KV sender/receiver (#35950) 2026-08-22 15:45:18 +08:00
Shangming Cai 15a4398320 refactor(disagg): hoist duplicated _handle_staging_req into a mixin (#35948) 2026-08-22 15:35:55 +08:00
Shangming Cai ac179eec11 fix(disagg): PD transfer-failure injection was silently inert (#35890) 2026-08-22 11:51:16 +08:00
Shangming Cai 729a050ea3 refactor(disagg): extract _all_reduce_polls helper (#35886) 2026-08-22 02:30:33 +08:00
Shangming Cai a41da991c8 refactor(disagg): collapse duplicated branches in get_kv_class (#35847) 2026-08-21 22:11:31 +08:00
Shangming Cai 4f343abc13 refactor(disagg): remove unreferenced dead code (#35838) 2026-08-21 22:09:25 +08:00
Shangming CaiandClaude Opus 5 dad6fd0f04 refactor(disagg): remove dead get_embedding_port (#35844) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-21 18:12:23 +08:00
Shangming Cai e7a37c8550 refactor(disagg): remove dead build_and_send_encode_request (#35843) 2026-08-21 17:58:35 +08:00
Shangming Cai 23cb04093c fix(multimodal): keep LLaVA image fetch off the CPU-preprocess timeout budget (flaky test_mixed_batch) (#35700) 2026-08-21 01:26:30 +08:00
Shangming Cai 628674a5c7 Remove unused MOONCAKE_COMPILE_ARG argument from Dockerfile (#35649) 2026-08-20 14:19:52 +08:00
Shangming Cai adca19c497 [PD] Deferred decode-side KV release for the NIXL backend (#35360) 2026-08-19 17:29:45 +08:00
Shangming CaiandClaude Opus 4.8 97dedd1ce9 [PD] Deferred decode-side KV release for aborts mid-transfer (#35049) 
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-08-18 23:56:33 +08:00
Shangming Cai 0772e79ee7 [CI][PD] Pin nccl rendezvous port per side to fix flaky disaggregation tests (#34755) 2026-08-14 01:44:20 +08:00
Shangming Cai f6ff5e8bb0 [PD] Handle abort requests in PP mode (#32797) 2026-07-30 14:39:44 +08:00
Shangming CaiClaude Fable 5Xinyuan Tong
 		1d350aaad3 fix(reasoning): let --enable-strict-thinking works for DeepSeek-V4 (#32400) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2026-07-27 18:15:47 +08:00
Shangming Cai bdd0698541 chore: bump mooncake version to 0.3.12.post1 (#32302) 2026-07-25 14:12:46 +08:00
Shangming Cai fa243fee40 [CI] Skip flaky test in CI for disaggregation group (#32324) 2026-07-24 21:59:20 +08:00
Shangming Cai 34d02bae47 Add CI permissions for Elastic EP contributor UNIDY2002 (#32306) 2026-07-24 17:20:15 +08:00
Shangming Cai 93926b9a95 Add Qwen3 MoE tests for PP compatibility with CP and DP (#29640) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-06-29 18:52:03 +08:00
Shangming Cai 0460f277b7 Fix manual chunked-prefill test to use req.fill_len after fill_ids refactor (#29032) 2026-06-23 17:52:26 +08:00
Shangming Cai a65c68c1fb Revert "[CI] Fix flaky optimistic test by adding contention handling" (#29023) 2026-06-23 16:53:32 +08:00
Shangming Cai 43b1fb95f6 [CI] Optimize nixl dependency installation script (#28950) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-06-23 01:58:22 +08:00
Shangming Cai ab21dc984a [CI] Fix flaky optimistic test by adding contention handling (#28947) 2026-06-23 00:35:13 +08:00
Shangming Cai 06e001347a [CI] Update nixl installation to include nixl-cu13 for h20 runner (#28930) 2026-06-22 20:35:14 +08:00
Shangming Cai b8e64fb56d [CI] Modify nixl installation to force reinstall (#28927) 2026-06-22 19:26:26 +08:00
Shangming Cai 378e66d248 [PD] Remove outdated backend whitelist for decode radix cache (#28238) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-06-15 22:36:05 +08:00
Shangming CaiandAbatom 6a012fbb2d [PD] Fix ZMQ stale socket reconnection in PD disaggregation (#27796) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Abatom <abzhonghua@gmail.com>
 2026-06-11 19:51:12 +08:00
Shangming Cai 6394a8b381 [EPD] Dynamic encoder registration cleanup (#27542) 2026-06-08 19:08:29 +08:00
Shangming Cai 8ff0c9fef9 [PD] Downgrade propagated rank failure logs from error to debug (#27534) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-06-08 16:56:46 +08:00
Shangming Cai 57909f731b [PD] Fix KV cache corruption on abort by notifying ongoing prefill (#27372) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-06-06 00:56:13 +08:00
Shangming Cai f3acb6d4de [CI] Fix multimodal-gen path filter for shared trace code (#27222) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-06-04 14:20:27 +08:00
Shangming Cai afd2d0b2f4 [PP][Bugfix] Handle input_ids assignment in prepare_for_extend (#26883) 2026-06-01 14:43:50 +08:00
Shangming Cai fba083c80f [CI] Split PP tests into base and extra suites (#26648) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-05-29 12:15:27 +08:00
Shangming Cai c8c1aed5e9 [PD] Fix cross-rank queue divergence by gating metadata readiness before all-reduce (#26394) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-05-26 21:59:42 +08:00
Shangming Cai 2aa6995308 [CI] Enable EPD CI for EPD architecture enhancements (#26281) 2026-05-25 23:52:58 +08:00
Shangming Cai bc8d64bf36 [CI] Align score threshold in dsv4 disaggregation test (#26268) 2026-05-25 17:48:45 +08:00
Shangming Cai 7f2829af39 chore: bump mooncake version to 0.3.11.post1 (#25989) 2026-05-25 16:51:03 +08:00
Shangming Cai a241659d18 [PD] Consolidate shared logic into common backend (#25979) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-05-23 10:41:20 +08:00
Shangming Cai fbebdd5105 [CI] Enable nixl disaggregation test for decode radix cache (#25990) 2026-05-21 19:29:24 +08:00
Shangming Cai 1fbee74fb6 [PD] Clean early abort logic in PD module (#25677) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-05-20 11:17:42 +08:00
Shangming Cai 86c6c77f2f [Bugfix] Fix missing group arg in get dp buffer (#25585) 2026-05-19 02:08:10 +08:00
Shangming Cai f04c522534 [PD] Add conclude_state to fake KV backend (#25599) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-05-18 19:56:42 +08:00
Shangming CaiandHanHan009527 4984552cc9 Fix tests for decode radix cache (#25145) 
Co-authored-by: HanHan009527 <hanhan7630@outlook.com>
 2026-05-13 20:31:44 +08:00
Shangming Cai 5227b07669 [CI] Add DSV4 Flash disaggregation test (#24973) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-05-13 18:42:42 +08:00
Shangming Cai d102b0cbc3 Add CODEOWNERS for EPD module (#24942) 2026-05-11 14:42:48 +08:00
Shangming Cai e264b5785d [PD] Centralize per-room cleanup in common backend (#24601) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-05-07 18:47:55 +08:00
Shangming Cai 32d9998b9d [PD] Prevent update_status to Failed from cleared entries (#24539) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-05-06 23:32:04 +08:00
Shangming Cai b8d883398d Revert "[Intel GPU] Enable pipeline parallelism on XPU" (#23641) 2026-04-24 17:36:35 +08:00
Shangming Cai 1c06a3d072 [CI] Move disaggregation basic CI back to 2-gpu suite (#23447) 2026-04-22 17:50:33 +08:00
Shangming Cai fa85bdf4ed chore: bump mooncake version to 0.3.10.post2 (#23439) 2026-04-22 15:01:47 +08:00
Shangming Cai a58c7f381e [PD] Fix clip logic when state indices lens are mismatch (#23323) 2026-04-21 13:22:20 +08:00
Shangming Cai 1c76f322df [HiCache] Add CP support for HiCache (#20977) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-04-10 17:52:51 +08:00
Shangming Cai 75ab75d027 Fix create_grammar_backend test calls with think_end_id (#22158) 2026-04-06 00:54:35 +08:00
Shangming Cai dccb11881f [PD] Fix staging warmup for GQA prefill decode different tp (#22153) 2026-04-05 23:13:06 +08:00
Shangming Cai d82097a0df [PD] Tiny register info field cleanup for mooncake backend (#22016) 2026-04-03 15:13:44 +08:00
Shangming Cai 7004df6094 chore: bump mooncake version to 0.3.10.post1 (#21844) 2026-04-02 10:54:22 +08:00
Shangming Cai ca2b2130ba [PD] Tiny cleanup after KVReceiver refactor (#21760) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-03-31 21:07:57 +08:00
Shangming Cai 166e9090ee [CI] Skip flaky elastic EP test (#21619) 2026-03-29 12:50:40 +08:00
Shangming Cai 1487f80158 chore: bump mooncake version to 0.3.10 (#20942) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-03-27 10:35:31 +08:00
Shangming Cai 6b0f1e3b43 Update skip condition for TestQwen35PPAccuracy (#21370) 2026-03-25 14:28:42 +08:00
Shangming Cai a094eb9c21 Temporarily disable flaky qwen3 cp test in CI (#21178) 2026-03-22 21:13:52 -07:00
Shangming Cai 4c52b7fcc6 [CI] Improve PP consistency check success rate (#20838) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-03-19 19:12:50 +08:00
Shangming Cai fd05532da1 Add logging for BootstrapServer for CI diagnosis (#20844) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-03-19 14:42:12 +08:00
Shangming Cai 8b46f1f4ec [PD] Add retry interval in ensure_prefill_info (#20832) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-03-18 16:02:20 +08:00
Shangming Caiandhnyls2002 2acb20f53b [Disagg] Non-blocking try_ensure_parallel_info in pending queue, consolidate rank mapping into PrefillServerInfo (#20785) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
 2026-03-17 17:26:18 -07:00
Shangming Cai 17c81a3e07 Revert "[PD] Make pending reqs resolving more robust" (#20779) 2026-03-17 20:31:12 +08:00
Shangming Cai 5d5c31c6e4 [PP] Add CP pyobj broadcasting when enable dynamic CPP (#20738) 2026-03-17 12:20:11 +08:00
Shangming Cai a4528a5737 [CI] Bring back CI test for Mamba PD Disaggregation (#20675) 2026-03-17 01:04:21 +08:00
Shangming Cai 738cbde902 [PD] Make pending reqs resolving more robust (#20505) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-03-16 14:12:13 +08:00
Shangming Cai 4f91b3069e Skip flaky test in CI for disaggregation decode offload (#20622) 2026-03-15 16:30:34 +08:00
Shangming Cai 99a3b25c9b [PP] Fix recv tensor dict potential race condition (#20341) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-03-14 13:35:01 +08:00
Shangming Cai af4c28904d [PD] Fix the infinite loop in deocde resolve_pending_reqs (#20371) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-03-11 14:11:19 -07:00
Shangming Cai ea6ac1d545 Add CODEOWNERS for mooncake transfer engine (#20226) 2026-03-09 20:22:06 -07:00
Shangming Cai c2b66d320d [HiCache] Add an env var to control transfer engine reuse (#19867) 2026-03-04 20:36:32 +08:00
Shangming Cai facde4c6d3 [PD] Enable all CP ranks for KVCache transfer (#19765) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-03-03 19:35:21 +08:00
Shangming Cai 3f36f27eae [Bugfix] Fix nixl and mori backend for missing decode tp size in PD module (#19690) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-03-02 19:55:19 +08:00
Shangming Cai 0a6678bf3a [PD] Remove unused server args for disaggregation (#19618) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-03-02 11:38:50 +08:00
Shangming Cai 366574b2b8 [PD] Cleanup BootstrapServer init and ready check (#19551) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-02-28 16:41:42 +08:00
Shangming CaiandVladislav Nosivskoy b01f3590be [PD] Support PD with context parallel after refactor (#19504) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Vladislav Nosivskoy <vladnosiv@gmail.com>
 2026-02-28 13:11:15 +08:00
Shangming Cai 288300aafd [PD] Tiny code cleanup for prefill info registering (#19414) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-02-27 01:35:26 +08:00
Shangming Cai e55e65535e [Bugfix] Add rids to the batch filtering for two batch overlap (#19418) 2026-02-26 06:57:25 -08:00
Shangming Cai 97f1fa5e6b [NPU] Fix disaggregation metadata buffer bootstrap_room_dtype for npu backend (#19423) 2026-02-26 21:10:50 +08:00
Shangming Cai 27fd014726 [PD] Add kv_cache_dtype consistency check for PD Disaggregation (#19407) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-02-26 17:15:58 +08:00
Shangming Cai 8d9ee6669e Fix comment for tp_rank calculation in dp_attention (#19306) 2026-02-25 15:19:10 +08:00
Shangming Cai 0fac2796b6 [PD-Disagg] Improve KVManager init across all backends (#19240) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-02-25 10:37:09 +08:00
Shangming Cai eccee4c48e [PD] Change bootstrap_room metadata dtype from int64 to uint64 (#19141) 2026-02-22 14:20:16 +08:00
Shangming Cai 1f39bf6523 [Bugfix] Add warnings when NSA indexer cache indice mismatch in PD module (#18727) 2026-02-13 15:20:05 +08:00
Shangming Cai 2a8a48c0ca Reuse initialized transfer engine in mooncake store (#18460) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-02-13 01:21:35 +08:00
Shangming Cai bffd765417 Refactoring Mooncake TE as a shared distributed component (#17810) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-02-09 10:53:11 +08:00
Shangming Cai 52401bec1d chore: bump mooncake version to 0.3.9 (#18316) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-02-07 17:30:01 +08:00