minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
14,605 Commits 2 Branches 0 Tags
bd3b252e0af0e7bfbcc1f329a672a7ee47a36721
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Shangming Cai 0460f277b7 Fix manual chunked-prefill test to use req.fill_len after fill_ids refactor (#29032) 2026-06-23 17:52:26 +08:00
Shangming Cai a65c68c1fb Revert "[CI] Fix flaky optimistic test by adding contention handling" (#29023) 2026-06-23 16:53:32 +08:00
Shangming Cai 43b1fb95f6 [CI] Optimize nixl dependency installation script (#28950) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-06-23 01:58:22 +08:00
Shangming Cai ab21dc984a [CI] Fix flaky optimistic test by adding contention handling (#28947) 2026-06-23 00:35:13 +08:00
Shangming Cai 06e001347a [CI] Update nixl installation to include nixl-cu13 for h20 runner (#28930) 2026-06-22 20:35:14 +08:00
Shangming Cai b8e64fb56d [CI] Modify nixl installation to force reinstall (#28927) 2026-06-22 19:26:26 +08:00
Shangming Cai 378e66d248 [PD] Remove outdated backend whitelist for decode radix cache (#28238) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-06-15 22:36:05 +08:00
Shangming CaiandAbatom 6a012fbb2d [PD] Fix ZMQ stale socket reconnection in PD disaggregation (#27796) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Abatom <abzhonghua@gmail.com>
 2026-06-11 19:51:12 +08:00
Shangming Cai 6394a8b381 [EPD] Dynamic encoder registration cleanup (#27542) 2026-06-08 19:08:29 +08:00
Shangming Cai 8ff0c9fef9 [PD] Downgrade propagated rank failure logs from error to debug (#27534) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-06-08 16:56:46 +08:00
Shangming Cai 57909f731b [PD] Fix KV cache corruption on abort by notifying ongoing prefill (#27372) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-06-06 00:56:13 +08:00
Shangming Cai f3acb6d4de [CI] Fix multimodal-gen path filter for shared trace code (#27222) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-06-04 14:20:27 +08:00
Shangming Cai afd2d0b2f4 [PP][Bugfix] Handle input_ids assignment in prepare_for_extend (#26883) 2026-06-01 14:43:50 +08:00
Shangming Cai fba083c80f [CI] Split PP tests into base and extra suites (#26648) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-05-29 12:15:27 +08:00
Shangming Cai c8c1aed5e9 [PD] Fix cross-rank queue divergence by gating metadata readiness before all-reduce (#26394) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-05-26 21:59:42 +08:00
Shangming Cai 2aa6995308 [CI] Enable EPD CI for EPD architecture enhancements (#26281) 2026-05-25 23:52:58 +08:00
Shangming Cai bc8d64bf36 [CI] Align score threshold in dsv4 disaggregation test (#26268) 2026-05-25 17:48:45 +08:00
Shangming Cai 7f2829af39 chore: bump mooncake version to 0.3.11.post1 (#25989) 2026-05-25 16:51:03 +08:00
Shangming Cai a241659d18 [PD] Consolidate shared logic into common backend (#25979) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-05-23 10:41:20 +08:00
Shangming Cai fbebdd5105 [CI] Enable nixl disaggregation test for decode radix cache (#25990) 2026-05-21 19:29:24 +08:00
Shangming Cai 1fbee74fb6 [PD] Clean early abort logic in PD module (#25677) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-05-20 11:17:42 +08:00
Shangming Cai 86c6c77f2f [Bugfix] Fix missing group arg in get dp buffer (#25585) 2026-05-19 02:08:10 +08:00
Shangming Cai f04c522534 [PD] Add conclude_state to fake KV backend (#25599) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-05-18 19:56:42 +08:00
Shangming CaiandHanHan009527 4984552cc9 Fix tests for decode radix cache (#25145) 
Co-authored-by: HanHan009527 <hanhan7630@outlook.com>
 2026-05-13 20:31:44 +08:00
Shangming Cai 5227b07669 [CI] Add DSV4 Flash disaggregation test (#24973) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-05-13 18:42:42 +08:00
Shangming Cai d102b0cbc3 Add CODEOWNERS for EPD module (#24942) 2026-05-11 14:42:48 +08:00
Shangming Cai e264b5785d [PD] Centralize per-room cleanup in common backend (#24601) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-05-07 18:47:55 +08:00
Shangming Cai 32d9998b9d [PD] Prevent update_status to Failed from cleared entries (#24539) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-05-06 23:32:04 +08:00
Shangming Cai b8d883398d Revert "[Intel GPU] Enable pipeline parallelism on XPU" (#23641) 2026-04-24 17:36:35 +08:00
Shangming Cai 1c06a3d072 [CI] Move disaggregation basic CI back to 2-gpu suite (#23447) 2026-04-22 17:50:33 +08:00
Shangming Cai fa85bdf4ed chore: bump mooncake version to 0.3.10.post2 (#23439) 2026-04-22 15:01:47 +08:00
Shangming Cai a58c7f381e [PD] Fix clip logic when state indices lens are mismatch (#23323) 2026-04-21 13:22:20 +08:00
Shangming Cai 1c76f322df [HiCache] Add CP support for HiCache (#20977) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-04-10 17:52:51 +08:00
Shangming Cai 75ab75d027 Fix create_grammar_backend test calls with think_end_id (#22158) 2026-04-06 00:54:35 +08:00
Shangming Cai dccb11881f [PD] Fix staging warmup for GQA prefill decode different tp (#22153) 2026-04-05 23:13:06 +08:00
Shangming Cai d82097a0df [PD] Tiny register info field cleanup for mooncake backend (#22016) 2026-04-03 15:13:44 +08:00
Shangming Cai 7004df6094 chore: bump mooncake version to 0.3.10.post1 (#21844) 2026-04-02 10:54:22 +08:00
Shangming Cai ca2b2130ba [PD] Tiny cleanup after KVReceiver refactor (#21760) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-03-31 21:07:57 +08:00
Shangming Cai 166e9090ee [CI] Skip flaky elastic EP test (#21619) 2026-03-29 12:50:40 +08:00
Shangming Cai 1487f80158 chore: bump mooncake version to 0.3.10 (#20942) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-03-27 10:35:31 +08:00
Shangming Cai 6b0f1e3b43 Update skip condition for TestQwen35PPAccuracy (#21370) 2026-03-25 14:28:42 +08:00
Shangming Cai a094eb9c21 Temporarily disable flaky qwen3 cp test in CI (#21178) 2026-03-22 21:13:52 -07:00
Shangming Cai 4c52b7fcc6 [CI] Improve PP consistency check success rate (#20838) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-03-19 19:12:50 +08:00
Shangming Cai fd05532da1 Add logging for BootstrapServer for CI diagnosis (#20844) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-03-19 14:42:12 +08:00
Shangming Cai 8b46f1f4ec [PD] Add retry interval in ensure_prefill_info (#20832) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-03-18 16:02:20 +08:00
Shangming Caiandhnyls2002 2acb20f53b [Disagg] Non-blocking try_ensure_parallel_info in pending queue, consolidate rank mapping into PrefillServerInfo (#20785) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
 2026-03-17 17:26:18 -07:00
Shangming Cai 17c81a3e07 Revert "[PD] Make pending reqs resolving more robust" (#20779) 2026-03-17 20:31:12 +08:00
Shangming Cai 5d5c31c6e4 [PP] Add CP pyobj broadcasting when enable dynamic CPP (#20738) 2026-03-17 12:20:11 +08:00
Shangming Cai a4528a5737 [CI] Bring back CI test for Mamba PD Disaggregation (#20675) 2026-03-17 01:04:21 +08:00
Shangming Cai 738cbde902 [PD] Make pending reqs resolving more robust (#20505) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-03-16 14:12:13 +08:00
Shangming Cai 4f91b3069e Skip flaky test in CI for disaggregation decode offload (#20622) 2026-03-15 16:30:34 +08:00
Shangming Cai 99a3b25c9b [PP] Fix recv tensor dict potential race condition (#20341) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-03-14 13:35:01 +08:00
Shangming Cai af4c28904d [PD] Fix the infinite loop in deocde resolve_pending_reqs (#20371) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-03-11 14:11:19 -07:00
Shangming Cai ea6ac1d545 Add CODEOWNERS for mooncake transfer engine (#20226) 2026-03-09 20:22:06 -07:00
Shangming Cai c2b66d320d [HiCache] Add an env var to control transfer engine reuse (#19867) 2026-03-04 20:36:32 +08:00
Shangming Cai facde4c6d3 [PD] Enable all CP ranks for KVCache transfer (#19765) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-03-03 19:35:21 +08:00
Shangming Cai 3f36f27eae [Bugfix] Fix nixl and mori backend for missing decode tp size in PD module (#19690) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-03-02 19:55:19 +08:00
Shangming Cai 0a6678bf3a [PD] Remove unused server args for disaggregation (#19618) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-03-02 11:38:50 +08:00
Shangming Cai 366574b2b8 [PD] Cleanup BootstrapServer init and ready check (#19551) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-02-28 16:41:42 +08:00
Shangming CaiandVladislav Nosivskoy b01f3590be [PD] Support PD with context parallel after refactor (#19504) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Vladislav Nosivskoy <vladnosiv@gmail.com>
 2026-02-28 13:11:15 +08:00
Shangming Cai 288300aafd [PD] Tiny code cleanup for prefill info registering (#19414) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-02-27 01:35:26 +08:00
Shangming Cai e55e65535e [Bugfix] Add rids to the batch filtering for two batch overlap (#19418) 2026-02-26 06:57:25 -08:00
Shangming Cai 97f1fa5e6b [NPU] Fix disaggregation metadata buffer bootstrap_room_dtype for npu backend (#19423) 2026-02-26 21:10:50 +08:00
Shangming Cai 27fd014726 [PD] Add kv_cache_dtype consistency check for PD Disaggregation (#19407) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-02-26 17:15:58 +08:00
Shangming Cai 8d9ee6669e Fix comment for tp_rank calculation in dp_attention (#19306) 2026-02-25 15:19:10 +08:00
Shangming Cai 0fac2796b6 [PD-Disagg] Improve KVManager init across all backends (#19240) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-02-25 10:37:09 +08:00
Shangming Cai eccee4c48e [PD] Change bootstrap_room metadata dtype from int64 to uint64 (#19141) 2026-02-22 14:20:16 +08:00
Shangming Cai 1f39bf6523 [Bugfix] Add warnings when NSA indexer cache indice mismatch in PD module (#18727) 2026-02-13 15:20:05 +08:00
Shangming Cai 2a8a48c0ca Reuse initialized transfer engine in mooncake store (#18460) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-02-13 01:21:35 +08:00
Shangming Cai bffd765417 Refactoring Mooncake TE as a shared distributed component (#17810) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-02-09 10:53:11 +08:00
Shangming Cai 52401bec1d chore: bump mooncake version to 0.3.9 (#18316) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-02-07 17:30:01 +08:00
Shangming Cai afae4c7178 [PD] Minor code cleanup for mooncake backend (#18279) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-02-05 17:38:09 +08:00
Shangming Cai 3ad3268e06 [CI] Skip PD hybrid attention test with different TP temporarily (#17791) 2026-01-27 10:59:33 +08:00
Shangming Cai 2262c5c9b5 Add ZhengWG to CI_Permission (#17572) 2026-01-22 17:24:06 +08:00
Shangming Cai 23d765d1f2 [Doc] Update pipeline parallelism documentation (#17414) 2026-01-20 20:10:56 +08:00
Shangming Cai 71cb9d0302 [PD] Fix PP dynamic chunking with DP attention (#17339) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-01-20 11:27:13 +08:00
Shangming Cai 0227db8926 [PD] Optimize MHA models pp util calculation logic (#17306) 2026-01-19 11:23:15 +08:00
Shangming Cai 4c59782e0f Fix hybrid attention PD Disaggregation test (#17099) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-01-15 23:38:58 +08:00
Shangming Cai 4b14f622e1 [CI] Add PD Disaggregation aarch64 test (#16572) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-01-10 14:44:54 +08:00
Shangming Cai 0c4e155a3c chore: bump mooncake version to 0.3.8.post1 (#16792) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-01-09 18:42:27 +08:00
Shangming Cai 973116e6bb [Doc] Optimize pipeline parallelism doc (#16630) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-01-07 14:52:42 +08:00
Shangming Cai 130f60ee78 Add CODEOWNERS for scheduler_pp_mixin.py (#16481) 2026-01-05 21:12:20 +08:00
Shangming Cai 9cac3c8628 Add liusy58 into CI_PERMISSION (#16373) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2026-01-03 21:36:35 -08:00
Shangming Cai db3821a9ef [PP] Add a minimum chunk value for PP dynamic chunking (#16140) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2025-12-30 16:35:39 +08:00
Shangming Cai 41addd2e08 chore: bump mooncake version to 0.3.8 (#15886) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2025-12-27 23:34:24 +08:00
Shangming Cai b3f83cc1c5 Fix pipeline parallelism doc typos (#15688) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2025-12-23 22:19:15 +08:00
Shangming Cai c15fa1c577 [2/N] Update doc of Pipeline Parallelism with case study (#15684) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2025-12-23 20:56:15 +08:00
Shangming Cai 17e81c750a [PP] Add dynamic chunking PP test (#15395) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2025-12-19 01:09:40 +08:00
Shangming Cai ee1ca51de8 [PP] Fix dynamic chunking strategy for PP (#15372) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2025-12-18 14:24:55 +08:00
Shangming Cai eeb2b9b259 [PP] Minor code cleanup for Pipeline Parallelism (#15329) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2025-12-17 23:31:25 +08:00
Shangming Cai 6682475124 [CI] Improve flaky 4 GPU test success rate (#15234) 2025-12-17 00:02:26 +08:00
Shangming Caiandybyang 36fcf71fff [Qwen3-next] Add PD disaggregation support for mamba with extra_buffer (#15180) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: ybyang <10629930+whybeyoung@users.noreply.github.com>
 2025-12-16 14:36:00 +08:00
Shangming Cai d277a86dea [CI] Add disaggregation decode PP test (#15114) 2025-12-14 23:54:25 +08:00
Shangming Caiandalpha-baby c8cf1cafdb [1/N] Update doc of Pipeline Parallelism (#14985) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: alpha-baby <fujianhao1997@qq.com>
 2025-12-12 19:32:52 +08:00
Shangming Cai a076d75e98 [Fix] Remove unused import from test_disaggregation_hicache.py (#14880) 2025-12-11 15:50:47 +08:00
Shangming Cai 93452a8252 [PD] Support decode pp for PD disaggregation (#14265) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2025-12-03 14:35:29 +08:00
Shangming Cai c6d34a0688 Move piecewise cuda graph test to manual dir to fix CI (#14121) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2025-11-29 00:52:01 -08:00
Shangming Cai b704b0a922 Optimize uneven PP layer distribution logic to improve PP performance (#13977) 2025-11-26 22:05:03 +08:00
Shangming Cai 4c3573e493 [PD] Clarify init method docstrings for kvsender and kvreceiver (#13476) 2025-11-18 12:20:19 +08:00
Shangming Cai c2d4716da0 chore: bump mooncake version to 0.3.7.post2 (#12599) 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
 2025-11-04 01:09:31 -08:00