 Shangming Cai
|
187b85b7f3
|
[PD] Optimize custom mem pool usage and bump mooncake version (#7393)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-06-20 09:50:39 -07:00 |
|
|
Shangming Cai
|
f88e70853e
|
[Bugfix][PD] Set conclude state before clear when failure happens (#7362)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-06-19 11:26:53 -07:00 |
|
|
shangmingc
|
c26d7349d3
|
[PD] Add custom memory pool option to support Mooncake PD with NVLink (#7264)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-06-17 17:21:37 -07:00 |
|
|
shangmingc
|
ceaa85c9e6
|
[PD] Support get local ip from NIC for PD disaggregation (#7237)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-06-17 17:19:26 -07:00 |
|
|
shangmingc
|
132dad874d
|
[PD] Optimize transfer queue forward logic for dummy rank (#6922)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-06-06 18:26:14 -07:00 |
|
|
shangmingc
|
dd1012fcbe
|
[PD] Fix potential perf spike caused by tracker gc and optimize doc (#6764)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-06-05 10:56:02 -07:00 |
|
|
shangmingc
|
6cb00c6398
|
[PD] Optimize time out logic and add env var doc for mooncake (#6761)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-05-30 00:45:02 -07:00 |
|
|
shangmingc
|
1dc6864f17
|
[PD] Support completion endpoint (#6729)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-05-29 16:26:18 +08:00 |
|
|
shangmingc
|
d63e76f735
|
[CI] Fix setup of disaggregation with different tp (#6706)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-05-28 11:17:27 -07:00 |
|
|
shangmingc
|
e9fd11c0d1
|
[Bugfix] Fix ChatCompletion endpoint of mini_lb when stream is set (#6703)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-05-28 21:33:36 +08:00 |
|
|
shangmingc
|
c7588d593e
|
[Bugfix] Fix slice operation when chunk size mismatch (#6697)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-05-28 21:15:00 +08:00 |
|
|
shangmingc
|
b1c8d4e9f3
|
[PD] Abort unbootstrapped prefill requests through timeout (#6685)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-05-28 00:40:54 -07:00 |
|
|
shangmingc
|
c25231c679
|
[CI] Fix flaky pp single node test (#6689)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-05-28 00:40:26 -07:00 |
|
|
shangmingc
|
fba03b29e3
|
[Bugfix] Fix missing abort finish reason for PD with ChatCompletion (#6693)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-05-28 00:39:46 -07:00 |
|
|
shangmingc
|
3ce94f71f9
|
[PD] Handle P/D failure and reconnect without affecting other instances (#6263)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-05-26 19:21:01 -07:00 |
|
|
shangmingc
|
58f10679e1
|
Fix missing http status import for PD failure handler (#6520)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-05-22 15:23:54 +08:00 |
|
|
shangmingc
|
f1c896007a
|
[PD] Add support for different TP sizes per DP rank (#5922)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-05-12 13:55:42 -07:00 |
|
|
shangmingc
|
3ee40ff919
|
[CI] Re-enable pd disaggregation test (#6231)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-05-12 10:09:12 -07:00 |
|
|
shangmingc
|
0f334945c6
|
[CI] Fix PD mooncake dependency error (#6212)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-05-12 10:08:49 -07:00 |
|
|
shangmingc
|
31d1f6e7f4
|
[PD] Add simple unit test for disaggregation feature (#5654)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-05-11 13:35:27 +08:00 |
|
|
shangmingc
|
56f6589ecb
|
[PD] Optimize disaggregation ib device help info (#5781)
|
2025-05-05 13:47:37 +08:00 |
|
|
shangmingc
|
50eda8398e
|
[PD] Add kvargs table and thread pool for kvcache sender of mooncake (#5738)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-04-25 18:15:01 +08:00 |
|
|
shangmingc
|
e0673969b9
|
[PD] Add support for dp attention with mooncake (#5530)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-04-23 17:20:27 +08:00 |
|
|
shangmingc
|
f29a718f63
|
[PD] Fix generate endpoint of min_lb for PD (#5598)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
|
2025-04-21 21:39:18 +08:00 |
|
|
shangmingc
|
dca90f1db8
|
[PD] Remove the requirement of config file for mooncake backend (#5460)
|
2025-04-19 19:31:00 +08:00 |
|
|
shangmingc
|
f1b3b75fc6
|
[PD] Remove unused bootstrap param and fix port table type (#5423)
|
2025-04-15 21:21:20 +08:00 |
|
|
shangmingcandybyang
|
ffde65a094
|
[PD] Fix dynamic port support and MLA buffer for Mooncake (#5415)
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
Co-authored-by: ybyang <ybyang7@iflytek.com>
|
2025-04-15 19:29:31 +08:00 |
|