mooncake_store: local_hostname 加 rank 后缀,让 master 的 preferred-segment 分配落到本进程段
build-sglang-image / build (push) Failing after 1s
build-sglang-image / build (push) Failing after 1s
两个 rank 进程共用默认 localhost 时,master 按名字匹配无法区分两个段, 副本会落到对方进程段;mooncake 的 local memcpy 快路径要求同进程 endpoint (跨进程虚拟地址无效),于是静默回退 loopback TCP(实测 ~0.78GB/s)。 唯一化后自写自读可命中 memcpy(干净集群探针实测 lo 流量归零)。 注意:memcpy 线程池 kDefaultMemcpyWorkers=1(mooncake 0.3.13 硬编码), 单流 ~1GB/s,进一步提升需改 mooncake 并重编译。
This commit is contained in:
@@ -515,6 +515,22 @@ class MooncakeStore(HiCacheStorage, MooncakeBaseStore):
|
||||
else:
|
||||
client_hostname = self.config.local_hostname
|
||||
transfer_engine = None
|
||||
# Each rank embeds its own Mooncake client process. Giving
|
||||
# every rank the same local_hostname makes the master's
|
||||
# preferred-segment allocation unable to tell their
|
||||
# segments apart, so replicas can land on a peer rank's
|
||||
# segment and the local-memcpy fast path (which requires
|
||||
# same-process endpoints) silently falls back to TCP.
|
||||
# Make the hostname unique per rank so own-writes land on
|
||||
# the own segment and reads can use local memcpy.
|
||||
if storage_config is not None:
|
||||
client_hostname = (
|
||||
f"{client_hostname}"
|
||||
f"-d{storage_config.dp_rank}"
|
||||
f"t{storage_config.tp_rank}"
|
||||
f"p{storage_config.pp_rank}"
|
||||
f"c{storage_config.attn_cp_rank}"
|
||||
)
|
||||
|
||||
setup_kwargs = {}
|
||||
if self.config.enable_ssd_offload:
|
||||
|
||||
Reference in New Issue
Block a user