From 0680f1b3d13fde8444e48e18b255f21dfb866b70 Mon Sep 17 00:00:00 2001 From: Lewis <63569348+TTThanos@users.noreply.github.com> Date: Thu, 14 May 2026 14:23:57 +0800 Subject: [PATCH] Add IntraNode NVLink configration in PD disaggregation docs (#23329) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: 百麒 --- docs/advanced_features/pd_disaggregation.md | 6 ++++++ docs_new/docs/advanced_features/pd_disaggregation.mdx | 6 ++++++ 2 files changed, 12 insertions(+) diff --git a/docs/advanced_features/pd_disaggregation.md b/docs/advanced_features/pd_disaggregation.md index e1edc56b8..7c12c26e0 100644 --- a/docs/advanced_features/pd_disaggregation.md +++ b/docs/advanced_features/pd_disaggregation.md @@ -133,6 +133,12 @@ To enable NVLink transport for KV cache transfers with the mooncake backend (rec export SGLANG_MOONCAKE_CUSTOM_MEM_POOL=NVLINK export MC_FORCE_MNNVL=True ``` +To utilize Intra-Node NVLink for KV cache transfers with the Mooncake backend (recommended for A100, H20, H100, etc.), set the following environment variables. Please note that auxiliary data still needs to be transferred via TCP. + +```bash +export SGLANG_MOONCAKE_CUSTOM_MEM_POOL=INTRA_NODE_NVLINK +export MC_INTRANODE_NVLINK=true +``` The `SGLANG_MOONCAKE_CUSTOM_MEM_POOL` environment variable enables the custom memory pool. Supported values are `NVLINK` (or `True`), `BAREX`, and `INTRA_NODE_NVLINK`. diff --git a/docs_new/docs/advanced_features/pd_disaggregation.mdx b/docs_new/docs/advanced_features/pd_disaggregation.mdx index 86f4bf025..ff92b55db 100644 --- a/docs_new/docs/advanced_features/pd_disaggregation.mdx +++ b/docs_new/docs/advanced_features/pd_disaggregation.mdx @@ -137,6 +137,12 @@ export SGLANG_MOONCAKE_CUSTOM_MEM_POOL=NVLINK export MC_FORCE_MNNVL=True ``` +To utilize Intra-Node NVLink for KV cache transfers with the Mooncake backend (recommended for A100, H20, H100, etc.), set the following environment variables. Please note that auxiliary data still needs to be transferred via TCP. + +```bash +export SGLANG_MOONCAKE_CUSTOM_MEM_POOL=INTRA_NODE_NVLINK +export MC_INTRANODE_NVLINK=true +``` The `SGLANG_MOONCAKE_CUSTOM_MEM_POOL` environment variable enables the custom memory pool. Supported values are `NVLINK` (or `True`), `BAREX`, and `INTRA_NODE_NVLINK`. #### Prefill Server Configuration