minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

Add IntraNode NVLink configration in PD disaggregation docs (#23329)

Browse Source 
Co-authored-by: 百麒 <yaozhong.lyz@alibaba-inc.com>
This commit is contained in:
Lewis
2026-05-13 23:23:57 -07:00
committed by GitHub
co-authored by 百麒
parent 3640116397
commit 0680f1b3d1
2 changed files with 12 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs_new/docs/advanced_features/pd_disaggregation.mdx
+6
View File
@@ -137,6 +137,12 @@ export SGLANG_MOONCAKE_CUSTOM_MEM_POOL=NVLINK
export MC_FORCE_MNNVL=True
```
To utilize Intra-Node NVLink for KV cache transfers with the Mooncake backend (recommended for A100, H20, H100, etc.), set the following environment variables. Please note that auxiliary data still needs to be transferred via TCP.
```bash
export SGLANG_MOONCAKE_CUSTOM_MEM_POOL=INTRA_NODE_NVLINK
export MC_INTRANODE_NVLINK=true
```
The `SGLANG_MOONCAKE_CUSTOM_MEM_POOL` environment variable enables the custom memory pool. Supported values are `NVLINK` (or `True`), `BAREX`, and `INTRA_NODE_NVLINK`.
#### Prefill Server Configuration