minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

[PD] docs: clarify disaggregation IB device formats (#25521)

Browse Source 
Co-authored-by: Zijie Xia <zijie.xia@radixark.ai>
This commit is contained in:
Teng Ma
2026-06-02 11:38:33 +08:00
committed by GitHub
co-authored by Zijie Xia
parent 9fe8b72912
commit b562da0d9f
2 changed files with 16 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs_new/docs/advanced_features/pd_disaggregation.mdx
+10
View File
@@ -36,6 +36,16 @@ For deploying PD disaggregation at scale with load balancing and fault tolerance
uv pip install mooncake-transfer-engine uv pip install mooncake-transfer-engine
``` ```
### IB Device Configuration
`--disaggregation-ib-device` supports the following formats when using the Mooncake backend:
1. Shared device list for all GPUs: `mlx5_0` or `mlx5_0,mlx5_1`
2. Per-GPU JSON mapping: `{"0": "mlx5_0,mlx5_1", "1": "mlx5_2,mlx5_3"}`
3. Path to a JSON file containing the same per-GPU mapping
Each JSON value uses the same comma-separated device list format as the shared configuration.
### Usage ### Usage
### Llama Single Node ### Llama Single Node
python/sglang/srt/server_args.py
+6 -3
View File
@@ -6844,9 +6844,12 @@ class ServerArgs:
"--disaggregation-ib-device", "--disaggregation-ib-device",
type=str, type=str,
default=ServerArgs.disaggregation_ib_device, default=ServerArgs.disaggregation_ib_device,
help="The InfiniBand devices for disaggregation transfer, accepts single device (e.g., --disaggregation-ib-device mlx5_0) " help="The InfiniBand devices for disaggregation transfer. Supports a single device "
"or multiple comma-separated devices (e.g., --disaggregation-ib-device mlx5_0,mlx5_1). " "(e.g., --disaggregation-ib-device mlx5_0), a shared comma-separated list "
"Default is None, which triggers automatic device detection when mooncake backend is enabled.", "(e.g., --disaggregation-ib-device mlx5_0,mlx5_1), a per-GPU JSON mapping "
'(e.g., --disaggregation-ib-device \'{"0": "mlx5_0,mlx5_1", "1": "mlx5_2"}\'), '
"or a path to a JSON file containing that mapping. Default is None, which triggers "
"automatic device detection when mooncake backend is enabled.",
) )
parser.add_argument( parser.add_argument(
"--disaggregation-decode-enable-radix-cache", "--disaggregation-decode-enable-radix-cache",