[NIXL] Add custom NIXL backend selection for KVManager (#17146)

Signed-off-by: Yoray Zack <yorayz@nvidia.com>
This commit is contained in:
zackyoray
2026-01-26 14:35:38 +08:00
committed by GitHub
parent 1e8db18290
commit d275d47973
3 changed files with 39 additions and 2 deletions
@@ -262,6 +262,26 @@ python -m sglang.launch_server \
--max-running-requests 128
```
### Advanced Configuration
#### NIXL Backend Selection
By default, NIXL uses the **UCX** backend for KV cache transfers. You can select a different NIXL plugin backend depending on your infrastructure using the environment variable `SGLANG_DISAGGREGATION_NIXL_BACKEND`.
Example: `export SGLANG_DISAGGREGATION_NIXL_BACKEND=LIBFABRIC`
**Available backends:** UCX (default), LIBFABRIC, or any installed NIXL plugin.
Example usage:
```bash
export SGLANG_DISAGGREGATION_NIXL_BACKEND=LIBFABRIC
python -m sglang.launch_server \
--model-path meta-llama/Llama-3.1-8B-Instruct \
--disaggregation-mode prefill \
--disaggregation-transfer-backend nixl \
--port 30000
```
## ASCEND
### Usage