[DOCS][NPU]Supplementary Notes (#28295)
This commit is contained in:
@@ -101,11 +101,9 @@ If you encounter an error like “Terms of Service have not been accepted” dur
|
||||
conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/
|
||||
conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/
|
||||
conda config --set show_channel_urls yes
|
||||
|
||||
# Edit the system-level conda config to remove any hardcoded defaults
|
||||
vi /root/miniconda3/.condarc
|
||||
conda config --remove channels defaults
|
||||
```
|
||||
Inside /root/miniconda3/.condarc, delete or comment out any lines containing defaults or official Anaconda URLs.
|
||||
Edit the system-level conda config to remove any hardcoded defaults, e.g. vi ~/miniconda3/.condarc
|
||||
Then remove the failed environment and recreate it:
|
||||
```bash Command
|
||||
conda clean -i
|
||||
@@ -123,7 +121,7 @@ Prior to start work with SGLang on Ascend you need to install CANN Toolkit, Kern
|
||||
If you want to use PD disaggregation mode, you need to install MemFabric-Hybrid. MemFabric-Hybrid is a drop-in replacement of Mooncake Transfer Engine that enables KV cache transfer on Ascend NPU clusters.
|
||||
|
||||
```bash Command
|
||||
pip install memfabric-hybrid==1.0.5
|
||||
pip install memfabric-hybrid==1.0.8
|
||||
```
|
||||
|
||||
#### Pytorch and Pytorch Framework Adaptor on Ascend
|
||||
@@ -220,8 +218,11 @@ git clone https://github.com/sgl-project/sglang.git
|
||||
cd sglang/docker
|
||||
|
||||
# Build the docker image
|
||||
# If there are network errors, please modify the Dockerfile to use offline dependencies or use a proxy
|
||||
# <arch_tag> is the target architecture of the image, e.g. amd64, arm64
|
||||
# Replace <arch_tag> with the target architecture, e.g. amd64, arm64.
|
||||
# Optional build arguments:
|
||||
# --build-arg DEVICE_TYPE=910b # Required for Atlas 800I A2
|
||||
# --build-arg APTMIRROR=<mirror_url> # Use a custom APT mirror too improve download speed
|
||||
# If there are network errors, please modify the Dockerfile to add ARG HTTP_PROXY/HTTPS_PROXY and set them as ENV.
|
||||
docker build --build-arg TARGETARCH=<arch_tag> -t <image_name> -f npu.Dockerfile .
|
||||
```
|
||||
|
||||
@@ -232,6 +233,7 @@ __Notice:__ `--privileged` and `--network=host` are required by RDMA, which is t
|
||||
<Tab title="Atlas 800I A3">
|
||||
|
||||
```bash Command
|
||||
# Create a shortcut 'drun' to launch a privileged Docker container
|
||||
alias drun='docker run -it --rm --privileged --network=host --ipc=host --shm-size=16g \
|
||||
--device=/dev/davinci0 --device=/dev/davinci1 --device=/dev/davinci2 --device=/dev/davinci3 \
|
||||
--device=/dev/davinci4 --device=/dev/davinci5 --device=/dev/davinci6 --device=/dev/davinci7 \
|
||||
@@ -244,6 +246,7 @@ alias drun='docker run -it --rm --privileged --network=host --ipc=host --shm-siz
|
||||
--volume /var/queue_schedule:/var/queue_schedule --volume ~/.cache/:/root/.cache/'
|
||||
|
||||
# Add HF_TOKEN env for download model by SGLang.
|
||||
# The container runs with the '--rm' flag, so it will be automatically removed after the command finishes (including Ctrl+C)
|
||||
drun --env "HF_TOKEN=<secret>" \
|
||||
<image_name> \
|
||||
python3 -m sglang.launch_server --model-path meta-llama/Llama-3.1-8B-Instruct --attention-backend ascend
|
||||
@@ -253,6 +256,7 @@ drun --env "HF_TOKEN=<secret>" \
|
||||
<Tab title="Atlas 800I A2">
|
||||
|
||||
```bash Command
|
||||
# Create a shortcut 'drun' to launch a privileged Docker container
|
||||
alias drun='docker run -it --rm --privileged --network=host --ipc=host --shm-size=16g \
|
||||
--device=/dev/davinci0 --device=/dev/davinci1 --device=/dev/davinci2 --device=/dev/davinci3 \
|
||||
--device=/dev/davinci4 --device=/dev/davinci5 --device=/dev/davinci6 --device=/dev/davinci7 \
|
||||
@@ -263,6 +267,7 @@ alias drun='docker run -it --rm --privileged --network=host --ipc=host --shm-siz
|
||||
--volume /var/queue_schedule:/var/queue_schedule --volume ~/.cache/:/root/.cache/'
|
||||
|
||||
# Add HF_TOKEN env for download model by SGLang.
|
||||
# The container runs with the '--rm' flag, so it will be automatically removed after the command finishes (including Ctrl+C)
|
||||
drun --env "HF_TOKEN=<secret>" \
|
||||
<image_name> \
|
||||
python3 -m sglang.launch_server --model-path meta-llama/Llama-3.1-8B-Instruct --attention-backend ascend
|
||||
@@ -501,8 +506,8 @@ curl http://127.0.0.1:8000/v1/chat/completions \
|
||||
|
||||
Some models return responses accompanied with thinking process content. To disable this output, configure parameters as follows:
|
||||
```bash Command
|
||||
curl http://127.0.0.1:8000/v1/chat/completions
|
||||
-H "Content-Type: application/json"
|
||||
curl http://127.0.0.1:8000/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"model": "Eco-Tech/Qwen3.5-27B-w8a8-mtp",
|
||||
"messages": [{"role": "user", "content": "What is the capital of France?"}],
|
||||
|
||||
Reference in New Issue
Block a user