minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

Doc/update ascend quickstart (#32205)

Browse Source
This commit is contained in:
Jinyan Yi
2026-07-23 20:25:20 +08:00
committed by GitHub
parent 70ac0c4b0e
commit b98a577fbe
3 changed files with 20 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs_new/docs/hardware-platforms/ascend-npus/ascend_npu_quick_start.mdx
+3 -3
View File
@@ -40,7 +40,7 @@ at least **30GB** of free space. If you need to download model weights, check th
</Warning>
<Tip>
We publish both **stable releases** and **daily builds**. Choose a stable release tag (e.g., `v0.5.13.post1-cann9.0.0-a3`)
We publish both **stable releases** and **daily builds**. Choose a stable release tag (e.g., `v0.5.14-cann9.0.0-a3`)
if you prefer a validated version, or a daily build tag (e.g., `main-cann9.0.0-a3`) if you need the latest development changes.
</Tip>
@@ -52,7 +52,7 @@ adding `--volume /path/to/model:/path/to/model` to the `docker run` command belo
```shell Command
# Choose one (uncomment the line you want):
export IMAGE=quay.io/ascend/sglang:v0.5.13.post1-cann9.0.0-a3 # Stable release
export IMAGE=quay.io/ascend/sglang:v0.5.14-cann9.0.0-a3 # Stable release
# export IMAGE=quay.io/ascend/sglang:main-cann9.0.0-a3 # Daily build
docker run -it --rm --privileged --network=host --ipc=host --shm-size=16g \
@@ -77,7 +77,7 @@ docker run -it --rm --privileged --network=host --ipc=host --shm-size=16g \
```shell Command
# Choose one (uncomment the line you want):
export IMAGE=quay.io/ascend/sglang:v0.5.13.post1-cann9.0.0-910b # Stable release
export IMAGE=quay.io/ascend/sglang:v0.5.14-cann9.0.0-910b # Stable release
# export IMAGE=quay.io/ascend/sglang:main-cann9.0.0-910b # Daily build
docker run -it --rm --privileged --network=host --ipc=host --shm-size=16g \
docs_new/docs/hardware-platforms/ascend-npus/best_practice/qwen3_6_35b_a3b.mdx
+9
View File
@@ -8,6 +8,8 @@ metatags:
This page focuses on optimal configuration and benchmark results for Qwen3.6-35B-A3B on the Ascend NPU. For environment setup, model weight download, feature configuration, and deployment instructions, etc., see the [Qwen3.6-35B-A3B Model Tutorial](/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_6_35b_a3b).
On A3 each card has 2 dies, so `--tp-size` is twice the card count; see [Ascend NPU Reference](/docs/hardware-platforms/ascend-npus/ascend_npu_reference#hardware) for details.
Use image **SGLang `>= v0.5.14`** for these NEXTN configurations. Without `--dataset-path`, `bench_serving --dataset-name random` downloads ShareGPT from Hugging Face; in offline environments, pass a local dataset path (for example a ShareGPT JSON file).
</Note>
### Low Latency
@@ -120,6 +122,7 @@ We tested it based on the `RANDOM` dataset.
```bash Command
python -m sglang.bench_serving \
--dataset-name random \
--dataset-path /path/to/dataset \
--backend sglang \
--host 127.0.0.1 \
--port 6688 \
@@ -217,6 +220,7 @@ We tested it based on the `RANDOM` dataset.
```bash Command
python -m sglang.bench_serving \
--dataset-name random \
--dataset-path /path/to/dataset \
--backend sglang \
--host 127.0.0.1 \
--port 6688 \
@@ -316,6 +320,7 @@ We tested it based on the `RANDOM` dataset.
```bash Command
python -m sglang.bench_serving \
--dataset-name random \
--dataset-path /path/to/dataset \
--backend sglang \
--host 127.0.0.1 \
--port 6688 \
@@ -410,6 +415,7 @@ We tested it based on the `RANDOM` dataset.
```bash Command
python -m sglang.bench_serving \
--dataset-name random \
--dataset-path /path/to/dataset \
--backend sglang \
--host 127.0.0.1 \
--port 6688 \
@@ -507,6 +513,7 @@ We tested it based on the `RANDOM` dataset.
```bash Command
python -m sglang.bench_serving \
--dataset-name random \
--dataset-path /path/to/dataset \
--backend sglang \
--host 127.0.0.1 \
--port 6688 \
@@ -604,6 +611,7 @@ We tested it based on the `RANDOM` dataset.
```bash Command
python -m sglang.bench_serving \
--dataset-name random \
--dataset-path /path/to/dataset \
--backend sglang \
--host 127.0.0.1 \
--port 6688 \
@@ -803,6 +811,7 @@ We tested it based on the `RANDOM` dataset.
```bash Command
python -m sglang.bench_serving \
--dataset-name random \
--dataset-path /path/to/dataset \
--backend sglang \
--host 127.0.0.1 \
--port 6688 \
docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_6_35b_a3b.mdx
+8 -7
View File
@@ -16,9 +16,10 @@ comprehension, and tool usage.
This document demonstrates the deployment of Qwen3.6-35B-A3B on Ascend NPUs using SGLang, including single-node PD mixed
mode, feature configuration, and performance optimization.
This document is validated and written based on **SGLang v0.5.13**. The current model (Qwen3.6-35B-A3B) is fully supported
in this version. To use the latest features (e.g., speculative decoding, multimodal), it is recommended to use
v0.5.13 or a later version.
This document is validated and written based on **SGLang v0.5.14**. The current model (Qwen3.6-35B-A3B) is fully supported
in this version. To use the latest features (e.g., speculative decoding / NEXTN, multimodal), use
**v0.5.14 or a later version**. Ascend images based on `v0.5.13(.post1)` can fail when NEXTN is enabled
(`NPUMHATokenToKVPool` missing `data_strides`).
## Supported features
@@ -87,7 +88,7 @@ For details, see [Docker image versions](/docs/hardware-platforms/ascend-npus/as
<Tab title="Atlas 800I A3">
```bash Command
docker pull quay.io/ascend/sglang:v0.5.13.post1-cann9.0.0-a3
docker pull quay.io/ascend/sglang:v0.5.14-cann9.0.0-a3
docker run -itd --shm-size=16g --name ${NAME} \
--privileged=true --net=host \
@@ -115,14 +116,14 @@ docker run -itd --shm-size=16g --name ${NAME} \
--device=/dev/davinci_manager:/dev/davinci_manager \
--device=/dev/hisi_hdc:/dev/hisi_hdc \
--entrypoint=bash \
quay.io/ascend/sglang:v0.5.13.post1-cann9.0.0-a3
quay.io/ascend/sglang:v0.5.14-cann9.0.0-a3
```
</Tab>
<Tab title="Atlas 800I A2">
```bash Command
docker pull quay.io/ascend/sglang:v0.5.13.post1-cann9.0.0-910b
docker pull quay.io/ascend/sglang:v0.5.14-cann9.0.0-910b
docker run -itd --shm-size=16g --name ${NAME} \
--privileged=true --net=host \
@@ -142,7 +143,7 @@ docker run -itd --shm-size=16g --name ${NAME} \
--device=/dev/davinci_manager:/dev/davinci_manager \
--device=/dev/hisi_hdc:/dev/hisi_hdc \
--entrypoint=bash \
quay.io/ascend/sglang:v0.5.13.post1-cann9.0.0-910b
quay.io/ascend/sglang:v0.5.14-cann9.0.0-910b
```
</Tab>