minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 10 Packages Projects Releases Wiki Activity

[NPU] [DOC] Fix TOC of Ascend NPU Docs (#29129)

Browse Source
This commit is contained in:
amote-i
2026-06-24 16:47:10 +08:00
committed by GitHub
parent 4992f7a108
commit 73d976e375
18 changed files with 40 additions and 52 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs_new/cookbook/autoregressive/DeepSeek/DeepSeek-V3.mdx
+1 -1
View File
@@ -425,7 +425,7 @@ For multi-node serving and hardware-specific examples:
- [16× A100 INT8](https://github.com/sgl-project/sglang/tree/main/benchmark/deepseek_v3#example-serving-with-16-a100a800-with-int8-quantization)
- [32× L40S INT8](https://github.com/sgl-project/sglang/tree/main/benchmark/deepseek_v3#example-serving-with-32-l40s-with-int8-quantization)
- [Xeon 6980P CPU](../../../docs/hardware-platforms/cpu_server#example-running-deepseek-r1)
- [4× Atlas 800I A3 (int8)](../../../docs/hardware-platforms/ascend-npus/ascend_npu_deepseek_example#running-deepseek-with-pd-disaggregation-on-4-x-atlas-800i-a3)
- [4× Atlas 800I A3 (int8)](../../../docs/hardware-platforms/ascend-npus/model-tutorials/deepseek_r1#multi-node-pd-disaggregation-deployment)
**Blog references for large-scale deployment:**
docs_new/docs.json
+13 -18
View File
@@ -435,11 +435,11 @@
},
{
"source": "/platforms/ascend/ascend_npu_best_practice.html",
"destination": "/docs/hardware-platforms/ascend-npus/ascend_npu_best_practice"
"destination": "/docs/hardware-platforms/ascend-npus/best_practice"
},
{
"source": "/platforms/ascend/ascend_npu_deepseek_example.html",
"destination": "/docs/hardware-platforms/ascend-npus/ascend_npu_deepseek_example"
"destination": "/docs/hardware-platforms/ascend-npus/model-tutorials/deepseek_v3_2"
},
{
"source": "/platforms/ascend/ascend_npu_environment_variables.html",
@@ -447,7 +447,7 @@
},
{
"source": "/platforms/ascend/ascend_npu_glm5_examples.html",
"destination": "/docs/hardware-platforms/ascend-npus/ascend_npu_glm5_examples"
"destination": "/docs/hardware-platforms/ascend-npus/model-tutorials/glm_5_1"
},
{
"source": "/platforms/ascend/ascend_npu_quantization.html",
@@ -459,11 +459,11 @@
},
{
"source": "/platforms/ascend/ascend_npu_qwen3_5_examples.html",
"destination": "/docs/hardware-platforms/ascend-npus/ascend_npu_qwen3_5_examples"
"destination": "/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_5_397b"
},
{
"source": "/platforms/ascend/ascend_npu_qwen3_examples.html",
"destination": "/docs/hardware-platforms/ascend-npus/ascend_npu_qwen3_examples"
"destination": "/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_235b_a22b"
},
{
"source": "/platforms/ascend/ascend_npu_support.html",
@@ -878,6 +878,14 @@
{
"group": "Ascend NPUs",
"pages": [
"docs/hardware-platforms/ascend-npus/ascend_npu_quick_start",
"docs/hardware-platforms/ascend-npus/ascend_npu",
"docs/hardware-platforms/ascend-npus/ascend_npu_support_features",
"docs/hardware-platforms/ascend-npus/ascend_npu_support_models",
"docs/hardware-platforms/ascend-npus/ascend_npu_quantization",
"docs/hardware-platforms/ascend-npus/mindspore_backend",
"docs/hardware-platforms/ascend-npus/ascend_contribution_guide",
"docs/hardware-platforms/ascend-npus/ascend_npu_support_new_models",
{
"group": "Model Tutorials",
"pages": [
@@ -896,17 +904,6 @@
"docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_next_80b_a3b_instruct"
]
},
"docs/hardware-platforms/ascend-npus/ascend_npu_quick_start",
"docs/hardware-platforms/ascend-npus/ascend_npu",
"docs/hardware-platforms/ascend-npus/ascend_npu_support_features",
"docs/hardware-platforms/ascend-npus/ascend_npu_support_models",
"docs/hardware-platforms/ascend-npus/ascend_npu_quantization",
"docs/hardware-platforms/ascend-npus/ascend_npu_deepseek_example",
"docs/hardware-platforms/ascend-npus/ascend_npu_qwen3_examples",
"docs/hardware-platforms/ascend-npus/mindspore_backend",
"docs/hardware-platforms/ascend-npus/ascend_contribution_guide",
"docs/hardware-platforms/ascend-npus/ascend_npu_support_new_models",
"docs/hardware-platforms/ascend-npus/ascend_npu_best_practice",
{
"group": "Best Practice",
"pages": [
@@ -929,8 +926,6 @@
"docs/hardware-platforms/ascend-npus/ascend_npu_accuracy_evaluation",
"docs/hardware-platforms/ascend-npus/ascend_npu_performance_testing",
"docs/hardware-platforms/ascend-npus/ascend_npu_ring_sp_performance",
"docs/hardware-platforms/ascend-npus/ascend_npu_qwen3_5_examples",
"docs/hardware-platforms/ascend-npus/ascend_npu_glm5_examples",
"docs/hardware-platforms/ascend-npus/ascend_npu_environment_variables",
"docs/hardware-platforms/ascend-npus/ascend_npu_faq",
"docs/hardware-platforms/ascend-npus/ascend_npu_operator_performance_optimizing",
docs_new/docs/hardware-platforms/ascend-npus/ascend_npu.mdx
+5 -5
View File
@@ -41,7 +41,7 @@ You can install SGLang using any of the methods below. Please go through `System
<tr>
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Triton</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>3.2.1.dev20260530</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>`pip install triton-ascend==3.2.1.dev20260530 \`<br/>`--extra-index-url=https://mirrors.huaweicloud.com/ascend/repos/pypi/nightly \`<br/>`--trusted-host triton-ascend.osinfra.cn`</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>`pip install triton-ascend==3.2.1.dev20260530 \`<br/>`--extra-index-url=https://mirrors.huaweicloud.com/ascend/repos/pypi/nightly \`<br/>`--trusted-host mirrors.huaweicloud.com`</td>
</tr>
<tr>
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>SGLang NPU Kernel</td>
@@ -143,7 +143,7 @@ We provide our own implementation of Triton for Ascend.
```bash Command
pip install triton-ascend==3.2.1.dev20260530 \
--extra-index-url=https://mirrors.huaweicloud.com/ascend/repos/pypi/nightly \
--trusted-host triton-ascend.osinfra.cn
--trusted-host mirrors.huaweicloud.com
```
For installation of Triton on Ascend nightly builds or from sources, follow [installation guide](https://gitcode.com/Ascend/triton-ascend/blob/master/docs/sources/getting-started/installation.md)
@@ -185,7 +185,7 @@ Ensure sufficient disk space before pulling images. Each Docker image requires a
1. Download SGLang image
<Tip>
We publish both **stable releases** and **daily builds**. Choose a stable release tag (e.g., `v0.5.10-npu.rc1-a3`) if you prefer a validated version, or a daily build tag (e.g., `main-cann9.0.0-a3`) if you need the latest development changes.
We publish both **stable releases** and **daily builds**. Choose a stable release tag (e.g., `v0.5.13.post1-cann9.0.0-a3`) if you prefer a validated version, or a daily build tag (e.g., `main-cann9.0.0-a3`) if you need the latest development changes.
</Tip>
<Tabs>
@@ -193,7 +193,7 @@ We publish both **stable releases** and **daily builds**. Choose a stable releas
```bash Command
# Stable release
docker pull quay.io/ascend/sglang:v0.5.10-npu.rc1-a3
docker pull quay.io/ascend/sglang:v0.5.13.post1-cann9.0.0-a3
# Daily build
docker pull quay.io/ascend/sglang:main-cann9.0.0-a3
@@ -204,7 +204,7 @@ docker pull quay.io/ascend/sglang:main-cann9.0.0-a3
```bash Command
# Stable release
docker pull quay.io/ascend/sglang:v0.5.10-npu.rc1-910b
docker pull quay.io/ascend/sglang:v0.5.13.post1-cann9.0.0-910b
# Daily build
docker pull quay.io/ascend/sglang:main-cann9.0.0-910b
docs_new/docs/hardware-platforms/ascend-npus/ascend_npu_accuracy_evaluation.mdx
+2 -2
View File
@@ -26,7 +26,7 @@ First, launch the SGLang environment using the provided container image:
<Tab title="Atlas 800I A3">
```shell Command
export IMAGE=quay.io/ascend/sglang:v0.5.10-npu.rc1-a3
export IMAGE=quay.io/ascend/sglang:v0.5.13.post1-cann9.0.0-a3
docker run -it --rm --privileged --network=host --ipc=host --shm-size=16g \
--device=/dev/davinci0 --device=/dev/davinci1 --device=/dev/davinci2 --device=/dev/davinci3 \
@@ -49,7 +49,7 @@ docker run -it --rm --privileged --network=host --ipc=host --shm-size=16g \
<Tab title="Atlas 800I A2">
```shell Command
export IMAGE=quay.io/ascend/sglang:v0.5.10-npu.rc1-910b
export IMAGE=quay.io/ascend/sglang:v0.5.13.post1-cann9.0.0-910b
docker run -it --rm --privileged --network=host --ipc=host --shm-size=16g \
--device=/dev/davinci0 --device=/dev/davinci1 --device=/dev/davinci2 --device=/dev/davinci3 \
docs_new/docs/hardware-platforms/ascend-npus/ascend_npu_faq.mdx
+1 -1
View File
@@ -293,7 +293,7 @@ Replace model weights, e.g., switch Qwen/Qwen3.5-27B-FP8 to Eco-Tech/Qwen3.5-27B
Docker images for Ascend NPU are available in two types:
- **Stable release** — validated version with a specific tag, e.g., `quay.io/ascend/sglang:v0.5.10-npu.rc1-a3`.
- **Stable release** — validated version with a specific tag, e.g., `quay.io/ascend/sglang:v0.5.13.post1-cann9.0.0-a3`.
Recommended for production deployments.
- **Daily build** — includes the latest development changes, e.g., `quay.io/ascend/sglang:main-cann8.5.0-a3`. Use this
if you need the latest features or bug fixes that have not yet been included in a stable release.
docs_new/docs/hardware-platforms/ascend-npus/ascend_npu_optimization.mdx
+3 -3
View File
@@ -5,9 +5,9 @@ metatags:
---
This guide explains the role of each parameter used in SGLang deployments on Ascend NPU. It uses
the [DeepSeek-V3.2 best practice configuration](/docs/hardware-platforms/ascend-npus/ascend_npu_best_practice#deepseek-v3-2-128k-1k-26ms-on-a3-32-cards-disaggregation-mode)
the [DeepSeek-V3.2 best practice configuration](/docs/hardware-platforms/ascend-npus/best_practice/deepseek_v3_2#pd-disaggregation)
as the reference example. For a complete list of tested deployment configurations, see the
[Ascend NPU Best Practice](/docs/hardware-platforms/ascend-npus/ascend_npu_best_practice) page.
[Ascend NPU Best Practice](/docs/hardware-platforms/ascend-npus/best_practice) page.
<Note>
Parameters in this guide fall into two categories:
@@ -794,7 +794,7 @@ Casts model weight tensors to the Ascend NPU ACL FRACTAL_NZ format (format 29) f
## See Also
- [Ascend NPU Best Practice](/docs/hardware-platforms/ascend-npus/ascend_npu_best_practice) — Complete deployment configurations and benchmark results for all supported models
- [Ascend NPU Best Practice](/docs/hardware-platforms/ascend-npus/best_practice) — Complete deployment configurations and benchmark results for all supported models
- [Ascend NPU Environment Variables](/docs/hardware-platforms/ascend-npus/ascend_npu_environment_variables) — Reference for all Ascend NPU-related environment variables
- [DeepSeek V3.2 Guide](/cookbook/autoregressive/DeepSeek/DeepSeek-V3_2) — Detailed usage guide for DeepSeek V3.2 deployment
- [Expert Parallelism](/docs/advanced_features/expert_parallelism) — DeepEP configuration and tuning guide
docs_new/docs/hardware-platforms/ascend-npus/best_practice/deepseek_r1.mdx
+2 -2
View File
@@ -25,8 +25,6 @@ This guide describes the best practice data for DeepSeek-R1 on the Ascend NPU.
## Optimal Configuration
<a id="pd-disaggregation" title="Referenced by external docs. Verify before removing."></a>
### DeepSeek-R1 W4A8 1P1D 16P IN3K5 OUT1K5 41ms
**Model**: DeepSeek-R1
@@ -325,6 +323,8 @@ python -m sglang.bench_serving \
--random-range-ratio 1
```
<a id="pd-disaggregation" title="Referenced by external docs. Verify before removing."></a>
### DeepSeek-R1 W8A8 2P1D 32P IN3K5 OUT1K5 18.9ms
**Model**: DeepSeek-R1
docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/deepseek_r1.mdx
+2 -2
View File
@@ -151,14 +151,14 @@ quay.io/ascend/sglang:v0.5.13.post1-cann9.0.0-910b
Single-node deployment completes both prefill and decode within the same node (PD mixed mode), suitable for scenarios
with limited hardware resources. This scenario is already covered in the best practice. For the complete, optimized
deployment commands and benchmark data, see
[DeepSeek-R1 Best Practice — W4A8 8P PD Mixed On A3](/docs/hardware-platforms/ascend-npus/best_practice/deepseek_r1#w4a8-8p-pd-mixed).
[DeepSeek-R1 Best Practice — W4A8 8P PD Mixed On A3](/docs/hardware-platforms/ascend-npus/best_practice/deepseek_r1#single-node-pd-mixed).
### Multi-node PD disaggregation deployment
PD disaggregation splits the prefill and decode stages onto separate nodes, reducing interference and improving
throughput for high-concurrency scenarios. This scenario is already covered in the best practice. For the complete, optimized
deployment commands and benchmark data, see
[DeepSeek-R1 Best Practice — W4A8 16P PD Disaggregation On A3](/docs/hardware-platforms/ascend-npus/best_practice/deepseek_r1#w4a8-16p-pd-disaggregation).
[DeepSeek-R1 Best Practice — W8A8 32P PD Disaggregation On A3](/docs/hardware-platforms/ascend-npus/best_practice/deepseek_r1#pd-disaggregation).
## Functional verification
docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/kimi_k2_6.mdx
+2 -2
View File
@@ -155,7 +155,7 @@ quay.io/ascend/sglang:v0.5.13.post1-cann9.0.0-910b
Single-node deployment completes both prefill and decode within the same node (PD mixed mode), suitable for scenarios
with limited hardware resources. This scenario is already covered in the best practice. For the complete, optimized
deployment commands and benchmark data, see
[Kimi K2.6 Best Practice — PD Mixed On A3](/docs/hardware-platforms/ascend-npus/best_practice/kimi_k2_6#single-node).
[Kimi K2.6 Best Practice — PD Mixed On A3](/docs/hardware-platforms/ascend-npus/best_practice/kimi_k2_6#single-node-pd-mixed).
### Multi-node online deployment
@@ -163,7 +163,7 @@ Multi-node deployment distributes the model across multiple Atlas 800I A3 nodes
prefill and decode on the same nodes (PD mixed mode), suitable for scenarios that need more device memory than a single
node can provide. This scenario is already covered in the best practice. For the complete, optimized
deployment commands and benchmark data, see
[Kimi-K2.6 Best Practice — Multi-node On A3](/docs/hardware-platforms/ascend-npus/best_practice/kimi_k2_6#multi-node).
[Kimi-K2.6 Best Practice — Multi-node On A3](/docs/hardware-platforms/ascend-npus/best_practice/kimi_k2_6#multi-node-pd-mixed).
### Multi-node PD disaggregation deployment
docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/minimax_m2_5.mdx
+1 -8
View File
@@ -149,14 +149,7 @@ quay.io/ascend/sglang:v0.5.13.post1-cann9.0.0-910b
Single-node deployment completes both prefill and decode within the same node (PD mixed mode), suitable for scenarios
with limited hardware resources. This scenario is already covered in the best practice. For the complete, optimized
deployment commands and benchmark data, see
[MiniMax-M2.5 Best Practice — PD Mixed On A3](/docs/hardware-platforms/ascend-npus/best_practice/minimax_m2_5#single-node).
### Multi-node PD disaggregation deployment
PD disaggregation splits the prefill and decode stages onto separate nodes, reducing interference and improving
throughput for high-concurrency scenarios. This scenario is already covered in the best practice. For the complete, optimized
deployment commands and benchmark data, see
[MiniMax-M2.5 Best Practice — PD Disaggregation On A3](/docs/hardware-platforms/ascend-npus/best_practice/minimax_m2_5#pd-disaggregation).
[MiniMax-M2.5 Best Practice — PD Mixed On A3](/docs/hardware-platforms/ascend-npus/best_practice/minimax_m2_5#single-node-pd-mixed).
## Functional verification
docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3-8b.mdx
+1 -1
View File
@@ -147,7 +147,7 @@ quay.io/ascend/sglang:v0.5.13.post1-cann9.0.0-910b
Single-node deployment completes both prefill and decode within the same node (PD mixed mode), suitable for scenarios
with limited hardware resources. This scenario is already covered in the best practice. For the complete, optimized
deployment commands and benchmark data, see
[Qwen3-8B Best Practice — PD Mixed On A3](/docs/hardware-platforms/ascend-npus/best_practice/qwen3-8b#single-node).
[Qwen3-8B Best Practice — PD Mixed On A3](/docs/hardware-platforms/ascend-npus/best_practice/qwen3-8b#single-node-pd-mixed).
## Functional verification
docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_235b_a22b.mdx
+1 -1
View File
@@ -153,7 +153,7 @@ quay.io/ascend/sglang:v0.5.13.post1-cann9.0.0-910b
Single-node deployment completes both prefill and decode within the same node (PD mixed mode), suitable for scenarios
with limited hardware resources. This scenario is already covered in the best practice. For the complete, optimized
deployment commands and benchmark data, see
[Qwen3-235B-A22B Best Practice — PD Mixed On A3](/docs/hardware-platforms/ascend-npus/best_practice/qwen3_235b_a22b#single-node).
[Qwen3-235B-A22B Best Practice — PD Mixed On A3](/docs/hardware-platforms/ascend-npus/best_practice/qwen3_235b_a22b#single-node-pd-mixed).
### Multi-node PD disaggregation deployment
docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_30b_a3b.mdx
+1 -1
View File
@@ -147,7 +147,7 @@ quay.io/ascend/sglang:v0.5.13.post1-cann9.0.0-910b
Single-node deployment completes both prefill and decode within the same node (PD mixed mode), suitable for scenarios
with limited hardware resources. This scenario is already covered in the best practice. For the complete, optimized
deployment commands and benchmark data, see
[Qwen3-30B-A3B Best Practice — PD Mixed On A3](/docs/hardware-platforms/ascend-npus/best_practice/qwen3_30b_a3b#single-node).
[Qwen3-30B-A3B Best Practice — PD Mixed On A3](/docs/hardware-platforms/ascend-npus/best_practice/qwen3_30b_a3b#single-node-pd-mixed).
## Functional verification
docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_32b.mdx
+1 -1
View File
@@ -145,7 +145,7 @@ quay.io/ascend/sglang:v0.5.13.post1-cann9.0.0-910b
Single-node deployment completes both prefill and decode within the same node (PD mixed mode), suitable for scenarios
with limited hardware resources. This scenario is already covered in the best practice. For the complete, optimized
deployment commands and benchmark data, see
[Qwen3-32B Best Practice — PD Mixed On A3](/docs/hardware-platforms/ascend-npus/best_practice/qwen3_32b#single-node).
[Qwen3-32B Best Practice — PD Mixed On A3](/docs/hardware-platforms/ascend-npus/best_practice/qwen3_32b#single-node-pd-mixed).
## Functional verification
docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_5_397b.mdx
+1 -1
View File
@@ -148,7 +148,7 @@ quay.io/ascend/sglang:v0.5.13.post1-cann9.0.0-910b
Single-node deployment completes both prefill and decode within the same node (PD mixed mode), suitable for scenarios
with limited hardware resources. This scenario is already covered in the best practice. For the complete, optimized
deployment commands and benchmark data, see
[Qwen3.5-397B Best Practice — PD Mixed On A3](/docs/hardware-platforms/ascend-npus/best_practice/qwen3_5_397b#single-node).
[Qwen3.5-397B Best Practice — PD Mixed On A3](/docs/hardware-platforms/ascend-npus/best_practice/qwen3_5_397b#single-node-pd-mixed).
## Functional verification
docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_6_27b.mdx
+1 -1
View File
@@ -145,7 +145,7 @@ quay.io/ascend/sglang:v0.5.13.post1-cann9.0.0-910b
Single-node deployment completes both prefill and decode within the same node (PD mixed mode), suitable for scenarios
with limited hardware resources. This scenario is already covered in the best practice. For the complete, optimized
deployment commands and benchmark data, see
[Qwen3.6-27B Best Practice — PD Mixed On A3](/docs/hardware-platforms/ascend-npus/best_practice/qwen3_6_27b#single-node).
[Qwen3.6-27B Best Practice — PD Mixed On A3](/docs/hardware-platforms/ascend-npus/best_practice/qwen3_6_27b#single-node-pd-mixed).
## Functional verification
docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_6_35b_a3b.mdx
+1 -1
View File
@@ -145,7 +145,7 @@ quay.io/ascend/sglang:v0.5.13.post1-cann9.0.0-910b
Single-node deployment completes both prefill and decode within the same node (PD mixed mode), suitable for scenarios
with limited hardware resources. This scenario is already covered in the best practice. For the complete, optimized
deployment commands and benchmark data, see
[Qwen3.6-35B-A3B Best Practice — PD Mixed On A3](/docs/hardware-platforms/ascend-npus/best_practice/qwen3_6_35b_a3b#single-node).
[Qwen3.6-35B-A3B Best Practice — PD Mixed On A3](/docs/hardware-platforms/ascend-npus/best_practice/qwen3_6_35b_a3b#single-node-pd-mixed).
## Functional verification
docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_next_80b_a3b_instruct.mdx
+1 -1
View File
@@ -152,7 +152,7 @@ quay.io/ascend/sglang:v0.5.13.post1-cann9.0.0-910b
Single-node deployment completes both prefill and decode within the same node (PD mixed mode), suitable for scenarios
with limited hardware resources. This scenario is already covered in the best practice. For the complete, optimized
deployment commands and benchmark data, see
[Qwen3-Next-80B-A3B-Instruct Best Practice — PD Mixed On A3](/docs/hardware-platforms/ascend-npus/best_practice/qwen3_next_80b_a3b_instruct#single-node).
[Qwen3-Next-80B-A3B-Instruct Best Practice — PD Mixed On A3](/docs/hardware-platforms/ascend-npus/best_practice/qwen3_next_80b_a3b_instruct#single-node-pd-mixed).
## Functional verification