From 7bd1a9d1638d5c21abf625f0e5ab5cfb77dff29d Mon Sep 17 00:00:00 2001 From: longxin9715 <59550463+longxin9715@users.noreply.github.com> Date: Mon, 15 Jun 2026 20:02:50 +0800 Subject: [PATCH] Update documentation for Ascend NPU Guide (#28284) --- .../ascend-npus/ascend_npu_optimization.mdx | 4 ++-- .../ascend-npus/ascend_npu_performance_testing.mdx | 7 ++++++- .../ascend-npus/ascend_npu_quick_start.mdx | 6 +++--- .../ascend-npus/ascend_npu_support_features.mdx | 2 +- 4 files changed, 12 insertions(+), 7 deletions(-) diff --git a/docs_new/docs/hardware-platforms/ascend-npus/ascend_npu_optimization.mdx b/docs_new/docs/hardware-platforms/ascend-npus/ascend_npu_optimization.mdx index 3a7fbbc7f..a560614e0 100644 --- a/docs_new/docs/hardware-platforms/ascend-npus/ascend_npu_optimization.mdx +++ b/docs_new/docs/hardware-platforms/ascend-npus/ascend_npu_optimization.mdx @@ -218,12 +218,12 @@ These arguments and environment variables are critical for tuning decode perform - `--dp-size` + `--dp-size` / `--data-parallel-size` Sets the data parallelism degree for the decode server. With DP attention enabled, attention layers are sharded across DP ranks while FFN/MoE layers use tensor parallelism. Higher values create more independent decode instances, increasing throughput through parallel request processing. Choose a value that divides evenly into your total card count, with remaining cards used for TP/EP. `8` - `--ep` + `--ep` / `--ep-size` / `--expert-parallel-size` Sets the expert parallelism degree. For MoE models, this distributes experts across cards, reducing per-card expert loading overhead and enabling all-to-all dispatch. The code default is `1`; set explicitly for MoE models. The optimal value depends on your model's expert count and architecture. DeepSeek V3.2 with 256 routed experts uses `ep=32`. For models with fewer experts, use a proportionally smaller value. `32` diff --git a/docs_new/docs/hardware-platforms/ascend-npus/ascend_npu_performance_testing.mdx b/docs_new/docs/hardware-platforms/ascend-npus/ascend_npu_performance_testing.mdx index 9c1d4c54a..7035dbd74 100644 --- a/docs_new/docs/hardware-platforms/ascend-npus/ascend_npu_performance_testing.mdx +++ b/docs_new/docs/hardware-platforms/ascend-npus/ascend_npu_performance_testing.mdx @@ -305,7 +305,12 @@ python -m sglang.bench_serving \ --max-concurrency 32 ``` -`--dataset-name random` samples token IDs from the ShareGPT dataset to generate realistic input; the first run downloads ShareGPT from Hugging Face automatically. Set `export HF_ENDPOINT=https://hf-mirror.com` if network is not available. + +`--dataset-name random` samples token IDs from the ShareGPT dataset to generate realistic input; the first run downloads ShareGPT from Hugging Face automatically. + +1. If you have network issues, set `export HF_ENDPOINT=https://hf-mirror.com` to use domestic mirror. +2. If downloading still fails, manually download the dataset file `ShareGPT_V3_unfiltered_cleaned_split.json` locally, upload it to your server, then specify the file directory via `--dataset-path` to run offline. + Set `--random-range-ratio 1` for fixed input/output lengths (recommended for consistent comparisons) or `0` (default) for uniform distribution. Add `--request-rate` to control the request rate. For all backends, datasets, and advanced options, see the full [Bench Serving Guide](/docs/developer_guide/bench_serving). diff --git a/docs_new/docs/hardware-platforms/ascend-npus/ascend_npu_quick_start.mdx b/docs_new/docs/hardware-platforms/ascend-npus/ascend_npu_quick_start.mdx index e7dcfbd4c..e3ab8cb10 100644 --- a/docs_new/docs/hardware-platforms/ascend-npus/ascend_npu_quick_start.mdx +++ b/docs_new/docs/hardware-platforms/ascend-npus/ascend_npu_quick_start.mdx @@ -18,7 +18,7 @@ Ensure sufficient disk space before proceeding. The Docker image requires at lea -We publish both **stable releases** and **daily builds**. Choose a stable release tag (e.g., `v0.5.10-npu.rc1-a3`) if you prefer a validated version, or a daily build tag (e.g., `main-cann9.0.0-a3`) if you need the latest development changes. +We publish both **stable releases** and **daily builds**. Choose a stable release tag (e.g., `v0.5.13.post1-cann9.0.0-a3`) if you prefer a validated version, or a daily build tag (e.g., `main-cann9.0.0-a3`) if you need the latest development changes. @@ -26,7 +26,7 @@ We publish both **stable releases** and **daily builds**. Choose a stable releas ```shell Command # Stable release -export IMAGE=quay.io/ascend/sglang:v0.5.10-npu.rc1-a3 +export IMAGE=quay.io/ascend/sglang:v0.5.13.post1-cann9.0.0-a3 # Daily build export IMAGE=quay.io/ascend/sglang:main-cann9.0.0-a3 @@ -53,7 +53,7 @@ docker run -it --rm --privileged --network=host --ipc=host --shm-size=16g \ ```shell Command # Stable release -export IMAGE=quay.io/ascend/sglang:v0.5.12-cann8.5.0-910b +export IMAGE=quay.io/ascend/sglang:v0.5.13.post1-cann9.0.0-910b # Daily build export IMAGE=quay.io/ascend/sglang:main-cann9.0.0-910b diff --git a/docs_new/docs/hardware-platforms/ascend-npus/ascend_npu_support_features.mdx b/docs_new/docs/hardware-platforms/ascend-npus/ascend_npu_support_features.mdx index ca10c07bb..9d0e9871e 100644 --- a/docs_new/docs/hardware-platforms/ascend-npus/ascend_npu_support_features.mdx +++ b/docs_new/docs/hardware-platforms/ascend-npus/ascend_npu_support_features.mdx @@ -1827,7 +1827,7 @@ click [Server Arguments](../../advanced_features/server_arguments). `--hicache-write-policy` `write_through` - Currently only `write_back` supported + `write_back`,
`write_through`,
`write_through_selective` A2, A3