[NPU] fix best practicce docs (#29303)

This commit is contained in:
jianzhao-xu
2026-06-26 11:17:50 +08:00
committed by GitHub
parent 30ea4c0f4b
commit cc294829aa
13 changed files with 44 additions and 19 deletions
@@ -4,7 +4,9 @@ metatags:
description: "Best Practice for DeepSeek-R1 on Ascend NPU" description: "Best Practice for DeepSeek-R1 on Ascend NPU"
--- ---
This guide describes the best practice data for DeepSeek-R1 on the Ascend NPU. <Note>
This page focuses on optimal configuration and benchmark results for DeepSeek-R1 on the Ascend NPU. For environment setup, model weight download, feature configuration, and deployment instructions, etc., see the [DeepSeek-R1 Model Tutorial](/docs/hardware-platforms/ascend-npus/model-tutorials/deepseek_r1).
</Note>
### Low Latency ### Low Latency
@@ -4,7 +4,9 @@ metatags:
description: "Best Practice for DeepSeek-V3.2 on Ascend NPU" description: "Best Practice for DeepSeek-V3.2 on Ascend NPU"
--- ---
This guide describes the best practice data for DeepSeek-V3.2 on the Ascend NPU. <Note>
This page focuses on optimal configuration and benchmark results for DeepSeek-V3.2 on the Ascend NPU. For environment setup, model weight download, feature configuration, and deployment instructions, etc., see the [DeepSeek-V3.2 Model Tutorial](/docs/hardware-platforms/ascend-npus/model-tutorials/deepseek_v3_2).
</Note>
### Low Latency ### Low Latency
@@ -4,7 +4,9 @@ metatags:
description: "Best Practice for GLM-5.1 on Ascend NPU" description: "Best Practice for GLM-5.1 on Ascend NPU"
--- ---
This guide describes the best practice data for GLM-5.1 on the Ascend NPU. <Note>
This page focuses on optimal configuration and benchmark results for GLM-5.1 on the Ascend NPU. For environment setup, model weight download, feature configuration, and deployment instructions, etc., see the [GLM-5.1 Model Tutorial](/docs/hardware-platforms/ascend-npus/model-tutorials/glm_5_1).
</Note>
### Low Latency ### Low Latency
@@ -4,7 +4,9 @@ metatags:
description: "Best Practice for Kimi-K2.6 on Ascend NPU" description: "Best Practice for Kimi-K2.6 on Ascend NPU"
--- ---
This guide describes the best practice data for Kimi-K2.6 on the Ascend NPU. <Note>
This page focuses on optimal configuration and benchmark results for Kimi-K2.6 on the Ascend NPU. For environment setup, model weight download, feature configuration, and deployment instructions, etc., see the [Kimi-K2.6 Model Tutorial](/docs/hardware-platforms/ascend-npus/model-tutorials/kimi_k2_6).
</Note>
### Low Latency ### Low Latency
@@ -1006,7 +1008,7 @@ python3 -m sglang.launch_server \
--host 127.0.0.1 --port 6688 \ --host 127.0.0.1 --port 6688 \
--quantization modelslim \ --quantization modelslim \
--dtype bfloat16 \ --dtype bfloat16 \
--model-loader-extra-config {"enable_multithread_load": true} \ --model-loader-extra-config '{"enable_multithread_load": true}' \
--trust-remote-code \ --trust-remote-code \
--device npu \ --device npu \
--attention-backend ascend \ --attention-backend ascend \
@@ -1116,7 +1118,7 @@ python3 -m sglang.launch_server \
--host 127.0.0.1 --port 6688 \ --host 127.0.0.1 --port 6688 \
--quantization modelslim \ --quantization modelslim \
--dtype bfloat16 \ --dtype bfloat16 \
--model-loader-extra-config {"enable_multithread_load": true} \ --model-loader-extra-config '{"enable_multithread_load": true}' \
--trust-remote-code \ --trust-remote-code \
--device npu \ --device npu \
--attention-backend ascend \ --attention-backend ascend \
@@ -1249,7 +1251,7 @@ python3 -m sglang.launch_server \
--deepep-mode auto \ --deepep-mode auto \
--cuda-graph-bs 1 2 3 4 5 \ --cuda-graph-bs 1 2 3 4 5 \
--disable-radix-cache \ --disable-radix-cache \
--model-loader-extra-config {"enable_multithread_load": true} \ --model-loader-extra-config '{"enable_multithread_load": true}' \
--speculative-algorithm EAGLE3 \ --speculative-algorithm EAGLE3 \
--speculative-draft-model-path $DRAFT_MODEL_PATH \ --speculative-draft-model-path $DRAFT_MODEL_PATH \
--speculative-num-steps 4 \ --speculative-num-steps 4 \
@@ -1358,7 +1360,7 @@ python3 -m sglang.launch_server \
--deepep-mode auto \ --deepep-mode auto \
--cuda-graph-bs 1 2 4 8 12 13 \ --cuda-graph-bs 1 2 4 8 12 13 \
--disable-radix-cache \ --disable-radix-cache \
--model-loader-extra-config {"enable_multithread_load": true} \ --model-loader-extra-config '{"enable_multithread_load": true}' \
--speculative-algorithm EAGLE3 \ --speculative-algorithm EAGLE3 \
--speculative-draft-model-path $DRAFT_MODEL_PATH \ --speculative-draft-model-path $DRAFT_MODEL_PATH \
--speculative-num-steps 4 \ --speculative-num-steps 4 \
@@ -4,7 +4,9 @@ metatags:
description: "Best Practice for MiniMax-M2.5 on Ascend NPU" description: "Best Practice for MiniMax-M2.5 on Ascend NPU"
--- ---
This guide describes the best practice data for MiniMax-M2.5 on the Ascend NPU. <Note>
This page focuses on optimal configuration and benchmark results for MiniMax-M2.5 on the Ascend NPU. For environment setup, model weight download, feature configuration, and deployment instructions, etc., see the [MiniMax-M2.5 Model Tutorial](/docs/hardware-platforms/ascend-npus/model-tutorials/minimax_m2_5).
</Note>
### Low Latency ### Low Latency
@@ -4,7 +4,9 @@ metatags:
description: "Best Practice for Qwen3-8B on Ascend NPU" description: "Best Practice for Qwen3-8B on Ascend NPU"
--- ---
This guide describes the best practice data for Qwen3-8B on the Ascend NPU. <Note>
This page focuses on optimal configuration and benchmark results for Qwen3-8B on the Ascend NPU. For environment setup, model weight download, feature configuration, and deployment instructions, etc., see the [Qwen3-8B Model Tutorial](/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3-8b).
</Note>
### Low Latency ### Low Latency
@@ -4,7 +4,9 @@ metatags:
description: "Best Practice for Qwen3-235B-A22B on Ascend NPU" description: "Best Practice for Qwen3-235B-A22B on Ascend NPU"
--- ---
This guide describes the best practice data for Qwen3-235B-A22B on the Ascend NPU. <Note>
This page focuses on optimal configuration and benchmark results for Qwen3-235B-A22B on the Ascend NPU. For environment setup, model weight download, feature configuration, and deployment instructions, etc., see the [Qwen3-235B-A22B Model Tutorial](/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_235b_a22b).
</Note>
### Low Latency ### Low Latency
@@ -192,7 +194,6 @@ python3 -m sglang.launch_server \
--max-prefill-tokens 458880 \ --max-prefill-tokens 458880 \
--sampling-backend ascend \ --sampling-backend ascend \
--ep-dispatch-algorithm static \ --ep-dispatch-algorithm static \
--init-expert-location /root/.cache/modelscope/hub/models/hot_map/235B_3_5k_decode.pt \
--disable-radix-cache \ --disable-radix-cache \
--moe-a2a-backend ascend_fuseep \ --moe-a2a-backend ascend_fuseep \
--speculative-algorithm EAGLE3 \ --speculative-algorithm EAGLE3 \
@@ -4,7 +4,9 @@ metatags:
description: "Best Practice for Qwen3-30B-A3B on Ascend NPU" description: "Best Practice for Qwen3-30B-A3B on Ascend NPU"
--- ---
This guide describes the best practice data for Qwen3-30B-A3B on the Ascend NPU. <Note>
This page focuses on optimal configuration and benchmark results for Qwen3-30B-A3B on the Ascend NPU. For environment setup, model weight download, feature configuration, and deployment instructions, etc., see the [Qwen3-30B-A3B Model Tutorial](/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_30b_a3b).
</Note>
### Low Latency ### Low Latency
@@ -71,7 +73,7 @@ source /usr/local/Ascend/nnal/atb/set_env.sh
export ASCEND_LAUNCH_BLOCKING=0 export ASCEND_LAUNCH_BLOCKING=0
export DP_ROUND_ROBIN=1 export DP_ROUND_ROBIN=1
export GLOO_SOCKET_IFNAME=<network-interface> export GLOO_SOCKET_IFNAME=<network-interface>
export HCCL_ALGO=level0:NA;level1:ring export HCCL_ALGO="level0:NA;level1:ring"
export HCCL_SOCKET_IFNAME=<network-interface> export HCCL_SOCKET_IFNAME=<network-interface>
export INF_NAN_MODE_FORCE_DISABLE=1 export INF_NAN_MODE_FORCE_DISABLE=1
export PYTORCH_NPU_ALLOC_CONF=expandable_segments:False export PYTORCH_NPU_ALLOC_CONF=expandable_segments:False
@@ -4,7 +4,9 @@ metatags:
description: "Best Practice for Qwen3-32B on Ascend NPU" description: "Best Practice for Qwen3-32B on Ascend NPU"
--- ---
This guide describes the best practice data for Qwen3-32B on the Ascend NPU. <Note>
This page focuses on optimal configuration and benchmark results for Qwen3-32B on the Ascend NPU. For environment setup, model weight download, feature configuration, and deployment instructions, etc., see the [Qwen3-32B Model Tutorial](/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_32b).
</Note>
### Low Latency ### Low Latency
@@ -4,7 +4,9 @@ metatags:
description: "Best Practice for Qwen3.5-397B on Ascend NPU" description: "Best Practice for Qwen3.5-397B on Ascend NPU"
--- ---
This guide describes the best practice data for Qwen3.5-397B on the Ascend NPU. <Note>
This page focuses on optimal configuration and benchmark results for Qwen3.5-397B on the Ascend NPU. For environment setup, model weight download, feature configuration, and deployment instructions, etc., see the [Qwen3.5-397B Model Tutorial](/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_5_397b).
</Note>
### Low Latency ### Low Latency
@@ -4,7 +4,9 @@ metatags:
description: "Best Practice for Qwen3.6-27B on Ascend NPU" description: "Best Practice for Qwen3.6-27B on Ascend NPU"
--- ---
This guide describes the best practice data for Qwen3.6-27B on the Ascend NPU. <Note>
This page focuses on optimal configuration and benchmark results for Qwen3.6-27B on the Ascend NPU. For environment setup, model weight download, feature configuration, and deployment instructions, etc., see the [Qwen3.6-27B Model Tutorial](/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_6_27b).
</Note>
### High Throughput ### High Throughput
@@ -4,7 +4,9 @@ metatags:
description: "Best Practice for Qwen3.6-35B-A3B on Ascend NPU" description: "Best Practice for Qwen3.6-35B-A3B on Ascend NPU"
--- ---
This guide describes the best practice data for Qwen3.6-35B-A3B on the Ascend NPU. <Note>
This page focuses on optimal configuration and benchmark results for Qwen3.6-35B-A3B on the Ascend NPU. For environment setup, model weight download, feature configuration, and deployment instructions, etc., see the [Qwen3.6-35B-A3B Model Tutorial](/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_6_35b_a3b).
</Note>
### Low Latency ### Low Latency
@@ -4,7 +4,9 @@ metatags:
description: "Best Practice for Qwen3-Next-80B-A3B-Instruct on Ascend NPU" description: "Best Practice for Qwen3-Next-80B-A3B-Instruct on Ascend NPU"
--- ---
This guide describes the best practice data for Qwen3-Next-80B-A3B-Instruct on the Ascend NPU. <Note>
This page focuses on optimal configuration and benchmark results for Qwen3-Next-80B-A3B-Instruct on the Ascend NPU. For environment setup, model weight download, feature configuration, and deployment instructions, etc., see the [Qwen3-Next-80B-A3B-Instruct Model Tutorial](/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_next_80b_a3b_instruct).
</Note>
### Low Latency ### Low Latency