[NPU] fix best practicce docs (#29303)
This commit is contained in:
@@ -4,7 +4,9 @@ metatags:
|
|||||||
description: "Best Practice for DeepSeek-R1 on Ascend NPU"
|
description: "Best Practice for DeepSeek-R1 on Ascend NPU"
|
||||||
---
|
---
|
||||||
|
|
||||||
This guide describes the best practice data for DeepSeek-R1 on the Ascend NPU.
|
<Note>
|
||||||
|
This page focuses on optimal configuration and benchmark results for DeepSeek-R1 on the Ascend NPU. For environment setup, model weight download, feature configuration, and deployment instructions, etc., see the [DeepSeek-R1 Model Tutorial](/docs/hardware-platforms/ascend-npus/model-tutorials/deepseek_r1).
|
||||||
|
</Note>
|
||||||
|
|
||||||
### Low Latency
|
### Low Latency
|
||||||
|
|
||||||
|
|||||||
@@ -4,7 +4,9 @@ metatags:
|
|||||||
description: "Best Practice for DeepSeek-V3.2 on Ascend NPU"
|
description: "Best Practice for DeepSeek-V3.2 on Ascend NPU"
|
||||||
---
|
---
|
||||||
|
|
||||||
This guide describes the best practice data for DeepSeek-V3.2 on the Ascend NPU.
|
<Note>
|
||||||
|
This page focuses on optimal configuration and benchmark results for DeepSeek-V3.2 on the Ascend NPU. For environment setup, model weight download, feature configuration, and deployment instructions, etc., see the [DeepSeek-V3.2 Model Tutorial](/docs/hardware-platforms/ascend-npus/model-tutorials/deepseek_v3_2).
|
||||||
|
</Note>
|
||||||
|
|
||||||
### Low Latency
|
### Low Latency
|
||||||
|
|
||||||
|
|||||||
@@ -4,7 +4,9 @@ metatags:
|
|||||||
description: "Best Practice for GLM-5.1 on Ascend NPU"
|
description: "Best Practice for GLM-5.1 on Ascend NPU"
|
||||||
---
|
---
|
||||||
|
|
||||||
This guide describes the best practice data for GLM-5.1 on the Ascend NPU.
|
<Note>
|
||||||
|
This page focuses on optimal configuration and benchmark results for GLM-5.1 on the Ascend NPU. For environment setup, model weight download, feature configuration, and deployment instructions, etc., see the [GLM-5.1 Model Tutorial](/docs/hardware-platforms/ascend-npus/model-tutorials/glm_5_1).
|
||||||
|
</Note>
|
||||||
|
|
||||||
### Low Latency
|
### Low Latency
|
||||||
|
|
||||||
|
|||||||
@@ -4,7 +4,9 @@ metatags:
|
|||||||
description: "Best Practice for Kimi-K2.6 on Ascend NPU"
|
description: "Best Practice for Kimi-K2.6 on Ascend NPU"
|
||||||
---
|
---
|
||||||
|
|
||||||
This guide describes the best practice data for Kimi-K2.6 on the Ascend NPU.
|
<Note>
|
||||||
|
This page focuses on optimal configuration and benchmark results for Kimi-K2.6 on the Ascend NPU. For environment setup, model weight download, feature configuration, and deployment instructions, etc., see the [Kimi-K2.6 Model Tutorial](/docs/hardware-platforms/ascend-npus/model-tutorials/kimi_k2_6).
|
||||||
|
</Note>
|
||||||
|
|
||||||
### Low Latency
|
### Low Latency
|
||||||
|
|
||||||
@@ -1006,7 +1008,7 @@ python3 -m sglang.launch_server \
|
|||||||
--host 127.0.0.1 --port 6688 \
|
--host 127.0.0.1 --port 6688 \
|
||||||
--quantization modelslim \
|
--quantization modelslim \
|
||||||
--dtype bfloat16 \
|
--dtype bfloat16 \
|
||||||
--model-loader-extra-config {"enable_multithread_load": true} \
|
--model-loader-extra-config '{"enable_multithread_load": true}' \
|
||||||
--trust-remote-code \
|
--trust-remote-code \
|
||||||
--device npu \
|
--device npu \
|
||||||
--attention-backend ascend \
|
--attention-backend ascend \
|
||||||
@@ -1116,7 +1118,7 @@ python3 -m sglang.launch_server \
|
|||||||
--host 127.0.0.1 --port 6688 \
|
--host 127.0.0.1 --port 6688 \
|
||||||
--quantization modelslim \
|
--quantization modelslim \
|
||||||
--dtype bfloat16 \
|
--dtype bfloat16 \
|
||||||
--model-loader-extra-config {"enable_multithread_load": true} \
|
--model-loader-extra-config '{"enable_multithread_load": true}' \
|
||||||
--trust-remote-code \
|
--trust-remote-code \
|
||||||
--device npu \
|
--device npu \
|
||||||
--attention-backend ascend \
|
--attention-backend ascend \
|
||||||
@@ -1249,7 +1251,7 @@ python3 -m sglang.launch_server \
|
|||||||
--deepep-mode auto \
|
--deepep-mode auto \
|
||||||
--cuda-graph-bs 1 2 3 4 5 \
|
--cuda-graph-bs 1 2 3 4 5 \
|
||||||
--disable-radix-cache \
|
--disable-radix-cache \
|
||||||
--model-loader-extra-config {"enable_multithread_load": true} \
|
--model-loader-extra-config '{"enable_multithread_load": true}' \
|
||||||
--speculative-algorithm EAGLE3 \
|
--speculative-algorithm EAGLE3 \
|
||||||
--speculative-draft-model-path $DRAFT_MODEL_PATH \
|
--speculative-draft-model-path $DRAFT_MODEL_PATH \
|
||||||
--speculative-num-steps 4 \
|
--speculative-num-steps 4 \
|
||||||
@@ -1358,7 +1360,7 @@ python3 -m sglang.launch_server \
|
|||||||
--deepep-mode auto \
|
--deepep-mode auto \
|
||||||
--cuda-graph-bs 1 2 4 8 12 13 \
|
--cuda-graph-bs 1 2 4 8 12 13 \
|
||||||
--disable-radix-cache \
|
--disable-radix-cache \
|
||||||
--model-loader-extra-config {"enable_multithread_load": true} \
|
--model-loader-extra-config '{"enable_multithread_load": true}' \
|
||||||
--speculative-algorithm EAGLE3 \
|
--speculative-algorithm EAGLE3 \
|
||||||
--speculative-draft-model-path $DRAFT_MODEL_PATH \
|
--speculative-draft-model-path $DRAFT_MODEL_PATH \
|
||||||
--speculative-num-steps 4 \
|
--speculative-num-steps 4 \
|
||||||
|
|||||||
@@ -4,7 +4,9 @@ metatags:
|
|||||||
description: "Best Practice for MiniMax-M2.5 on Ascend NPU"
|
description: "Best Practice for MiniMax-M2.5 on Ascend NPU"
|
||||||
---
|
---
|
||||||
|
|
||||||
This guide describes the best practice data for MiniMax-M2.5 on the Ascend NPU.
|
<Note>
|
||||||
|
This page focuses on optimal configuration and benchmark results for MiniMax-M2.5 on the Ascend NPU. For environment setup, model weight download, feature configuration, and deployment instructions, etc., see the [MiniMax-M2.5 Model Tutorial](/docs/hardware-platforms/ascend-npus/model-tutorials/minimax_m2_5).
|
||||||
|
</Note>
|
||||||
|
|
||||||
### Low Latency
|
### Low Latency
|
||||||
|
|
||||||
|
|||||||
@@ -4,7 +4,9 @@ metatags:
|
|||||||
description: "Best Practice for Qwen3-8B on Ascend NPU"
|
description: "Best Practice for Qwen3-8B on Ascend NPU"
|
||||||
---
|
---
|
||||||
|
|
||||||
This guide describes the best practice data for Qwen3-8B on the Ascend NPU.
|
<Note>
|
||||||
|
This page focuses on optimal configuration and benchmark results for Qwen3-8B on the Ascend NPU. For environment setup, model weight download, feature configuration, and deployment instructions, etc., see the [Qwen3-8B Model Tutorial](/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3-8b).
|
||||||
|
</Note>
|
||||||
|
|
||||||
### Low Latency
|
### Low Latency
|
||||||
|
|
||||||
|
|||||||
@@ -4,7 +4,9 @@ metatags:
|
|||||||
description: "Best Practice for Qwen3-235B-A22B on Ascend NPU"
|
description: "Best Practice for Qwen3-235B-A22B on Ascend NPU"
|
||||||
---
|
---
|
||||||
|
|
||||||
This guide describes the best practice data for Qwen3-235B-A22B on the Ascend NPU.
|
<Note>
|
||||||
|
This page focuses on optimal configuration and benchmark results for Qwen3-235B-A22B on the Ascend NPU. For environment setup, model weight download, feature configuration, and deployment instructions, etc., see the [Qwen3-235B-A22B Model Tutorial](/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_235b_a22b).
|
||||||
|
</Note>
|
||||||
|
|
||||||
### Low Latency
|
### Low Latency
|
||||||
|
|
||||||
@@ -192,7 +194,6 @@ python3 -m sglang.launch_server \
|
|||||||
--max-prefill-tokens 458880 \
|
--max-prefill-tokens 458880 \
|
||||||
--sampling-backend ascend \
|
--sampling-backend ascend \
|
||||||
--ep-dispatch-algorithm static \
|
--ep-dispatch-algorithm static \
|
||||||
--init-expert-location /root/.cache/modelscope/hub/models/hot_map/235B_3_5k_decode.pt \
|
|
||||||
--disable-radix-cache \
|
--disable-radix-cache \
|
||||||
--moe-a2a-backend ascend_fuseep \
|
--moe-a2a-backend ascend_fuseep \
|
||||||
--speculative-algorithm EAGLE3 \
|
--speculative-algorithm EAGLE3 \
|
||||||
|
|||||||
@@ -4,7 +4,9 @@ metatags:
|
|||||||
description: "Best Practice for Qwen3-30B-A3B on Ascend NPU"
|
description: "Best Practice for Qwen3-30B-A3B on Ascend NPU"
|
||||||
---
|
---
|
||||||
|
|
||||||
This guide describes the best practice data for Qwen3-30B-A3B on the Ascend NPU.
|
<Note>
|
||||||
|
This page focuses on optimal configuration and benchmark results for Qwen3-30B-A3B on the Ascend NPU. For environment setup, model weight download, feature configuration, and deployment instructions, etc., see the [Qwen3-30B-A3B Model Tutorial](/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_30b_a3b).
|
||||||
|
</Note>
|
||||||
|
|
||||||
### Low Latency
|
### Low Latency
|
||||||
|
|
||||||
@@ -71,7 +73,7 @@ source /usr/local/Ascend/nnal/atb/set_env.sh
|
|||||||
export ASCEND_LAUNCH_BLOCKING=0
|
export ASCEND_LAUNCH_BLOCKING=0
|
||||||
export DP_ROUND_ROBIN=1
|
export DP_ROUND_ROBIN=1
|
||||||
export GLOO_SOCKET_IFNAME=<network-interface>
|
export GLOO_SOCKET_IFNAME=<network-interface>
|
||||||
export HCCL_ALGO=level0:NA;level1:ring
|
export HCCL_ALGO="level0:NA;level1:ring"
|
||||||
export HCCL_SOCKET_IFNAME=<network-interface>
|
export HCCL_SOCKET_IFNAME=<network-interface>
|
||||||
export INF_NAN_MODE_FORCE_DISABLE=1
|
export INF_NAN_MODE_FORCE_DISABLE=1
|
||||||
export PYTORCH_NPU_ALLOC_CONF=expandable_segments:False
|
export PYTORCH_NPU_ALLOC_CONF=expandable_segments:False
|
||||||
|
|||||||
@@ -4,7 +4,9 @@ metatags:
|
|||||||
description: "Best Practice for Qwen3-32B on Ascend NPU"
|
description: "Best Practice for Qwen3-32B on Ascend NPU"
|
||||||
---
|
---
|
||||||
|
|
||||||
This guide describes the best practice data for Qwen3-32B on the Ascend NPU.
|
<Note>
|
||||||
|
This page focuses on optimal configuration and benchmark results for Qwen3-32B on the Ascend NPU. For environment setup, model weight download, feature configuration, and deployment instructions, etc., see the [Qwen3-32B Model Tutorial](/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_32b).
|
||||||
|
</Note>
|
||||||
|
|
||||||
### Low Latency
|
### Low Latency
|
||||||
|
|
||||||
|
|||||||
@@ -4,7 +4,9 @@ metatags:
|
|||||||
description: "Best Practice for Qwen3.5-397B on Ascend NPU"
|
description: "Best Practice for Qwen3.5-397B on Ascend NPU"
|
||||||
---
|
---
|
||||||
|
|
||||||
This guide describes the best practice data for Qwen3.5-397B on the Ascend NPU.
|
<Note>
|
||||||
|
This page focuses on optimal configuration and benchmark results for Qwen3.5-397B on the Ascend NPU. For environment setup, model weight download, feature configuration, and deployment instructions, etc., see the [Qwen3.5-397B Model Tutorial](/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_5_397b).
|
||||||
|
</Note>
|
||||||
|
|
||||||
### Low Latency
|
### Low Latency
|
||||||
|
|
||||||
|
|||||||
@@ -4,7 +4,9 @@ metatags:
|
|||||||
description: "Best Practice for Qwen3.6-27B on Ascend NPU"
|
description: "Best Practice for Qwen3.6-27B on Ascend NPU"
|
||||||
---
|
---
|
||||||
|
|
||||||
This guide describes the best practice data for Qwen3.6-27B on the Ascend NPU.
|
<Note>
|
||||||
|
This page focuses on optimal configuration and benchmark results for Qwen3.6-27B on the Ascend NPU. For environment setup, model weight download, feature configuration, and deployment instructions, etc., see the [Qwen3.6-27B Model Tutorial](/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_6_27b).
|
||||||
|
</Note>
|
||||||
|
|
||||||
### High Throughput
|
### High Throughput
|
||||||
|
|
||||||
|
|||||||
@@ -4,7 +4,9 @@ metatags:
|
|||||||
description: "Best Practice for Qwen3.6-35B-A3B on Ascend NPU"
|
description: "Best Practice for Qwen3.6-35B-A3B on Ascend NPU"
|
||||||
---
|
---
|
||||||
|
|
||||||
This guide describes the best practice data for Qwen3.6-35B-A3B on the Ascend NPU.
|
<Note>
|
||||||
|
This page focuses on optimal configuration and benchmark results for Qwen3.6-35B-A3B on the Ascend NPU. For environment setup, model weight download, feature configuration, and deployment instructions, etc., see the [Qwen3.6-35B-A3B Model Tutorial](/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_6_35b_a3b).
|
||||||
|
</Note>
|
||||||
|
|
||||||
### Low Latency
|
### Low Latency
|
||||||
|
|
||||||
|
|||||||
+3
-1
@@ -4,7 +4,9 @@ metatags:
|
|||||||
description: "Best Practice for Qwen3-Next-80B-A3B-Instruct on Ascend NPU"
|
description: "Best Practice for Qwen3-Next-80B-A3B-Instruct on Ascend NPU"
|
||||||
---
|
---
|
||||||
|
|
||||||
This guide describes the best practice data for Qwen3-Next-80B-A3B-Instruct on the Ascend NPU.
|
<Note>
|
||||||
|
This page focuses on optimal configuration and benchmark results for Qwen3-Next-80B-A3B-Instruct on the Ascend NPU. For environment setup, model weight download, feature configuration, and deployment instructions, etc., see the [Qwen3-Next-80B-A3B-Instruct Model Tutorial](/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_next_80b_a3b_instruct).
|
||||||
|
</Note>
|
||||||
|
|
||||||
### Low Latency
|
### Low Latency
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user