[NPU] [DOC] Polish English wording in NPU docs (#36123)
This commit is contained in:
@@ -235,7 +235,7 @@ After locating the divergent node (e.g., a specific Conv layer or torch API with
|
|||||||
|
|
||||||
### No Dump Files Generated
|
### No Dump Files Generated
|
||||||
|
|
||||||
1. To confirm if MSProbe is installed, use `pip show mindstudio_probe` to troubleshoot. If it is installed, the MSProbe
|
1. To confirm if MSProbe is installed, use `pip show mindstudio-probe` to troubleshoot. If it is installed, the MSProbe
|
||||||
version information will be printed. If it is confirmed that it has not been installed, please
|
version information will be printed. If it is confirmed that it has not been installed, please
|
||||||
use `pip install mindstudio-probe --pre` for installation;
|
use `pip install mindstudio-probe --pre` for installation;
|
||||||
2. Confirm the `--msprobe-dump-config` parameter points to the **correct JSON file path**.
|
2. Confirm the `--msprobe-dump-config` parameter points to the **correct JSON file path**.
|
||||||
|
|||||||
@@ -12,7 +12,7 @@ SGLang. It also covers how to test new models and register external implementati
|
|||||||
To support a new model in SGLang, you only need to add a single file under
|
To support a new model in SGLang, you only need to add a single file under
|
||||||
the [SGLang Models Directory](https://github.com/sgl-project/sglang/tree/main/python/sglang/srt/models). You can learn
|
the [SGLang Models Directory](https://github.com/sgl-project/sglang/tree/main/python/sglang/srt/models). You can learn
|
||||||
from existing model implementations and create a new file for your model. For most models, you should be able to find a
|
from existing model implementations and create a new file for your model. For most models, you should be able to find a
|
||||||
similar model to start with (e.g., starting from Llama). Also refer how
|
similar model to start with (e.g., starting from Llama). Also refer to how
|
||||||
to [port a Model from vLLM to SGLang](#port-a-model-from-vllm-to-sglang)
|
to [port a Model from vLLM to SGLang](#port-a-model-from-vllm-to-sglang)
|
||||||
|
|
||||||
NPU adaptations are embedded in existing model files (e.g., `llama.py`, `qwen3_vl.py`) through `_is_npu` conditional
|
NPU adaptations are embedded in existing model files (e.g., `llama.py`, `qwen3_vl.py`) through `_is_npu` conditional
|
||||||
|
|||||||
@@ -16,7 +16,7 @@ You can install SGLang using any of the methods below. Please go through `System
|
|||||||
<tr style={{borderBottom: "2px solid #d55816"}}>
|
<tr style={{borderBottom: "2px solid #d55816"}}>
|
||||||
<th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Component</th>
|
<th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Component</th>
|
||||||
<th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>Version</th>
|
<th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>Version</th>
|
||||||
<th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Obtain Way</th>
|
<th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Obtain Method</th>
|
||||||
</tr>
|
</tr>
|
||||||
</thead>
|
</thead>
|
||||||
<tbody>
|
<tbody>
|
||||||
@@ -119,7 +119,7 @@ conda activate sglang_npu
|
|||||||
|
|
||||||
#### CANN
|
#### CANN
|
||||||
|
|
||||||
Prior to start work with SGLang on Ascend you need to install CANN Toolkit, Kernels operator package and NNAL version 9.0.0, check the [installation guide](https://www.hiascend.com/document/detail/zh/CANNCommunityEdition/900/softwareinst/instg/instg_0008.html?OS=openEuler&InstallType=local)
|
Prior to starting work with SGLang on Ascend, you need to install CANN Toolkit, Kernels operator package and NNAL version 9.0.0, check the [installation guide](https://www.hiascend.com/document/detail/zh/CANNCommunityEdition/900/softwareinst/instg/instg_0008.html?OS=openEuler&InstallType=local)
|
||||||
|
|
||||||
#### MemFabric-Hybrid
|
#### MemFabric-Hybrid
|
||||||
|
|
||||||
@@ -140,7 +140,7 @@ installed in addition to MemFabric-Hybrid.
|
|||||||
pip install memfabric-zbal==1.1.1
|
pip install memfabric-zbal==1.1.1
|
||||||
```
|
```
|
||||||
|
|
||||||
#### PyTorch and PyTorch Framework Adaptor on Ascend
|
#### PyTorch and PyTorch Framework Adapter on Ascend
|
||||||
|
|
||||||
```bash Command
|
```bash Command
|
||||||
PYTORCH_VERSION=2.10.0
|
PYTORCH_VERSION=2.10.0
|
||||||
@@ -270,7 +270,7 @@ alias drun='docker run -it --rm --privileged --network=host --ipc=host --shm-siz
|
|||||||
--volume /etc/ascend_install.info:/etc/ascend_install.info \
|
--volume /etc/ascend_install.info:/etc/ascend_install.info \
|
||||||
--volume /var/queue_schedule:/var/queue_schedule --volume ~/.cache/:/root/.cache/'
|
--volume /var/queue_schedule:/var/queue_schedule --volume ~/.cache/:/root/.cache/'
|
||||||
|
|
||||||
# Add HF_TOKEN env for download model by SGLang.
|
# Add HF_TOKEN env for downloading models by SGLang.
|
||||||
# The container runs with the '--rm' flag, so it will be automatically removed after the command finishes (including Ctrl+C)
|
# The container runs with the '--rm' flag, so it will be automatically removed after the command finishes (including Ctrl+C)
|
||||||
drun --env "HF_TOKEN=<secret>" \
|
drun --env "HF_TOKEN=<secret>" \
|
||||||
<image_name> \
|
<image_name> \
|
||||||
@@ -291,7 +291,7 @@ alias drun='docker run -it --rm --privileged --network=host --ipc=host --shm-siz
|
|||||||
--volume /etc/ascend_install.info:/etc/ascend_install.info \
|
--volume /etc/ascend_install.info:/etc/ascend_install.info \
|
||||||
--volume /var/queue_schedule:/var/queue_schedule --volume ~/.cache/:/root/.cache/'
|
--volume /var/queue_schedule:/var/queue_schedule --volume ~/.cache/:/root/.cache/'
|
||||||
|
|
||||||
# Add HF_TOKEN env for download model by SGLang.
|
# Add HF_TOKEN env for downloading models by SGLang.
|
||||||
# The container runs with the '--rm' flag, so it will be automatically removed after the command finishes (including Ctrl+C)
|
# The container runs with the '--rm' flag, so it will be automatically removed after the command finishes (including Ctrl+C)
|
||||||
drun --env "HF_TOKEN=<secret>" \
|
drun --env "HF_TOKEN=<secret>" \
|
||||||
<image_name> \
|
<image_name> \
|
||||||
@@ -539,7 +539,7 @@ curl http://127.0.0.1:30000/v1/chat/completions \
|
|||||||
}'
|
}'
|
||||||
```
|
```
|
||||||
|
|
||||||
Some models return responses accompanied with thinking process content. To disable this output, configure parameters as follows:
|
Some models return responses accompanied by thinking process content. To disable this output, configure parameters as follows:
|
||||||
|
|
||||||
```bash Command
|
```bash Command
|
||||||
curl http://127.0.0.1:30000/v1/chat/completions \
|
curl http://127.0.0.1:30000/v1/chat/completions \
|
||||||
|
|||||||
@@ -19,7 +19,7 @@ This document provides a list of commonly used environment variables and aims to
|
|||||||
<tbody>
|
<tbody>
|
||||||
<tr>
|
<tr>
|
||||||
<td><code>SGLANG_NPU_USE_MLAPO</code></td>
|
<td><code>SGLANG_NPU_USE_MLAPO</code></td>
|
||||||
<td>Adopts the <code>MLAPO</code> fusion operator in attention <br/> preprocessing stage of the MLA model.</td>
|
<td>Adopts the <code>MLAPO</code> fusion operator in the attention <br/> preprocessing stage of the MLA model.</td>
|
||||||
<td><code>false</code></td>
|
<td><code>false</code></td>
|
||||||
</tr>
|
</tr>
|
||||||
<tr>
|
<tr>
|
||||||
@@ -34,7 +34,7 @@ This document provides a list of commonly used environment variables and aims to
|
|||||||
</tr>
|
</tr>
|
||||||
<tr>
|
<tr>
|
||||||
<td><code>SGLANG_NPU_DISABLE_ACL_FORMAT_WEIGHT</code></td>
|
<td><code>SGLANG_NPU_DISABLE_ACL_FORMAT_WEIGHT</code></td>
|
||||||
<td>Disable cast model weight tensor to a specific NPU <br/> ACL format.</td>
|
<td>Disable casting model weight tensor to a specific NPU <br/> ACL format.</td>
|
||||||
<td><code>false</code></td>
|
<td><code>false</code></td>
|
||||||
</tr>
|
</tr>
|
||||||
<tr>
|
<tr>
|
||||||
@@ -109,7 +109,7 @@ This document provides a list of commonly used environment variables and aims to
|
|||||||
<tbody>
|
<tbody>
|
||||||
<tr>
|
<tr>
|
||||||
<td><code>TASK_QUEUE_ENABLE</code></td>
|
<td><code>TASK_QUEUE_ENABLE</code></td>
|
||||||
<td>Used to control the optimization level of the dispatch queue<br/> about the task_queue operator. <a href="https://www.hiascend.com/document/detail/zh/Pytorch/730/comref/Envvariables/docs/zh/environment_variable_reference/TASK_QUEUE_ENABLE.md">Detail</a></td>
|
<td>Used to control the optimization level of the dispatch queue<br/> for the task_queue operator. <a href="https://www.hiascend.com/document/detail/zh/Pytorch/730/comref/Envvariables/docs/zh/environment_variable_reference/TASK_QUEUE_ENABLE.md">Detail</a></td>
|
||||||
<td><code>1</code></td>
|
<td><code>1</code></td>
|
||||||
</tr>
|
</tr>
|
||||||
<tr>
|
<tr>
|
||||||
|
|||||||
Reference in New Issue
Block a user