[NPU] [DOC] Add environment prerequisites to model tutorials (#29293)
This commit is contained in:
@@ -124,6 +124,17 @@ If you want to use PD disaggregation mode, you need to install MemFabric-Hybrid.
|
|||||||
pip install memfabric-hybrid==1.0.8
|
pip install memfabric-hybrid==1.0.8
|
||||||
```
|
```
|
||||||
|
|
||||||
|
#### MemFabric-zbal
|
||||||
|
|
||||||
|
MemFabric-zbal is a Zero Buffer Acceleration Library of high-performance operators for LLM inference and training on Ascend,
|
||||||
|
accelerating computation by eliminating intermediate memory buffers; it is required only on **aarch64** clusters and is
|
||||||
|
installed in addition to MemFabric-Hybrid.
|
||||||
|
|
||||||
|
```bash Command
|
||||||
|
# Only needed on aarch64 (arm64) hosts
|
||||||
|
pip install memfabric-zbal==1.1.1
|
||||||
|
```
|
||||||
|
|
||||||
#### Pytorch and Pytorch Framework Adaptor on Ascend
|
#### Pytorch and Pytorch Framework Adaptor on Ascend
|
||||||
|
|
||||||
```bash Command
|
```bash Command
|
||||||
|
|||||||
@@ -47,6 +47,18 @@ see [Feature Compatibility](/docs/hardware-platforms/ascend-npus/ascend_npu_opti
|
|||||||
|
|
||||||
## Prerequisites
|
## Prerequisites
|
||||||
|
|
||||||
|
### Environment
|
||||||
|
|
||||||
|
Before following this tutorial, complete the environment setup in the documents below:
|
||||||
|
|
||||||
|
- [Ascend NPU Quickstart](/docs/hardware-platforms/ascend-npus/ascend_npu_quick_start) — the fastest way to get started.
|
||||||
|
It walks you through launching the official container image, starting the SGLang server, and sending a test request.
|
||||||
|
Recommended if you are new to SGLang on Ascend.
|
||||||
|
- [SGLang Installation with NPU Support](/docs/hardware-platforms/ascend-npus/ascend_npu) — the full installation guide.
|
||||||
|
It covers the component version mapping (CANN, PyTorch adapter, Triton, kernels, etc.), building from source or from a
|
||||||
|
Dockerfile, and recommended system settings (CPU power scheme, NUMA, swap). Use it when you need to install or customize
|
||||||
|
the environment instead of using the official image.
|
||||||
|
|
||||||
### Model weights
|
### Model weights
|
||||||
|
|
||||||
<Warning>
|
<Warning>
|
||||||
|
|||||||
@@ -46,6 +46,18 @@ see [Feature Compatibility](/docs/hardware-platforms/ascend-npus/ascend_npu_opti
|
|||||||
|
|
||||||
## Prerequisites
|
## Prerequisites
|
||||||
|
|
||||||
|
### Environment
|
||||||
|
|
||||||
|
Before following this tutorial, complete the environment setup in the documents below:
|
||||||
|
|
||||||
|
- [Ascend NPU Quickstart](/docs/hardware-platforms/ascend-npus/ascend_npu_quick_start) — the fastest way to get started.
|
||||||
|
It walks you through launching the official container image, starting the SGLang server, and sending a test request.
|
||||||
|
Recommended if you are new to SGLang on Ascend.
|
||||||
|
- [SGLang Installation with NPU Support](/docs/hardware-platforms/ascend-npus/ascend_npu) — the full installation guide.
|
||||||
|
It covers the component version mapping (CANN, PyTorch adapter, Triton, kernels, etc.), building from source or from a
|
||||||
|
Dockerfile, and recommended system settings (CPU power scheme, NUMA, swap). Use it when you need to install or customize
|
||||||
|
the environment instead of using the official image.
|
||||||
|
|
||||||
### Model weights
|
### Model weights
|
||||||
|
|
||||||
<Warning>
|
<Warning>
|
||||||
|
|||||||
@@ -48,6 +48,18 @@ see [Feature Compatibility](/docs/hardware-platforms/ascend-npus/ascend_npu_opti
|
|||||||
|
|
||||||
## Prerequisites
|
## Prerequisites
|
||||||
|
|
||||||
|
### Environment
|
||||||
|
|
||||||
|
Before following this tutorial, complete the environment setup in the documents below:
|
||||||
|
|
||||||
|
- [Ascend NPU Quickstart](/docs/hardware-platforms/ascend-npus/ascend_npu_quick_start) — the fastest way to get started.
|
||||||
|
It walks you through launching the official container image, starting the SGLang server, and sending a test request.
|
||||||
|
Recommended if you are new to SGLang on Ascend.
|
||||||
|
- [SGLang Installation with NPU Support](/docs/hardware-platforms/ascend-npus/ascend_npu) — the full installation guide.
|
||||||
|
It covers the component version mapping (CANN, PyTorch adapter, Triton, kernels, etc.), building from source or from a
|
||||||
|
Dockerfile, and recommended system settings (CPU power scheme, NUMA, swap). Use it when you need to install or customize
|
||||||
|
the environment instead of using the official image.
|
||||||
|
|
||||||
### Model weights
|
### Model weights
|
||||||
|
|
||||||
<Warning>
|
<Warning>
|
||||||
|
|||||||
@@ -48,6 +48,18 @@ see [Feature Compatibility](/docs/hardware-platforms/ascend-npus/ascend_npu_opti
|
|||||||
|
|
||||||
## Prerequisites
|
## Prerequisites
|
||||||
|
|
||||||
|
### Environment
|
||||||
|
|
||||||
|
Before following this tutorial, complete the environment setup in the documents below:
|
||||||
|
|
||||||
|
- [Ascend NPU Quickstart](/docs/hardware-platforms/ascend-npus/ascend_npu_quick_start) — the fastest way to get started.
|
||||||
|
It walks you through launching the official container image, starting the SGLang server, and sending a test request.
|
||||||
|
Recommended if you are new to SGLang on Ascend.
|
||||||
|
- [SGLang Installation with NPU Support](/docs/hardware-platforms/ascend-npus/ascend_npu) — the full installation guide.
|
||||||
|
It covers the component version mapping (CANN, PyTorch adapter, Triton, kernels, etc.), building from source or from a
|
||||||
|
Dockerfile, and recommended system settings (CPU power scheme, NUMA, swap). Use it when you need to install or customize
|
||||||
|
the environment instead of using the official image.
|
||||||
|
|
||||||
### Model weights
|
### Model weights
|
||||||
|
|
||||||
<Warning>
|
<Warning>
|
||||||
|
|||||||
@@ -45,6 +45,18 @@ see [Feature Compatibility](/docs/hardware-platforms/ascend-npus/ascend_npu_opti
|
|||||||
|
|
||||||
## Prerequisites
|
## Prerequisites
|
||||||
|
|
||||||
|
### Environment
|
||||||
|
|
||||||
|
Before following this tutorial, complete the environment setup in the documents below:
|
||||||
|
|
||||||
|
- [Ascend NPU Quickstart](/docs/hardware-platforms/ascend-npus/ascend_npu_quick_start) — the fastest way to get started.
|
||||||
|
It walks you through launching the official container image, starting the SGLang server, and sending a test request.
|
||||||
|
Recommended if you are new to SGLang on Ascend.
|
||||||
|
- [SGLang Installation with NPU Support](/docs/hardware-platforms/ascend-npus/ascend_npu) — the full installation guide.
|
||||||
|
It covers the component version mapping (CANN, PyTorch adapter, Triton, kernels, etc.), building from source or from a
|
||||||
|
Dockerfile, and recommended system settings (CPU power scheme, NUMA, swap). Use it when you need to install or customize
|
||||||
|
the environment instead of using the official image.
|
||||||
|
|
||||||
### Model weights
|
### Model weights
|
||||||
|
|
||||||
<Warning>
|
<Warning>
|
||||||
|
|||||||
@@ -42,6 +42,18 @@ see [Feature Compatibility](/docs/hardware-platforms/ascend-npus/ascend_npu_opti
|
|||||||
|
|
||||||
## Prerequisites
|
## Prerequisites
|
||||||
|
|
||||||
|
### Environment
|
||||||
|
|
||||||
|
Before following this tutorial, complete the environment setup in the documents below:
|
||||||
|
|
||||||
|
- [Ascend NPU Quickstart](/docs/hardware-platforms/ascend-npus/ascend_npu_quick_start) — the fastest way to get started.
|
||||||
|
It walks you through launching the official container image, starting the SGLang server, and sending a test request.
|
||||||
|
Recommended if you are new to SGLang on Ascend.
|
||||||
|
- [SGLang Installation with NPU Support](/docs/hardware-platforms/ascend-npus/ascend_npu) — the full installation guide.
|
||||||
|
It covers the component version mapping (CANN, PyTorch adapter, Triton, kernels, etc.), building from source or from a
|
||||||
|
Dockerfile, and recommended system settings (CPU power scheme, NUMA, swap). Use it when you need to install or customize
|
||||||
|
the environment instead of using the official image.
|
||||||
|
|
||||||
### Model weights
|
### Model weights
|
||||||
|
|
||||||
<Warning>
|
<Warning>
|
||||||
|
|||||||
@@ -48,6 +48,18 @@ see [Feature Compatibility](/docs/hardware-platforms/ascend-npus/ascend_npu_opti
|
|||||||
|
|
||||||
## Prerequisites
|
## Prerequisites
|
||||||
|
|
||||||
|
### Environment
|
||||||
|
|
||||||
|
Before following this tutorial, complete the environment setup in the documents below:
|
||||||
|
|
||||||
|
- [Ascend NPU Quickstart](/docs/hardware-platforms/ascend-npus/ascend_npu_quick_start) — the fastest way to get started.
|
||||||
|
It walks you through launching the official container image, starting the SGLang server, and sending a test request.
|
||||||
|
Recommended if you are new to SGLang on Ascend.
|
||||||
|
- [SGLang Installation with NPU Support](/docs/hardware-platforms/ascend-npus/ascend_npu) — the full installation guide.
|
||||||
|
It covers the component version mapping (CANN, PyTorch adapter, Triton, kernels, etc.), building from source or from a
|
||||||
|
Dockerfile, and recommended system settings (CPU power scheme, NUMA, swap). Use it when you need to install or customize
|
||||||
|
the environment instead of using the official image.
|
||||||
|
|
||||||
### Model weights
|
### Model weights
|
||||||
|
|
||||||
<Warning>
|
<Warning>
|
||||||
|
|||||||
@@ -42,6 +42,18 @@ see [Feature Compatibility](/docs/hardware-platforms/ascend-npus/ascend_npu_opti
|
|||||||
|
|
||||||
## Prerequisites
|
## Prerequisites
|
||||||
|
|
||||||
|
### Environment
|
||||||
|
|
||||||
|
Before following this tutorial, complete the environment setup in the documents below:
|
||||||
|
|
||||||
|
- [Ascend NPU Quickstart](/docs/hardware-platforms/ascend-npus/ascend_npu_quick_start) — the fastest way to get started.
|
||||||
|
It walks you through launching the official container image, starting the SGLang server, and sending a test request.
|
||||||
|
Recommended if you are new to SGLang on Ascend.
|
||||||
|
- [SGLang Installation with NPU Support](/docs/hardware-platforms/ascend-npus/ascend_npu) — the full installation guide.
|
||||||
|
It covers the component version mapping (CANN, PyTorch adapter, Triton, kernels, etc.), building from source or from a
|
||||||
|
Dockerfile, and recommended system settings (CPU power scheme, NUMA, swap). Use it when you need to install or customize
|
||||||
|
the environment instead of using the official image.
|
||||||
|
|
||||||
### Model weights
|
### Model weights
|
||||||
|
|
||||||
<Warning>
|
<Warning>
|
||||||
|
|||||||
@@ -40,6 +40,18 @@ see [Feature Compatibility](/docs/hardware-platforms/ascend-npus/ascend_npu_opti
|
|||||||
|
|
||||||
## Prerequisites
|
## Prerequisites
|
||||||
|
|
||||||
|
### Environment
|
||||||
|
|
||||||
|
Before following this tutorial, complete the environment setup in the documents below:
|
||||||
|
|
||||||
|
- [Ascend NPU Quickstart](/docs/hardware-platforms/ascend-npus/ascend_npu_quick_start) — the fastest way to get started.
|
||||||
|
It walks you through launching the official container image, starting the SGLang server, and sending a test request.
|
||||||
|
Recommended if you are new to SGLang on Ascend.
|
||||||
|
- [SGLang Installation with NPU Support](/docs/hardware-platforms/ascend-npus/ascend_npu) — the full installation guide.
|
||||||
|
It covers the component version mapping (CANN, PyTorch adapter, Triton, kernels, etc.), building from source or from a
|
||||||
|
Dockerfile, and recommended system settings (CPU power scheme, NUMA, swap). Use it when you need to install or customize
|
||||||
|
the environment instead of using the official image.
|
||||||
|
|
||||||
### Model weights
|
### Model weights
|
||||||
|
|
||||||
<Warning>
|
<Warning>
|
||||||
|
|||||||
@@ -45,6 +45,18 @@ see [Feature Compatibility](/docs/hardware-platforms/ascend-npus/ascend_npu_opti
|
|||||||
|
|
||||||
## Prerequisites
|
## Prerequisites
|
||||||
|
|
||||||
|
### Environment
|
||||||
|
|
||||||
|
Before following this tutorial, complete the environment setup in the documents below:
|
||||||
|
|
||||||
|
- [Ascend NPU Quickstart](/docs/hardware-platforms/ascend-npus/ascend_npu_quick_start) — the fastest way to get started.
|
||||||
|
It walks you through launching the official container image, starting the SGLang server, and sending a test request.
|
||||||
|
Recommended if you are new to SGLang on Ascend.
|
||||||
|
- [SGLang Installation with NPU Support](/docs/hardware-platforms/ascend-npus/ascend_npu) — the full installation guide.
|
||||||
|
It covers the component version mapping (CANN, PyTorch adapter, Triton, kernels, etc.), building from source or from a
|
||||||
|
Dockerfile, and recommended system settings (CPU power scheme, NUMA, swap). Use it when you need to install or customize
|
||||||
|
the environment instead of using the official image.
|
||||||
|
|
||||||
### Model weights
|
### Model weights
|
||||||
|
|
||||||
<Warning>
|
<Warning>
|
||||||
|
|||||||
@@ -42,6 +42,18 @@ see [Feature Compatibility](/docs/hardware-platforms/ascend-npus/ascend_npu_opti
|
|||||||
|
|
||||||
## Prerequisites
|
## Prerequisites
|
||||||
|
|
||||||
|
### Environment
|
||||||
|
|
||||||
|
Before following this tutorial, complete the environment setup in the documents below:
|
||||||
|
|
||||||
|
- [Ascend NPU Quickstart](/docs/hardware-platforms/ascend-npus/ascend_npu_quick_start) — the fastest way to get started.
|
||||||
|
It walks you through launching the official container image, starting the SGLang server, and sending a test request.
|
||||||
|
Recommended if you are new to SGLang on Ascend.
|
||||||
|
- [SGLang Installation with NPU Support](/docs/hardware-platforms/ascend-npus/ascend_npu) — the full installation guide.
|
||||||
|
It covers the component version mapping (CANN, PyTorch adapter, Triton, kernels, etc.), building from source or from a
|
||||||
|
Dockerfile, and recommended system settings (CPU power scheme, NUMA, swap). Use it when you need to install or customize
|
||||||
|
the environment instead of using the official image.
|
||||||
|
|
||||||
### Model weights
|
### Model weights
|
||||||
|
|
||||||
<Warning>
|
<Warning>
|
||||||
|
|||||||
@@ -42,6 +42,18 @@ see [Feature Compatibility](/docs/hardware-platforms/ascend-npus/ascend_npu_opti
|
|||||||
|
|
||||||
## Prerequisites
|
## Prerequisites
|
||||||
|
|
||||||
|
### Environment
|
||||||
|
|
||||||
|
Before following this tutorial, complete the environment setup in the documents below:
|
||||||
|
|
||||||
|
- [Ascend NPU Quickstart](/docs/hardware-platforms/ascend-npus/ascend_npu_quick_start) — the fastest way to get started.
|
||||||
|
It walks you through launching the official container image, starting the SGLang server, and sending a test request.
|
||||||
|
Recommended if you are new to SGLang on Ascend.
|
||||||
|
- [SGLang Installation with NPU Support](/docs/hardware-platforms/ascend-npus/ascend_npu) — the full installation guide.
|
||||||
|
It covers the component version mapping (CANN, PyTorch adapter, Triton, kernels, etc.), building from source or from a
|
||||||
|
Dockerfile, and recommended system settings (CPU power scheme, NUMA, swap). Use it when you need to install or customize
|
||||||
|
the environment instead of using the official image.
|
||||||
|
|
||||||
### Model weights
|
### Model weights
|
||||||
|
|
||||||
<Warning>
|
<Warning>
|
||||||
|
|||||||
+12
@@ -44,6 +44,18 @@ see [Feature Compatibility](/docs/hardware-platforms/ascend-npus/ascend_npu_opti
|
|||||||
|
|
||||||
## Prerequisites
|
## Prerequisites
|
||||||
|
|
||||||
|
### Environment
|
||||||
|
|
||||||
|
Before following this tutorial, complete the environment setup in the documents below:
|
||||||
|
|
||||||
|
- [Ascend NPU Quickstart](/docs/hardware-platforms/ascend-npus/ascend_npu_quick_start) — the fastest way to get started.
|
||||||
|
It walks you through launching the official container image, starting the SGLang server, and sending a test request.
|
||||||
|
Recommended if you are new to SGLang on Ascend.
|
||||||
|
- [SGLang Installation with NPU Support](/docs/hardware-platforms/ascend-npus/ascend_npu) — the full installation guide.
|
||||||
|
It covers the component version mapping (CANN, PyTorch adapter, Triton, kernels, etc.), building from source or from a
|
||||||
|
Dockerfile, and recommended system settings (CPU power scheme, NUMA, swap). Use it when you need to install or customize
|
||||||
|
the environment instead of using the official image.
|
||||||
|
|
||||||
### Model weights
|
### Model weights
|
||||||
|
|
||||||
<Warning>
|
<Warning>
|
||||||
|
|||||||
Reference in New Issue
Block a user