diff --git a/docs_new/docs/hardware-platforms/ascend-npus/ascend_npu.mdx b/docs_new/docs/hardware-platforms/ascend-npus/ascend_npu.mdx index ce65bbd2a..395568d49 100644 --- a/docs_new/docs/hardware-platforms/ascend-npus/ascend_npu.mdx +++ b/docs_new/docs/hardware-platforms/ascend-npus/ascend_npu.mdx @@ -124,6 +124,17 @@ If you want to use PD disaggregation mode, you need to install MemFabric-Hybrid. pip install memfabric-hybrid==1.0.8 ``` +#### MemFabric-zbal + +MemFabric-zbal is a Zero Buffer Acceleration Library of high-performance operators for LLM inference and training on Ascend, +accelerating computation by eliminating intermediate memory buffers; it is required only on **aarch64** clusters and is +installed in addition to MemFabric-Hybrid. + +```bash Command +# Only needed on aarch64 (arm64) hosts +pip install memfabric-zbal==1.1.1 +``` + #### Pytorch and Pytorch Framework Adaptor on Ascend ```bash Command diff --git a/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/deepseek_r1.mdx b/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/deepseek_r1.mdx index e19d51ba4..018ba26d4 100644 --- a/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/deepseek_r1.mdx +++ b/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/deepseek_r1.mdx @@ -47,6 +47,18 @@ see [Feature Compatibility](/docs/hardware-platforms/ascend-npus/ascend_npu_opti ## Prerequisites +### Environment + +Before following this tutorial, complete the environment setup in the documents below: + +- [Ascend NPU Quickstart](/docs/hardware-platforms/ascend-npus/ascend_npu_quick_start) — the fastest way to get started. + It walks you through launching the official container image, starting the SGLang server, and sending a test request. + Recommended if you are new to SGLang on Ascend. +- [SGLang Installation with NPU Support](/docs/hardware-platforms/ascend-npus/ascend_npu) — the full installation guide. + It covers the component version mapping (CANN, PyTorch adapter, Triton, kernels, etc.), building from source or from a + Dockerfile, and recommended system settings (CPU power scheme, NUMA, swap). Use it when you need to install or customize + the environment instead of using the official image. + ### Model weights diff --git a/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/deepseek_v3_2.mdx b/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/deepseek_v3_2.mdx index 63676919f..c91dd99a9 100644 --- a/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/deepseek_v3_2.mdx +++ b/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/deepseek_v3_2.mdx @@ -46,6 +46,18 @@ see [Feature Compatibility](/docs/hardware-platforms/ascend-npus/ascend_npu_opti ## Prerequisites +### Environment + +Before following this tutorial, complete the environment setup in the documents below: + +- [Ascend NPU Quickstart](/docs/hardware-platforms/ascend-npus/ascend_npu_quick_start) — the fastest way to get started. + It walks you through launching the official container image, starting the SGLang server, and sending a test request. + Recommended if you are new to SGLang on Ascend. +- [SGLang Installation with NPU Support](/docs/hardware-platforms/ascend-npus/ascend_npu) — the full installation guide. + It covers the component version mapping (CANN, PyTorch adapter, Triton, kernels, etc.), building from source or from a + Dockerfile, and recommended system settings (CPU power scheme, NUMA, swap). Use it when you need to install or customize + the environment instead of using the official image. + ### Model weights diff --git a/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/glm_5_1.mdx b/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/glm_5_1.mdx index ae8807096..e36c331a6 100644 --- a/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/glm_5_1.mdx +++ b/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/glm_5_1.mdx @@ -48,6 +48,18 @@ see [Feature Compatibility](/docs/hardware-platforms/ascend-npus/ascend_npu_opti ## Prerequisites +### Environment + +Before following this tutorial, complete the environment setup in the documents below: + +- [Ascend NPU Quickstart](/docs/hardware-platforms/ascend-npus/ascend_npu_quick_start) — the fastest way to get started. + It walks you through launching the official container image, starting the SGLang server, and sending a test request. + Recommended if you are new to SGLang on Ascend. +- [SGLang Installation with NPU Support](/docs/hardware-platforms/ascend-npus/ascend_npu) — the full installation guide. + It covers the component version mapping (CANN, PyTorch adapter, Triton, kernels, etc.), building from source or from a + Dockerfile, and recommended system settings (CPU power scheme, NUMA, swap). Use it when you need to install or customize + the environment instead of using the official image. + ### Model weights diff --git a/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/kimi_k2_6.mdx b/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/kimi_k2_6.mdx index dd82ad300..9d9ef5516 100644 --- a/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/kimi_k2_6.mdx +++ b/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/kimi_k2_6.mdx @@ -48,6 +48,18 @@ see [Feature Compatibility](/docs/hardware-platforms/ascend-npus/ascend_npu_opti ## Prerequisites +### Environment + +Before following this tutorial, complete the environment setup in the documents below: + +- [Ascend NPU Quickstart](/docs/hardware-platforms/ascend-npus/ascend_npu_quick_start) — the fastest way to get started. + It walks you through launching the official container image, starting the SGLang server, and sending a test request. + Recommended if you are new to SGLang on Ascend. +- [SGLang Installation with NPU Support](/docs/hardware-platforms/ascend-npus/ascend_npu) — the full installation guide. + It covers the component version mapping (CANN, PyTorch adapter, Triton, kernels, etc.), building from source or from a + Dockerfile, and recommended system settings (CPU power scheme, NUMA, swap). Use it when you need to install or customize + the environment instead of using the official image. + ### Model weights diff --git a/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/minimax_m2_5.mdx b/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/minimax_m2_5.mdx index dcd713130..7ac83c7f4 100644 --- a/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/minimax_m2_5.mdx +++ b/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/minimax_m2_5.mdx @@ -45,6 +45,18 @@ see [Feature Compatibility](/docs/hardware-platforms/ascend-npus/ascend_npu_opti ## Prerequisites +### Environment + +Before following this tutorial, complete the environment setup in the documents below: + +- [Ascend NPU Quickstart](/docs/hardware-platforms/ascend-npus/ascend_npu_quick_start) — the fastest way to get started. + It walks you through launching the official container image, starting the SGLang server, and sending a test request. + Recommended if you are new to SGLang on Ascend. +- [SGLang Installation with NPU Support](/docs/hardware-platforms/ascend-npus/ascend_npu) — the full installation guide. + It covers the component version mapping (CANN, PyTorch adapter, Triton, kernels, etc.), building from source or from a + Dockerfile, and recommended system settings (CPU power scheme, NUMA, swap). Use it when you need to install or customize + the environment instead of using the official image. + ### Model weights diff --git a/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3-8b.mdx b/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3-8b.mdx index 83f911359..71bf1b5e3 100644 --- a/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3-8b.mdx +++ b/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3-8b.mdx @@ -42,6 +42,18 @@ see [Feature Compatibility](/docs/hardware-platforms/ascend-npus/ascend_npu_opti ## Prerequisites +### Environment + +Before following this tutorial, complete the environment setup in the documents below: + +- [Ascend NPU Quickstart](/docs/hardware-platforms/ascend-npus/ascend_npu_quick_start) — the fastest way to get started. + It walks you through launching the official container image, starting the SGLang server, and sending a test request. + Recommended if you are new to SGLang on Ascend. +- [SGLang Installation with NPU Support](/docs/hardware-platforms/ascend-npus/ascend_npu) — the full installation guide. + It covers the component version mapping (CANN, PyTorch adapter, Triton, kernels, etc.), building from source or from a + Dockerfile, and recommended system settings (CPU power scheme, NUMA, swap). Use it when you need to install or customize + the environment instead of using the official image. + ### Model weights diff --git a/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_235b_a22b.mdx b/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_235b_a22b.mdx index db57d9719..bad84e227 100644 --- a/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_235b_a22b.mdx +++ b/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_235b_a22b.mdx @@ -48,6 +48,18 @@ see [Feature Compatibility](/docs/hardware-platforms/ascend-npus/ascend_npu_opti ## Prerequisites +### Environment + +Before following this tutorial, complete the environment setup in the documents below: + +- [Ascend NPU Quickstart](/docs/hardware-platforms/ascend-npus/ascend_npu_quick_start) — the fastest way to get started. + It walks you through launching the official container image, starting the SGLang server, and sending a test request. + Recommended if you are new to SGLang on Ascend. +- [SGLang Installation with NPU Support](/docs/hardware-platforms/ascend-npus/ascend_npu) — the full installation guide. + It covers the component version mapping (CANN, PyTorch adapter, Triton, kernels, etc.), building from source or from a + Dockerfile, and recommended system settings (CPU power scheme, NUMA, swap). Use it when you need to install or customize + the environment instead of using the official image. + ### Model weights diff --git a/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_30b_a3b.mdx b/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_30b_a3b.mdx index 7db7396c9..9666ccd81 100644 --- a/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_30b_a3b.mdx +++ b/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_30b_a3b.mdx @@ -42,6 +42,18 @@ see [Feature Compatibility](/docs/hardware-platforms/ascend-npus/ascend_npu_opti ## Prerequisites +### Environment + +Before following this tutorial, complete the environment setup in the documents below: + +- [Ascend NPU Quickstart](/docs/hardware-platforms/ascend-npus/ascend_npu_quick_start) — the fastest way to get started. + It walks you through launching the official container image, starting the SGLang server, and sending a test request. + Recommended if you are new to SGLang on Ascend. +- [SGLang Installation with NPU Support](/docs/hardware-platforms/ascend-npus/ascend_npu) — the full installation guide. + It covers the component version mapping (CANN, PyTorch adapter, Triton, kernels, etc.), building from source or from a + Dockerfile, and recommended system settings (CPU power scheme, NUMA, swap). Use it when you need to install or customize + the environment instead of using the official image. + ### Model weights diff --git a/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_32b.mdx b/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_32b.mdx index 20cc54c59..302e7c19a 100644 --- a/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_32b.mdx +++ b/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_32b.mdx @@ -40,6 +40,18 @@ see [Feature Compatibility](/docs/hardware-platforms/ascend-npus/ascend_npu_opti ## Prerequisites +### Environment + +Before following this tutorial, complete the environment setup in the documents below: + +- [Ascend NPU Quickstart](/docs/hardware-platforms/ascend-npus/ascend_npu_quick_start) — the fastest way to get started. + It walks you through launching the official container image, starting the SGLang server, and sending a test request. + Recommended if you are new to SGLang on Ascend. +- [SGLang Installation with NPU Support](/docs/hardware-platforms/ascend-npus/ascend_npu) — the full installation guide. + It covers the component version mapping (CANN, PyTorch adapter, Triton, kernels, etc.), building from source or from a + Dockerfile, and recommended system settings (CPU power scheme, NUMA, swap). Use it when you need to install or customize + the environment instead of using the official image. + ### Model weights diff --git a/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_5_397b.mdx b/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_5_397b.mdx index 3c59571ab..0ee319f7f 100644 --- a/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_5_397b.mdx +++ b/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_5_397b.mdx @@ -45,6 +45,18 @@ see [Feature Compatibility](/docs/hardware-platforms/ascend-npus/ascend_npu_opti ## Prerequisites +### Environment + +Before following this tutorial, complete the environment setup in the documents below: + +- [Ascend NPU Quickstart](/docs/hardware-platforms/ascend-npus/ascend_npu_quick_start) — the fastest way to get started. + It walks you through launching the official container image, starting the SGLang server, and sending a test request. + Recommended if you are new to SGLang on Ascend. +- [SGLang Installation with NPU Support](/docs/hardware-platforms/ascend-npus/ascend_npu) — the full installation guide. + It covers the component version mapping (CANN, PyTorch adapter, Triton, kernels, etc.), building from source or from a + Dockerfile, and recommended system settings (CPU power scheme, NUMA, swap). Use it when you need to install or customize + the environment instead of using the official image. + ### Model weights diff --git a/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_6_27b.mdx b/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_6_27b.mdx index 3656c88e2..ea51dd415 100644 --- a/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_6_27b.mdx +++ b/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_6_27b.mdx @@ -42,6 +42,18 @@ see [Feature Compatibility](/docs/hardware-platforms/ascend-npus/ascend_npu_opti ## Prerequisites +### Environment + +Before following this tutorial, complete the environment setup in the documents below: + +- [Ascend NPU Quickstart](/docs/hardware-platforms/ascend-npus/ascend_npu_quick_start) — the fastest way to get started. + It walks you through launching the official container image, starting the SGLang server, and sending a test request. + Recommended if you are new to SGLang on Ascend. +- [SGLang Installation with NPU Support](/docs/hardware-platforms/ascend-npus/ascend_npu) — the full installation guide. + It covers the component version mapping (CANN, PyTorch adapter, Triton, kernels, etc.), building from source or from a + Dockerfile, and recommended system settings (CPU power scheme, NUMA, swap). Use it when you need to install or customize + the environment instead of using the official image. + ### Model weights diff --git a/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_6_35b_a3b.mdx b/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_6_35b_a3b.mdx index a766ee11f..413d8ecf8 100644 --- a/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_6_35b_a3b.mdx +++ b/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_6_35b_a3b.mdx @@ -42,6 +42,18 @@ see [Feature Compatibility](/docs/hardware-platforms/ascend-npus/ascend_npu_opti ## Prerequisites +### Environment + +Before following this tutorial, complete the environment setup in the documents below: + +- [Ascend NPU Quickstart](/docs/hardware-platforms/ascend-npus/ascend_npu_quick_start) — the fastest way to get started. + It walks you through launching the official container image, starting the SGLang server, and sending a test request. + Recommended if you are new to SGLang on Ascend. +- [SGLang Installation with NPU Support](/docs/hardware-platforms/ascend-npus/ascend_npu) — the full installation guide. + It covers the component version mapping (CANN, PyTorch adapter, Triton, kernels, etc.), building from source or from a + Dockerfile, and recommended system settings (CPU power scheme, NUMA, swap). Use it when you need to install or customize + the environment instead of using the official image. + ### Model weights diff --git a/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_next_80b_a3b_instruct.mdx b/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_next_80b_a3b_instruct.mdx index 829670080..327cdf5b0 100644 --- a/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_next_80b_a3b_instruct.mdx +++ b/docs_new/docs/hardware-platforms/ascend-npus/model-tutorials/qwen3_next_80b_a3b_instruct.mdx @@ -44,6 +44,18 @@ see [Feature Compatibility](/docs/hardware-platforms/ascend-npus/ascend_npu_opti ## Prerequisites +### Environment + +Before following this tutorial, complete the environment setup in the documents below: + +- [Ascend NPU Quickstart](/docs/hardware-platforms/ascend-npus/ascend_npu_quick_start) — the fastest way to get started. + It walks you through launching the official container image, starting the SGLang server, and sending a test request. + Recommended if you are new to SGLang on Ascend. +- [SGLang Installation with NPU Support](/docs/hardware-platforms/ascend-npus/ascend_npu) — the full installation guide. + It covers the component version mapping (CANN, PyTorch adapter, Triton, kernels, etc.), building from source or from a + Dockerfile, and recommended system settings (CPU power scheme, NUMA, swap). Use it when you need to install or customize + the environment instead of using the official image. + ### Model weights