minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

fix issues for npu docs (#22307)

Browse Source
This commit is contained in:
amote-i
2026-04-09 16:27:34 +08:00
committed by GitHub
parent 8ec0934f8f
commit 7965573eb4
4 changed files with 12 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/platforms/ascend/ascend_contribution_guide.md
+1 -1
View File
@@ -38,7 +38,7 @@ If you add a new feature or fix a bug, please add corresponding unit tests to en
SGLang uses Python's built-in [unittest](https://docs.python.org/3/library/unittest.html) framework. SGLang uses Python's built-in [unittest](https://docs.python.org/3/library/unittest.html) framework.
For detailed instructions on running tests and integrating them into CI, refer to [test/README.md](https://github.com/sgl-project/sglang/tree/main/test/README.md). For detailed instructions on running tests and integrating them into CI, refer to [test/README.md](https://github.com/sgl-project/sglang/tree/main/test/README.md).
If you need to use model which is not in ```python/sglang/test/ascend/test_ascend_utils.py`` list. Follow these steps: If you need to use model which is not in `python/sglang/test/ascend/test_ascend_utils.py` list. Follow these steps:
1. Register account and upload your model to [modelscope](https://modelscope.cn/models). 1. Register account and upload your model to [modelscope](https://modelscope.cn/models).
2. Make sure your model is pre-cached on the CI server and is on the way "/data/ascend-ci-share-pkking-sglang/modelscope/hub/models/{your_model_repo}/{your_model}". 2. Make sure your model is pre-cached on the CI server and is on the way "/data/ascend-ci-share-pkking-sglang/modelscope/hub/models/{your_model_repo}/{your_model}".
If this is not the case, use following command on CI server: If this is not the case, use following command on CI server:
docs/platforms/ascend/ascend_npu.md
+1 -1
View File
@@ -38,7 +38,7 @@ conda activate sglang_npu
#### CANN #### CANN
Prior to start work with SGLang on Ascend you need to install CANN Toolkit, Kernels operator package and NNAL version 8.3.RC2 or higher, check the [installation guide](https://www.hiascend.com/document/detail/zh/CANNCommunityEdition/83RC1/softwareinst/instg/instg_0008.html?Mode=PmIns&InstallType=local&OS=openEuler&Software=cannToolKit) Prior to start work with SGLang on Ascend you need to install CANN Toolkit, Kernels operator package and NNAL version 8.5.0, check the [installation guide](https://www.hiascend.com/document/detail/zh/CANNCommunityEdition/850/softwareinst/instg/instg_0008.html?Mode=PmIns&InstallType=local&OS=openEuler&Software=cannToolKit)
#### MemFabric-Hybrid #### MemFabric-Hybrid
docs/platforms/ascend/ascend_npu_quantization.md
+1 -1
View File
@@ -2,7 +2,7 @@
To load already quantized models, simply load the model weights and config. Again, if the model has been quantized offline, there's no need to add `--quantization` argument when starting the engine. The quantization method will be automatically parsed from the downloaded `quant_model_description.json` or `config.json` config. To load already quantized models, simply load the model weights and config. Again, if the model has been quantized offline, there's no need to add `--quantization` argument when starting the engine. The quantization method will be automatically parsed from the downloaded `quant_model_description.json` or `config.json` config.
SGLang support **mix-bits** quantization (independently defines and loads each layer depending on the type of quantification specified in the `quant_model_description'.json`). [Advanced mix-bits for MoE](https://github.com/sgl-project/sglang/pull/17361) in progress, will add independent quantization determination for the w13 (up-gate) and w2 (down) layers). SGLang support **mix-bits** quantization (independently defines and loads each layer depending on the type of quantification specified in the `quant_model_description'.json`). [Advanced mix-bits for MoE](https://github.com/sgl-project/sglang/pull/17361) in progress, will add independent quantization determination for the w13 (up-gate) and w2 (down) layers.
[ModelSlim on Ascend support](https://github.com/sgl-project/sglang/pull/14504) [ModelSlim on Ascend support](https://github.com/sgl-project/sglang/pull/14504)
| Quantization scheme | Layer type | A2 Supported | A3 Supported | A5 Supported | Diffusion models | | Quantization scheme | Layer type | A2 Supported | A3 Supported | A5 Supported | Diffusion models |
docs/platforms/ascend/ascend_npu_support_features.md
+9 -7
View File
@@ -86,7 +86,7 @@ click [Server Arguments](https://docs.sglang.io/advanced_features/server_argumen
|----------------------------------------------------|----------|---------------------------|:----------------:| |----------------------------------------------------|----------|---------------------------|:----------------:|
| `--device` | `None` | Type: str | A2, A3 | | `--device` | `None` | Type: str | A2, A3 |
| `--tensor-parallel-size`<br/>`--tp-size` | `1` | Type: int | A2, A3 | | `--tensor-parallel-size`<br/>`--tp-size` | `1` | Type: int | A2, A3 |
| `--pipeline-parallel-size`<br/>`--pp-size` | `1` | Type: int | A2, A3 | | `--pipeline-parallel-size`<br/>`--pp-size` | `1` | Type: int; Currently `2` not supported | A2, A3 |
| `--attention-context-parallel-size`<br/>`--attn-cp-size` | `1` | Type: int | A2, A3 | | `--attention-context-parallel-size`<br/>`--attn-cp-size` | `1` | Type: int | A2, A3 |
| `--moe-data-parallel-size`<br/>`--moe-dp-size` | `1` | Type: int | A2, A3 | | `--moe-data-parallel-size`<br/>`--moe-dp-size` | `1` | Type: int | A2, A3 |
| `--pp-max-micro-batch-size` | `None` | Type: int | A2, A3 | | `--pp-max-micro-batch-size` | `None` | Type: int | A2, A3 |
@@ -184,6 +184,7 @@ click [Server Arguments](https://docs.sglang.io/advanced_features/server_argumen
| Argument | Defaults | Options | Server supported | | Argument | Defaults | Options | Server supported |
|--------------------------|----------|-------------------------------------|:----------------:| |--------------------------|----------|-------------------------------------|:----------------:|
| `--enable-lora` | `False` | Bool flag <br/>(set to enable) | A2, A3 | | `--enable-lora` | `False` | Bool flag <br/>(set to enable) | A2, A3 |
| `--enable-lora-overlap-loading` | `False` | Bool flag <br/>(set to enable) | A2, A3 |
| `--max-lora-rank` | `None` | Type: int | A2, A3 | | `--max-lora-rank` | `None` | Type: int | A2, A3 |
| `--lora-target-modules` | `None` | `all` | A2, A3 | | `--lora-target-modules` | `None` | `all` | A2, A3 |
| `--lora-paths` | `None` | Type: List[str] /<br/> JSON objects | A2, A3 | | `--lora-paths` | `None` | Type: List[str] /<br/> JSON objects | A2, A3 |
@@ -252,16 +253,17 @@ click [Server Arguments](https://docs.sglang.io/advanced_features/server_argumen
| `--deepep-mode` | `auto` | `normal`, <br/>`low_latency`,<br/> `auto` | A2, A3 | | `--deepep-mode` | `auto` | `normal`, <br/>`low_latency`,<br/> `auto` | A2, A3 |
| `--deepep-config` | `None` | Type: str | Special for GPU | | `--deepep-config` | `None` | Type: str | Special for GPU |
| `--ep-num-redundant-experts` | `0` | Type: int | A2, A3 | | `--ep-num-redundant-experts` | `0` | Type: int | A2, A3 |
| `--ep-dispatch-algorithm` | `None` | Type: str | A2, A3 | | `--ep-dispatch-algorithm` | `None` | `static`,<br/> `dynamic`,<br/> `fake` | A2, A3 |
| `--init-expert-location` | `trivial` | Type: str | A2, A3 | | `--init-expert-location` | `trivial` | Type: str | A2, A3 |
| `--enable-eplb` | `False` | bool flag<br/> (set to enable) | A2, A3 | | `--enable-eplb` | `False` | bool flag<br/> (set to enable) | A2, A3 |
| `--eplb-algorithm` | `auto` | Type: str | A2, A3 | | `--eplb-algorithm` | `deepseek`| Type: str | A2, A3 |
| `--eplb-rebalance-num-iterations` | `1000` | Type: int | A2, A3 |
| `--eplb-rebalance-layers-`<br/>`per-chunk` | `None` | Type: int | A2, A3 | | `--eplb-rebalance-layers-`<br/>`per-chunk` | `None` | Type: int | A2, A3 |
| `--eplb-min-rebalancing-`<br/>`utilization-threshold` | `1.0` | Type: float | A2, A3 | | `--eplb-min-rebalancing-`<br/>`utilization-threshold` | `1.0` | Type: float | A2, A3 |
| `--expert-distribution-`<br/>`recorder-mode` | `None` | Type: str | A2, A3 | | `--expert-distribution-`<br/>`recorder-mode` | `None` | `stat`,<br/> `stat_approx`,<br/> `per_pass`,<br/> `per_token` | A2, A3 |
| `--expert-distribution-`<br/>`recorder-buffer-size` | `None` | Type: int | A2, A3 | | `--expert-distribution-`<br/>`recorder-buffer-size` | `None` | Type: int | A2, A3 |
| `--enable-expert-distribution-`<br/>`metrics` | `False` | bool flag (set to enable) | A2, A3 | | `--enable-expert-distribution-`<br/>`metrics` | `False` | bool flag (set to enable) | A2, A3 |
| `--moe-dense-tp-size` | `None` | Type: int | A2, A3 | | `--moe-dense-tp-size` | `None` | `1` | A2, A3 |
| `--elastic-ep-backend` | `None` | `none`, `mooncake` | Special for GPU | | `--elastic-ep-backend` | `None` | `none`, `mooncake` | Special for GPU |
| `--mooncake-ib-device` | `None` | Type: str | Special for GPU | | `--mooncake-ib-device` | `None` | Type: str | Special for GPU |
@@ -357,8 +359,8 @@ click [Server Arguments](https://docs.sglang.io/advanced_features/server_argumen
| `--allow-auto-truncate` | `False` | bool flag<br/> (set to enable) | A2, A3 | | `--allow-auto-truncate` | `False` | bool flag<br/> (set to enable) | A2, A3 |
| `--enable-custom-`<br/>`logit-processor` | `False` | bool flag<br/> (set to enable) | A2, A3 | | `--enable-custom-`<br/>`logit-processor` | `False` | bool flag<br/> (set to enable) | A2, A3 |
| `--flashinfer-mla-`<br/>`disable-ragged` | `False` | bool flag<br/> (set to enable) | Special for GPU | | `--flashinfer-mla-`<br/>`disable-ragged` | `False` | bool flag<br/> (set to enable) | Special for GPU |
| `--disable-shared-`<br/>`experts-fusion` | `False` | bool flag<br/> (set to enable) | A2, A3 | | `--disable-shared-`<br/>`experts-fusion` | `True` | bool flag<br/> (set to enable) | A2, A3 |
| `--disable-chunked-`<br/>`prefix-cache` | `False` | bool flag<br/> (set to enable) | A2, A3 | | `--disable-chunked-`<br/>`prefix-cache` | `True` | bool flag<br/> (set to enable) | A2, A3 |
| `--disable-fast-`<br/>`image-processor` | `False` | bool flag<br/> (set to enable) | A2, A3 | | `--disable-fast-`<br/>`image-processor` | `False` | bool flag<br/> (set to enable) | A2, A3 |
| `--keep-mm-feature-`<br/>`on-device` | `False` | bool flag<br/> (set to enable) | A2, A3 | | `--keep-mm-feature-`<br/>`on-device` | `False` | bool flag<br/> (set to enable) | A2, A3 |
| `--enable-return-`<br/>`hidden-states` | `False` | bool flag<br/> (set to enable) | A2, A3 | | `--enable-return-`<br/>`hidden-states` | `False` | bool flag<br/> (set to enable) | A2, A3 |