minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

Modify the optional values and constraints of parameter. (#22705)

Browse Source
This commit is contained in:
chx96642264
2026-04-13 22:50:48 +08:00
committed by GitHub
parent b6a91b1afe
commit c6403a11cb
1 changed files with 12 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/platforms/ascend/ascend_npu_support_features.md
+12 -12
View File
@@ -274,22 +274,22 @@ click [Server Arguments](https://docs.sglang.io/advanced_features/server_argumen
| `--max-mamba-cache-size` | `None` | Type: int | A2, A3 |
| `--mamba-ssm-dtype` | `float32` | `float32`,<br/>`bfloat16`,<br/>`float16` | A2, A3 |
| `--mamba-full-memory-ratio` | `0.9` | Type: float | A2, A3 |
| `--mamba-scheduler-strategy` | `auto` | Only `auto`, `no_buffer` supported | A2, A3 |
| `--mamba-scheduler-strategy` | `auto` | `auto`,<br/>`no_buffer`,<br/>`extra_buffer` | A2, A3 |
| `--mamba-track-interval` | `256` | Type: int | A2, A3 |
## Hierarchical cache
| Argument | Defaults | Options | Server supported |
|-------------------------------------------------|-----------------|---------------------------------------------------------------------|:----------------:|
| `--enable-hierarchical-`<br/>`cache` | `False` | bool flag<br/> (set to enable) | A2, A3 |
| `--hicache-ratio` | `2.0` | Type: float | A2, A3 |
| `--hicache-size` | `0` | Type: int | A2, A3 |
| `--hicache-write-policy` | `write_through` | Currently only `write_back` supported | A2, A3 |
| `--hicache-io-backend` | `kernel` | `kernel_ascend`,<br/> `direct` | A2, A3 |
| `--hicache-mem-layout` | `layer_first` | `page_first_direct`,<br/> `page_first_kv_split` | A2, A3 |
| `--hicache-storage-`<br/>`backend` | `None` | `file` | A2, A3 |
| `--hicache-storage-`<br/>`prefetch-policy` | `best_effort` | `best_effort`,<br/> `wait_complete`,<br/> `timeout` | Special for GPU |
| `--hicache-storage-`<br/>`backend-extra-config` | `None` | Type: str | Special for GPU |
| Argument | Defaults | Options | Server supported |
|-------------------------------------------------|-----------------|-------------------------------------------------------------------------------|:----------------:|
| `--enable-hierarchical-`<br/>`cache` | `False` | bool flag<br/> (set to enable).<br/> Currently, mamba cache is not supported. | A2, A3 |
| `--hicache-ratio` | `2.0` | Type: float | A2, A3 |
| `--hicache-size` | `0` | Type: int | A2, A3 |
| `--hicache-write-policy` | `write_through` | Currently only `write_back` supported | A2, A3 |
| `--hicache-io-backend` | `kernel` | `kernel_ascend`,<br/> `direct` | A2, A3 |
| `--hicache-mem-layout` | `layer_first` | `page_first_direct`,<br/> `page_first_kv_split` | A2, A3 |
| `--hicache-storage-`<br/>`backend` | `None` | `file` | A2, A3 |
| `--hicache-storage-`<br/>`prefetch-policy` | `best_effort` | `best_effort`,<br/> `wait_complete`,<br/> `timeout` | Special for GPU |
| `--hicache-storage-`<br/>`backend-extra-config` | `None` | Type: str | Special for GPU |
## LMCache