diff --git a/docs_new/docs/hardware-platforms/ascend-npus/ascend_npu_support_features.mdx b/docs_new/docs/hardware-platforms/ascend-npus/ascend_npu_support_features.mdx index 63efb4f1c..ca10c07bb 100644 --- a/docs_new/docs/hardware-platforms/ascend-npus/ascend_npu_support_features.mdx +++ b/docs_new/docs/hardware-platforms/ascend-npus/ascend_npu_support_features.mdx @@ -46,12 +46,24 @@ click [Server Arguments](../../advanced_features/server_arguments). `auto`, `slow` A2, A3 + + `--tokenizer-backend` + `huggingface` + `huggingface`, `fastokens` + A2, A3 + `--tokenizer-worker-num` `1` Type: int A2, A3 + + `--detokenizer-worker-num` + `1` + Type: int + A2, A3 + `--skip-tokenizer-init` `False` @@ -106,6 +118,12 @@ click [Server Arguments](../../advanced_features/server_arguments). `auto`, `sglang`,
`transformers` A2, A3 + + `--model-config-parser` + `auto` + Type: str + A2, A3 + @@ -471,6 +489,18 @@ click [Server Arguments](../../advanced_features/server_arguments). List[float] A2, A3 + + `--prefill-delayer-queue-min-ratio` + `None` + Type: float + A2, A3 + + + `--prefill-delayer-max-delay-ms` + `None` + Type: float + A2, A3 + `--abort-on-priority-`
`when-disabled` `False` @@ -553,6 +583,12 @@ click [Server Arguments](../../advanced_features/server_arguments). Type: int A2, A3 + + `--batch-notify-size` + `16` + Type: int + A2, A3 + `--incremental-streaming-output` `False` @@ -637,6 +673,12 @@ click [Server Arguments](../../advanced_features/server_arguments). bool flag (set to enable) A2, A3 + + `--load-snapshot-publish-interval` + `15` + Type: int + A2, A3 + `--use-ray` `False` @@ -804,6 +846,24 @@ click [Server Arguments](../../advanced_features/server_arguments). Type: str Special for GPU + + `--enable-forward-pass-metrics` + `False` + bool flag (set to enable) + A2, A3 + + + `--forward-pass-metrics-worker-id` + `` + Type: str + A2, A3 + + + `--forward-pass-metrics-ipc-name` + `None` + Type: str + A2, A3 + `--enable-trace` `False` @@ -811,7 +871,13 @@ click [Server Arguments](../../advanced_features/server_arguments). A2, A3 - `--oltp-traces-endpoint` + `--trace-modules` + `request` + Type: str + A2, A3 + + + `--otlp-traces-endpoint` `localhost:4317` Type: str A2, A3 @@ -944,6 +1010,18 @@ click [Server Arguments](../../advanced_features/server_arguments). `deepseek-r1`
`deepseek-v3`
`glm45`
`gpt-oss`
`kimi`
`qwen3`
`qwen3-thinking`
`step3` A2, A3 + + `--strip-thinking-cache` + `False` + bool flag (set to enable) + A2, A3 + + + `--enable-strict-thinking` + `False` + bool flag (set to enable) + A2, A3 + `--tool-call-parser` `None` @@ -956,6 +1034,18 @@ click [Server Arguments](../../advanced_features/server_arguments). `openai`, `model` A2, A3 + + `--asr-max-buffer-seconds` + `60` + Type: int + A2, A3 + + + `--asr-max-concurrent-sessions` + `32` + Type: int + A2, A3 + @@ -1159,6 +1249,12 @@ click [Server Arguments](../../advanced_features/server_arguments). Type: bool A2, A3 + + `--lora-drain-wait-threshold` + `0.0` + Type: float + A2, A3 + `--max-lora-chunk-size` `16` @@ -1313,6 +1409,12 @@ click [Server Arguments](../../advanced_features/server_arguments). Type: int A2, A3 + + `--speculative-dflash-block-size` + `None` + Type: int + A2, A3 + `--speculative-accept-`
`threshold-single` `1.0` @@ -1355,6 +1457,12 @@ click [Server Arguments](../../advanced_features/server_arguments). `ascend` A2, A3 + + `--speculative-dflash-draft-window-size`
`--speculative-draft-window-size` + `None` + Type: int + A2, A3 + `--speculative-draft-model-quantization` `None` @@ -1444,6 +1552,24 @@ click [Server Arguments](../../advanced_features/server_arguments). Type: int Experimental + + `--speculative-adaptive` + `False` + bool flag (set to enable) + A2, A3 + + + `--speculative-adaptive-config` + `None` + Type: str + A2, A3 + + + `--speculative-skip-dp-mlp-sync` + `False` + bool flag (set to enable) + A2, A3 + @@ -1503,6 +1629,12 @@ click [Server Arguments](../../advanced_features/server_arguments). `normal`,
`low_latency`,
`auto` A2, A3 + + `--deepep-dispatcher-output-dtype` + `auto` + `auto`,
`bf16`,
`fp8`,
`int8`,
`nvfp4` + A2, A3 + `--deepep-config` `None` @@ -1593,6 +1725,12 @@ click [Server Arguments](../../advanced_features/server_arguments). Type: str Special for GPU + + `--enable-deepep-waterfill` + `False` + bool flag (set to enable) + A2, A3 + @@ -1708,7 +1846,7 @@ click [Server Arguments](../../advanced_features/server_arguments). `--hicache-storage-`
`backend` `None` `file` - A2, A3 + Planned `--hicache-storage-`
`prefetch-policy` @@ -1885,6 +2023,12 @@ click [Server Arguments](../../advanced_features/server_arguments). List[int] A2, A3 + + `--disable-piecewise-cuda-graph` + `False` + bool flag (set to enable) + A2, A3 + `--disable-cuda-graph` `False` @@ -1951,6 +2095,12 @@ click [Server Arguments](../../advanced_features/server_arguments). bool flag
(set to enable) Special for GPU + + `--pre-warm-nccl` + `False` + bool flag (set to enable) + A2, A3 + `--enable-torch-`
`symm-mem` `False` @@ -1997,7 +2147,7 @@ click [Server Arguments](../../advanced_features/server_arguments). `--enable-single-`
`batch-overlap` `False` bool flag
(set to enable) - A2, A3 + Special for GPU `--tbo-token-`
`distribution-threshold` @@ -2185,6 +2335,18 @@ click [Server Arguments](../../advanced_features/server_arguments). `fsdp` Planned + + `--disable-attn-tp-gather` + `False` + bool flag (set to enable) + A2, A3 + + + `--enable-nsa-prefill-context-parallel` + `False` + bool flag (set to enable) + A2, A3 + `--enable-layerwise-`
`nvtx-marker` `False` @@ -2209,6 +2371,12 @@ click [Server Arguments](../../advanced_features/server_arguments). bool flag
(set to enable) A2, A3 + + `--dsa-prefill-cp-mode` + `round-robin-split` + `in-seq-split`, `round-robin-split` + A2, A3 + `--prefill-cp-mode` `in-seq-split` @@ -2378,6 +2546,12 @@ click [Server Arguments](../../advanced_features/server_arguments). Type: int A2, A3 + + `--optimistic-prefill-retries` + `0` + Type: int + A2, A3 + @@ -2501,6 +2675,12 @@ click [Server Arguments](../../advanced_features/server_arguments). `transfer_engine`,
`nccl` Special for GPU + + `--weight-loader-drop-cache-after-load` + `False` + bool flag (set to enable) + A2, A3 + `--remote-instance-weight-`
`loader-start-seed-via-transfer-engine` `False` @@ -2663,6 +2843,40 @@ click [Server Arguments](../../advanced_features/server_arguments). Type: JSON list A2, A3 + + `--enable-quant-communications` + `False` + bool flag (set to enable) + A2, A3 + + + + +## msProbe + + + + + + + + + + + + + + + + + + + + + + + +
ArgumentDefaultsOptionsServer supported
`--msprobe-dump-config``None`Type: strA2, A3