[NPU][DOCS]Add faq and feature Compatibilit (#26464)

2026-05-27 17:48:47 +08:00
parent 3afc80d781
commit a1ebc4917a
4 changed files with 443 additions and 18 deletions
@@ -494,6 +494,17 @@ curl http://127.0.0.1:8000/v1/chat/completions \
  }'
 ```

+Some models return responses accompanied with thinking process content. To disable this output, configure parameters as follows:
+```bash Command
+curl http://127.0.0.1:8000/v1/chat/completions
+  -H "Content-Type: application/json"
+  -d '{
+    "model": "Eco-Tech/Qwen3.5-27B-w8a8-mtp",
+    "messages": [{"role": "user", "content": "What is the capital of France?"}],
+    "chat_template_kwargs": {"enable_thinking": false}
+  }'
+```
+
 The expected output should contain "Paris".

 ### Multimodal Chat Completions
@@ -375,34 +375,34 @@ you encounter issues or have any questions, please [open an issue](https://githu
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}><a href="#qwen3-14b-3_5k-1_5k-9ms-on-a3-1-cards-mixed-mode">Optimal Configuration</a></td>
    </tr>
    <tr>
-      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Qwen3-27B</td>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Qwen3.5-27B</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Atlas 800I A3</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>2</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>PD Mixed</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>3.5K+1.5K</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>20ms</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>W8A8 INT8</td>
-      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}><a href="#qwen3-27b-3_5k-1_5k-20ms-on-a3-2-cards-mixed-mode">Optimal Configuration</a></td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}><a href="#qwen35-27b-3_5k-1_5k-20ms-on-a3-2-cards-mixed-mode">Optimal Configuration</a></td>
    </tr>
    <tr>
-      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Qwen3-27B</td>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Qwen3.5-27B</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Atlas 800I A3</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>1</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>PD Mixed</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>16K+1K</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>20ms</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>W8A8 INT8</td>
-      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}><a href="#qwen3-27b-16k-1k-20ms-on-a3-1-cards-mixed-mode">Optimal Configuration</a></td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}><a href="#qwen35-27b-16k-1k-20ms-on-a3-1-cards-mixed-mode">Optimal Configuration</a></td>
    </tr>
    <tr>
-      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Qwen3-27B</td>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Qwen3.5-27B</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Atlas 800I A3</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>1</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>PD Mixed</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>64K+1K</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>20ms</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>W8A8 INT8</td>
-      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}><a href="#qwen3-27b-64k-1k-20ms-on-a3-1-cards-mixed-mode">Optimal Configuration</a></td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}><a href="#qwen35-27b-64k-1k-20ms-on-a3-1-cards-mixed-mode">Optimal Configuration</a></td>
    </tr>
 </tbody>
 </table>
@@ -584,34 +584,34 @@ you encounter issues or have any questions, please [open an issue](https://githu
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}><a href="#qwen3-8b-3_5k-1_5k-50ms-on-a3-1-cards-mixed-mode">Optimal Configuration</a></td>
    </tr>
    <tr>
-      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Qwen3-27B</td>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Qwen3.5-27B</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Atlas 800I A3</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>1</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>PD Mixed</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>3.5K+1.5K</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>50ms</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>W8A8 INT8</td>
-      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}><a href="#qwen3-27b-3_5k-1_5k-50ms-on-a3-1-cards-mixed-mode">Optimal Configuration</a></td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}><a href="#qwen35-27b-3_5k-1_5k-50ms-on-a3-1-cards-mixed-mode">Optimal Configuration</a></td>
    </tr>
    <tr>
-      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Qwen3-27B</td>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Qwen3.5-27B</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Atlas 800I A3</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>2</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>PD Mixed</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>16K+1K</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>50ms</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>W8A8 INT8</td>
-      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}><a href="#qwen3-27b-16k-1k-50ms-on-a3-2-cards-mixed-mode">Optimal Configuration</a></td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}><a href="#qwen35-27b-16k-1k-50ms-on-a3-2-cards-mixed-mode">Optimal Configuration</a></td>
    </tr>
    <tr>
-      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Qwen3-27B</td>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Qwen3.5-27B</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Atlas 800I A3</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>2</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>PD Mixed</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>64K+1K</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>50ms</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>W8A8 INT8</td>
-      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}><a href="#qwen3-27b-64k-1k-50ms-on-a3-2-cards-mixed-mode">Optimal Configuration</a></td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}><a href="#qwen35-27b-64k-1k-50ms-on-a3-2-cards-mixed-mode">Optimal Configuration</a></td>
    </tr>
 </tbody>
 </table>
@@ -4215,7 +4215,7 @@ We tested it based on the `RANDOM` dataset.
 python3 -m sglang.bench_serving --dataset-name random --backend sglang --host 127.0.0.1 --port 6699 --random-range-ratio 1 --max-concurrency 1 --random-output-len 1500 --random-input-len 3500 --num-prompts 1
 ```

-### Qwen3-27B 3_5K-1_5K 20ms on A3 2 Cards Mixed Mode
+### Qwen3.5-27B 3_5K-1_5K 20ms on A3 2 Cards Mixed Mode

 Model: Eco-Tech/Qwen3.5-27B-w8a8-mtp

@@ -4297,7 +4297,7 @@ We tested it based on the `RANDOM` dataset.
 python3 -m sglang.bench_serving --backend sglang --host 127.0.0.1 --port 6699 --dataset-name random  --max-concurrency 38 --num-prompts 152 --random-range-ratio 1  --random-output-len 1500 --random-input-len 3500
 ```

-### Qwen3-27B 16K-1K 20ms on A3 1 Cards Mixed Mode
+### Qwen3.5-27B 16K-1K 20ms on A3 1 Cards Mixed Mode

 Model: Eco-Tech/Qwen3.5-27B-w8a8-mtp

@@ -4370,7 +4370,7 @@ We tested it based on the `RANDOM` dataset.
 python3 -m sglang.bench_serving --backend sglang --host 127.0.0.1 --port 8001 --dataset-name random  --max-concurrency 32 --num-prompts 128 --random-range-ratio 1  --random-output-len 1000 --random-input-len 16000
 ```

-### Qwen3-27B 64K-1K 20ms on A3 1 Cards Mixed Mode
+### Qwen3.5-27B 64K-1K 20ms on A3 1 Cards Mixed Mode

 Model: Eco-Tech/Qwen3.5-27B-w8a8-mtp

@@ -4442,7 +4442,7 @@ We tested it based on the `RANDOM` dataset.
 python3 -m sglang.bench_serving --backend sglang --host 127.0.0.1 --port 8004 --dataset-name random  --max-concurrency 9 --num-prompts 36 --random-range-ratio 1  --random-output-len 1000 --random-input-len 64000
 ```

-### Qwen3-27B 3_5K-1_5K 50ms on A3 1 Cards Mixed Mode
+### Qwen3.5-27B 3_5K-1_5K 50ms on A3 1 Cards Mixed Mode

 Model: Eco-Tech/Qwen3.5-27B-w8a8-mtp

@@ -4513,7 +4513,7 @@ We tested it based on the `RANDOM` dataset.
 python3 -m sglang.bench_serving --backend sglang --host 127.0.0.1 --port 8000 --dataset-name random  --max-concurrency 48 --num-prompts 192 --random-range-ratio 1  --random-output-len 1500 --random-input-len 3500
 ```

-### Qwen3-27B 16K-1K 50ms on A3 2 Cards Mixed Mode
+### Qwen3.5-27B 16K-1K 50ms on A3 2 Cards Mixed Mode

 Model: Eco-Tech/Qwen3.5-27B-w8a8-mtp

@@ -4586,7 +4586,7 @@ We tested it based on the `RANDOM` dataset.
 python3 -m sglang.bench_serving --backend sglang --host 127.0.0.1 --port 8001 --dataset-name random  --max-concurrency 28 --num-prompts 152 --random-range-ratio 1  --random-output-len 1000 --random-input-len 16000
 ```

-### Qwen3-27B 64K-1K 50ms on A3 2 Cards Mixed Mode
+### Qwen3.5-27B 64K-1K 50ms on A3 2 Cards Mixed Mode

 Model: Eco-Tech/Qwen3.5-27B-w8a8-mtp

@@ -281,3 +281,10 @@ The machine cannot directly access the HuggingFace server due to network restric
    --dataset-path /path/to/ShareGPT_V3_unfiltered_cleaned_split.json \
    ...
  ```
+## 7. Unexpected type fp8
+
+### Cause
+FP8 model is not supported
+
+### Solution
+Replace model weights, e.g., switch Qwen/Qwen3.5-27B-FP8 to Eco-Tech/Qwen3.5-27B-w8a8-mtp.
@@ -390,6 +390,413 @@ The following environment variables are used in other best practice configuratio
  </tbody>
 </table>

+## Feature Compatibility
+
+The table shows feature-by-feature compatibility of SGLang capabilities on Ascend NPU. Features that have performance impact on Ascend are included. Each cell at the intersection of two features indicates whether they can be used together.
+
+The symbols used are defined as follows:
+- 🟢 = Full compatibility
+- 🟠 = Partial compatibility
+- ❌ = No compatibility
+- ❔ = Unknown or TBD
+
+<table>
+  <thead>
+    <tr>
+      <th>Feature</th>
+      <th>Tensor Parallelism</th>
+      <th>Data Parallelism</th>
+      <th>Expert Parallelism</th>
+      <th>Context Parallelism</th>
+      <th>PD Disaggregation</th>
+      <th>Quantization</th>
+      <th>Chunked Prefill</th>
+      <th>NPU Graph</th>
+      <th>Speculative Decoding</th>
+      <th>PrefixCache</th>
+      <th>Overlap Schedule</th>
+      <th>DP LM Head</th>
+      <th>MLAPO</th>
+      <th>Multistream MoE</th>
+      <th>EPLB</th>
+      <th>NZ Weight Format</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td>Tensor Parallelism</td>
+      <td></td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟠</td>
+      <td>🟠</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+    </tr>
+    <tr>
+      <td>Data Parallelism</td>
+      <td>🟢</td>
+      <td></td>
+      <td>🟢</td>
+      <td>🟠</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟠</td>
+      <td>🟠</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+    </tr>
+    <tr>
+      <td>Expert Parallelism</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td></td>
+      <td>🟠</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟠</td>
+      <td>🟠</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟠</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+    </tr>
+    <tr>
+      <td>Context Parallelism</td>
+      <td>🟢</td>
+      <td>🟠</td>
+      <td>🟠</td>
+      <td></td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟠</td>
+      <td>🟠</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟠</td>
+      <td>❔</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+    </tr>
+    <tr>
+      <td>PD Disaggregation</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td></td>
+      <td>🟢</td>
+      <td>🟠</td>
+      <td>🟠</td>
+      <td>🟢</td>
+      <td>🟠</td>
+      <td>🟢</td>
+      <td>🟠</td>
+      <td>🟠</td>
+      <td>🟢</td>
+      <td>🟠</td>
+      <td>🟢</td>
+    </tr>
+    <tr>
+      <td>Quantization</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td></td>
+      <td>🟠</td>
+      <td>🟠</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟠</td>
+      <td>🟠</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+    </tr>
+    <tr>
+      <td>Chunked Prefill</td>
+      <td>🟠</td>
+      <td>🟠</td>
+      <td>🟠</td>
+      <td>🟠</td>
+      <td>🟠</td>
+      <td>🟠</td>
+      <td></td>
+      <td>🟠</td>
+      <td>🟠</td>
+      <td>🟢</td>
+      <td>🟠</td>
+      <td>🟠</td>
+      <td>🟠</td>
+      <td>🟠</td>
+      <td>🟠</td>
+      <td>🟢</td>
+    </tr>
+    <tr>
+      <td>NPU Graph</td>
+      <td>🟠</td>
+      <td>🟠</td>
+      <td>🟠</td>
+      <td>🟠</td>
+      <td>🟠</td>
+      <td>🟠</td>
+      <td>🟠</td>
+      <td></td>
+      <td>🟠</td>
+      <td>🟢</td>
+      <td>🟠</td>
+      <td>🟠</td>
+      <td>🟠</td>
+      <td>🟠</td>
+      <td>🟠</td>
+      <td>🟠</td>
+    </tr>
+    <tr>
+      <td>Speculative Decoding</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟠</td>
+      <td>🟠</td>
+      <td></td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟠</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+    </tr>
+    <tr>
+      <td>PrefixCache</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟠</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td></td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+    </tr>
+    <tr>
+      <td>Overlap Schedule</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟠</td>
+      <td>🟢</td>
+      <td>🟠</td>
+      <td>🟠</td>
+      <td>🟠</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td></td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+    </tr>
+    <tr>
+      <td>DP LM Head</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟠</td>
+      <td>❔</td>
+      <td>🟠</td>
+      <td>🟠</td>
+      <td>🟠</td>
+      <td>🟠</td>
+      <td>🟠</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td></td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟠</td>
+      <td>🟢</td>
+    </tr>
+    <tr>
+      <td>MLAPO</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟠</td>
+      <td>🟢</td>
+      <td>🟠</td>
+      <td>🟠</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td></td>
+      <td>🟢</td>
+      <td>❔</td>
+      <td>🟢</td>
+    </tr>
+    <tr>
+      <td>Multistream MoE</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟠</td>
+      <td>🟠</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td></td>
+      <td>❔</td>
+      <td>🟢</td>
+    </tr>
+    <tr>
+      <td>EPLB</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟠</td>
+      <td>🟢</td>
+      <td>🟠</td>
+      <td>🟠</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟠</td>
+      <td>❔</td>
+      <td>❔</td>
+      <td></td>
+      <td>🟠</td>
+    </tr>
+    <tr>
+      <td>NZ Weight Format</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟠</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟢</td>
+      <td>🟠</td>
+      <td></td>
+    </tr>
+  </tbody>
+</table>
+
+## Feature descriptions
+
+### Tensor Parallelism (`--tp-size`)
+
+Splits model weights across multiple NPU devices so that large models can be loaded and run cooperatively. Each device holds a shard of every weight tensor and communicates via HCCL all-reduce. This is the primary mechanism for deploying models that exceed single-device memory.
+
+### Data Parallelism (`--dp-size`, `--enable-dp-attention`)
+
+Replicates the model across independent device groups to increase throughput by processing multiple requests in parallel. With --enable-dp-attention, the attention layers are replicated across DP ranks while the FFN/MoE layers remain tensor-parallel/expert-parallel. This reduces the communication overhead of attention layers in TP (which is significant during small-batch decode) while keeping the heavy compute and memory demands of MoE layers distributed.
+
+### Expert Parallelism (`--ep-size`)
+
+Each device holds a subset of experts and routes tokens via all-to-all communication based on the model's gating network. Supports load balancing methods including default round-robin expert placement and dynamic Expert Parallelism Load Balancing (EPLB).
+
+### Context Parallelism (`--attn-cp-size`)
+
+Splits long input sequences across devices so that the KV cache and attention computation for a single request are distributed. This enables serving very long context lengths (e.g., 128K tokens) that exceed single-device memory. On Ascend, `--attn-cp-size` must equal `--tp-size`.
+
+### PD Disaggregation (`--disaggregation-mode`)
+
+Separates the prefill (P) and decode (D) phases onto different device groups, allowing each phase to be independently optimized for its compute and memory characteristics. Prefill nodes handle long input processing at high throughput, while decode nodes focus on low-latency token generation. Uses the Ascend MemFabric transfer backend (`--disaggregation-transfer-backend ascend`).
+
+### Quantization (`--quantization`)
+
+Reduces model weight and activation precision (e.g., W8A8, W4A8, W4A16) to decrease memory usage and increase throughput. On Ascend, the natively supported and highly optimized quantization method is ModelSlim (supporting W4A4, W8A8, W4A8 dynamic/static). Support for other community formats (such as AWQ, GPTQ, Auto-round, and Compressed-tensors) depends on the availability of specific Ascend custom kernels in your environment; see quantization for details and compatibility matrices.
+
+### Chunked Prefill (`--chunked-prefill-size`)
+
+Breaks large prefill computations into smaller, fixed-size chunks for better scheduling interleaving with decode batches. This prevents long prefill requests from blocking decode latency. Use `-1` to disable chunked prefill on dedicated prefill nodes. Has partial compatibility with most features because chunk boundaries introduce scheduling complexity.
+
+### NPU Graph (`--cuda-graph-bs`)
+
+Captures the compute graph on NPU and replays it to eliminate kernel launch overhead, analogous to CUDA Graph on NVIDIA GPUs. Internally uses `torch.npu.NPUGraph`. Most effective for decode with stable, predictable batch sizes (via `--cuda-graph-bs`). Has partial compatibility with most features because graph capture requires fixed control flow and tensor shapes.
+
+### Speculative Decoding (`--speculative-algorithm`)
+
+Reduces per-token latency by predicting multiple future tokens in a single forward pass, then verifying them against the model. Ascend supports `NEXTN` (for DeepSeek models, using the model's own hidden states) and `EAGLE3` (for Qwen MoE models, using a separate draft model). Controlled by `--speculative-num-steps`, `--speculative-eagle-topk`, and `--speculative-num-draft-tokens`. On Ascend, `SGLANG_ENABLE_SPEC_V2=1` and `SGLANG_ENABLE_OVERLAP_PLAN_STREAM=1` enable the optimized overlap scheduler for speculative decoding.
+
+### PrefixCache (`--disable-radix-cache`)
+
+Reuses KV cache across requests that share common prompt prefixes (Radix Cache), reducing repeated prefill computation and lowering time-to-first-token. Enabled by default; disable with `--disable-radix-cache` when prefix reuse is not expected (e.g., dedicated prefill nodes in PD disaggregation, or random-input benchmarks).
+
+### Overlap Schedule
+
+On Ascend, this is primarily controlled via environment variable SGLANG_ENABLE_OVERLAP_PLAN_STREAM=1. For speculative decoding scenarios, combining it with SGLANG_ENABLE_SPEC_V2=1 enables the optimized overlap scheduler. Most effective during decode with speculative decoding and DP attention, where expert dispatch/reduce communication can be overlapped with the next batch's computation.
+
+### DP LM Head (`--enable-dp-lm-head`)
+
+Shards the language model head (the final vocabulary projection layer) across DP ranks to reduce memory consumption and improve decode throughput. Without this, each DP rank holds a full copy of the LM head, which is wasteful for large vocabularies. Typically used together with `--enable-dp-attention`.
+
+### MLAPO (`SGLANG_NPU_USE_MLAPO=1`)
+
+Enables the MLAPO (MLA Pre-Processing Optimization) fusion operator for DeepSeek-style Multi-head Latent Attention models, replacing multiple separate kernels with a single fused kernel for Q/K/V projection and absorption. Reduces prefill latency and memory bandwidth usage. Used in most DeepSeek best-practice configurations.
+
+### Multistream MoE (`SGLANG_NPU_USE_MULTI_STREAM=1`)
+
+Enables dual-stream parallel execution for MoE layers, where shared experts and routing experts run concurrently on separate NPU streams. This overlaps shared-expert computation with routed-expert dispatch, improving MoE layer throughput.
+
+### EPLB (`--enable-eplb`)
+
+Enables Expert Parallelism Load Balancing to dynamically redistribute experts across devices based on workload, ensuring even expert utilization and preventing hotspots. Uses the `deepseek` balancing algorithm. Compatible with the `deepep` MoE A2A backend, but not with `ascend_fuseep`.
+
+### NZ Weight Format (`SGLANG_NPU_DISABLE_ACL_FORMAT_WEIGHT`)
+
+Casts model weight tensors to the Ascend NPU ACL FRACTAL_NZ format (format 29) for improved memory access efficiency on Da Vinci AI cores. Enabled by default globally; disable with `SGLANG_NPU_DISABLE_ACL_FORMAT_WEIGHT=1` when the format conflicts with other features (e.g., EPLB). Applied to linear weights, MoE weights, quantized weights, MLA QKV projections, and KV Cache (via `PA_NZ` / `nzcache` modes for FIA). Related env var: `SGLANG_USE_FIA_NZ=1` (requires `SGLANG_NPU_USE_MLAPO=1`).
+
+
 ## See Also

 - [Ascend NPU Best Practice](/docs/hardware-platforms/ascend-npus/ascend_npu_best_practice) — Complete deployment configurations and benchmark results for all supported models