[NPU] ascend backend support qwen3 moe attention cp (#21685)

2026-04-29 19:25:17 +08:00
parent ae0c036c24
commit 4c1eefca4f
3 changed files with 331 additions and 8 deletions
@@ -185,6 +185,86 @@ python3 -m sglang_router.launch_router \
   --prometheus-port 29010
 ```

+#### Running Qwen3-235B-A22B-Instruct-2507-W8A8 with Prefill Context Parallel (CP) on 2 x Atlas 800I A3
+
+This example enables **Prefill Context Parallel** (`--enable-prefill-context-parallel`) to split the context across CP ranks during prefill, reducing per-device memory pressure and improving TTFT for long sequences. PD disaggregation is required.
+
+> **Constraints**
+> - Prefill side must set `--max-running-requests 1` (PCP only supports batch_size=1)
+> - `--attn-cp-size` must evenly divide `--tp-size`; each CP rank occupies `tp_size / cp_size` NPUs
+
+**Prefill node <PREFILL_HOST_IP>:**
+
+```shell
+export SGLANG_SET_CPU_AFFINITY=1
+export ASCEND_MF_STORE_URL="tcp://<PREFILL_HOST_IP>:23456"
+export ASCEND_USE_FIA=True
+
+python3 -m sglang.launch_server \
+  --model-path /mnt/share/weights/Qwen3-235B-A22B-Instruct-2507-W8A8 \
+  --trust-remote-code \
+  --disaggregation-mode prefill \
+  --disaggregation-transfer-backend ascend \
+  --disaggregation-bootstrap-port 8995 \
+  --quantization modelslim \
+  --attention-backend ascend \
+  --skip-server-warmup \
+  --mem-fraction-static 0.7 \
+  --chunked-prefill-size 32768 \
+  --device npu \
+  --base-gpu-id 0 \
+  --tp-size 16 \
+  --enable-prefill-context-parallel \
+  --attn-cp-size 2 \
+  --moe-dp-size 2 \
+  --max-running-requests 1 \
+  --host <PREFILL_HOST_IP> \
+  --port 8000 \
+  --nnodes 1 \
+  --node-rank 0 \
+  --dist-init-addr <PREFILL_HOST_IP>:6688
+```
+
+Key parameters for PCP:
+
+| Parameter | Value | Description |
+|-----------|-------|-------------|
+| `--enable-prefill-context-parallel` | flag | Enable PCP feature |
+| `--attn-cp-size` | 2 | Split context across 2 CP ranks (each rank handles half the sequence) |
+| `--moe-dp-size` | 2 | MoE DP size, should match `--attn-cp-size` |
+| `--max-running-requests` | 1 | Required by PCP (batch_size=1 constraint) |
+
+**Decode node (<DECODE_HOST_IP>):**
+
+```shell
+export ASCEND_MF_STORE_URL="tcp://141.61.39.231:23456"
+export ASCEND_USE_FIA=True
+
+python3 -m sglang.launch_server \
+  --model-path /mnt/share/weights/Qwen3-235B-A22B-Instruct-2507-W8A8 \
+  --trust-remote-code \
+  --disaggregation-mode decode \
+  --disaggregation-transfer-backend ascend \
+  --quantization modelslim \
+  --attention-backend ascend \
+  --disable-radix-cache \
+  --disable-cuda-graph \
+  --mem-fraction-static 0.7 \
+  --chunked-prefill-size 32768 \
+  --skip-server-warmup \
+  --device npu \
+  --base-gpu-id 0 \
+  --tp-size 8 \
+  --max-running-requests 32 \
+  --host <DECODE_HOST_IP> \
+  --port 8001 \
+  --nnodes 1 \
+  --node-rank 0 \
+  --dist-init-addr <DECODE_HOST_IP>:6688
+```
+
+> **Note:** `ASCEND_MF_STORE_URL` on both nodes must point to the same KV store (typically the Prefill node IP). `ASCEND_USE_FIA=True` enables fast interconnect aggregation for KV transfer. PCP is a Prefill-only feature; the Decode side needs no CP-related flags.
+
 #### Running Qwen3-VL-8B-Instruct on 1 x Atlas 800I A3.

 Model weights could be found [here](https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct)