minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 13 Packages Projects Releases Wiki Activity

[CP V1 Deprecation 5/5] Update prefill CP documentation (#36230)

Browse Source
This commit is contained in:
Baizhou Zhang
2026-09-08 19:27:49 -07:00
committed by GitHub
parent 8ab9982851
commit e54ff1efb9
7 changed files with 30 additions and 73 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/cookbook/autoregressive/DeepSeek/DeepSeek-V3_2.mdx
+14 -35
View File
@@ -432,25 +432,19 @@ For production deployments (RBG / LWS-based, DeepEP EP parallelism), see [multi_
#### 4.2.5 DSA Long-Sequence Context Parallel and PP/CP
SGLang provides two context parallel (CP) modes for long-sequence workloads, controlled with `--dsa-prefill-cp-mode`.
Enable prefill context parallelism (CP) on CUDA with `--enable-prefill-cp --cp-strategy interleave` for long-sequence workloads.
**In-sequence splitting** (`--dsa-prefill-cp-mode in-seq-split`): Each CP rank handles a uniform shard of the sequence; KV cache is gathered via all-gather. Batch size is restricted to 1 during prefill. See [PR #12065](https://github.com/sgl-project/sglang/pull/12065).
<Warning>
Zigzag prefill CP (`--cp-strategy zigzag`) is temporarily unavailable for DeepSeek V3.2, GLM-5, GLM-5.1, GLM-5.2, and GLM-5.3. Use `interleave` for these models and keep `--dp 1`; interleave DSA CP does not support `--dp` greater than 1.
</Warning>
**Interleave** (`--cp-strategy interleave`): Distributes tokens by `token_idx % cp_size`. It supports fused MoE, FP8 KV cache, and multi-batch prefill.
```bash Command
# In-seq splitting mode — EP + DP, batch size 1
# Interleave — FusedMoE + CP8
python -m sglang.launch_server --model deepseek-ai/DeepSeek-V3.2-Exp \
--tp 8 --ep 8 --dp 2 --enable-dp-attention \
--enable-dsa-prefill-context-parallel --attn-cp-size 4 \
--dsa-prefill-cp-mode in-seq-split --max-running-requests 32
```
**Round-robin splitting** (`--dsa-prefill-cp-mode round-robin-split`, default): Distributes tokens by `token_idx % cp_size`. Supports fused MoE, FP8 KV cache, and multi-batch prefill. Cannot be combined with DP attention. See [PR #13959](https://github.com/sgl-project/sglang/pull/13959).
```bash Command
# Round-robin splitting — FusedMoE + CP8
python -m sglang.launch_server --model deepseek-ai/DeepSeek-V3.2-Exp \
--tp 8 --enable-dsa-prefill-context-parallel --attn-cp-size 8 \
--dsa-prefill-cp-mode round-robin-split --max-running-requests 32
--tp 8 --enable-prefill-cp --attn-cp-size 8 \
--cp-strategy interleave --max-running-requests 32
```
**PP + CP (multi-node):** Combines Pipeline Parallelism and Context Parallelism for cross-node scaling. The production-optimized configurations below have been verified on Hopper:
@@ -499,8 +493,8 @@ sglang_args=$(echo serve \
--attention-backend dsa \
--dsa-prefill-backend flashmla_sparse \
--dsa-decode-backend flashmla_sparse \
--enable-dsa-prefill-context-parallel \
--dsa-prefill-cp-mode round-robin-split \
--enable-prefill-cp \
--cp-strategy interleave \
--cuda-graph-max-bs-decode 128 \
--max-running-requests 128 \
--trust-remote-code --host "0.0.0.0" --port 30000 \
@@ -518,34 +512,19 @@ sglang "${sglang_args[@]}" 2>&1 | tee $LOG_DIR/$RANK.log
**fp8 KV + CP + PP**
With FP8 KV, we can have less memory footprint. This can be combined with various parallel schemes:
FP8 KV reduces the memory footprint. For DeepSeek V3.2, combine it with interleave CP and PP while keeping `--dp 1`:
```shell Command
# verified in Hopper platform
dp=1
dp_config=" \
--dp 1 --enable-dp-attention \
"
cp_config=" \
--enable-dsa-prefill-context-parallel \
--enable-prefill-cp \
--cp-strategy interleave \
"
if [ "$dp" -eq 1 ]; then
cp_config=" \
$cp_config \
--dsa-prefill-cp-mode round-robin-split \
"
else
cp_config=" \
$cp_config \
--dsa-prefill-cp-mode in-seq-split \
"
fi
# see discussion : https://github.com/sgl-project/sglang/pull/12065
sglang_args=$(echo serve \
--model-path $MAPPED_MODEL_PATH \