(1/n)support context parallel with deepseekv3.2-DSA (#12065)

This commit is contained in:
lixiaolx
2025-11-16 20:12:25 -08:00
committed by GitHub
parent 7e626d12b7
commit d368c7451a
17 changed files with 1247 additions and 54 deletions
@@ -396,6 +396,7 @@ Please consult the documentation below and [server_args.py](https://github.com/s
| `--numa-node` | Sets the numa node for the subprocesses. i-th element corresponds to i-th subprocess. | `None` | List[int] |
| `--enable-layerwise-nvtx-marker` | Enable layerwise NVTX profiling annotations for the model. This adds NVTX markers to every layer for detailed per-layer performance analysis with Nsight Systems. | `False` | bool flag (set to enable) |
| `--enable-attn-tp-input-scattered` | Allow input of attention to be scattered when only using tensor parallelism, to reduce the computational load of operations such as qkv latent. | `False` | bool flag (set to enable) |
| `--enable-nsa-prefill-context-parallel` | Context parallelism used in the long sequence prefill phase of DeepSeek v3.2 | `False` | bool flag (set to enable) |
## Debug tensor dumps
| Argument | Description | Defaults | Options |