(1/n)support context parallel with deepseekv3.2-DSA (#12065)
This commit is contained in:
@@ -396,6 +396,7 @@ Please consult the documentation below and [server_args.py](https://github.com/s
|
||||
| `--numa-node` | Sets the numa node for the subprocesses. i-th element corresponds to i-th subprocess. | `None` | List[int] |
|
||||
| `--enable-layerwise-nvtx-marker` | Enable layerwise NVTX profiling annotations for the model. This adds NVTX markers to every layer for detailed per-layer performance analysis with Nsight Systems. | `False` | bool flag (set to enable) |
|
||||
| `--enable-attn-tp-input-scattered` | Allow input of attention to be scattered when only using tensor parallelism, to reduce the computational load of operations such as qkv latent. | `False` | bool flag (set to enable) |
|
||||
| `--enable-nsa-prefill-context-parallel` | Context parallelism used in the long sequence prefill phase of DeepSeek v3.2 | `False` | bool flag (set to enable) |
|
||||
|
||||
## Debug tensor dumps
|
||||
| Argument | Description | Defaults | Options |
|
||||
|
||||
Reference in New Issue
Block a user