deprecate prefill-round-robin-balance (#16195)
Signed-off-by: Chang Huaixin (OpenAnolis) <changhuaixin@linux.alibaba.com> Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
This commit is contained in:
co-authored by
Liangsheng Yin
parent
3b3c5a05c1
commit
c1dfbc777b
@@ -207,7 +207,6 @@ Please consult the documentation below and [server_args.py](https://github.com/s
|
||||
| `--data-parallel-size`<br>`--dp-size` | The data parallelism size. | `1` | Type: int |
|
||||
| `--load-balance-method` | The load balancing strategy for data parallelism. The Minimum Token algorithm can only be used when DP attention is applied. This algorithm performs load balancing based on the real-time token load of the DP workers. | `auto` | `auto`, `round_robin`, `follow_bootstrap_room`, `shortest_queue`, `minimum_tokens` |
|
||||
| `--load-watch-interval` | The interval of load watching in seconds. | `0.1` | Type: float |
|
||||
| `--prefill-round-robin-balance` | Prefill is round robin balanced. This is used to promise decode server can get the correct dp rank. | `False` | bool flag (set to enable) |
|
||||
|
||||
## Multi-node distributed serving
|
||||
| Argument | Description | Defaults | Options |
|
||||
@@ -461,6 +460,7 @@ Please consult the documentation below and [server_args.py](https://github.com/s
|
||||
| --- | --- | --- | --- |
|
||||
| `--enable-ep-moe` | NOTE: --enable-ep-moe is deprecated. Please set `--ep-size` to the same value as `--tp-size` instead. | `None` | N/A |
|
||||
| `--enable-deepep-moe` | NOTE: --enable-deepep-moe is deprecated. Please set `--moe-a2a-backend` to 'deepep' instead. | `None` | N/A |
|
||||
| `--prefill-round-robin-balance` | Note: Note: --prefill-round-robin-balance is deprecated now. | `None` | N/A |
|
||||
| `--enable-flashinfer-cutlass-moe` | NOTE: --enable-flashinfer-cutlass-moe is deprecated. Please set `--moe-runner-backend` to 'flashinfer_cutlass' instead. | `None` | N/A |
|
||||
| `--enable-flashinfer-cutedsl-moe` | NOTE: --enable-flashinfer-cutedsl-moe is deprecated. Please set `--moe-runner-backend` to 'flashinfer_cutedsl' instead. | `None` | N/A |
|
||||
| `--enable-flashinfer-trtllm-moe` | NOTE: --enable-flashinfer-trtllm-moe is deprecated. Please set `--moe-runner-backend` to 'flashinfer_trtllm' instead. | `None` | N/A |
|
||||
|
||||
@@ -157,7 +157,6 @@ python -m sglang.launch_server \
|
||||
--speculative-num-steps 3 \
|
||||
--speculative-eagle-topk 1 \
|
||||
--speculative-num-draft-tokens 4 \
|
||||
--prefill-round-robin-balance \
|
||||
--disable-shared-experts-fusion \
|
||||
--dtype bfloat16 \
|
||||
--tokenizer-worker-num 4
|
||||
@@ -293,7 +292,6 @@ do
|
||||
--speculative-eagle-topk 1 \
|
||||
--speculative-num-draft-tokens 3 \
|
||||
--tokenizer-worker-num 4 \
|
||||
--prefill-round-robin-balance \
|
||||
--disable-shared-experts-fusion \
|
||||
--dtype bfloat16
|
||||
done
|
||||
|
||||
@@ -132,7 +132,6 @@ If you want to know the meaning and usage of each parameter, click [Service Argu
|
||||
|----------------------------------------|---------------|-------------------------------------------------------------|:----------------------------------------:|:----------------------------------------:|
|
||||
| `--data-parallel-size`<br/>`--dp-size` | `1` | Type: int | **<span style="color: green;">√</span>** | **<span style="color: green;">√</span>** |
|
||||
| `--load-balance-method` | `round_robin` | `round_robin`,<br/> `shortest_queue`,<br/> `minimum_tokens` | **<span style="color: green;">√</span>** | **<span style="color: green;">√</span>** |
|
||||
| `--prefill-round-robin-balance` | `False` | bool flag<br/> (set to enable) | **<span style="color: green;">√</span>** | **<span style="color: green;">√</span>** |
|
||||
|
||||
## Multi-node distributed serving
|
||||
|
||||
|
||||
@@ -260,7 +260,6 @@ spec:
|
||||
- mlx5_0,mlx5_1,mlx5_2,mlx5_3,mlx5_4,mlx5_5,mlx5_6,mlx5_7
|
||||
- --chunked-prefill-size
|
||||
- "131072"
|
||||
- --prefill-round-robin-balance
|
||||
- --eplb-rebalance-layers-per-chunk
|
||||
- "29"
|
||||
- --page-size
|
||||
@@ -332,7 +331,6 @@ spec:
|
||||
- /log
|
||||
- --chunked-prefill-size
|
||||
- "262144"
|
||||
- --prefill-round-robin-balance
|
||||
- --eplb-rebalance-layers-per-chunk
|
||||
- "29"
|
||||
- --page-size
|
||||
|
||||
Reference in New Issue
Block a user