[dLLM] Make FDFO a framework capability for all dLLM algorithms (#27551)
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
This commit is contained in:
co-authored by
Xiaoyu Zhang
parent
fc2ef35308
commit
e3ceccf781
@@ -16,6 +16,19 @@ python3 -m sglang.launch_server \
|
||||
--port 30000
|
||||
```
|
||||
|
||||
## First-Done-First-Out (FDFO) Scheduling
|
||||
|
||||
FDFO scheduling is **enabled by default**: each request leaves the batch as soon as its block is resolved, instead of advancing in lockstep where fast-converging requests must wait for slow long-tail requests before leaving the batch (head-of-line blocking). This improves throughput and is orthogonal to `--dllm-algorithm`, so it works with any dLLM algorithm. Pass `--no-dllm-fdfo` to fall back to synchronous lockstep scheduling:
|
||||
|
||||
```bash Command
|
||||
python3 -m sglang.launch_server \
|
||||
--model-path inclusionAI/LLaDA2.0-mini \
|
||||
--dllm-algorithm LowConfidence \
|
||||
--no-dllm-fdfo \
|
||||
--host 0.0.0.0 \
|
||||
--port 30000
|
||||
```
|
||||
|
||||
## Example Configuration File
|
||||
|
||||
Depending on the algorithm selected, the configuration parameters vary.
|
||||
|
||||
Reference in New Issue
Block a user