minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

Tiny fix eos handling for PD disaggregation (#12334)

Browse Source 
Signed-off-by: Shangming Cai <csmthu@gmail.com>
This commit is contained in:
Shangming Cai
2025-10-31 17:57:10 -07:00
committed by GitHub
parent 41c10e67fc
commit 358ae3563d
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
python/sglang/srt/disaggregation/decode.py
+1 -1
View File
@@ -776,7 +776,7 @@ class DecodeTransferQueue:
# special handling for corner cases # special handling for corner cases
should_finish = ( should_finish = (
decode_req.req.sampling_params.max_new_tokens == 1 decode_req.req.sampling_params.max_new_tokens == 1
or output_id in decode_req.req.eos_token_ids or decode_req.req.output_ids[-1] in decode_req.req.eos_token_ids
) )
if should_finish: if should_finish:
# finish immediately # finish immediately