minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

Fix dp run error with fp8-kv enable in high concurrency test (#15241)

Browse Source 
Co-authored-by: wunhuang <wunhuang@amd.com>
This commit is contained in:
kk
2025-12-17 11:49:18 -08:00
committed by GitHub
co-authored by wunhuang
parent 8451e22758
commit 5d087891c9
1 changed files with 1 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files
python/sglang/srt/models/deepseek_v2.py
+1 -7
View File
@@ -411,13 +411,7 @@ def handle_attention_trtllm_mla(attn, forward_batch):
def handle_attention_aiter(attn, forward_batch): def handle_attention_aiter(attn, forward_batch):
if forward_batch.forward_mode.is_extend_without_speculative(): if forward_batch.forward_mode.is_extend_without_speculative():
if is_dp_attention_enabled(): return AttnForwardMethod.MHA
if sum(forward_batch.extend_prefix_lens_cpu) == 0:
return AttnForwardMethod.MHA
else:
return AttnForwardMethod.MLA
else:
return AttnForwardMethod.MHA
else: else:
return AttnForwardMethod.MLA return AttnForwardMethod.MLA