minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

[DSV4] perf: Enable alt stream during BCG prefill (#29070)

Browse Source 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
This commit is contained in:
Kaixi
2026-08-11 05:10:19 -07:00
committed by GitHub
co-authored by github-actions[bot] Claude Fable 5 Brayden Zhong
parent 546965fc72
commit a0a76e4485
1 changed files with 4 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
python/sglang/srt/models/deepseek_v4.py
+4 -1
View File
@@ -1262,7 +1262,10 @@ class MQALayer(MqaAttentionBase):
envs.SGLANG_OPT_USE_MULTI_STREAM_OVERLAP.get()
and self.alt_streams is not None
and get_is_capture_mode()
and x.shape[0] <= self._multi_stream_bs_limit
and (
is_in_breakable_cuda_graph()
or x.shape[0] <= self._multi_stream_bs_limit
)
and not (self.dsa_enable_prefill_cp and dsa_use_prefill_cp(forward_batch))
and not (_is_hip and self.compressor is None)
)