minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

[spec] Consolidate the per-decode KV alloc reserve into one helper (#27458)

Browse Source
This commit is contained in:
Liangsheng Yin
2026-06-06 14:58:50 -07:00
committed by GitHub
parent 4989d6691d
commit 1c7acba579
6 changed files with 74 additions and 73 deletions
Download Patch File Download Diff File Expand all files Collapse all files
python/sglang/srt/debug_utils/pr_fix_toggle.py
+5 -10
View File
@@ -89,20 +89,15 @@ patches:
_PR_REVERT_YAML_26972 = """
patches:
- target: sglang.srt.model_executor.model_runner_kv_cache_mixin.ModelRunnerKVCacheMixin._init_pools
- target: sglang.srt.mem_cache.common.get_req_to_token_extra_context_len
edits:
- match: |
if (
self.server_args.speculative_algorithm is not None
and self.server_args.page_size > 1
and (self.server_args.speculative_eagle_topk or 1) > 1
server_args.speculative_algorithm is not None
and server_args.page_size > 1
and (server_args.speculative_eagle_topk or 1) > 1
):
from sglang.srt.managers.utils import get_alloc_len_per_decode
extra_max_context_len = max(
extra_max_context_len,
2 * get_alloc_len_per_decode(self.server_args),
)
extra = max(extra, get_alloc_reserve_per_decode(server_args))
replacement: ""
"""