minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 13 Packages Projects Releases Wiki Activity

Increase post-capture decode memory reserve (#34996)

Browse Source
This commit is contained in:
Lianmin Zheng
2026-08-16 15:31:36 -07:00
committed by GitHub
parent f61f584347
commit b7eccd642f
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
python/sglang/srt/server_args.py
+1 -1
View File
@@ -4913,7 +4913,7 @@ class ServerArgs:
if self.post_capture_kv_sizing_planned(): if self.post_capture_kv_sizing_planned():
# Post-capture sizing measures free memory after graph capture, so # Post-capture sizing measures free memory after graph capture, so
# skip the graph/activation reserve; keep only the floor + parallel slack. # skip the graph/activation reserve; keep only the floor + parallel slack.
reserved_mem = 512 reserved_mem = 1536
reserved_mem += self.tp_size * self.pp_size / 8 * 1024 reserved_mem += self.tp_size * self.pp_size / 8 * 1024
else: else:
# Tokens the activation working set scales with (per serving mode). # Tokens the activation working set scales with (per serving mode).