minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

Ensure GPU work is finished when release memory occupation call is finished (#12592)

Browse Source
This commit is contained in:
fzyzcjy
2025-11-04 18:01:27 +08:00
committed by GitHub
parent d84790db39
commit ff0b64e1e6
1 changed files with 2 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
python/sglang/srt/managers/scheduler_update_weights_mixin.py
+2
View File
@@ -132,6 +132,8 @@ class SchedulerUpdateWeightsMixin:
if GPU_MEMORY_TYPE_CUDA_GRAPH in tags: if GPU_MEMORY_TYPE_CUDA_GRAPH in tags:
self.memory_saver_adapter.pause(GPU_MEMORY_TYPE_CUDA_GRAPH) self.memory_saver_adapter.pause(GPU_MEMORY_TYPE_CUDA_GRAPH)
torch.cuda.synchronize()
return ReleaseMemoryOccupationReqOutput() return ReleaseMemoryOccupationReqOutput()
def resume_memory_occupation( def resume_memory_occupation(