[Refactor] Cuda Graph Runner/Backend Refactor (#23906)

Co-authored-by: BBuf <1182563586@qq.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
This commit is contained in:
Yuwei An
2026-06-09 21:36:57 -07:00
committed by GitHub
co-authored by BBuf Cheng Wan Lianmin Zheng
parent 56f06278c6
commit 2495c02c2c
160 changed files with 5212 additions and 3083 deletions
@@ -325,7 +325,7 @@ def _run_server_and_generate(
"--mem-fraction-static",
"0.5",
"--disable-cuda-graph",
"--disable-piecewise-cuda-graph",
"--cuda-graph-backend-prefill=disabled",
"--disable-radix-cache",
]
if extra_server_args: