[Refactor] Cuda Graph Runner/Backend Refactor (#23906)

Co-authored-by: BBuf <1182563586@qq.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
This commit is contained in:
Yuwei An
2026-06-09 21:36:57 -07:00
committed by GitHub
co-authored by BBuf Cheng Wan Lianmin Zheng
parent 56f06278c6
commit 2495c02c2c
160 changed files with 5212 additions and 3083 deletions
@@ -230,7 +230,7 @@ def _spawn_server_process(
kv_canary="raise",
kv_canary_real_data="partial",
kv_canary_sweep_interval=100,
disable_piecewise_cuda_graph=True,
disable_prefill_cuda_graph=True,
)
launch_kwargs.update(engine_kwargs)
http_port = launch_kwargs["port"]