mem_cache: allow disabling the DSv4.1 SWA prefix-match gate via env

The unified-cache SWA component gates prefix matching on a live SWA window,
so a long prefix only matches up to the last sliding window and the tail is
recomputed. Add SGLANG_DSV4_SWA_GATES_PREFIX_MATCH (default True) so the gate
can be turned off for measurement; the SWA window is then rebuilt/recovered
downstream as for the req-ring paths. Measured on 4xB300 DP4 at 160k prefix:
c32 hit tokens 40.6% -> ~100% (with HiCache L2/L3 enabled), needle test
(cold vs hit) identical.
This commit is contained in:
2026-10-26 04:06:25 +00:00
parent 0a3c370451
commit f6fc6ac3e7
2 changed files with 5 additions and 1 deletions
+4
View File
@@ -633,6 +633,10 @@ class Envs:
# size the KV pool after CUDA-graph capture
SGLANG_ENABLE_POST_CAPTURE_KV_SIZING = EnvBool(False)
# DSv4.1 unified cache: allow a FULL prefix match even when the SWA component has
# no live value (the SWA window is rebuilt/recovered downstream).
SGLANG_DSV4_SWA_GATES_PREFIX_MATCH = EnvBool(True)
# ===================================================================
# Scheduler token budgeting and admission
# ===================================================================
@@ -352,7 +352,7 @@ def build_kv_cache(
# SWA the tree does not own must not gate prefix matching: a per-request ring,
# or paged SWA beside the target's encoder-replay request window, which then
# only the draft uses.
swa_gates_prefix_match = not (
swa_gates_prefix_match = envs.SGLANG_DSV4_SWA_GATES_PREFIX_MATCH.get() and not (
is_swa_req_ring(token_to_kv_pool_allocator)
or (
isinstance(token_to_kv_pool, DeepSeekV4TokenToKVPool)