[Refactor] Encapsulate SWA loc translation inside SWAKVPool with per-batch cache invalidation (#25824)

Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-05-20 21:26:32 -07:00
parent 90efa9c83f
commit 79b937aefb
18 changed files with 298 additions and 174 deletions
@@ -33,6 +33,7 @@ def _make_self(*, page_size: int, full_available: int, swa_available: int):
        ),
        translate_loc_from_full_to_swa=lambda last_loc: last_loc,
        full_to_swa_index_mapping=torch.zeros(64, dtype=torch.int64),
+        _kvcache=SimpleNamespace(invalidate_loc_cache=lambda: None),
    )