fix【hicache】fix the KV cache resource occupation and invalid loading from prefetch when pending requests are aborted. (#16369)

Signed-off-by: CLFutureX <chenyongqyl@163.com> Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
2026-01-15 15:14:38 -08:00
parent 69822c7271
commit 8b22deef5b
3 changed files with 11 additions and 1 deletions
@@ -726,7 +726,6 @@ class HiCacheController:
                operation = self.prefetch_queue.get(block=True, timeout=1)
                if operation is None:
                    continue
-
                hash_value, storage_hit_count = self._storage_hit_query(operation)
                if self.tp_world_size > 1:
                    storage_hit_count_tensor = torch.tensor(