fix: wait for HiCache load-back before hybrid MLA KV reads (#38217)

Co-authored-by: zhouzechun <zhouzechun@kuaishou.com>
Co-authored-by: alphabetc1 <2508695655@qq.com>
This commit is contained in:
LoCinder
2026-10-11 09:50:13 +08:00
committed by GitHub
co-authored by zhouzechun alphabetc1
parent 9336681310
commit 156d54e56c
2 changed files with 21 additions and 1 deletions
@@ -4502,6 +4502,8 @@ class HybridLinearKVPool(KVCache):
dst_dtype: Optional[torch.dtype] = None,
):
assert self.use_mla, "get_mla_kv_buffer called when use_mla is False"
# Wait for HiCache load-back here; the inner pool's counter is disabled.
self._wait_for_layer(layer.layer_id)
# Read door -- same physical-id contract as the write door: `loc` is
# a read-index tensor already translated at its production site
# (fetch_mha_one_shot_kv_indices / prepare_chunked_kv_indices); the