minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

Fix regression caused by fa3 block_table (#15009)

Browse Source
This commit is contained in:
Shu Wang
2025-12-12 19:11:09 -08:00
committed by GitHub
parent 7160283800
commit 665cb02003
1 changed files with 8 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
python/sglang/srt/layers/attention/flashattention_backend.py
+8 -1
View File
@@ -263,7 +263,14 @@ def make_local_attention_virtual_batches(
np.arange(actual_batch_size, dtype=np.int32), np.arange(actual_batch_size, dtype=np.int32),
local_blocks * pages_per_local_batch, local_blocks * pages_per_local_batch,
) )
block_table_local = block_table[batch_indices, block_indices].view(
# NOTE: https://github.com/pytorch/pytorch/pull/160256 causes performance
# regression when using numpy arrays (batch and block indices) to index into
# torch tensor (block_table). As a workaround, convert numpy arrays to torch
# tensor first, which recovers perf.
batch_indices_torch = torch.from_numpy(batch_indices)
block_indices_torch = torch.from_numpy(block_indices)
block_table_local = block_table[batch_indices_torch, block_indices_torch].view(
virtual_batches, -1 virtual_batches, -1
) )