[DFlash] Keep grouped-conv taps inside each request's block so NaN/Inf cannot leak across requests (#41736)
Co-authored-by: Rahul Chalamala <22563365+rchalamala@users.noreply.github.com> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This commit is contained in:
co-authored by
Rahul Chalamala
Devin AI
parent
42f2e4fcb7
commit
b51d4a04f0
@@ -473,7 +473,8 @@ def _grouped_conv(hidden_states, delta, base, block_size, num_groups, group_size
|
||||
position = position % block_size
|
||||
for tap in range(1, taps):
|
||||
shifted = F.pad(blocks[:-tap], (0, 0, 0, 0, tap, 0))
|
||||
out = out + coefficients[:, tap] * shifted * (position >= tap).view(-1, 1, 1)
|
||||
shifted = torch.where((position >= tap).view(-1, 1, 1), shifted, 0)
|
||||
out = out + coefficients[:, tap] * shifted
|
||||
return out.flatten(-2)
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user