[Perf] Mamba2 selective_state_update up to 2x faster on B200 via 8x1 launch config for dstate 128 (+7.5% Nemotron-3-Super serving) (#41223)
Co-authored-by: hexu.zhao <zhaohexu2001@gmail.com>
This commit is contained in:
@@ -474,7 +474,7 @@ def selective_state_update(
|
||||
else (
|
||||
(16, 4)
|
||||
if dstate <= 32
|
||||
else ((8, 4) if dstate <= 64 else ((4, 4) if dstate <= 128 else ((4, 8))))
|
||||
else ((8, 4) if dstate <= 64 else ((8, 1) if dstate <= 128 else ((4, 8))))
|
||||
)
|
||||
)
|
||||
tie_hdim = (
|
||||
|
||||
Reference in New Issue
Block a user