[Mamba] Describe the track snapshot dtype by its kernel instead of fp32 (#43192)
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com> Co-authored-by: jiayisuse <jiayisuse@users.noreply.github.com>
This commit is contained in:
co-authored by
Hanming Lu
jiayisuse
parent
e3c5719713
commit
075fd8fa0b
@@ -1002,8 +1002,9 @@ class MambaAttnBackendBase(AttentionBackend):
|
||||
"""Copy extend SSM state at the last chunk boundary to track slots (source
|
||||
depends on chunk alignment; see `_init_track_ssm_indices`).
|
||||
|
||||
Unaligned rows read the fp32 ``h_track_buf`` snapshot written in-kernel
|
||||
when given (its rows follow the batch, selected by the integer index
|
||||
Unaligned rows read the ``h_track_buf`` snapshot written in-kernel
|
||||
when given (in the kernel's native state dtype; its rows follow the batch,
|
||||
selected by the integer index
|
||||
``track_ssm_h_batch_src`` — a boolean mask would nonzero() and sync the
|
||||
stream once per layer); otherwise they fall back to the per-chunk
|
||||
states ``h`` (already rounded to the activation dtype)."""
|
||||
|
||||
Reference in New Issue
Block a user