Add Inkling model support (#31681)
Co-authored-by: Chunan Zeng <zcnrex@gmail.com> Co-authored-by: Ke Bao <ispobaoke@gmail.com> Co-authored-by: Yanbin Jiang <jybsuper@gmail.com> Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com> Co-authored-by: Qiaolin Yu <qiaolin.yu@radixark.ai> Co-authored-by: Zhichen Zeng <zczeng@uw.edu> Co-authored-by: Aurick Qiao <aurick@thinkingmachines.ai> Co-authored-by: Joseph <jk@thinkingmachines.ai>
This commit is contained in:
co-authored by
Chunan Zeng
Ke Bao
Yanbin Jiang
Yuhao Yang
Qiaolin Yu
Zhichen Zeng
Aurick Qiao
Joseph
parent
829e9ce9d5
commit
02236fa38c
@@ -309,6 +309,7 @@ def _init_cuda_graph_replay_metadata(backend, capture_batch_size: int, batch):
|
||||
seq_lens_sum=batch.seq_lens_sum,
|
||||
seq_lens_cpu=batch.seq_lens_cpu,
|
||||
encoder_lens=batch.encoder_lens,
|
||||
extend_seq_lens=getattr(batch, "extend_seq_lens", None),
|
||||
out_cache_loc=getattr(batch, "out_cache_loc", None),
|
||||
spec_info=batch.spec_info,
|
||||
)
|
||||
|
||||
Reference in New Issue
Block a user