minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

[mtp] add rejection sampling for speculative decoding (#26312)

Browse Source 
Co-authored-by: lyc508653 <lyc508653@alibaba-inc.com>
Co-authored-by: Qiaolin-Yu <liin1211@outlook.com>
Co-authored-by: Huiqiang Jiang <30883354+iofu728@users.noreply.github.com>
Co-authored-by: Yi Zhang <25844240+yizhang2077@users.noreply.github.com>
Co-authored-by: Yizhong Cao <114661107+cao1zhg@users.noreply.github.com>
This commit is contained in:
Yucheng Li
2026-06-20 15:10:42 -07:00
committed by GitHub
co-authored by lyc508653 Qiaolin-Yu Huiqiang Jiang Yi Zhang Yizhong Cao
parent 95fb1ef697
commit f42ec350b4
11 changed files with 566 additions and 20 deletions
Download Patch File Download Diff File Expand all files Collapse all files
python/sglang/srt/model_executor/forward_batch_info.py
+4
View File
@@ -1336,6 +1336,10 @@ class ForwardBatch(ForwardBatchDeepSeekMHAMixin):
spec_info.topk_index = self._pad_tensor_to_size(
spec_info.topk_index, bs
)
if getattr(spec_info, "draft_probs", None) is not None:
spec_info.draft_probs = self._pad_tensor_to_size(
spec_info.draft_probs, bs
)
if getattr(spec_info, "num_correct_drafts", None) is not None:
spec_info.num_correct_drafts = self._pad_tensor_to_size(
spec_info.num_correct_drafts, bs