[Speculative] Support block verification for the DFlash family (#43012)

Co-authored-by: maocheng <mao.cheng@MacBook-Pro-MD663G60VV.local>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
maocheng23
2026-10-08 13:59:52 -07:00
committed by GitHub
co-authored by maocheng Claude Opus 5.5
parent d1ba943016
commit e902207a4e
9 changed files with 141 additions and 7 deletions
@@ -413,6 +413,14 @@ For stochastic EAGLE, EAGLE3, or NEXTN decoding on NVIDIA CUDA or AMD ROCm, add
with `--speculative-eagle-topk 1`. The draft model must produce a full target-vocabulary
proposal distribution, as required by the existing rejection sampler.
The same flag applies to the DFlash family (`DFLASH` and `DSPARK`) on CUDA or ROCm
when the draft samples its proposals: DSpark drafts, DFlash checkpoints with a
candidate selector, and [LiLiCorr](#lilicorr-reranked-dflash-drafts) drafts with
`SGLANG_ENABLE_LILICORR_SAMPLING=1`. These paths already publish their proposal
distribution, so `--speculative-use-rejection-sampling` is not needed. A DFlash draft
that proposes greedy tokens is a point mass, where block and token-wise verification
accept the same prefixes; the server warns that the flag has no effect.
[Block verification](https://arxiv.org/abs/2403.10444) checks every draft prefix and
keeps the longest accepted prefix, including when an earlier prefix was rejected.
It uses a corresponding correction distribution to preserve target sampling.