minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity

Update dspark draft path in Inkling small cookbook (#34250)

Browse Source
This commit is contained in:
Ke Bao
2026-08-10 17:12:51 +08:00
committed by GitHub
parent a76a167812
commit 430f38ea25
2 changed files with 5 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/cookbook/autoregressive/ThinkingMachines/Inkling-Small.mdx
+1 -1
View File
@@ -307,6 +307,6 @@ To try it, select the **Long Context** strategy in the Deploy panel above for an
### 3.7 DSpark (Speculative Decoding)
The **DSpark** deploy strategy is the second speculative-decoding path for Inkling-Small. Unlike **MTP**, which drives Inkling-Small's own multi-layer draft head, DSpark runs a **separate draft checkpoint** — `RadixArk/Inkling-Small-DSpark-Preview` — served unquantized alongside the NVFP4 target.
The **DSpark** deploy strategy is the second speculative-decoding path for Inkling-Small. Unlike **MTP**, which drives Inkling-Small's own multi-layer draft head, DSpark runs a **separate draft checkpoint** — `RadixArk/Inkling-Small-DSpark` — served unquantized alongside the NVFP4 target.
DSpark support ships in the images listed in §1 (`dev-inkling-dspark` for CUDA 13, `dev-cu12-inkling-dspark` for CUDA 12), so no separate build is needed. Verified end-to-end on B200 (TP=8, NVFP4).
docs/src/snippets/configs/thinkingmachines/inkling-small.jsx
+4 -4
View File
@@ -751,7 +751,7 @@ export const config = {
"--tool-call-parser inkling",
"--skip-server-warmup",
"--speculative-algorithm DSPARK",
"--speculative-draft-model-path RadixArk/Inkling-Small-DSpark-Preview",
"--speculative-draft-model-path RadixArk/Inkling-Small-DSpark",
"--speculative-draft-model-quantization unquant",
"--chunked-prefill-size 8192",
"--cuda-graph-max-bs-prefill 8192",
@@ -786,7 +786,7 @@ export const config = {
"--tool-call-parser inkling",
"--skip-server-warmup",
"--speculative-algorithm DSPARK",
"--speculative-draft-model-path RadixArk/Inkling-Small-DSpark-Preview",
"--speculative-draft-model-path RadixArk/Inkling-Small-DSpark",
"--speculative-draft-model-quantization unquant",
"--chunked-prefill-size 8192",
"--cuda-graph-max-bs-prefill 8192",
@@ -821,7 +821,7 @@ export const config = {
"--tool-call-parser inkling",
"--skip-server-warmup",
"--speculative-algorithm DSPARK",
"--speculative-draft-model-path RadixArk/Inkling-Small-DSpark-Preview",
"--speculative-draft-model-path RadixArk/Inkling-Small-DSpark",
"--speculative-draft-model-quantization unquant",
"--chunked-prefill-size 8192",
"--cuda-graph-max-bs-prefill 8192",
@@ -856,7 +856,7 @@ export const config = {
"--tool-call-parser inkling",
"--skip-server-warmup",
"--speculative-algorithm DSPARK",
"--speculative-draft-model-path RadixArk/Inkling-Small-DSpark-Preview",
"--speculative-draft-model-path RadixArk/Inkling-Small-DSpark",
"--speculative-draft-model-quantization unquant",
"--chunked-prefill-size 8192",
"--cuda-graph-max-bs-prefill 8192",