minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

docs: link dots3.note checkpoints, add H100 cells (#34797)

Browse Source 
Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
Yuhao Yang
2026-08-14 02:42:27 +00:00
committed by GitHub
co-authored by Claude
parent 456c5551cc
commit 6ad3f2d8fd
4 changed files with 128 additions and 21 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/cookbook/autoregressive/RedNote/Dots3-Note.mdx
+2 -12
View File
@@ -46,16 +46,12 @@ For how to launch the image, see [Install → Method 3: Using Docker](../../../d
</Accordion>
Pick the checkpoint precision — the only deployment choice. The recipe runs on a single 8-GPU H200 node with DP8 attention × TP8 × EP8 and DeepEP as the MoE all-to-all transport. Blackwell is not supported yet.
Pick the hardware and the checkpoint precision. The recipe runs on a single 8-GPU Hopper node with DP8 attention × TP8 × EP8 and DeepEP as the MoE all-to-all transport. Blackwell is not supported yet.
**Precision** — selects the MoE path, not just the weights. The BF16 cells pin `--moe-runner-backend deep_gemm` with BF16 DeepEP dispatch output (JIT DeepGEMM is enabled via `SGLANG_ENABLE_JIT_DEEPGEMM=1`). The FP8 cells leave both at `auto` and let SGLang resolve the runner from the checkpoint's quantization config.
**Spec Decode** — NEXTN is on in every cell: 3 draft steps, 4 draft tokens per step, and the draft model path pointing at the target checkpoint itself. dots3's MTP layer is full-sharing — it carries the dots3 sliding-window attention geometry and reuses the target LM head — so no separate draft checkpoint is needed. Target verification and draft extension run on the paged, absorbed SWA-MLA FA3 path.
<Note>
Every cell in the Deploy panel above is currently **unverified**: the recipe runs, but no serving round on public weights has landed (the checkpoint is not yet released). Treat the cells as starting points and re-measure throughput and accuracy on your workload.
</Note>
import { Deployment } from "/src/snippets/_deployment.jsx";
import { config } from "/src/snippets/configs/rednote/dots3-note.jsx";
@@ -70,13 +66,7 @@ dots3.note is RedNote's native multimodal omni model, built on the dots3 languag
- **Hybrid attention** — dots3 combines MLA with full-attention and sliding-window layers of different geometry, attention gates, and optional DSA indexing on full-attention layers.
- **MTP speculative decoding** — a full-sharing MTP/NextN architecture exposes one recursively shared, SWA-shaped MTP layer and shares the target LM head.
<Note>
The dots3.note checkpoint is **not yet publicly released**. The recipes on this page were validated against [SGLang PR #33829](https://github.com/sgl-project/sglang/pull/33829); a Hugging Face repository will be linked here at launch.
</Note>
**Resources:** [SGLang PR #33829](https://github.com/sgl-project/sglang/pull/33829)
{/* TODO: Add the Hugging Face link once the checkpoint is released. */}
**Resources:** [Hugging Face](https://huggingface.co/dots-studio/dots3-note-prev) · [SGLang PR #33829](https://github.com/sgl-project/sglang/pull/33829)
## 2. Configuration Tips
docs/src/snippets/_deployment.jsx
+7 -1
View File
@@ -39,7 +39,8 @@
// `verificationStatus` overrides it with a third state —
// "verified" | "in-progress" | "unverified" — for a recipe
// whose verification round is open rather than absent.
// modelNames HF slug lookup, `hw|variant|quant` then `variant|quant`
// modelNames HF slug lookup, `hw|variant|quant`, `variant|quant`,
// `hw|quant`, `quant`, `hw`, then `default`
// placeholders {{KEY}} → {target: 'command'|'curl', label, default?}
// curl cURL template (uses {{MODEL_NAME}} + placeholders), or
// `(selection, cell) => template` when the request payload
@@ -643,10 +644,15 @@ export const Deployment = ({ config, benchmarks }) => {
// Lookup walks most-specific to least so a config that drops the variant/quant
// dims can key its HF slug on `hw` alone, or on the single "default" entry.
// The `hw|quant` and bare `quant` rungs cover a `matchDims` config that declares
// no variant dim at all — there `sel.variant` is undefined, so the two leading
// keys can never hit.
const resolveModelName = (sel) => {
const keys = [
`${sel.hw}|${sel.variant}|${sel.quant}`,
`${sel.variant}|${sel.quant}`,
`${sel.hw}|${sel.quant}`,
sel.quant,
sel.hw,
"default",
];
docs/src/snippets/_playground.jsx
+12 -4
View File
@@ -171,11 +171,19 @@ export const Playground = ({ config }) => {
return null;
};
// hw|variant|quant → variant|quant → "".
// hw|variant|quant → variant|quant → hw|quant → quant → "".
const resolveModelName = (sel) => {
const triple = `${sel.hw}|${sel.variant}|${sel.quant}`;
const pair = `${sel.variant}|${sel.quant}`;
return config.modelNames[triple] ?? config.modelNames[pair] ?? "";
const keys = [
`${sel.hw}|${sel.variant}|${sel.quant}`,
`${sel.variant}|${sel.quant}`,
`${sel.hw}|${sel.quant}`,
sel.quant,
];
for (const k of keys) {
const hit = config.modelNames[k];
if (hit) return hit;
}
return "";
};
const interpolate = (text, env, modelName) =>
docs/src/snippets/configs/rednote/dots3-note.jsx
+107 -4
View File
@@ -9,7 +9,7 @@ export const config = {
showPlaygroundLink: false,
// Hopper only for now — no Blackwell support.
supportedHardware: ["h200"],
supportedHardware: ["h200", "h100"],
// One model and one node shape — only the checkpoint precision is a real choice.
matchDims: [
@@ -24,8 +24,8 @@ export const config = {
],
modelNames: {
// TODO: replace with the public repo id once the checkpoint is released.
default: "<dots-note-checkpoint>",
bf16: "dots-studio/dots3-note-prev",
fp8: "dots-studio/dots3-note-prev-fp8",
},
placeholders: {
@@ -55,6 +55,7 @@ export const config = {
dockerImages: {
h200: "lmsysorg/sglang:dev",
h100: "lmsysorg/sglang:dev",
},
@@ -62,6 +63,108 @@ export const config = {
{
match: { hw: "h200", quant: "bf16" },
nnodes: 1,
verified: true,
env: [
"SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1",
"SGLANG_ENABLE_JIT_DEEPGEMM=1",
"SGLANG_CHUNKED_PREFIX_CACHE_THRESHOLD=8192",
"SGLANG_MAX_KV_CHUNK_CAPACITY=8192",
"SGLANG_DEEPEP_NUM_MAX_DISPATCH_TOKENS_PER_RANK=128",
"SGLANG_WARMUP_TIMEOUT=1800",
],
flags: [
"--model-path {{MODEL_NAME}}",
"--context-length 524288",
"--enable-dp-attention",
"--dp-size 8",
"--tp-size 8",
"--ep-size 8",
"--mem-fraction-static 0.87",
"--max-running-requests 256",
"--chunked-prefill-size 16384",
"--trust-remote-code",
"--swa-full-tokens-ratio 0.03",
"--prefill-attention-backend fa3",
"--decode-attention-backend fa3",
"--page-size 64",
"--moe-dense-tp-size 1",
"--cuda-graph-backend-decode full",
"--cuda-graph-backend-prefill disabled",
"--cuda-graph-max-bs-decode 32",
"--speculative-algorithm NEXTN",
"--speculative-num-steps 3",
"--speculative-eagle-topk 1",
"--speculative-num-draft-tokens 4",
"--speculative-draft-model-path {{MODEL_NAME}}",
"--speculative-draft-attention-backend fa3",
"--moe-a2a-backend deepep",
"--moe-runner-backend deep_gemm",
"--deepep-dispatcher-output-dtype bf16",
"--deepep-mode auto",
"--enable-nccl-nvls",
"--enable-multimodal",
"--enable-metrics",
"--tool-call-parser dots",
"--reasoning-parser qwen3",
"--watchdog-timeout 1800",
"--host {{HOST_IP}}",
"--port {{PORT}}",
],
},
{
match: { hw: "h200", quant: "fp8" },
nnodes: 1,
verified: true,
env: [
"SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1",
"SGLANG_ENABLE_JIT_DEEPGEMM=1",
"SGLANG_CHUNKED_PREFIX_CACHE_THRESHOLD=8192",
"SGLANG_MAX_KV_CHUNK_CAPACITY=8192",
"SGLANG_DEEPEP_NUM_MAX_DISPATCH_TOKENS_PER_RANK=128",
"SGLANG_WARMUP_TIMEOUT=1800",
],
flags: [
"--model-path {{MODEL_NAME}}",
"--context-length 524288",
"--enable-dp-attention",
"--dp-size 8",
"--tp-size 8",
"--ep-size 8",
"--mem-fraction-static 0.87",
"--max-running-requests 256",
"--chunked-prefill-size 16384",
"--trust-remote-code",
"--swa-full-tokens-ratio 0.03",
"--prefill-attention-backend fa3",
"--decode-attention-backend fa3",
"--page-size 64",
"--moe-dense-tp-size 1",
"--cuda-graph-backend-decode full",
"--cuda-graph-backend-prefill disabled",
"--cuda-graph-max-bs-decode 32",
"--speculative-algorithm NEXTN",
"--speculative-num-steps 3",
"--speculative-eagle-topk 1",
"--speculative-num-draft-tokens 4",
"--speculative-draft-model-path {{MODEL_NAME}}",
"--speculative-draft-attention-backend fa3",
"--moe-a2a-backend deepep",
"--moe-runner-backend auto",
"--deepep-dispatcher-output-dtype auto",
"--deepep-mode auto",
"--enable-nccl-nvls",
"--enable-multimodal",
"--enable-metrics",
"--reasoning-parser qwen3",
"--tool-call-parser dots",
"--watchdog-timeout 1800",
"--host {{HOST_IP}}",
"--port {{PORT}}",
],
},
{
match: { hw: "h100", quant: "bf16" },
nnodes: 1,
verified: false,
env: [
"SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1",
@@ -111,7 +214,7 @@ export const config = {
],
},
{
match: { hw: "h200", quant: "fp8" },
match: { hw: "h100", quant: "fp8" },
nnodes: 1,
verified: false,
env: [