From 88cf5c954193ba61a287aed5c8e0700979822107 Mon Sep 17 00:00:00 2001 From: zijiexia <37504505+zijiexia@users.noreply.github.com> Date: Mon, 31 Aug 2026 14:52:28 -0700 Subject: [PATCH] [Cookbook] Add DeepSeek-V4-Flash-Vision-Exp to the DeepSeek-V4 page (#37293) --- .claude/skills/cookbook-add-model/SKILL.md | 6 +- .../references/authoring-reference.md | 2 +- .../templates/config.jsx.tmpl | 14 +- .../autoregressive/DeepSeek/DeepSeek-V4.mdx | 75 ++++- docs/src/snippets/_deployment.jsx | 13 +- docs/src/snippets/_playground.jsx | 16 +- .../deepseek-ai/deepseek-v4-benchmarks.jsx | 24 ++ .../configs/deepseek-ai/deepseek-v4.jsx | 293 +++++++++++++++++- 8 files changed, 417 insertions(+), 26 deletions(-) diff --git a/.claude/skills/cookbook-add-model/SKILL.md b/.claude/skills/cookbook-add-model/SKILL.md index c0d2e5988..2d5e722fd 100644 --- a/.claude/skills/cookbook-add-model/SKILL.md +++ b/.claude/skills/cookbook-add-model/SKILL.md @@ -164,8 +164,10 @@ least one real deployment or capability boundary. Put orthogonal runtime feature 4. **Fill `cells[]`** with the verified recipes from Phase 1 (replace every EXAMPLE cell; set `verified: true` only on tested combos), and `modelNames` with real HF slugs, `dockerImages` for your hw (use the Phase-1 tag, or default `lmsysorg/sglang:dev` — never - a guessed release; key by `hw`, or `hw|quant` when one quant on a shared GPU needs its own - image), `multiNodeHints` only for fabric-specific hw (e.g. gb200). + a guessed release; key by `hw`, or a more specific key — `hw|variant|quant`, `variant|quant`, + `hw|quant|strategy`, `hw|quant`, resolved in that order before `hw` — when one + variant/quant/strategy needs its own image), `multiNodeHints` only for fabric-specific hw + (e.g. gb200). ### Site-wiring (do all three) diff --git a/.claude/skills/cookbook-add-model/references/authoring-reference.md b/.claude/skills/cookbook-add-model/references/authoring-reference.md index 118ba83e6..9ee9a7bba 100644 --- a/.claude/skills/cookbook-add-model/references/authoring-reference.md +++ b/.claude/skills/cookbook-add-model/references/authoring-reference.md @@ -44,7 +44,7 @@ the full contract): | Field | Type | Purpose | |---|---|---| | `multiNodeHints` | `{[hwId]: string[]}` | Lines prepended as `# ...` comments to multi-node commands (env-var hints). Per-hw, and only for hw whose **cluster fabric needs manual NIC config** (e.g. `gb200` NVL72/MNNVL → NVSHMEM/Gloo hints). NOT every multi-N hw needs an entry — standard-IB DeepEP (h200) auto-detects the HCA, and Marlin multi-node (h100) uses no DeepEP/NVSHMEM at all. Hints render above **both** run modes, so keep them mode-agnostic: `docker run` flags belong in the hardware entry's `multiNodeDockerFlags` (above), which the engine puts in the command itself. | -| `dockerImages` | `{[key]: string}` | Image for `docker run` framing, keyed by `hw\|quant` (most specific) then `hw`. Use a `hw\|quant` key only when one quant on a shared GPU needs a different image (e.g. an NVFP4 dev build on b300/gb300 while FP8/BF16 stay on the release image); otherwise key by plain `hw`. **Ask the user which sglang build the recipes ran on; don't guess a supporting release.** Falls back to `lmsysorg/sglang:dev` if missing — also the sensible default when unsure. | +| `dockerImages` | `{[key]: string}` | Image for `docker run` framing, keyed by `hw\|variant\|quant` (most specific), then `variant\|quant`, `hw\|quant\|strategy`, `hw\|quant`, `hw`. Key by plain `hw` unless a subset needs a different image: `hw\|quant` when one quant on a shared GPU needs its own build (e.g. an NVFP4 dev build on b300/gb300 while FP8/BF16 stay on the release image); the `variant` keys when one checkpoint needs its own build (e.g. a preview image for a not-yet-released variant); the `strategy` key when one tier needs one (e.g. a spec-decoding preview). **Ask the user which sglang build the recipes ran on; don't guess a supporting release.** Falls back to `lmsysorg/sglang:dev` if missing — also the sensible default when unsure. | | `playgroundFeatures` | `{[axisId]: {...}}` | Opts into the Playground widget. See §2.3. | | `benchmarkCommands` | `{speed: string, accuracy: {[accKey]: string \| {[variant]: string}}, numPromptsByConc?: {[c]: number}}` | Powers the benchmark card's **"⚡ Reproduce"** modal. `speed` is ONE `bench_serving` template; the engine fills `{{DATASET}}`/`{{ISL}}`/`{{OSL}}` from each cell's `speed[].workload`, the chip-picked `{{MAX_CONCURRENCY}}`, and `{{NUM_PROMPTS}}` (resolved `workload.num_prompts ?? numPromptsByConc[c] ?? max(c*2, 200)`). `accuracy` maps an accuracy field (e.g. `gsm8k_pct`) to a per-eval template — a string, OR a `{flash, pro, …}` object keyed by variant when the command differs per variant (e.g. GPQA/AIME `--max-tokens`). The modal renders a chip per eval (one command area, like Speed). Both also use `{{MODEL_NAME}}` + `{{CURL_HOST}}`/`{{CURL_PORT}}` like `curl`. `speed` should carry `--flush-cache` (bench_serving's `random` prompts are deterministic — warm reruns hit the radix cache and inflate throughput; measure cache-cold). Optional; the button only appears when this AND `benchmarks` are present. | | `defaultAccuracy` | `{[variant]: {[accKey]: number}}` | Model-level accuracy applied to **every** cell of a variant (e.g. GPQA Diamond / AIME25 — hardware-independent). Merged UNDER each cell's measured `accuracy` (a per-cell value wins), so you set a variant's score once instead of copying it onto every benchmark entry. Keys must match `accuracyLabels` (below) + `benchmarkCommands.accuracy`. | diff --git a/.claude/skills/cookbook-add-model/templates/config.jsx.tmpl b/.claude/skills/cookbook-add-model/templates/config.jsx.tmpl index 6cfaf3d28..e3a8d1634 100644 --- a/.claude/skills/cookbook-add-model/templates/config.jsx.tmpl +++ b/.claude/skills/cookbook-add-model/templates/config.jsx.tmpl @@ -141,12 +141,14 @@ sgl-eval run gsm8k \\ ], }, - // Image for `docker run` framing, keyed by `hw` (or `hw|quant`, resolved first, when one - // quant on a shared GPU needs its own image — e.g. an FP4 dev build while FP8/BF16 use the - // release tag). ASK the user which sglang build the recipes ran on; don't guess a supporting - // release. Default below is :dev (nightly) — replace the tag with the user's release if they - // give one. NVIDIA share one image; AMD uses ROCm tags. GB200/GB300/B300 may need a - // `-cu130` (CUDA 13) tag — confirm per release. + // Image for `docker run` framing, keyed by `hw` (or, resolved before it: `hw|variant|quant`, + // `variant|quant`, `hw|quant|strategy`, `hw|quant` — use a more specific key only when a + // variant/quant/strategy subset needs its own build, e.g. an FP4 dev build while FP8/BF16 use + // the release tag, or a preview image for a not-yet-released variant). ASK the user which + // sglang build the recipes ran on; don't guess a supporting release. Default below is :dev + // (nightly) — replace the tag with the user's release if they give one. NVIDIA share one + // image; AMD uses ROCm tags. GB200/GB300/B300 may need a `-cu130` (CUDA 13) tag — confirm + // per release. dockerImages: { h100: "lmsysorg/sglang:dev", h200: "lmsysorg/sglang:dev", diff --git a/docs/cookbook/autoregressive/DeepSeek/DeepSeek-V4.mdx b/docs/cookbook/autoregressive/DeepSeek/DeepSeek-V4.mdx index b8cc007f0..cc01fab62 100644 --- a/docs/cookbook/autoregressive/DeepSeek/DeepSeek-V4.mdx +++ b/docs/cookbook/autoregressive/DeepSeek/DeepSeek-V4.mdx @@ -1,6 +1,6 @@ --- title: DeepSeek-V4 -description: "Deploy DeepSeek-V4 with SGLang — verified launch commands, benchmarks, and tuning for Flash Official (0731), Flash, Pro, and Pro Official (0813)." +description: "Deploy DeepSeek-V4 with SGLang — verified launch commands, benchmarks, and tuning for Flash Official (0731), Flash, Flash Vision (Exp), Pro, and Pro Official (0813)." tag: NEW --- @@ -32,7 +32,7 @@ For how to launch the image, see [Install → Method 3: Using Docker](../../../d **NVIDIA GPUs** -A single image — `lmsysorg/sglang:latest` — covers the **datacenter GPUs** in this cookbook (B200 / B300 / GB200 / GB300 / H100 / H200 / RTX PRO 6000). +A single image — `lmsysorg/sglang:latest` — covers the **datacenter GPUs** in this cookbook (B200 / B300 / GB200 / GB300 / H100 / H200 / RTX PRO 6000). The one exception is **Flash Vision (Exp)**, whose support has not shipped in a release yet: its cells use the preview image `lmsysorg/sglang:dev-dsv4-flash-vision` (the command panel picks it automatically — see the [Flash Vision notes](#vision-note)). ```bash Command docker pull lmsysorg/sglang:latest @@ -86,6 +86,10 @@ import { Deployment } from "/src/snippets/_deployment.jsx"; import { config } from "/src/snippets/configs/deepseek-ai/deepseek-v4.jsx"; import { benchmarks } from "/src/snippets/configs/deepseek-ai/deepseek-v4-benchmarks.jsx"; + + The **Flash Vision (Exp)** variant runs on a dedicated preview Docker image, `lmsysorg/sglang:dev-dsv4-flash-vision` — its support ([sgl-project/sglang#37253](https://github.com/sgl-project/sglang/pull/37253)) has not shipped in a release yet. The command panel's Docker mode emits that image automatically for Flash Vision cells; see the [Flash Vision notes](#vision-note). + + @@ -127,7 +131,7 @@ import { Playground } from "/src/snippets/_playground.jsx"; ## 1. Model Introduction -**DeepSeek-V4** is the next-generation Mixture-of-Experts model from DeepSeek, released 2026-04-24 under an **MIT License**. The 0731 Flash and 0813 Pro refreshes add checkpoints with a bundled DSpark draft head: +**DeepSeek-V4** is the next-generation Mixture-of-Experts model from DeepSeek, released 2026-04-24 under an **MIT License**. The 0731 Flash and 0813 Pro refreshes add checkpoints with a bundled DSpark draft head, and the experimental Flash Vision checkpoint builds image understanding on top of the 0731 Flash base: @@ -157,6 +161,12 @@ import { Playground } from "/src/snippets/_playground.jsx"; + + + + + + @@ -178,7 +188,7 @@ The Instruct checkpoints ship as **FP4 MoE experts + FP8 attention / dense** (on **Recommended generation:** `temperature=1.0`, `top_p=1.0`. -**Resources:** HuggingFace · [Flash Official (0731)](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731) · [Flash](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash) · [Pro](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro) · [Pro Official (0813)](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro-0813) · ModelScope · [Flash](https://modelscope.cn/models/deepseek-ai/DeepSeek-V4-Flash) · [Pro](https://modelscope.cn/models/deepseek-ai/DeepSeek-V4-Pro). +**Resources:** HuggingFace · [Flash Official (0731)](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731) · [Flash](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash) · [Flash Vision (Exp)](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp) · [Pro](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro) · [Pro Official (0813)](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro-0813) · ModelScope · [Flash](https://modelscope.cn/models/deepseek-ai/DeepSeek-V4-Flash) · [Pro](https://modelscope.cn/models/deepseek-ai/DeepSeek-V4-Pro). ## 2. Configuration Tips @@ -205,6 +215,18 @@ For the original Flash and Pro checkpoints: - `high-throughput`: MTP disabled — at saturation the verify step costs more than it saves. - MTP runs on the v2 speculative path. + + +**DeepSeek-V4-Flash-Vision-Exp (Experimental)** + +[`deepseek-ai/DeepSeek-V4-Flash-Vision-Exp`](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp) is DeepSeek's first experimental multimodal V4 checkpoint: the 0731 Flash base plus a vision encoder and aligner, served through the same `sglang serve` flow with OpenAI-style `image_url` inputs (see [Vision](#3-5-vision-image-inputs) below). Select the **Flash Vision** variant in the Deploy panel for its recipes. + +- **Preview build required** — support lands via [sgl-project/sglang#37253](https://github.com/sgl-project/sglang/pull/37253) and has not shipped in a release. Docker mode on the Flash Vision cells already emits the preview image `lmsysorg/sglang:dev-dsv4-flash-vision`; for a Python environment, install SGLang from that PR's branch. +- **Verified matrix** — MMMU-Pro via sgl-eval at `temperature 1.0`, `top-p 0.95`, `--reasoning-effort max`. +- **Engine auto-configuration** — the engine picks the `flashinfer_mxfp4` MoE runner and auto-disables shared-experts fusion for this checkpoint (its HashTopK routing rejects fused shared experts); don't pass `--enforce-shared-experts-fusion`. +- **Chunked prefill & radix cache stay enabled** — the scheduler keeps image spans consistent automatically: chunked-prefill truncation points are span-aligned (an image span always prefills within a single extend, overshooting the chunk budget by at most one span), and a radix-cache prefix match ending deep inside an image span is re-issued from the span start. +- **Speculative decoding** — the checkpoint bundles a DSpark head, but MTP/DSpark with image batches is not yet verified, so all Flash Vision recipes run target-only. As on the 0731/0813 checkpoints, do not pass the EAGLE flags. + **Shared experts fusion (Blackwell, flashinfer_mxfp4)** On the Blackwell fp4 recipes (`--moe-runner-backend flashinfer_mxfp4`), the shared expert runs as a separate FP8 MLP on an alternate stream by default. Adding: @@ -215,7 +237,7 @@ On the Blackwell fp4 recipes (`--moe-runner-backend flashinfer_mxfp4`), the shar routes it as one extra MXFP4 expert through the same trtllm-gen MoE kernel, so the whole MoE runs on a single stream (~4 fewer kernel launches and 2 fewer stream syncs per MoE layer). The shared expert is requantized from FP8 to MXFP4 at load time. Measured on GB200 tp4: gsm8k and AIME25 accuracy on par with the unfused baseline; Mean TTFT -13% to -21% and P99 ITL -15% to -53% at QPS 1-8 with neutral throughput. -Only for deployments without expert parallelism (e.g. the single-node low-latency recipes): with `moe_ep_size > 1` the flag is rejected at startup, unless the DeepEP/MegaMOE per-rank shared-slot path is in use. +Only for deployments without expert parallelism (e.g. the single-node low-latency recipes): with `moe_ep_size > 1` the flag is rejected at startup, unless the DeepEP/MegaMOE per-rank shared-slot path is in use. Not applicable to [Flash Vision (Exp)](#vision-note) — the engine auto-disables the fusion on that checkpoint. **Compressed attention state dtype** @@ -603,6 +625,8 @@ For more details, see the [HiCache documentation](../../../docs/advanced_feature Flash Official (0731) and Pro Official (0813) bundle a DSpark draft head in `deepseek-ai/DeepSeek-V4-Flash-0731` and `deepseek-ai/DeepSeek-V4-Pro-0813`. The target and draft weights therefore come from the same checkpoint: enable DSpark with `--speculative-algorithm DSPARK` and do not set a separate `--speculative-draft-model-path`. +The experimental [Flash Vision checkpoint](#vision-note) also bundles a DSpark head, but speculative decoding with image batches is not yet verified on it — the Flash Vision recipes run target-only for now, and the Playground greys the DSpark chip out on that variant. + Unlike the EAGLE recipes for the original Flash and Pro checkpoints, this recipe omits `--speculative-num-steps`, `--speculative-eagle-topk`, and `--speculative-num-draft-tokens`. SGLang reads the DSpark shape from the checkpoint. @@ -634,3 +658,44 @@ Larger blocks can improve decode latency when acceptance stays high, but they al For every candidate, compare with the same recipe without `--speculative-algorithm DSPARK`. Restart the server between the DSpark and non-speculative legs, keep the request corpus, sampling, concurrency, and warmup identical, and give each `bench_serving` leg its own `--flush-cache`. Leave `--speculative-draft-attention-backend` unset unless a separate profiling run justifies an override. DSpark currently requires CUDA, `pp_size == 1`, and DP Attention disabled. It is not compatible with PD disaggregation on current SGLang releases; selecting a prefill or decode role in the Playground automatically removes the inherited DSpark flags. The DP-Attention and MI355X Flash Official recipes therefore run target-only. If a larger draft block or concurrency causes graph-capture OOM, lower `--mem-fraction-static`, the draft block size, or the configured maximum running requests, then rerun both performance and accuracy gates. + +### 3.5 Vision (Image Inputs) + +The experimental [`DeepSeek-V4-Flash-Vision-Exp`](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp) checkpoint (the **Flash Vision** variant in the Deploy panel — see the [configuration notes](#vision-note)) takes images via the OpenAI-compatible `image_url` content type, as public URLs or base64 `data:` URIs; text and images mix freely in one message. Vision input works with the same server the Deploy panel produces — no extra model-specific flags needed. + + + +```python Example +from openai import OpenAI + +client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY") + +response = client.chat.completions.create( + model="deepseek-ai/DeepSeek-V4-Flash-Vision-Exp", + messages=[{ + "role": "user", + "content": [ + {"type": "image_url", + "image_url": {"url": "https://raw.githubusercontent.com/sgl-project/sglang/main/examples/assets/example_image.png"}}, + {"type": "text", "text": "Describe this image in a few sentences."}, + ], + }], +) + +message = response.choices[0].message +if getattr(message, "reasoning_content", None): + print("=============== Thinking =================") + print(message.reasoning_content) +print("=============== Content =================") +print(message.content) +``` + + + + + +```text Output +Pending update... +``` + + diff --git a/docs/src/snippets/_deployment.jsx b/docs/src/snippets/_deployment.jsx index 58a95d828..a8e265321 100644 --- a/docs/src/snippets/_deployment.jsx +++ b/docs/src/snippets/_deployment.jsx @@ -80,6 +80,7 @@ // Legacy "Mean" data is being re-measured to P50; drop once done // multiNodeHints optional — {[hwId]: string[]} prepended as `# ...` lines // dockerImages optional — `docker run` image, keyed by +// `hw|variant|quant` then `variant|quant` then // `hw|quant|strategy` then `hw|quant` then `hw`; // falls back to `lmsysorg/sglang:dev` // dockerHostNetworkWhen optional — `(selection, {flags, env}) => boolean` @@ -780,11 +781,15 @@ export const Deployment = ({ config, benchmarks }) => { let cmd; if (mode === "docker") { - // Image keyed by `hw|quant|strategy` (most specific), then `hw|quant`, - // then `hw`; `:dev` if unmapped. The strategy key covers a tier that - // needs its own build (e.g. a spec-decoding preview image). + // Image keyed by `hw|variant|quant` (most specific), then `variant|quant`, + // then `hw|quant|strategy`, `hw|quant`, `hw`; `:dev` if unmapped. The + // variant keys cover a checkpoint that needs its own build (e.g. a + // new-variant preview image); the strategy key covers a tier that needs + // one (e.g. a spec-decoding preview image). const di = config.dockerImages || {}; - const image = di[`${sel.hw}|${sel.quant}|${sel.strategy}`] + const image = di[`${sel.hw}|${sel.variant}|${sel.quant}`] + || di[`${sel.variant}|${sel.quant}`] + || di[`${sel.hw}|${sel.quant}|${sel.strategy}`] || di[`${sel.hw}|${sel.quant}`] || di[sel.hw] || "lmsysorg/sglang:dev"; const dockerRunCommand = typeof config.dockerRunCommand === "function" ? config.dockerRunCommand(sel) diff --git a/docs/src/snippets/_playground.jsx b/docs/src/snippets/_playground.jsx index b1b7ae9cc..cb961a267 100644 --- a/docs/src/snippets/_playground.jsx +++ b/docs/src/snippets/_playground.jsx @@ -1520,13 +1520,17 @@ export const Playground = ({ config }) => { } let cmd; if (mode === "docker") { - // Image keyed by `hw|quant|strategy` (most specific), then `hw|quant`, then - // `hw`; `:dev` if unmapped (matches _deployment.jsx). The strategy key covers - // a tier that needs its own build (e.g. a spec-decoding preview image), so the - // playground base must resolve it too or it hands back an image that cannot - // run the command. + // Image keyed by `hw|variant|quant` (most specific), then `variant|quant`, + // then `hw|quant|strategy`, `hw|quant`, `hw`; `:dev` if unmapped (matches + // _deployment.jsx). The variant keys cover a checkpoint that needs its own + // build (e.g. a new-variant preview image) and the strategy key a tier that + // needs one (e.g. a spec-decoding preview image), so the playground base + // must resolve them too or it hands back an image that cannot run the + // command. const di = config.dockerImages || {}; - const image = di[`${sel.hw}|${sel.quant}|${sel.strategy}`] + const image = di[`${sel.hw}|${sel.variant}|${sel.quant}`] + || di[`${sel.variant}|${sel.quant}`] + || di[`${sel.hw}|${sel.quant}|${sel.strategy}`] || di[`${sel.hw}|${sel.quant}`] || di[sel.hw] || "lmsysorg/sglang:dev"; const dockerRunCommand = typeof config.dockerRunCommand === "function" ? config.dockerRunCommand(sel) diff --git a/docs/src/snippets/configs/deepseek-ai/deepseek-v4-benchmarks.jsx b/docs/src/snippets/configs/deepseek-ai/deepseek-v4-benchmarks.jsx index 633903b6d..34598c46f 100644 --- a/docs/src/snippets/configs/deepseek-ai/deepseek-v4-benchmarks.jsx +++ b/docs/src/snippets/configs/deepseek-ai/deepseek-v4-benchmarks.jsx @@ -634,4 +634,28 @@ export const benchmarks = [ { match: { hw: "mi355x", variant: "pro", quant: "fp8", strategy: "low-latency", nodes: "single" } }, { match: { hw: "mi355x", variant: "pro", quant: "fp8", strategy: "balanced", nodes: "single" } }, { match: { hw: "mi355x", variant: "pro", quant: "fp8", strategy: "high-throughput", nodes: "single" } }, + // ==================================================================== + // B200 + FP4 — Flash Vision (Exp) + // ==================================================================== + { + match: { hw: "b200", variant: "flash-vision", quant: "fp4", strategy: "low-latency", nodes: "single" }, + sglang_version: "dev-dsv4-flash-vision", + accuracy: { mmmu_pro_pct: 74.96 }, + notes: "MMMU-Pro (standard, 10-option) measured with sgl-eval on 4×B200 (TP=4) at temperature 1.0, top-p 0.95, --reasoning-effort max.", + }, + { match: { hw: "b200", variant: "flash-vision", quant: "fp4", strategy: "balanced", nodes: "single" } }, + { match: { hw: "b200", variant: "flash-vision", quant: "fp4", strategy: "high-throughput", nodes: "single" } }, + // B300 / GB200 / GB300 / H200 / H100 — Flash Vision (Exp), all pending + { match: { hw: "b300", variant: "flash-vision", quant: "fp4", strategy: "low-latency", nodes: "single" } }, + { match: { hw: "b300", variant: "flash-vision", quant: "fp4", strategy: "balanced", nodes: "single" } }, + { match: { hw: "b300", variant: "flash-vision", quant: "fp4", strategy: "high-throughput", nodes: "single" } }, + { match: { hw: "gb200", variant: "flash-vision", quant: "fp4", strategy: "low-latency", nodes: "single" } }, + { match: { hw: "gb200", variant: "flash-vision", quant: "fp4", strategy: "balanced", nodes: "single" } }, + { match: { hw: "gb200", variant: "flash-vision", quant: "fp4", strategy: "high-throughput", nodes: "single" } }, + { match: { hw: "gb300", variant: "flash-vision", quant: "fp4", strategy: "low-latency", nodes: "single" } }, + { match: { hw: "gb300", variant: "flash-vision", quant: "fp4", strategy: "balanced", nodes: "single" } }, + { match: { hw: "gb300", variant: "flash-vision", quant: "fp4", strategy: "high-throughput", nodes: "single" } }, + { match: { hw: "h200", variant: "flash-vision", quant: "fp4", strategy: "low-latency", nodes: "single" } }, + { match: { hw: "h200", variant: "flash-vision", quant: "fp4", strategy: "balanced", nodes: "single" } }, + { match: { hw: "h100", variant: "flash-vision", quant: "fp4", strategy: "balanced", nodes: "single" } }, ]; diff --git a/docs/src/snippets/configs/deepseek-ai/deepseek-v4.jsx b/docs/src/snippets/configs/deepseek-ai/deepseek-v4.jsx index 1993e9b06..0f53e3ff9 100644 --- a/docs/src/snippets/configs/deepseek-ai/deepseek-v4.jsx +++ b/docs/src/snippets/configs/deepseek-ai/deepseek-v4.jsx @@ -25,6 +25,7 @@ export const config = { variants: [ { id: "flash", label: "Flash", subtitle: "284B" }, { id: "flash-official", label: "Flash Official", subtitle: "284B · 0731" }, + { id: "flash-vision", label: "Flash Vision", subtitle: "305B · Exp" }, { id: "pro", label: "Pro", subtitle: "1.6T" }, { id: "pro-official", label: "Pro Official", subtitle: "1.6T · 0813" }, ], @@ -49,6 +50,7 @@ export const config = { "flash|fp8": "deepseek-ai/DeepSeek-V4-Flash", "flash|nvfp4": "nvidia/DeepSeek-V4-Flash-NVFP4", "flash-official|fp4": "deepseek-ai/DeepSeek-V4-Flash-0731", + "flash-vision|fp4": "deepseek-ai/DeepSeek-V4-Flash-Vision-Exp", "pro|fp4": "deepseek-ai/DeepSeek-V4-Pro", "pro|fp8": "deepseek-ai/DeepSeek-V4-Pro", "pro|nvfp4": "nvidia/DeepSeek-V4-Pro-NVFP4", @@ -144,6 +146,14 @@ sgl-eval run aime25 \\ --n-repeats 16 --max-tokens 400000 \\ --temperature 1.0 --top-p 1.0 --thinking \\ --out-dir /sgl-workspace/logs \\ + --base-url http://{{CURL_HOST}}:{{CURL_PORT}}/v1`, + }, + mmmu_pro_pct: { + "flash-vision": +`# To install sgl-eval: pip install git+https://github.com/sgl-project/sgl-eval +sgl-eval run mmmu_pro \\ + --reasoning-effort max \\ + --temperature 1.0 --top-p 0.95 \\ --base-url http://{{CURL_HOST}}:{{CURL_PORT}}/v1`, }, }, @@ -162,6 +172,7 @@ sgl-eval run aime25 \\ ["gpqa_pct", "GPQA Diamond", "%"], ["aime25_pct", "AIME25", "%"], ["gsm8k_pct", "GSM8K (1-shot)", "%"], + ["mmmu_pro_pct", "MMMU-Pro (standard, 10-option)", "%"], ], // Prepended as `# ...` comments above multi-node commands. @@ -175,6 +186,10 @@ sgl-eval run aime25 \\ }, dockerImages: { + // Flash Vision (Exp) support has not shipped in a release yet + // (sgl-project/sglang#37253) — until it does, the variant needs this + // preview build on every hardware. + "flash-vision|fp4": "lmsysorg/sglang:dev-dsv4-flash-vision", h100: "lmsysorg/sglang:latest", h200: "lmsysorg/sglang:latest", b200: "lmsysorg/sglang:latest", @@ -289,15 +304,17 @@ sgl-eval run aime25 \\ { id: "mtp-314", label: "EAGLE / MTP 3-1-4", flags: ["--speculative-algorithm EAGLE", "--speculative-num-steps 3", "--speculative-eagle-topk 1", "--speculative-num-draft-tokens 4"], - hide: { variant: ["flash-official", "pro-official"] } }, + hide: { variant: ["flash-official", "flash-vision", "pro-official"] } }, { id: "mtp-112", label: "EAGLE / MTP 1-1-2", flags: ["--speculative-algorithm EAGLE", "--speculative-num-steps 1", "--speculative-eagle-topk 1", "--speculative-num-draft-tokens 2"], - hide: { variant: ["flash-official", "pro-official"] } }, + hide: { variant: ["flash-official", "flash-vision", "pro-official"] } }, { id: "dspark", label: "DSpark", flags: ["--speculative-algorithm DSPARK"], hide: { variant: ["flash", "pro"] }, disable: [ + { when: { variant: ["flash-vision"] }, + reason: "The Flash Vision checkpoint bundles a DSpark head, but speculative decoding is not yet verified with image inputs — the cookbook recipes run target-only for now." }, { when: { dpAttnOn: [true] }, reason: "DSpark is not compatible with DP Attention on the current release." }, { when: { hw: ["mi300x", "mi355x"] }, @@ -2826,5 +2843,277 @@ sgl-eval run aime25 \\ "--port {{PORT}}", ], }, + + // ==================================================================== + // B200 + FP4 — Flash Vision (Exp) + // + // DeepSeek-V4-Flash-Vision-Exp (sgl-project/sglang#37253): the 0731 + // Flash base plus a vision encoder + aligner. The checkpoint bundles a + // DSpark head, but speculative decoding is not yet verified with image + // batches, so every recipe runs target-only. Low-latency is the serving + // shape the MMMU-Pro round ran on (4×B200); balanced / high-throughput + // mirror the Flash Official recipes on the same 4-GPU topology — final + // verification in progress. + // ==================================================================== + { + match: { hw: "b200", variant: "flash-vision", quant: "fp4", strategy: "low-latency", nodes: "single" }, + verified: true, + warn: "DeepSeek-V4-Flash-Vision-Exp support has not shipped in an SGLang release yet (sglang PR 37253): Docker mode already points at the preview image; for Python mode install SGLang from that PR. See [Flash Vision notes](#vision-note).", + env: [], + flags: [ + "--model-path {{MODEL_NAME}}", + "--tp 4", + "--mem-fraction-static 0.85", + "--host {{HOST_IP}}", + "--port {{PORT}}", + ], + }, + { + match: { hw: "b200", variant: "flash-vision", quant: "fp4", strategy: "balanced", nodes: "single" }, + verified: false, + verificationStatus: "in-progress", + warn: "DeepSeek-V4-Flash-Vision-Exp support has not shipped in an SGLang release yet (sglang PR 37253): Docker mode already points at the preview image; for Python mode install SGLang from that PR. See [Flash Vision notes](#vision-note).", + env: ["SGLANG_DEEPEP_NUM_MAX_DISPATCH_TOKENS_PER_RANK=1024"], + flags: [ + "--model-path {{MODEL_NAME}}", + "--tp 4", + "--dp 4", + "--enable-dp-attention", + "--moe-a2a-backend deepep", + "--mem-fraction-static 0.85", + "--deepep-config '{\"normal_dispatch\":{\"num_sms\":96},\"normal_combine\":{\"num_sms\":96}}'", + "--host {{HOST_IP}}", + "--port {{PORT}}", + ], + }, + { + match: { hw: "b200", variant: "flash-vision", quant: "fp4", strategy: "high-throughput", nodes: "single" }, + verified: false, + verificationStatus: "in-progress", + warn: "DeepSeek-V4-Flash-Vision-Exp support has not shipped in an SGLang release yet (sglang PR 37253): Docker mode already points at the preview image; for Python mode install SGLang from that PR. See [Flash Vision notes](#vision-note).", + env: [ + "SGLANG_OPT_DEEPGEMM_MEGA_MOE_NUM_MAX_TOKENS_PER_RANK=8320", + ], + flags: [ + "--model-path {{MODEL_NAME}}", + "--tp 4", + "--dp 4", + "--enable-dp-attention", + "--moe-a2a-backend megamoe", + "--mem-fraction-static 0.85", + "--host {{HOST_IP}}", + "--port {{PORT}}", + ], + }, + + // ==================================================================== + // B300 / GB200 / GB300 + FP4 — Flash Vision (Exp) + // ==================================================================== + { + match: { hw: "b300", variant: "flash-vision", quant: "fp4", strategy: "low-latency", nodes: "single" }, + verified: false, + verificationStatus: "in-progress", + warn: "DeepSeek-V4-Flash-Vision-Exp support has not shipped in an SGLang release yet (sglang PR 37253): Docker mode already points at the preview image; for Python mode install SGLang from that PR. See [Flash Vision notes](#vision-note).", + env: [], + flags: [ + "--model-path {{MODEL_NAME}}", + "--tp 4", + "--mem-fraction-static 0.85", + "--host {{HOST_IP}}", + "--port {{PORT}}", + ], + }, + { + match: { hw: "b300", variant: "flash-vision", quant: "fp4", strategy: "balanced", nodes: "single" }, + verified: false, + verificationStatus: "in-progress", + warn: "DeepSeek-V4-Flash-Vision-Exp support has not shipped in an SGLang release yet (sglang PR 37253): Docker mode already points at the preview image; for Python mode install SGLang from that PR. See [Flash Vision notes](#vision-note).", + env: ["SGLANG_DEEPEP_NUM_MAX_DISPATCH_TOKENS_PER_RANK=1024"], + flags: [ + "--model-path {{MODEL_NAME}}", + "--tp 4", + "--dp 4", + "--enable-dp-attention", + "--moe-a2a-backend deepep", + "--mem-fraction-static 0.85", + "--deepep-config '{\"normal_dispatch\":{\"num_sms\":96},\"normal_combine\":{\"num_sms\":96}}'", + "--host {{HOST_IP}}", + "--port {{PORT}}", + ], + }, + { + match: { hw: "b300", variant: "flash-vision", quant: "fp4", strategy: "high-throughput", nodes: "single" }, + verified: false, + verificationStatus: "in-progress", + warn: "DeepSeek-V4-Flash-Vision-Exp support has not shipped in an SGLang release yet (sglang PR 37253): Docker mode already points at the preview image; for Python mode install SGLang from that PR. See [Flash Vision notes](#vision-note).", + env: [ + "SGLANG_OPT_DEEPGEMM_MEGA_MOE_NUM_MAX_TOKENS_PER_RANK=8320", + ], + flags: [ + "--model-path {{MODEL_NAME}}", + "--tp 4", + "--dp 4", + "--enable-dp-attention", + "--moe-a2a-backend megamoe", + "--mem-fraction-static 0.85", + "--host {{HOST_IP}}", + "--port {{PORT}}", + ], + }, + { + match: { hw: "gb200", variant: "flash-vision", quant: "fp4", strategy: "low-latency", nodes: "single" }, + verified: false, + verificationStatus: "in-progress", + warn: "DeepSeek-V4-Flash-Vision-Exp support has not shipped in an SGLang release yet (sglang PR 37253): Docker mode already points at the preview image; for Python mode install SGLang from that PR. See [Flash Vision notes](#vision-note).", + env: [], + flags: [ + "--model-path {{MODEL_NAME}}", + "--tp 4", + "--mem-fraction-static 0.85", + "--host {{HOST_IP}}", + "--port {{PORT}}", + ], + }, + { + match: { hw: "gb200", variant: "flash-vision", quant: "fp4", strategy: "balanced", nodes: "single" }, + verified: false, + verificationStatus: "in-progress", + warn: "DeepSeek-V4-Flash-Vision-Exp support has not shipped in an SGLang release yet (sglang PR 37253): Docker mode already points at the preview image; for Python mode install SGLang from that PR. See [Flash Vision notes](#vision-note).", + env: ["SGLANG_DEEPEP_NUM_MAX_DISPATCH_TOKENS_PER_RANK=1024"], + flags: [ + "--model-path {{MODEL_NAME}}", + "--tp 4", + "--dp 4", + "--enable-dp-attention", + "--moe-a2a-backend deepep", + "--mem-fraction-static 0.85", + "--deepep-config '{\"normal_dispatch\":{\"num_sms\":96},\"normal_combine\":{\"num_sms\":96}}'", + "--host {{HOST_IP}}", + "--port {{PORT}}", + ], + }, + { + match: { hw: "gb200", variant: "flash-vision", quant: "fp4", strategy: "high-throughput", nodes: "single" }, + verified: false, + verificationStatus: "in-progress", + warn: "DeepSeek-V4-Flash-Vision-Exp support has not shipped in an SGLang release yet (sglang PR 37253): Docker mode already points at the preview image; for Python mode install SGLang from that PR. See [Flash Vision notes](#vision-note).", + env: [ + "SGLANG_OPT_DEEPGEMM_MEGA_MOE_NUM_MAX_TOKENS_PER_RANK=8320", + ], + flags: [ + "--model-path {{MODEL_NAME}}", + "--tp 4", + "--dp 4", + "--enable-dp-attention", + "--moe-a2a-backend megamoe", + "--mem-fraction-static 0.85", + "--host {{HOST_IP}}", + "--port {{PORT}}", + ], + }, + { + match: { hw: "gb300", variant: "flash-vision", quant: "fp4", strategy: "low-latency", nodes: "single" }, + verified: false, + verificationStatus: "in-progress", + warn: "DeepSeek-V4-Flash-Vision-Exp support has not shipped in an SGLang release yet (sglang PR 37253): Docker mode already points at the preview image; for Python mode install SGLang from that PR. See [Flash Vision notes](#vision-note).", + env: [], + flags: [ + "--model-path {{MODEL_NAME}}", + "--tp 4", + "--mem-fraction-static 0.85", + "--host {{HOST_IP}}", + "--port {{PORT}}", + ], + }, + { + match: { hw: "gb300", variant: "flash-vision", quant: "fp4", strategy: "balanced", nodes: "single" }, + verified: false, + verificationStatus: "in-progress", + warn: "DeepSeek-V4-Flash-Vision-Exp support has not shipped in an SGLang release yet (sglang PR 37253): Docker mode already points at the preview image; for Python mode install SGLang from that PR. See [Flash Vision notes](#vision-note).", + env: ["SGLANG_DEEPEP_NUM_MAX_DISPATCH_TOKENS_PER_RANK=1024"], + flags: [ + "--model-path {{MODEL_NAME}}", + "--tp 4", + "--dp 4", + "--enable-dp-attention", + "--moe-a2a-backend deepep", + "--mem-fraction-static 0.85", + "--deepep-config '{\"normal_dispatch\":{\"num_sms\":96},\"normal_combine\":{\"num_sms\":96}}'", + "--host {{HOST_IP}}", + "--port {{PORT}}", + ], + }, + { + match: { hw: "gb300", variant: "flash-vision", quant: "fp4", strategy: "high-throughput", nodes: "single" }, + verified: false, + verificationStatus: "in-progress", + warn: "DeepSeek-V4-Flash-Vision-Exp support has not shipped in an SGLang release yet (sglang PR 37253): Docker mode already points at the preview image; for Python mode install SGLang from that PR. See [Flash Vision notes](#vision-note).", + env: [ + "SGLANG_OPT_DEEPGEMM_MEGA_MOE_NUM_MAX_TOKENS_PER_RANK=8320", + ], + flags: [ + "--model-path {{MODEL_NAME}}", + "--tp 4", + "--dp 4", + "--enable-dp-attention", + "--moe-a2a-backend megamoe", + "--mem-fraction-static 0.85", + "--host {{HOST_IP}}", + "--port {{PORT}}", + ], + }, + + // ==================================================================== + // H200 + FP4 — Flash Vision (Exp) + // ==================================================================== + { + match: { hw: "h200", variant: "flash-vision", quant: "fp4", strategy: "low-latency", nodes: "single" }, + verified: false, + verificationStatus: "in-progress", + warn: "DeepSeek-V4-Flash-Vision-Exp support has not shipped in an SGLang release yet (sglang PR 37253): Docker mode already points at the preview image; for Python mode install SGLang from that PR. See [Flash Vision notes](#vision-note).", + env: [], + flags: [ + "--model-path {{MODEL_NAME}}", + "--tp 4", + "--moe-runner-backend marlin", + "--mem-fraction-static 0.85", + "--host {{HOST_IP}}", + "--port {{PORT}}", + ], + }, + { + match: { hw: "h200", variant: "flash-vision", quant: "fp4", strategy: "balanced", nodes: "single" }, + verified: false, + verificationStatus: "in-progress", + warn: "DeepSeek-V4-Flash-Vision-Exp support has not shipped in an SGLang release yet (sglang PR 37253): Docker mode already points at the preview image; for Python mode install SGLang from that PR. See [Flash Vision notes](#vision-note).", + env: [], + flags: [ + "--model-path {{MODEL_NAME}}", + "--tp 4", + "--moe-runner-backend flashinfer_mxfp4", + "--mem-fraction-static 0.85", + "--host {{HOST_IP}}", + "--port {{PORT}}", + ], + }, + + // ==================================================================== + // H100 + FP4 — Flash Vision (Exp) + // ==================================================================== + { + match: { hw: "h100", variant: "flash-vision", quant: "fp4", strategy: "balanced", nodes: "single" }, + verified: false, + verificationStatus: "in-progress", + warn: "DeepSeek-V4-Flash-Vision-Exp support has not shipped in an SGLang release yet (sglang PR 37253): Docker mode already points at the preview image; for Python mode install SGLang from that PR. See [Flash Vision notes](#vision-note).", + env: [], + flags: [ + "--model-path {{MODEL_NAME}}", + "--tp 8", + "--moe-runner-backend marlin", + "--mem-fraction-static 0.85", + "--host {{HOST_IP}}", + "--port {{PORT}}", + ], + }, ], };
13B Flash Official (0731), with a bundled DSpark draft head; verified on 8×B200, 4×GB300, and 4×H200
DeepSeek-V4-Flash-Vision-Exp305B13BFlash Vision (Exp) — experimental multimodal (image-text-to-text): the 0731 Flash base + vision encoder & aligner; verified on 4×B200 (TP=4), requires the preview build
DeepSeek-V4-Pro 1.6T