@@ -157,6 +161,12 @@ import { Playground } from "/src/snippets/_playground.jsx";
| 13B |
Flash Official (0731), with a bundled DSpark draft head; verified on 8×B200, 4×GB300, and 4×H200 |
+
+ | DeepSeek-V4-Flash-Vision-Exp |
+ 305B |
+ 13B |
+ Flash Vision (Exp) — experimental multimodal (image-text-to-text): the 0731 Flash base + vision encoder & aligner; verified on 4×B200 (TP=4), requires the preview build |
+
| DeepSeek-V4-Pro |
1.6T |
@@ -178,7 +188,7 @@ The Instruct checkpoints ship as **FP4 MoE experts + FP8 attention / dense** (on
**Recommended generation:** `temperature=1.0`, `top_p=1.0`.
-**Resources:** HuggingFace · [Flash Official (0731)](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731) · [Flash](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash) · [Pro](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro) · [Pro Official (0813)](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro-0813) · ModelScope · [Flash](https://modelscope.cn/models/deepseek-ai/DeepSeek-V4-Flash) · [Pro](https://modelscope.cn/models/deepseek-ai/DeepSeek-V4-Pro).
+**Resources:** HuggingFace · [Flash Official (0731)](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731) · [Flash](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash) · [Flash Vision (Exp)](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp) · [Pro](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro) · [Pro Official (0813)](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro-0813) · ModelScope · [Flash](https://modelscope.cn/models/deepseek-ai/DeepSeek-V4-Flash) · [Pro](https://modelscope.cn/models/deepseek-ai/DeepSeek-V4-Pro).
## 2. Configuration Tips
@@ -205,6 +215,18 @@ For the original Flash and Pro checkpoints:
- `high-throughput`: MTP disabled — at saturation the verify step costs more than it saves.
- MTP runs on the v2 speculative path.
+
+
+**DeepSeek-V4-Flash-Vision-Exp (Experimental)**
+
+[`deepseek-ai/DeepSeek-V4-Flash-Vision-Exp`](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp) is DeepSeek's first experimental multimodal V4 checkpoint: the 0731 Flash base plus a vision encoder and aligner, served through the same `sglang serve` flow with OpenAI-style `image_url` inputs (see [Vision](#3-5-vision-image-inputs) below). Select the **Flash Vision** variant in the Deploy panel for its recipes.
+
+- **Preview build required** — support lands via [sgl-project/sglang#37253](https://github.com/sgl-project/sglang/pull/37253) and has not shipped in a release. Docker mode on the Flash Vision cells already emits the preview image `lmsysorg/sglang:dev-dsv4-flash-vision`; for a Python environment, install SGLang from that PR's branch.
+- **Verified matrix** — MMMU-Pro via sgl-eval at `temperature 1.0`, `top-p 0.95`, `--reasoning-effort max`.
+- **Engine auto-configuration** — the engine picks the `flashinfer_mxfp4` MoE runner and auto-disables shared-experts fusion for this checkpoint (its HashTopK routing rejects fused shared experts); don't pass `--enforce-shared-experts-fusion`.
+- **Chunked prefill & radix cache stay enabled** — the scheduler keeps image spans consistent automatically: chunked-prefill truncation points are span-aligned (an image span always prefills within a single extend, overshooting the chunk budget by at most one span), and a radix-cache prefix match ending deep inside an image span is re-issued from the span start.
+- **Speculative decoding** — the checkpoint bundles a DSpark head, but MTP/DSpark with image batches is not yet verified, so all Flash Vision recipes run target-only. As on the 0731/0813 checkpoints, do not pass the EAGLE flags.
+
**Shared experts fusion (Blackwell, flashinfer_mxfp4)**
On the Blackwell fp4 recipes (`--moe-runner-backend flashinfer_mxfp4`), the shared expert runs as a separate FP8 MLP on an alternate stream by default. Adding:
@@ -215,7 +237,7 @@ On the Blackwell fp4 recipes (`--moe-runner-backend flashinfer_mxfp4`), the shar
routes it as one extra MXFP4 expert through the same trtllm-gen MoE kernel, so the whole MoE runs on a single stream (~4 fewer kernel launches and 2 fewer stream syncs per MoE layer). The shared expert is requantized from FP8 to MXFP4 at load time. Measured on GB200 tp4: gsm8k and AIME25 accuracy on par with the unfused baseline; Mean TTFT -13% to -21% and P99 ITL -15% to -53% at QPS 1-8 with neutral throughput.
-Only for deployments without expert parallelism (e.g. the single-node low-latency recipes): with `moe_ep_size > 1` the flag is rejected at startup, unless the DeepEP/MegaMOE per-rank shared-slot path is in use.
+Only for deployments without expert parallelism (e.g. the single-node low-latency recipes): with `moe_ep_size > 1` the flag is rejected at startup, unless the DeepEP/MegaMOE per-rank shared-slot path is in use. Not applicable to [Flash Vision (Exp)](#vision-note) — the engine auto-disables the fusion on that checkpoint.
**Compressed attention state dtype**
@@ -603,6 +625,8 @@ For more details, see the [HiCache documentation](../../../docs/advanced_feature
Flash Official (0731) and Pro Official (0813) bundle a DSpark draft head in `deepseek-ai/DeepSeek-V4-Flash-0731` and `deepseek-ai/DeepSeek-V4-Pro-0813`. The target and draft weights therefore come from the same checkpoint: enable DSpark with `--speculative-algorithm DSPARK` and do not set a separate `--speculative-draft-model-path`.
+The experimental [Flash Vision checkpoint](#vision-note) also bundles a DSpark head, but speculative decoding with image batches is not yet verified on it — the Flash Vision recipes run target-only for now, and the Playground greys the DSpark chip out on that variant.
+
Unlike the EAGLE recipes for the original Flash and Pro checkpoints, this recipe omits `--speculative-num-steps`, `--speculative-eagle-topk`, and `--speculative-num-draft-tokens`. SGLang reads the DSpark shape from the checkpoint.
@@ -634,3 +658,44 @@ Larger blocks can improve decode latency when acceptance stays high, but they al
For every candidate, compare with the same recipe without `--speculative-algorithm DSPARK`. Restart the server between the DSpark and non-speculative legs, keep the request corpus, sampling, concurrency, and warmup identical, and give each `bench_serving` leg its own `--flush-cache`. Leave `--speculative-draft-attention-backend` unset unless a separate profiling run justifies an override.
DSpark currently requires CUDA, `pp_size == 1`, and DP Attention disabled. It is not compatible with PD disaggregation on current SGLang releases; selecting a prefill or decode role in the Playground automatically removes the inherited DSpark flags. The DP-Attention and MI355X Flash Official recipes therefore run target-only. If a larger draft block or concurrency causes graph-capture OOM, lower `--mem-fraction-static`, the draft block size, or the configured maximum running requests, then rerun both performance and accuracy gates.
+
+### 3.5 Vision (Image Inputs)
+
+The experimental [`DeepSeek-V4-Flash-Vision-Exp`](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp) checkpoint (the **Flash Vision** variant in the Deploy panel — see the [configuration notes](#vision-note)) takes images via the OpenAI-compatible `image_url` content type, as public URLs or base64 `data:` URIs; text and images mix freely in one message. Vision input works with the same server the Deploy panel produces — no extra model-specific flags needed.
+
+
+
+```python Example
+from openai import OpenAI
+
+client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")
+
+response = client.chat.completions.create(
+ model="deepseek-ai/DeepSeek-V4-Flash-Vision-Exp",
+ messages=[{
+ "role": "user",
+ "content": [
+ {"type": "image_url",
+ "image_url": {"url": "https://raw.githubusercontent.com/sgl-project/sglang/main/examples/assets/example_image.png"}},
+ {"type": "text", "text": "Describe this image in a few sentences."},
+ ],
+ }],
+)
+
+message = response.choices[0].message
+if getattr(message, "reasoning_content", None):
+ print("=============== Thinking =================")
+ print(message.reasoning_content)
+print("=============== Content =================")
+print(message.content)
+```
+
+
+
+
+
+```text Output
+Pending update...
+```
+
+
diff --git a/docs/src/snippets/_deployment.jsx b/docs/src/snippets/_deployment.jsx
index 58a95d828..a8e265321 100644
--- a/docs/src/snippets/_deployment.jsx
+++ b/docs/src/snippets/_deployment.jsx
@@ -80,6 +80,7 @@
// Legacy "Mean" data is being re-measured to P50; drop once done
// multiNodeHints optional — {[hwId]: string[]} prepended as `# ...` lines
// dockerImages optional — `docker run` image, keyed by
+// `hw|variant|quant` then `variant|quant` then
// `hw|quant|strategy` then `hw|quant` then `hw`;
// falls back to `lmsysorg/sglang:dev`
// dockerHostNetworkWhen optional — `(selection, {flags, env}) => boolean`
@@ -780,11 +781,15 @@ export const Deployment = ({ config, benchmarks }) => {
let cmd;
if (mode === "docker") {
- // Image keyed by `hw|quant|strategy` (most specific), then `hw|quant`,
- // then `hw`; `:dev` if unmapped. The strategy key covers a tier that
- // needs its own build (e.g. a spec-decoding preview image).
+ // Image keyed by `hw|variant|quant` (most specific), then `variant|quant`,
+ // then `hw|quant|strategy`, `hw|quant`, `hw`; `:dev` if unmapped. The
+ // variant keys cover a checkpoint that needs its own build (e.g. a
+ // new-variant preview image); the strategy key covers a tier that needs
+ // one (e.g. a spec-decoding preview image).
const di = config.dockerImages || {};
- const image = di[`${sel.hw}|${sel.quant}|${sel.strategy}`]
+ const image = di[`${sel.hw}|${sel.variant}|${sel.quant}`]
+ || di[`${sel.variant}|${sel.quant}`]
+ || di[`${sel.hw}|${sel.quant}|${sel.strategy}`]
|| di[`${sel.hw}|${sel.quant}`] || di[sel.hw] || "lmsysorg/sglang:dev";
const dockerRunCommand = typeof config.dockerRunCommand === "function"
? config.dockerRunCommand(sel)
diff --git a/docs/src/snippets/_playground.jsx b/docs/src/snippets/_playground.jsx
index b1b7ae9cc..cb961a267 100644
--- a/docs/src/snippets/_playground.jsx
+++ b/docs/src/snippets/_playground.jsx
@@ -1520,13 +1520,17 @@ export const Playground = ({ config }) => {
}
let cmd;
if (mode === "docker") {
- // Image keyed by `hw|quant|strategy` (most specific), then `hw|quant`, then
- // `hw`; `:dev` if unmapped (matches _deployment.jsx). The strategy key covers
- // a tier that needs its own build (e.g. a spec-decoding preview image), so the
- // playground base must resolve it too or it hands back an image that cannot
- // run the command.
+ // Image keyed by `hw|variant|quant` (most specific), then `variant|quant`,
+ // then `hw|quant|strategy`, `hw|quant`, `hw`; `:dev` if unmapped (matches
+ // _deployment.jsx). The variant keys cover a checkpoint that needs its own
+ // build (e.g. a new-variant preview image) and the strategy key a tier that
+ // needs one (e.g. a spec-decoding preview image), so the playground base
+ // must resolve them too or it hands back an image that cannot run the
+ // command.
const di = config.dockerImages || {};
- const image = di[`${sel.hw}|${sel.quant}|${sel.strategy}`]
+ const image = di[`${sel.hw}|${sel.variant}|${sel.quant}`]
+ || di[`${sel.variant}|${sel.quant}`]
+ || di[`${sel.hw}|${sel.quant}|${sel.strategy}`]
|| di[`${sel.hw}|${sel.quant}`] || di[sel.hw] || "lmsysorg/sglang:dev";
const dockerRunCommand = typeof config.dockerRunCommand === "function"
? config.dockerRunCommand(sel)
diff --git a/docs/src/snippets/configs/deepseek-ai/deepseek-v4-benchmarks.jsx b/docs/src/snippets/configs/deepseek-ai/deepseek-v4-benchmarks.jsx
index 633903b6d..34598c46f 100644
--- a/docs/src/snippets/configs/deepseek-ai/deepseek-v4-benchmarks.jsx
+++ b/docs/src/snippets/configs/deepseek-ai/deepseek-v4-benchmarks.jsx
@@ -634,4 +634,28 @@ export const benchmarks = [
{ match: { hw: "mi355x", variant: "pro", quant: "fp8", strategy: "low-latency", nodes: "single" } },
{ match: { hw: "mi355x", variant: "pro", quant: "fp8", strategy: "balanced", nodes: "single" } },
{ match: { hw: "mi355x", variant: "pro", quant: "fp8", strategy: "high-throughput", nodes: "single" } },
+ // ====================================================================
+ // B200 + FP4 — Flash Vision (Exp)
+ // ====================================================================
+ {
+ match: { hw: "b200", variant: "flash-vision", quant: "fp4", strategy: "low-latency", nodes: "single" },
+ sglang_version: "dev-dsv4-flash-vision",
+ accuracy: { mmmu_pro_pct: 74.96 },
+ notes: "MMMU-Pro (standard, 10-option) measured with sgl-eval on 4×B200 (TP=4) at temperature 1.0, top-p 0.95, --reasoning-effort max.",
+ },
+ { match: { hw: "b200", variant: "flash-vision", quant: "fp4", strategy: "balanced", nodes: "single" } },
+ { match: { hw: "b200", variant: "flash-vision", quant: "fp4", strategy: "high-throughput", nodes: "single" } },
+ // B300 / GB200 / GB300 / H200 / H100 — Flash Vision (Exp), all pending
+ { match: { hw: "b300", variant: "flash-vision", quant: "fp4", strategy: "low-latency", nodes: "single" } },
+ { match: { hw: "b300", variant: "flash-vision", quant: "fp4", strategy: "balanced", nodes: "single" } },
+ { match: { hw: "b300", variant: "flash-vision", quant: "fp4", strategy: "high-throughput", nodes: "single" } },
+ { match: { hw: "gb200", variant: "flash-vision", quant: "fp4", strategy: "low-latency", nodes: "single" } },
+ { match: { hw: "gb200", variant: "flash-vision", quant: "fp4", strategy: "balanced", nodes: "single" } },
+ { match: { hw: "gb200", variant: "flash-vision", quant: "fp4", strategy: "high-throughput", nodes: "single" } },
+ { match: { hw: "gb300", variant: "flash-vision", quant: "fp4", strategy: "low-latency", nodes: "single" } },
+ { match: { hw: "gb300", variant: "flash-vision", quant: "fp4", strategy: "balanced", nodes: "single" } },
+ { match: { hw: "gb300", variant: "flash-vision", quant: "fp4", strategy: "high-throughput", nodes: "single" } },
+ { match: { hw: "h200", variant: "flash-vision", quant: "fp4", strategy: "low-latency", nodes: "single" } },
+ { match: { hw: "h200", variant: "flash-vision", quant: "fp4", strategy: "balanced", nodes: "single" } },
+ { match: { hw: "h100", variant: "flash-vision", quant: "fp4", strategy: "balanced", nodes: "single" } },
];
diff --git a/docs/src/snippets/configs/deepseek-ai/deepseek-v4.jsx b/docs/src/snippets/configs/deepseek-ai/deepseek-v4.jsx
index 1993e9b06..0f53e3ff9 100644
--- a/docs/src/snippets/configs/deepseek-ai/deepseek-v4.jsx
+++ b/docs/src/snippets/configs/deepseek-ai/deepseek-v4.jsx
@@ -25,6 +25,7 @@ export const config = {
variants: [
{ id: "flash", label: "Flash", subtitle: "284B" },
{ id: "flash-official", label: "Flash Official", subtitle: "284B · 0731" },
+ { id: "flash-vision", label: "Flash Vision", subtitle: "305B · Exp" },
{ id: "pro", label: "Pro", subtitle: "1.6T" },
{ id: "pro-official", label: "Pro Official", subtitle: "1.6T · 0813" },
],
@@ -49,6 +50,7 @@ export const config = {
"flash|fp8": "deepseek-ai/DeepSeek-V4-Flash",
"flash|nvfp4": "nvidia/DeepSeek-V4-Flash-NVFP4",
"flash-official|fp4": "deepseek-ai/DeepSeek-V4-Flash-0731",
+ "flash-vision|fp4": "deepseek-ai/DeepSeek-V4-Flash-Vision-Exp",
"pro|fp4": "deepseek-ai/DeepSeek-V4-Pro",
"pro|fp8": "deepseek-ai/DeepSeek-V4-Pro",
"pro|nvfp4": "nvidia/DeepSeek-V4-Pro-NVFP4",
@@ -144,6 +146,14 @@ sgl-eval run aime25 \\
--n-repeats 16 --max-tokens 400000 \\
--temperature 1.0 --top-p 1.0 --thinking \\
--out-dir /sgl-workspace/logs \\
+ --base-url http://{{CURL_HOST}}:{{CURL_PORT}}/v1`,
+ },
+ mmmu_pro_pct: {
+ "flash-vision":
+`# To install sgl-eval: pip install git+https://github.com/sgl-project/sgl-eval
+sgl-eval run mmmu_pro \\
+ --reasoning-effort max \\
+ --temperature 1.0 --top-p 0.95 \\
--base-url http://{{CURL_HOST}}:{{CURL_PORT}}/v1`,
},
},
@@ -162,6 +172,7 @@ sgl-eval run aime25 \\
["gpqa_pct", "GPQA Diamond", "%"],
["aime25_pct", "AIME25", "%"],
["gsm8k_pct", "GSM8K (1-shot)", "%"],
+ ["mmmu_pro_pct", "MMMU-Pro (standard, 10-option)", "%"],
],
// Prepended as `# ...` comments above multi-node commands.
@@ -175,6 +186,10 @@ sgl-eval run aime25 \\
},
dockerImages: {
+ // Flash Vision (Exp) support has not shipped in a release yet
+ // (sgl-project/sglang#37253) — until it does, the variant needs this
+ // preview build on every hardware.
+ "flash-vision|fp4": "lmsysorg/sglang:dev-dsv4-flash-vision",
h100: "lmsysorg/sglang:latest",
h200: "lmsysorg/sglang:latest",
b200: "lmsysorg/sglang:latest",
@@ -289,15 +304,17 @@ sgl-eval run aime25 \\
{ id: "mtp-314", label: "EAGLE / MTP 3-1-4",
flags: ["--speculative-algorithm EAGLE", "--speculative-num-steps 3",
"--speculative-eagle-topk 1", "--speculative-num-draft-tokens 4"],
- hide: { variant: ["flash-official", "pro-official"] } },
+ hide: { variant: ["flash-official", "flash-vision", "pro-official"] } },
{ id: "mtp-112", label: "EAGLE / MTP 1-1-2",
flags: ["--speculative-algorithm EAGLE", "--speculative-num-steps 1",
"--speculative-eagle-topk 1", "--speculative-num-draft-tokens 2"],
- hide: { variant: ["flash-official", "pro-official"] } },
+ hide: { variant: ["flash-official", "flash-vision", "pro-official"] } },
{ id: "dspark", label: "DSpark",
flags: ["--speculative-algorithm DSPARK"],
hide: { variant: ["flash", "pro"] },
disable: [
+ { when: { variant: ["flash-vision"] },
+ reason: "The Flash Vision checkpoint bundles a DSpark head, but speculative decoding is not yet verified with image inputs — the cookbook recipes run target-only for now." },
{ when: { dpAttnOn: [true] },
reason: "DSpark is not compatible with DP Attention on the current release." },
{ when: { hw: ["mi300x", "mi355x"] },
@@ -2826,5 +2843,277 @@ sgl-eval run aime25 \\
"--port {{PORT}}",
],
},
+
+ // ====================================================================
+ // B200 + FP4 — Flash Vision (Exp)
+ //
+ // DeepSeek-V4-Flash-Vision-Exp (sgl-project/sglang#37253): the 0731
+ // Flash base plus a vision encoder + aligner. The checkpoint bundles a
+ // DSpark head, but speculative decoding is not yet verified with image
+ // batches, so every recipe runs target-only. Low-latency is the serving
+ // shape the MMMU-Pro round ran on (4×B200); balanced / high-throughput
+ // mirror the Flash Official recipes on the same 4-GPU topology — final
+ // verification in progress.
+ // ====================================================================
+ {
+ match: { hw: "b200", variant: "flash-vision", quant: "fp4", strategy: "low-latency", nodes: "single" },
+ verified: true,
+ warn: "DeepSeek-V4-Flash-Vision-Exp support has not shipped in an SGLang release yet (sglang PR 37253): Docker mode already points at the preview image; for Python mode install SGLang from that PR. See [Flash Vision notes](#vision-note).",
+ env: [],
+ flags: [
+ "--model-path {{MODEL_NAME}}",
+ "--tp 4",
+ "--mem-fraction-static 0.85",
+ "--host {{HOST_IP}}",
+ "--port {{PORT}}",
+ ],
+ },
+ {
+ match: { hw: "b200", variant: "flash-vision", quant: "fp4", strategy: "balanced", nodes: "single" },
+ verified: false,
+ verificationStatus: "in-progress",
+ warn: "DeepSeek-V4-Flash-Vision-Exp support has not shipped in an SGLang release yet (sglang PR 37253): Docker mode already points at the preview image; for Python mode install SGLang from that PR. See [Flash Vision notes](#vision-note).",
+ env: ["SGLANG_DEEPEP_NUM_MAX_DISPATCH_TOKENS_PER_RANK=1024"],
+ flags: [
+ "--model-path {{MODEL_NAME}}",
+ "--tp 4",
+ "--dp 4",
+ "--enable-dp-attention",
+ "--moe-a2a-backend deepep",
+ "--mem-fraction-static 0.85",
+ "--deepep-config '{\"normal_dispatch\":{\"num_sms\":96},\"normal_combine\":{\"num_sms\":96}}'",
+ "--host {{HOST_IP}}",
+ "--port {{PORT}}",
+ ],
+ },
+ {
+ match: { hw: "b200", variant: "flash-vision", quant: "fp4", strategy: "high-throughput", nodes: "single" },
+ verified: false,
+ verificationStatus: "in-progress",
+ warn: "DeepSeek-V4-Flash-Vision-Exp support has not shipped in an SGLang release yet (sglang PR 37253): Docker mode already points at the preview image; for Python mode install SGLang from that PR. See [Flash Vision notes](#vision-note).",
+ env: [
+ "SGLANG_OPT_DEEPGEMM_MEGA_MOE_NUM_MAX_TOKENS_PER_RANK=8320",
+ ],
+ flags: [
+ "--model-path {{MODEL_NAME}}",
+ "--tp 4",
+ "--dp 4",
+ "--enable-dp-attention",
+ "--moe-a2a-backend megamoe",
+ "--mem-fraction-static 0.85",
+ "--host {{HOST_IP}}",
+ "--port {{PORT}}",
+ ],
+ },
+
+ // ====================================================================
+ // B300 / GB200 / GB300 + FP4 — Flash Vision (Exp)
+ // ====================================================================
+ {
+ match: { hw: "b300", variant: "flash-vision", quant: "fp4", strategy: "low-latency", nodes: "single" },
+ verified: false,
+ verificationStatus: "in-progress",
+ warn: "DeepSeek-V4-Flash-Vision-Exp support has not shipped in an SGLang release yet (sglang PR 37253): Docker mode already points at the preview image; for Python mode install SGLang from that PR. See [Flash Vision notes](#vision-note).",
+ env: [],
+ flags: [
+ "--model-path {{MODEL_NAME}}",
+ "--tp 4",
+ "--mem-fraction-static 0.85",
+ "--host {{HOST_IP}}",
+ "--port {{PORT}}",
+ ],
+ },
+ {
+ match: { hw: "b300", variant: "flash-vision", quant: "fp4", strategy: "balanced", nodes: "single" },
+ verified: false,
+ verificationStatus: "in-progress",
+ warn: "DeepSeek-V4-Flash-Vision-Exp support has not shipped in an SGLang release yet (sglang PR 37253): Docker mode already points at the preview image; for Python mode install SGLang from that PR. See [Flash Vision notes](#vision-note).",
+ env: ["SGLANG_DEEPEP_NUM_MAX_DISPATCH_TOKENS_PER_RANK=1024"],
+ flags: [
+ "--model-path {{MODEL_NAME}}",
+ "--tp 4",
+ "--dp 4",
+ "--enable-dp-attention",
+ "--moe-a2a-backend deepep",
+ "--mem-fraction-static 0.85",
+ "--deepep-config '{\"normal_dispatch\":{\"num_sms\":96},\"normal_combine\":{\"num_sms\":96}}'",
+ "--host {{HOST_IP}}",
+ "--port {{PORT}}",
+ ],
+ },
+ {
+ match: { hw: "b300", variant: "flash-vision", quant: "fp4", strategy: "high-throughput", nodes: "single" },
+ verified: false,
+ verificationStatus: "in-progress",
+ warn: "DeepSeek-V4-Flash-Vision-Exp support has not shipped in an SGLang release yet (sglang PR 37253): Docker mode already points at the preview image; for Python mode install SGLang from that PR. See [Flash Vision notes](#vision-note).",
+ env: [
+ "SGLANG_OPT_DEEPGEMM_MEGA_MOE_NUM_MAX_TOKENS_PER_RANK=8320",
+ ],
+ flags: [
+ "--model-path {{MODEL_NAME}}",
+ "--tp 4",
+ "--dp 4",
+ "--enable-dp-attention",
+ "--moe-a2a-backend megamoe",
+ "--mem-fraction-static 0.85",
+ "--host {{HOST_IP}}",
+ "--port {{PORT}}",
+ ],
+ },
+ {
+ match: { hw: "gb200", variant: "flash-vision", quant: "fp4", strategy: "low-latency", nodes: "single" },
+ verified: false,
+ verificationStatus: "in-progress",
+ warn: "DeepSeek-V4-Flash-Vision-Exp support has not shipped in an SGLang release yet (sglang PR 37253): Docker mode already points at the preview image; for Python mode install SGLang from that PR. See [Flash Vision notes](#vision-note).",
+ env: [],
+ flags: [
+ "--model-path {{MODEL_NAME}}",
+ "--tp 4",
+ "--mem-fraction-static 0.85",
+ "--host {{HOST_IP}}",
+ "--port {{PORT}}",
+ ],
+ },
+ {
+ match: { hw: "gb200", variant: "flash-vision", quant: "fp4", strategy: "balanced", nodes: "single" },
+ verified: false,
+ verificationStatus: "in-progress",
+ warn: "DeepSeek-V4-Flash-Vision-Exp support has not shipped in an SGLang release yet (sglang PR 37253): Docker mode already points at the preview image; for Python mode install SGLang from that PR. See [Flash Vision notes](#vision-note).",
+ env: ["SGLANG_DEEPEP_NUM_MAX_DISPATCH_TOKENS_PER_RANK=1024"],
+ flags: [
+ "--model-path {{MODEL_NAME}}",
+ "--tp 4",
+ "--dp 4",
+ "--enable-dp-attention",
+ "--moe-a2a-backend deepep",
+ "--mem-fraction-static 0.85",
+ "--deepep-config '{\"normal_dispatch\":{\"num_sms\":96},\"normal_combine\":{\"num_sms\":96}}'",
+ "--host {{HOST_IP}}",
+ "--port {{PORT}}",
+ ],
+ },
+ {
+ match: { hw: "gb200", variant: "flash-vision", quant: "fp4", strategy: "high-throughput", nodes: "single" },
+ verified: false,
+ verificationStatus: "in-progress",
+ warn: "DeepSeek-V4-Flash-Vision-Exp support has not shipped in an SGLang release yet (sglang PR 37253): Docker mode already points at the preview image; for Python mode install SGLang from that PR. See [Flash Vision notes](#vision-note).",
+ env: [
+ "SGLANG_OPT_DEEPGEMM_MEGA_MOE_NUM_MAX_TOKENS_PER_RANK=8320",
+ ],
+ flags: [
+ "--model-path {{MODEL_NAME}}",
+ "--tp 4",
+ "--dp 4",
+ "--enable-dp-attention",
+ "--moe-a2a-backend megamoe",
+ "--mem-fraction-static 0.85",
+ "--host {{HOST_IP}}",
+ "--port {{PORT}}",
+ ],
+ },
+ {
+ match: { hw: "gb300", variant: "flash-vision", quant: "fp4", strategy: "low-latency", nodes: "single" },
+ verified: false,
+ verificationStatus: "in-progress",
+ warn: "DeepSeek-V4-Flash-Vision-Exp support has not shipped in an SGLang release yet (sglang PR 37253): Docker mode already points at the preview image; for Python mode install SGLang from that PR. See [Flash Vision notes](#vision-note).",
+ env: [],
+ flags: [
+ "--model-path {{MODEL_NAME}}",
+ "--tp 4",
+ "--mem-fraction-static 0.85",
+ "--host {{HOST_IP}}",
+ "--port {{PORT}}",
+ ],
+ },
+ {
+ match: { hw: "gb300", variant: "flash-vision", quant: "fp4", strategy: "balanced", nodes: "single" },
+ verified: false,
+ verificationStatus: "in-progress",
+ warn: "DeepSeek-V4-Flash-Vision-Exp support has not shipped in an SGLang release yet (sglang PR 37253): Docker mode already points at the preview image; for Python mode install SGLang from that PR. See [Flash Vision notes](#vision-note).",
+ env: ["SGLANG_DEEPEP_NUM_MAX_DISPATCH_TOKENS_PER_RANK=1024"],
+ flags: [
+ "--model-path {{MODEL_NAME}}",
+ "--tp 4",
+ "--dp 4",
+ "--enable-dp-attention",
+ "--moe-a2a-backend deepep",
+ "--mem-fraction-static 0.85",
+ "--deepep-config '{\"normal_dispatch\":{\"num_sms\":96},\"normal_combine\":{\"num_sms\":96}}'",
+ "--host {{HOST_IP}}",
+ "--port {{PORT}}",
+ ],
+ },
+ {
+ match: { hw: "gb300", variant: "flash-vision", quant: "fp4", strategy: "high-throughput", nodes: "single" },
+ verified: false,
+ verificationStatus: "in-progress",
+ warn: "DeepSeek-V4-Flash-Vision-Exp support has not shipped in an SGLang release yet (sglang PR 37253): Docker mode already points at the preview image; for Python mode install SGLang from that PR. See [Flash Vision notes](#vision-note).",
+ env: [
+ "SGLANG_OPT_DEEPGEMM_MEGA_MOE_NUM_MAX_TOKENS_PER_RANK=8320",
+ ],
+ flags: [
+ "--model-path {{MODEL_NAME}}",
+ "--tp 4",
+ "--dp 4",
+ "--enable-dp-attention",
+ "--moe-a2a-backend megamoe",
+ "--mem-fraction-static 0.85",
+ "--host {{HOST_IP}}",
+ "--port {{PORT}}",
+ ],
+ },
+
+ // ====================================================================
+ // H200 + FP4 — Flash Vision (Exp)
+ // ====================================================================
+ {
+ match: { hw: "h200", variant: "flash-vision", quant: "fp4", strategy: "low-latency", nodes: "single" },
+ verified: false,
+ verificationStatus: "in-progress",
+ warn: "DeepSeek-V4-Flash-Vision-Exp support has not shipped in an SGLang release yet (sglang PR 37253): Docker mode already points at the preview image; for Python mode install SGLang from that PR. See [Flash Vision notes](#vision-note).",
+ env: [],
+ flags: [
+ "--model-path {{MODEL_NAME}}",
+ "--tp 4",
+ "--moe-runner-backend marlin",
+ "--mem-fraction-static 0.85",
+ "--host {{HOST_IP}}",
+ "--port {{PORT}}",
+ ],
+ },
+ {
+ match: { hw: "h200", variant: "flash-vision", quant: "fp4", strategy: "balanced", nodes: "single" },
+ verified: false,
+ verificationStatus: "in-progress",
+ warn: "DeepSeek-V4-Flash-Vision-Exp support has not shipped in an SGLang release yet (sglang PR 37253): Docker mode already points at the preview image; for Python mode install SGLang from that PR. See [Flash Vision notes](#vision-note).",
+ env: [],
+ flags: [
+ "--model-path {{MODEL_NAME}}",
+ "--tp 4",
+ "--moe-runner-backend flashinfer_mxfp4",
+ "--mem-fraction-static 0.85",
+ "--host {{HOST_IP}}",
+ "--port {{PORT}}",
+ ],
+ },
+
+ // ====================================================================
+ // H100 + FP4 — Flash Vision (Exp)
+ // ====================================================================
+ {
+ match: { hw: "h100", variant: "flash-vision", quant: "fp4", strategy: "balanced", nodes: "single" },
+ verified: false,
+ verificationStatus: "in-progress",
+ warn: "DeepSeek-V4-Flash-Vision-Exp support has not shipped in an SGLang release yet (sglang PR 37253): Docker mode already points at the preview image; for Python mode install SGLang from that PR. See [Flash Vision notes](#vision-note).",
+ env: [],
+ flags: [
+ "--model-path {{MODEL_NAME}}",
+ "--tp 8",
+ "--moe-runner-backend marlin",
+ "--mem-fraction-static 0.85",
+ "--host {{HOST_IP}}",
+ "--port {{PORT}}",
+ ],
+ },
],
};