minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 10 Packages Projects Releases Wiki Activity

[diffusion] docs: add verified DGX Spark recipe for Qwen-Image 2.1 (#40487)

Browse Source 
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
This commit is contained in:
Mick
2026-09-21 08:49:35 +08:00
committed by GitHub
co-authored by Mick Qian
parent b912db67ea
commit 6ad78f2281
2 changed files with 54 additions and 28 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/cookbook/diffusion/Qwen-Image/Qwen-Image-2.1.mdx
+26 -5
View File
@@ -14,9 +14,10 @@ import { config } from '/src/snippets/configs/Qwen/qwen-image-2.1.jsx';
Install the runtime dependencies with `uv pip install "sglang[diffusion]" --prerelease=allow`,
then install this integration from its source checkout with
`uv pip install -e "python[diffusion]"`. Use an authorized checkpoint directory in
place of `/models/qwen-image-2.1`. The recipes below target NVIDIA CUDA on Linux;
the hardware picker selects a tested single-GPU recipe for the full checkpoint.
`uv pip install -e "python[diffusion]"`. The picker uses `Qwen/Qwen-Image-2.1`;
you can also set a local checkpoint directory under **Variables**. The recipes
target NVIDIA CUDA on Linux; the picker marks which single-GPU workloads have
been verified with the full checkpoint.
<Deployment config={config} />
@@ -27,7 +28,7 @@ steps, and output count. Set reference PNG paths under **Variables**; edits
upload files from the machine running cURL, so they need not exist on the server.
Hardware selection applies the recommended placement for that GPU. H200,
B200, and RTX PRO 6000 96GB keep weights resident; RTX 5090 and RTX 4090
B200, RTX PRO 6000 96GB, and DGX Spark keep weights resident; RTX 5090 and RTX 4090
offload selected components to fit the full pipeline.
Untested topologies and feature combinations remain selectable and are labeled
**Unverified**. Invalid topology combinations disable Copy. This integration
@@ -58,6 +59,7 @@ execution, and full-image VAE decoding.
| B200 192GB | Resident / FlashAttention | 2.46 s | 3.02 s | 38.5 GiB |
| RTX PRO 6000 96GB | Resident / Torch SDPA | 8.03 s | 9.63 s | 38.4 GiB |
| RTX 4090 24GB | DiT and VAE resident, encoder layerwise offload / FlashAttention | 18.68 s | 21.68 s | 22.7 GiB |
| DGX Spark 128GB unified | Resident / Torch SDPA | 35.36 s | 42.23 s | — (unified) |
Measured on 2026-09-20 at 1024×1024, 40 steps, CFG 1, and one RGBA PNG per
request. Times are median HTTP latency after warmup, including PNG serialization
@@ -68,6 +70,25 @@ RTX 5090 uses DiT layerwise offload and Torch SDPA; its recipe has not been
retested with the updated checkpoint. Both RTX 5090 and RTX PRO 6000 use SDPA
when FlashAttention is selected in this runtime. CPU offload requires host RAM.
### DGX Spark
Select **DGX Spark** for one GB10 GPU on Linux ARM64 with CUDA 13. Use the
source installation above. The recommended configuration keeps all components
resident, uses native BF16/FP32 precision, and lets the runtime select Torch SDPA:
```bash Command
sglang serve \
--model-path Qwen/Qwen-Image-2.1 \
--performance-mode speed
```
The [128 GB unified memory](https://docs.nvidia.com/dgx/dgx-spark/hardware.html)
is shared by the CPU and GPU. CPU offload is unnecessary for the verified
single-image 1024×1024 workload. Keep full-image VAE decoding and eager execution.
Generation, editing, transparent generation, and transparent editing were
verified with PyTorch 2.13.0+cu130. Spark reports no separate VRAM usage in `nvidia-smi`.
This recipe covers one Spark; multi-node deployment and batching remain unverified.
### Batching
Keep **Request batching → Off** and **Outputs → 1** for interactive use.
@@ -203,7 +224,7 @@ See the [GGUF guide](/docs/sglang-diffusion/quantization#gguf).
NVFP4 requires Blackwell and compatible ModelOpt exports. Select the component
directories using `--component-paths.transformer` and/or
`--component-paths.text_encoder`. Keep the FlashInfer backend at `auto` on
RTX 5090 and RTX PRO 6000: TensorRT-LLM FP4 GEMM does not support SM120.
RTX 5090, RTX PRO 6000, and DGX Spark: TensorRT-LLM FP4 GEMM does not support SM12.x.
These GPUs remain unverified for this model's NVFP4 exports. See the
[NVFP4 guide](/docs/sglang-diffusion/quantization#modelopt-nvfp4).
docs/src/snippets/configs/Qwen/qwen-image-2.1.jsx
+28 -23
View File
@@ -1,11 +1,11 @@
export const config = (() => {
const sm120Hardware = ["rtx5090", "rtxpro6000"];
const platformAttention = (s) => sm120Hardware.includes(s.hw) ? "sdpa" : "fa";
const effectiveAttention = (s) => s.attention === "platform" || (sm120Hardware.includes(s.hw) && s.attention === "fa") ? platformAttention(s) : s.attention;
const sm12Hardware = ["rtx5090", "rtxpro6000", "dgx-spark"];
const platformAttention = (s) => sm12Hardware.includes(s.hw) ? "sdpa" : "fa";
const effectiveAttention = (s) => s.attention === "platform" || (sm12Hardware.includes(s.hw) && s.attention === "fa") ? platformAttention(s) : s.attention;
const config = {
modelName: "Qwen-Image 2.1",
supportedHardware: ["h200", "b200", "rtxpro6000", "rtx5090", "rtx4090"],
supportedHardware: ["h200", "b200", "rtxpro6000", "rtx5090", "rtx4090", "dgx-spark"],
hardware: [
{ id: "rtxpro6000", label: "RTX PRO 6000", vram: "96GB", vendor: "consumer" },
{ id: "rtx5090", label: "RTX 5090", vram: "32GB", vendor: "consumer" },
@@ -19,7 +19,7 @@ const config = {
id: "weights",
title: "Checkpoint weights",
scope: "base",
description: "One checkpoint serves generation and editing. Set its authorized local path under Variables.",
description: "One checkpoint serves generation and editing. Set its Hugging Face repository or local path under Variables.",
default: "default",
options: [{ id: "default", label: "Qwen-Image 2.1", flags: [] }],
},
@@ -45,11 +45,11 @@ const config = {
options: [
{
id: "resident", label: "Resident",
recommendedWhen: (s) => ["h200", "b200", "rtxpro6000"].includes(s.hw),
recommendedWhen: (s) => ["h200", "b200", "rtxpro6000", "dgx-spark"].includes(s.hw),
disabled: (s) => ["rtx5090", "rtx4090"].includes(s.hw) && Number(s.gpus_per_node) === 1,
disableReason: "The full resident pipeline exceeds one consumer GPU's memory. Select CPU offload.",
flags: ["--performance-mode speed"],
description: "Keep all components on the GPU. Recommended for H200, B200, and RTX PRO 6000 96GB. RTX 5090 and RTX 4090 need offload.",
description: "Keep all components resident. Recommended for H200, B200, RTX PRO 6000 96GB, and DGX Spark 128GB unified memory. RTX 5090 and RTX 4090 need offload.",
},
{
id: "offload", label: "CPU offload",
@@ -80,10 +80,10 @@ const config = {
options: [
{
id: "platform", label: "Automatic", recommended: true,
flags: (s) => [`--attention-backend ${platformAttention(s) === "sdpa" ? "torch_sdpa" : "fa"}`],
description: "Uses SDPA on RTX PRO 6000 and RTX 5090, and FlashAttention on the other listed GPUs.",
flags: (s) => s.hw === "b200" ? ["--attention-backend fa"] : [],
description: "Uses SDPA on RTX PRO 6000, RTX 5090, and DGX Spark, and FlashAttention on the other listed GPUs.",
},
{ id: "fa", label: "FlashAttention", flags: ["--attention-backend fa"], description: "Exact attention with a fused kernel. This runtime falls back to Torch SDPA on RTX PRO 6000 and RTX 5090." },
{ id: "fa", label: "FlashAttention", flags: ["--attention-backend fa"], description: "Exact attention with a fused kernel. This runtime falls back to Torch SDPA on RTX PRO 6000, RTX 5090, and DGX Spark." },
{
id: "sdpa", label: "Torch SDPA", flags: ["--attention-backend torch_sdpa"],
soft: (s) => !config.commandBuilder.resource.verifiedRecipes.some((r) => r.hw === s.hw && r.placement === s.placement && r.attentions.includes("sdpa") && Number(s.gpus_per_node) === r.gpus_per_node),
@@ -143,19 +143,19 @@ const config = {
},
{
id: "nvfp4_dit", label: "NVFP4 DiT", flags: ['--component-paths.transformer "{{NVFP4_DIT_PATH}}"'],
disabled: (s) => !["b200", "rtxpro6000", "rtx5090"].includes(s.hw),
disabled: (s) => !["b200", "rtxpro6000", "rtx5090", "dgx-spark"].includes(s.hw),
disableReason: "Native NVFP4 requires a Blackwell GPU (compute capability 10.0 or newer).",
soft: true, softReason: "A calibrated ModelOpt-format DiT export passed 1024px/40-step generation, editing, and transparent output on B200. Other exports, RTX PRO 6000, and RTX 5090 need validation.",
},
{
id: "nvfp4_encoder", label: "NVFP4 encoder", flags: ['--component-paths.text_encoder "{{NVFP4_ENCODER_PATH}}"'],
disabled: (s) => !["b200", "rtxpro6000", "rtx5090"].includes(s.hw),
disabled: (s) => !["b200", "rtxpro6000", "rtx5090", "dgx-spark"].includes(s.hw),
disableReason: "Native NVFP4 requires a Blackwell GPU (compute capability 10.0 or newer).",
soft: true, softReason: "A calibrated language-encoder export passed generation, editing, and transparent output on B200; vision weights retain native precision. Output quality requires validation.",
},
{
id: "nvfp4_both", label: "NVFP4 DiT + encoder", flags: ['--component-paths.transformer "{{NVFP4_DIT_PATH}}"', '--component-paths.text_encoder "{{NVFP4_ENCODER_PATH}}"'],
disabled: (s) => !["b200", "rtxpro6000", "rtx5090"].includes(s.hw),
disabled: (s) => !["b200", "rtxpro6000", "rtx5090", "dgx-spark"].includes(s.hw),
disableReason: "Native NVFP4 requires a Blackwell GPU (compute capability 10.0 or newer).",
soft: true, softReason: "Combined exports passed generation, editing, transparent output, offload, and TP2 on B200. The small max-calibration sample changes image and alpha values; validate your exported checkpoint.",
},
@@ -169,7 +169,7 @@ const config = {
learnMore: "#5-runtime-features",
default: "auto",
options: [
{ id: "auto", label: "Auto", flags: ["--encoder-parallel auto"], recommended: true },
{ id: "auto", label: "Auto", recommended: true },
{ id: "replicate", label: "Replicate", flags: ["--encoder-parallel replicate"], soft: true, softReason: "Explicit replication has not been verified for this server recipe." },
{ id: "fold", label: "Fold", flags: ["--encoder-parallel fold"], soft: true, softReason: "Native encoder TP and full-checkpoint TP2 × SP2 editing passed on B200. Requires node-local P2P; this HTTP recipe is unverified." },
],
@@ -224,7 +224,7 @@ const config = {
learnMore: "#batching",
default: "off",
options: [
{ id: "off", label: "Off", recommended: true, flags: ["--batching-max-size 1"], description: "Recommended for interactive latency. Resident H200, B200, and RTX PRO 6000 batching did not materially improve throughput in the measured workload." },
{ id: "off", label: "Off", recommended: true, description: "Recommended for interactive latency. Resident H200, B200, and RTX PRO 6000 batching did not materially improve throughput in the measured workload." },
{
id: "2", label: "Up to 2 images",
flags: ["--batching-max-size 2", "--batching-delay-ms 20"],
@@ -289,8 +289,9 @@ const config = {
{ id: "b200-2-ulysses", hw: "b200", nodes: 1, gpus_per_node: 2, placement: "resident", tp_size: 1, ulysses_degree: 2, ring_degree: 1, encoder: "auto", attentions: ["fa"], batchSizes: [1, 2] },
{ id: "rtxpro6000-1-resident", hw: "rtxpro6000", nodes: 1, gpus_per_node: 1, placement: "resident", tp_size: 1, ulysses_degree: 1, ring_degree: 1, encoder: "auto", attentions: ["sdpa"], batchSizes: [1, 2, 4], default: true },
{ id: "rtxpro6000-1-offload", hw: "rtxpro6000", nodes: 1, gpus_per_node: 1, placement: "offload", tp_size: 1, ulysses_degree: 1, ring_degree: 1, encoder: "auto", attentions: ["sdpa"] },
{ id: "rtx5090-1-offload", hw: "rtx5090", nodes: 1, gpus_per_node: 1, placement: "offload", tp_size: 1, ulysses_degree: 1, ring_degree: 1, encoder: "auto", attentions: ["sdpa"], default: true },
{ id: "rtx5090-1-offload", hw: "rtx5090", nodes: 1, gpus_per_node: 1, placement: "offload", tp_size: 1, ulysses_degree: 1, ring_degree: 1, encoder: "auto", attentions: ["sdpa"], default: true, unverified: true },
{ id: "rtx4090-1-offload", hw: "rtx4090", nodes: 1, gpus_per_node: 1, placement: "offload", tp_size: 1, ulysses_degree: 1, ring_degree: 1, encoder: "auto", attentions: ["fa", "sdpa"], batchSizes: [1, 2], batchAttentions: ["fa"], default: true },
{ id: "dgx-spark-1-resident", hw: "dgx-spark", nodes: 1, gpus_per_node: 1, placement: "resident", tp_size: 1, ulysses_degree: 1, ring_degree: 1, encoder: "auto", attentions: ["sdpa"], batchSizes: [1], default: true },
],
autoTopology: (s) => ({ tp_size: 1, ulysses_degree: Number(s.gpus_per_node), ring_degree: 1 }),
validateTopology: (s, topology) => {
@@ -300,11 +301,12 @@ const config = {
const { tp_size: tp, ulysses_degree: ulysses, ring_degree: ring } = topology;
if (nodes !== 1) errors.push("This picker covers single-node deployment only.");
if (![1, 2, 4].includes(perNode)) errors.push("Select one, two, or four GPUs per node.");
if (s.hw === "dgx-spark" && perNode !== 1) errors.push("DGX Spark has one GPU per node. This picker covers one Spark only.");
if (![tp, ulysses, ring].every((n) => [1, 2, 4].includes(n))) errors.push("TP, Ulysses and Ring must each be 1, 2, or 4.");
if (nodes * perNode !== tp * ulysses * ring) errors.push(`World size ${nodes * perNode} must equal TP × Ulysses × Ring (${tp * ulysses * ring}).`);
if (32 % (tp * ulysses) !== 0) errors.push("32 attention heads must be divisible by TP × Ulysses.");
if (ring > 1 && effectiveAttention(s) === "sdpa") errors.push("Ring requires FlashAttention or SageAttention; Torch SDPA is unsupported.");
if (s.precision?.startsWith("nvfp4_") && !["b200", "rtxpro6000", "rtx5090"].includes(s.hw)) errors.push("Native NVFP4 requires a Blackwell GPU. Select B200, RTX PRO 6000, or RTX 5090.");
if (s.precision?.startsWith("nvfp4_") && !["b200", "rtxpro6000", "rtx5090", "dgx-spark"].includes(s.hw)) errors.push("Native NVFP4 requires a Blackwell GPU. Select B200, RTX PRO 6000, RTX 5090, or DGX Spark.");
if (perNode === 1 && ["rtx5090", "rtx4090"].includes(s.hw) && s.placement === "resident") errors.push("The full resident pipeline exceeds this GPU's memory. Select CPU offload.");
return errors;
},
@@ -319,7 +321,7 @@ const config = {
&& entry.nodes === Number(s.nodes) && entry.gpus_per_node === Number(s.gpus_per_node)
&& entry.placement === s.placement && entry.tp_size === topology.tp_size
&& entry.ulysses_degree === topology.ulysses_degree && entry.ring_degree === topology.ring_degree);
const serveVerified = !!recipe && errors.length === 0 && s.encoder === "auto"
const serveVerified = !!recipe && !recipe.unverified && errors.length === 0 && s.encoder === "auto"
&& recipe.attentions.includes(effectiveAttention(s)) && s.precision === "native"
&& s.execution === "eager" && s.vae === "full"
&& (!s.batching || s.batching === "off" || ((recipe.batchSizes || [1]).includes(Number(s.batching))
@@ -333,15 +335,18 @@ const config = {
|| (s.hw === "h200" && s.background === "scene" && s.mode === "text" && s.resolution === "512" && Number(s.steps) === 4 && Number(s.outputs) === 2)
|| (s.hw === "h200" && s.background === "scene" && s.mode === "multi" && s.resolution === "512" && Number(s.steps) === 4 && Number(s.outputs) === 1));
const world = Number(s.nodes) * Number(s.gpus_per_node);
const flags = ['--model-path "{{MODEL_PATH}}"', "--model-id Qwen-Image-2.1", `--num-gpus ${world}`];
const flags = ['--model-path "{{MODEL_PATH}}"'];
if (world > 1) flags.push(`--num-gpus ${world}`);
if (topology.tp_size > 1) flags.push(`--tp-size ${topology.tp_size}`);
flags.push(`--ulysses-degree ${topology.ulysses_degree}`);
if (world > 1) flags.push(`--ulysses-degree ${topology.ulysses_degree}`);
if (topology.ring_degree > 1) flags.push(`--ring-degree ${topology.ring_degree}`);
flags.push("--host {{HOST_IP}}", "--port {{PORT}}");
const warnings = [];
if (s.hw === "rtx4090" && (Number(s.outputs) > 1 || (s.batching && s.batching !== "off"))) warnings.push("This recipe streams DiT layers for batch memory headroom. Restart with the updated Server command when changing output count or request batching.");
if (s.batching && s.batching !== "off" && s.mode !== "text") warnings.push("Cross-request batching applies to text-to-image requests. Image edits run separately; use Outputs for multiple images in one edit request.");
if (!serveVerified && !errors.length) warnings.push("This server combination has not completed an exact HTTP verification run.");
if (!serveVerified && !errors.length) warnings.push(s.hw === "rtx5090"
? "This RTX 5090 recipe has not been retested with the updated checkpoint."
: "This server combination has not completed an exact HTTP verification run.");
if (!requestVerified && !errors.length) warnings.push("This request shape is outside the verified HTTP matrix.");
return {
match: { hw: s.hw }, nnodes: Number(s.nodes), verified: serveVerified, flags,
@@ -355,7 +360,7 @@ const config = {
},
resolvedSettings: {
attention: s.attention === "platform" ? `${platformAttention(s) === "sdpa" ? "Torch SDPA" : "FlashAttention"} (auto)`
: sm120Hardware.includes(s.hw) && s.attention === "fa" ? "Torch SDPA (FA fallback)" : undefined,
: sm12Hardware.includes(s.hw) && s.attention === "fa" ? "Torch SDPA (FA fallback)" : undefined,
encoder: s.encoder === "auto" && world === 1 ? "Single GPU (auto)" : undefined,
},
},
@@ -365,7 +370,7 @@ const config = {
modelNames: { default: "Qwen-Image-2.1" },
placeholders: {
MODEL_PATH: { target: "command", label: "Authorized checkpoint directory", default: "/models/qwen-image-2.1" },
MODEL_PATH: { target: "command", label: "Checkpoint repository or directory", default: "Qwen/Qwen-Image-2.1" },
FP8_DIT_PATH: { target: "command", label: "Serialized FP8 DiT directory", default: "/models/qwen-image-2.1-fp8/transformer" },
FP8_ENCODER_PATH: { target: "command", label: "Serialized FP8 encoder directory", default: "/models/qwen-image-2.1-fp8/text_encoder" },
GGUF_DIT_PATH: { target: "command", label: "GGUF DiT file", default: "/models/qwen-image-2.1-gguf/transformer-Q4_0.gguf" },