[diffusion] model: support qwen-image-2.1 (#39983)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com> Co-authored-by: BBuf <1182563586@qq.com>
This commit is contained in:
Mick
@@ -0,0 +1,547 @@
|
||||
---
|
||||
title: Qwen-Image 2.1
|
||||
description: "Run Qwen-Image 2.1 text-to-image and image-conditioned generation with SGLang Diffusion."
|
||||
tag: NEW
|
||||
---
|
||||
|
||||
import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';
|
||||
import { Deployment } from '/src/snippets/_deployment.jsx';
|
||||
import { config } from '/src/snippets/configs/Qwen/qwen-image-2.1.jsx';
|
||||
|
||||
<DiffusionModelTags tags={["RGBA image", "text-to-image", "image editing", "multi-image references", "block-causal attention"]} />
|
||||
|
||||
## 1. Quick start
|
||||
|
||||
Install the runtime dependencies with `uv pip install "sglang[diffusion]" --prerelease=allow`,
|
||||
then install this integration from its source checkout with
|
||||
`uv pip install -e "python[diffusion]"`. Use an authorized checkpoint directory in
|
||||
place of `/models/qwen-image-2.1`. The recipes below target NVIDIA CUDA on Linux;
|
||||
the hardware picker selects a tested single-GPU recipe for the full checkpoint.
|
||||
|
||||
<Deployment config={config} />
|
||||
|
||||
Use **Setup** to select text-to-image, single-image editing, or multi-image
|
||||
editing. **Server** controls placement, attention, encoder scheduling, VAE
|
||||
tiling, and graph execution. **Request** controls the background, resolution,
|
||||
steps, and output count. Set reference PNG paths under **Variables**; edits
|
||||
upload files from the machine running cURL, so they need not exist on the server.
|
||||
|
||||
Hardware selection applies the recommended placement for that GPU. H200,
|
||||
B200, and RTX PRO 6000 96GB keep weights resident; RTX 5090 and RTX 4090 use
|
||||
offload to fit the full pipeline.
|
||||
Custom two- and four-GPU topologies and unverified feature combinations remain selectable and are labeled
|
||||
**Unverified**. Invalid topology combinations disable Copy. This integration
|
||||
currently uses the Python/source command; no published Docker image is verified.
|
||||
|
||||
Both request modes return base64 PNGs. To save all returned images, append
|
||||
`> response.json` to the request command, then run:
|
||||
|
||||
```bash Command
|
||||
python - <<'PY'
|
||||
import base64
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
for i, item in enumerate(json.loads(Path("response.json").read_text())["data"]):
|
||||
Path(f"output-{i}.png").write_bytes(base64.b64decode(item["b64_json"]))
|
||||
PY
|
||||
```
|
||||
|
||||
### Platform measurements
|
||||
|
||||
The following four-platform comparison and the fusion measurements below precede
|
||||
the training-template and VAE normalization corrections in `c2a31b2693c`;
|
||||
their output comparisons should not be treated as baselines for that revision.
|
||||
The separate RTX PRO 6000 measurement uses the corrected implementation.
|
||||
|
||||
| GPU | Recommended placement / attention | Generation median | Single edit | Peak device memory |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| H200 141GB | Resident / FlashAttention | Functional verification only | Passed | Not measured in this comparison |
|
||||
| B200 192GB | Resident / FlashAttention | 3.44 s | 3.84 s | 40.1 GiB |
|
||||
| RTX 5090 32GB | DiT layerwise offload / SDPA | 14.30 s | 16.84 s | 26.9 GiB |
|
||||
| RTX 4090 24GB | DiT layerwise + encoder CPU offload / FlashAttention | 24.60 s | 25.59 s | 21.4 GiB |
|
||||
|
||||
The recommendations compare exact attention backends and memory placement on
|
||||
one GPU per platform. Each run warms up with one 512px, 4-step request, then
|
||||
measures three 1024px, 40-step generations, one single-image edit, and one
|
||||
transparent generation. All use seed 42, CFG 1, eager execution, full-image VAE
|
||||
decoding, and PNG output. Generation latency is the median of three sequential
|
||||
HTTP requests; editing is one request. Times include encoding and PNG response
|
||||
serialization, but exclude server startup. Device memory is the highest sampled
|
||||
`nvidia-smi` usage across loading and requests, sampled every 0.5 seconds.
|
||||
|
||||
Measured on 2026-09-16 with source revision `128ae46cc`, PyTorch 2.13.0+cu130,
|
||||
Transformers 5.12.1, and Diffusers 0.37.0. SGLang's native encoder uses the
|
||||
Transformers 4.57.3 numerical semantics described below. The RTX 5090 runs used
|
||||
a 50 GiB process-group memory limit on a roughly 60 GiB host; this is a tested
|
||||
budget, not a minimum host-memory requirement.
|
||||
|
||||
B200 FlashAttention was faster than SDPA in this comparison (3.44 vs 3.70 s).
|
||||
On RTX 5090, both commands used Torch SDPA: this runtime falls back to SDPA
|
||||
when `--attention-backend fa` is selected on SM120. The measured 14.30 s
|
||||
(explicit SDPA) and 14.39 s (FA selection with SDPA fallback) therefore do not
|
||||
compare different backends. The picker defaults to SDPA and rejects Ring with
|
||||
either selection on RTX 5090. Keeping eight DiT layers resident
|
||||
did not improve the RTX 5090 generation median, so that flag is omitted.
|
||||
On RTX 4090, DiT offload alone passed generation but ran out of memory during
|
||||
editing. The recommended command also sets `--text-encoder-cpu-offload true`;
|
||||
this complete recipe passed generation, editing, and transparent PNG output.
|
||||
|
||||
These are measurements of this small workload, not universal latency or image
|
||||
quality guarantees. Different prompts, reference sizes, batching, and software
|
||||
versions can change memory use and latency. Multi-reference and batched request
|
||||
recipes retain their separate H200 verification scope in the picker.
|
||||
|
||||
### RTX PRO 6000 Blackwell 96GB
|
||||
|
||||
The recommended single-GPU command keeps all weights resident and selects Torch
|
||||
SDPA. This is the 96GB Blackwell Server Edition (SM120). This runtime also maps
|
||||
`--attention-backend fa` to SDPA on this GPU; Ring therefore requires another
|
||||
supported backend and is rejected with either selection in the picker.
|
||||
|
||||
Source revision `1eab5de5990` was measured on 2026-09-18:
|
||||
|
||||
| Placement | Generation median | Edit median | Peak device memory |
|
||||
| --- | --- | --- | --- |
|
||||
| Resident (recommended) | 8.23 s | 9.85 s | 40.1 GiB |
|
||||
| DiT layerwise offload | 10.28 s | 10.66 s | 26.1 GiB |
|
||||
|
||||
Both runs used PyTorch 2.13.0+cu130, Transformers 5.12.1, Diffusers 0.37.0,
|
||||
native precision, eager execution, and full-image VAE decoding.
|
||||
After two 1024px/40-step warmups, each measured five generations and three edits
|
||||
at that same resolution and step count, with seed 42, CFG 1, and CPU noise
|
||||
generation. HTTP latency includes PNG serialization and excludes server startup;
|
||||
device memory was sampled every 0.5 seconds across startup and requests.
|
||||
|
||||
Transparent generation and two repeated edits of the same transparent input passed
|
||||
with both placements, retaining alpha values from 0 to 255. Repeated requests
|
||||
and corresponding outputs across placements produced identical RGBA pixels for
|
||||
this workload. Quantized checkpoints and multi-GPU recipes on RTX PRO 6000 remain
|
||||
unverified.
|
||||
|
||||
### Lossless RoPE fusion
|
||||
|
||||
The native DiT fuses the float conversion, complex rotary multiplication, and
|
||||
output cast on supported CUDA tensors. Its first eager call checks exact
|
||||
agreement with the original PyTorch operation; a mismatch disables the fusion.
|
||||
No additional command flag is needed.
|
||||
|
||||
A separate comparison on 2026-09-17 used native revision `6b190085c48` as the
|
||||
baseline and `63ed20bbedb` with the fusion. Both used the software versions
|
||||
listed above, full-image VAE decode, eager execution, and the recommended
|
||||
placement and attention backend for each GPU:
|
||||
|
||||
- B200: generation **3.42 → 3.27 s** (4.5% lower latency), editing
|
||||
**4.03 → 3.89 s** (3.4% lower).
|
||||
- RTX 5090: generation **14.49 → 14.20 s** (2.0% lower), editing
|
||||
**16.97 → 16.68 s** (1.7% lower).
|
||||
|
||||
Each GPU ran four fresh servers in optimized/baseline/baseline/optimized order.
|
||||
Each startup used two full-size warmups followed by five generations and three
|
||||
edits. The medians pool 10 generations and six edits per variant, all at
|
||||
1024px, 40 steps, seed 42, CFG 1, CPU noise generation, and one RGBA PNG per
|
||||
request. The workload generated a red teapot and edited the same reference
|
||||
image to blue. HTTP times include PNG serialization and exclude startup.
|
||||
All corresponding output pixels were identical between revisions on each GPU.
|
||||
These measurements cover this fixed workload; other prompts and configurations
|
||||
can have different gains.
|
||||
|
||||
### Lossless MLP and residual fusion
|
||||
|
||||
The native DiT also uses the shared BF16 SiLU-multiply and gated-residual
|
||||
kernels, preserving the eager operations' intermediate rounding. SiLU-multiply
|
||||
checks its first eager call and falls back on mismatch. These optimizations
|
||||
are automatic on supported CUDA inputs.
|
||||
|
||||
A second B200 comparison on 2026-09-17 used `f874eae18be` (already including
|
||||
the RoPE fusion) versus `a3d14531474`. With resident weights, FlashAttention,
|
||||
and the same four-startup protocol and workload above, generation decreased
|
||||
from **3.272 to 3.134 s** (4.23%) and editing from **3.886 to 3.762 s** (3.18%).
|
||||
All corresponding RGBA pixels were identical across the 10 generation and six
|
||||
editing samples per variant. These are additional gains over the RoPE baseline;
|
||||
this comparison does not establish the gain on other GPUs.
|
||||
|
||||
### Lossless Q/K normalization
|
||||
|
||||
Q/K RMSNorm fuses the input conversion and square, then the normalization,
|
||||
output cast, and weight multiply. It retains the original FP32 mean reduction
|
||||
with the same tensor shape, preserving the eager reduction order and
|
||||
cast-before-weight rounding. The native DiT verifies its first eager call and
|
||||
uses the original implementation if the outputs differ. No flag is needed.
|
||||
|
||||
A B200 comparison on 2026-09-17 used revision `4e5459e0eda` (including the
|
||||
RoPE, MLP, and residual fusions) versus `d9e1e5dac96`. With resident weights,
|
||||
FlashAttention, and the four-startup protocol above, generation decreased from
|
||||
**3.114 to 2.828 s** (9.17%) and editing from **3.742 to 3.450 s** (7.81%).
|
||||
Each variant has 10 generation and six editing measurements at 1024px,
|
||||
40 steps, seed 42, and CFG 1. Every corresponding RGBA pixel was identical.
|
||||
These gains apply to this fixed B200 workload; other GPUs were not measured
|
||||
in this comparison.
|
||||
|
||||
### Lossless LayerNorm modulation
|
||||
|
||||
The DiT fuses affine-free LayerNorm and `* (1 + scale)` while retaining the
|
||||
eager Welford reduction and BF16 rounding order. Scale-only modulation skips
|
||||
the shift addition, including its effect on signed zeros. The first eager call
|
||||
checks the fused result against the native path and falls back on a mismatch.
|
||||
|
||||
A B200 comparison on 2026-09-17 used `5bddbfca9b1` (including the preceding
|
||||
fusions) versus `162181ff0ec`. With resident weights, FlashAttention, eager
|
||||
execution, and the same four-startup protocol, generation decreased from
|
||||
**2.831 to 2.748 s** (2.92%) and editing from **3.436 to 3.358 s** (2.26%).
|
||||
Each variant has 10 generation and six editing measurements at 1024px,
|
||||
40 steps, seed 42, and CFG 1. Every corresponding RGBA pixel was identical.
|
||||
This comparison measures this B200 workload only.
|
||||
|
||||
## 2. Model capabilities
|
||||
|
||||
Qwen-Image 2.1 supports text-to-image generation and image-conditioned editing
|
||||
through one pipeline. Qwen3-VL encodes the instruction and reference images;
|
||||
a single-stream transformer inserts each reference image's latents into its
|
||||
corresponding position in that sequence. Block-causal attention keeps each
|
||||
image internally bidirectional while respecting the order of text and images.
|
||||
|
||||
For successive edits, send the previous output as the next request's reference
|
||||
image. Requests do not retain dialogue history. Conditional KV is reused across
|
||||
denoising steps within one request and released afterward; cross-request caching
|
||||
and incremental dialogue-history caching are not implemented.
|
||||
|
||||
Choose this pipeline for checkpoints declaring `QwenImage21Pipeline`,
|
||||
`QwenImage21Transformer2DModel`, and `AutoencoderKLQwenImage21`. The older
|
||||
Qwen-Image and Qwen-Image-Edit checkpoints use different components and latent
|
||||
packing. They cannot share this model's VAE or transformer weights. Text and
|
||||
condition-image activations use timestep zero, allowing their attention keys
|
||||
and values to be reused for the remaining denoising steps.
|
||||
|
||||
## 3. Checkpoint layout
|
||||
|
||||
The checkpoint directory must contain `model_index.json` and the `processor`,
|
||||
`text_encoder`, `transformer`, `vae`, and `scheduler` subdirectories. The
|
||||
processor must include the Qwen3-VL tokenizer assets. SGLang loads all three
|
||||
neural components natively. A separate tokenizer directory is not required.
|
||||
|
||||
The checkpoint's VAE uses RGBA input and output with 64-channel latents. PNG
|
||||
reference images retain their alpha channel; RGB inputs receive an opaque
|
||||
alpha channel. Save generated images as PNG to preserve transparency.
|
||||
|
||||
Text conditioning uses the last decoder layer's output before the final
|
||||
normalization, matching the reference implementation with Transformers
|
||||
4.57.3. Vision position interpolation also follows its BF16 rounding order.
|
||||
SGLang selects these native semantics explicitly, so keep the
|
||||
repository's installed dependencies instead of downgrading the entire runtime.
|
||||
The updated [Diffusers reference](https://github.com/huggingface/diffusers/pull/14804)
|
||||
also selects pre-normalization hidden states explicitly on newer Transformers.
|
||||
|
||||
Editing uses the training markers `<image1>`, `<image2>`, and so on. The vision
|
||||
encoder sees alpha composited over white, while the VAE receives the original
|
||||
RGBA pixels. Empty prompts become a space. The VAE normalizes features in
|
||||
FP32 before casting back to the activation dtype and compresses spatial
|
||||
dimensions by a factor of 16.
|
||||
|
||||
Use `--model-id Qwen-Image-2.1` when the checkpoint directory has a different
|
||||
name. The model ID is a routing identifier; it does not grant access to model
|
||||
weights. Keep checkpoint access credentials in your environment.
|
||||
|
||||
### Two-GPU end-to-end test
|
||||
|
||||
The `qwen_image21_t2i_tp2` case is temporarily disabled until the checkpoint is
|
||||
accessible to fork PR CI. Its configuration and pinned reference image are
|
||||
retained for re-enabling the test.
|
||||
|
||||
The case uses TP 2 with sequence
|
||||
parallelism disabled, 1024 × 1024 PNG output, 40 steps, CFG 1, and seed 42.
|
||||
It sends two consecutive requests and checks the model API and image consistency.
|
||||
This case does not enforce a latency baseline or run a component accuracy check.
|
||||
|
||||
### Transparent PNG output
|
||||
|
||||
Choose **Transparent / alpha** under Request to generate an isolated subject
|
||||
or preserve a transparent reference during editing. The picker adds the
|
||||
transparency instruction to the prompt and sets `output_format: "png"`.
|
||||
`background: "transparent"` alone only selects an output format; it does not
|
||||
remove the background or change model conditioning. JPEG cannot retain alpha.
|
||||
|
||||
The model predicts continuous alpha values, including partly transparent edges.
|
||||
No thresholding or background-removal postprocessing is applied. Transparent
|
||||
generation and transparent-input editing were compared against the reference
|
||||
at 1024 × 1024 and 40 steps; that check does not guarantee perfect cutouts for
|
||||
every prompt. Transparent generation and single-image editing also passed on
|
||||
the recommended one-H200 and one-RTX PRO 6000 servers at that resolution and
|
||||
step count, with one output per request.
|
||||
|
||||
## 4. Offline requests
|
||||
|
||||
### Text-to-image
|
||||
|
||||
```bash Command
|
||||
sglang generate \
|
||||
--model-path /models/qwen-image-2.1 \
|
||||
--model-id Qwen-Image-2.1 \
|
||||
--prompt "A capybara reading a book by candlelight" \
|
||||
--width 1024 --height 1024 \
|
||||
--num-inference-steps 40 --guidance-scale 1 \
|
||||
--seed 0 --save-output
|
||||
```
|
||||
|
||||
### Image-conditioned editing
|
||||
|
||||
```bash Command
|
||||
sglang generate \
|
||||
--model-path /models/qwen-image-2.1 \
|
||||
--model-id Qwen-Image-2.1 \
|
||||
--image-path /path/to/input.png \
|
||||
--prompt "Move the scene to a snowy mountain at sunrise" \
|
||||
--width 1024 --height 1024 \
|
||||
--num-inference-steps 40 --guidance-scale 1 \
|
||||
--seed 0 --save-output
|
||||
```
|
||||
|
||||
Height and width must be positive multiples of 32. Reference images preserve
|
||||
their aspect ratio and are resized to approximately the requested output area;
|
||||
the same resized image feeds the VLM and VAE. Image labels are deterministic
|
||||
(`Picture 1`, `Picture 2`, and so on). Multiple outputs receive independent
|
||||
noise seeds and independent prefix caches.
|
||||
|
||||
## 5. Runtime features
|
||||
|
||||
The API requires a text prompt; precomputed embeddings alone do not provide
|
||||
the image-token positions needed by this pipeline.
|
||||
|
||||
The default is 40 Euler flow-matching steps with CFG disabled. To use CFG,
|
||||
provide `--negative-prompt` and a `--guidance-scale` greater than one. CFG uses
|
||||
the ordinary linear combination without the older Qwen-Image norm correction.
|
||||
Positive and negative prompts have separate request-owned prefix caches.
|
||||
|
||||
TP uses native parallel projections. Ulysses and Ring shard target-image
|
||||
attention while keeping the condition prefix replicated. The target token
|
||||
count, `(height / 16) × (width / 16)`, must be divisible by the SP degree. Encoder
|
||||
folding shards Qwen3-VL's language projections using the native encoder TP group.
|
||||
Full-checkpoint editing passed with TP2 × Ulysses2 and TP2 × Ring2 + FlashAttention
|
||||
on four B200 GPUs. These CLI checks do not mark every HTTP topology as verified.
|
||||
|
||||
VAE tiling is disabled by default for both encoding and decoding. Enable
|
||||
`--vae-tiling true` for tiled encoding and decoding; `--vae-sp true` also distributes tiles
|
||||
across the configured GPUs. These paths use the standard VAE runtime; tiled
|
||||
decode can differ from full image decode near tile boundaries.
|
||||
|
||||
For full-image spatial parallel decode, select **Spatial shard** or pass
|
||||
`--vae-config.parallel-decode-mode spatial_shard` with at least two GPUs.
|
||||
This mode splits feature-map height, exchanges convolution halos, and gathers
|
||||
the full map for VAE attention. It does not require `--vae-tiling` or `--vae-sp`.
|
||||
Two-B200 checks cover TP2, CFG parallelism, and all-component layerwise offload.
|
||||
FP64 component comparisons match full decode; BF16 full-checkpoint output can
|
||||
differ through floating-point rounding.
|
||||
|
||||
Select **All components layerwise** or pass `--layerwise-offload-components all`
|
||||
to stream repeated blocks in the DiT, Qwen3-VL language and vision encoders, and
|
||||
VAE encoder/decoder. Full-checkpoint 512px editing passed on one B200 and on
|
||||
two B200s with TP2 plus spatial VAE decode. This setting reduces device memory
|
||||
at the cost of host-device transfers; it is not the measured default for the
|
||||
consumer-GPU recipes above.
|
||||
|
||||
Revision `f1f3366c7c` fixes CPU/GPU initialization rounding in the vision
|
||||
encoder's rotary frequencies after device transfer. On one B200, native
|
||||
1024px/40-step generation, editing, and transparent output with all-component
|
||||
layerwise offload matched resident RGBA pixels exactly. Repeated editing after
|
||||
a transparent-generation request also matched. Resident output was unchanged
|
||||
from revision `6ee35b52fb`. These checks use FlashAttention, seed 42, and CFG 1.
|
||||
|
||||
Revision `81c8c550fa` also preserves the loader's FP8 weights and FP32 rotary
|
||||
buffers when moving the whole encoder between CPU and GPU. With that fix,
|
||||
`--text-encoder-cpu-offload true` matched resident generation, editing, and
|
||||
transparent RGBA pixels for both native precision and the combined serialized
|
||||
FP8 export in the same B200 workload, including repeated editing.
|
||||
|
||||
The pipeline also supports the shared
|
||||
[disaggregated runtime](/docs/sglang-diffusion/disaggregation). The encoder role
|
||||
loads both Qwen3-VL and the VAE to prepare reference-image conditioning; nested
|
||||
condition tensors and complex RoPE tensors transfer with the request. Separate
|
||||
encoder, denoiser, and decoder processes matched monolithic RGBA output for
|
||||
512px/4-step generation, editing, different prompt lengths, and CFG on B200.
|
||||
That check used same-host Mooncake TCP; multi-host RDMA remains unverified.
|
||||
|
||||
Online FP8 is available independently for the DiT and encoder through
|
||||
`--component-quantizations.transformer fp8` and
|
||||
`--component-quantizations.text_encoder fp8`. Each component and the combination
|
||||
passed 1024px/40-step HTTP generation and editing on a resident B200. FP8 changes
|
||||
the output: in one generation/edit pair, DiT-only FP8 gave RGBA PSNR
|
||||
37.56/41.07 dB against native precision; quantizing both gave 32.66/40.99 dB.
|
||||
These samples do not establish general image or alpha quality. Native precision
|
||||
remains the default.
|
||||
|
||||
### Serialized FP8 components
|
||||
|
||||
Select a **Serialized FP8** precision option in the picker and set the component
|
||||
directories under **Variables**. The tested format is E4M3FN weights with one
|
||||
FP32 `weight_scale` per linear and dynamic activation quantization. Each
|
||||
component directory contains its own architecture `config.json`, weight shards,
|
||||
and index; merge this top-level quantization configuration into its `config.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"quantization_config": {
|
||||
"quant_method": "fp8",
|
||||
"activation_scheme": "dynamic"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Load compatible exported components through the shared loader:
|
||||
|
||||
```bash Command
|
||||
sglang serve \
|
||||
--model-path /models/qwen-image-2.1 \
|
||||
--model-id Qwen-Image-2.1 \
|
||||
--component-paths.transformer /models/qwen-image-2.1-fp8/transformer \
|
||||
--component-paths.text_encoder /models/qwen-image-2.1-fp8/text_encoder \
|
||||
--num-gpus 1 --performance-mode speed --attention-backend fa \
|
||||
--host 0.0.0.0 --port 30010
|
||||
```
|
||||
|
||||
Use either override independently, or both as shown. Omit online quantization
|
||||
flags: the component metadata selects serialized loading. Adding metadata to
|
||||
BF16 weights does not convert them. The validated export quantizes 224 DiT
|
||||
attention/MLP matrices and 252 Qwen3-VL language matrices; the vision encoder,
|
||||
embeddings, output head, other DiT projections, and VAE retain native precision.
|
||||
All 476 loaded matrices and scales matched their serialized values.
|
||||
|
||||
At revision `5a117c9f3f`, DiT-only, encoder-only, and combined exports passed
|
||||
1024px/40-step generation, editing, and transparent PNG requests on B200 with
|
||||
FlashAttention, seed 42, and CFG 1. The combined export also passed TP2 with
|
||||
encoder folding and single-GPU `--layerwise-offload-components all`.
|
||||
At that revision, offload matched resident generation and transparent output
|
||||
exactly, but editing differed at 49.50 dB RGBA PSNR. Revision `f1f3366c7c` fixes
|
||||
the vision rotary initialization difference: a new 1024px/40-step comparison
|
||||
matched resident generation, editing, and transparent RGBA pixels exactly
|
||||
with all-component layerwise offload. Resident outputs were unchanged. TP2
|
||||
still changes numerical results.
|
||||
|
||||
| Serialized FP8 scope | Generation RGBA PSNR vs native | Edit RGBA PSNR vs native |
|
||||
| --- | --- | --- |
|
||||
| DiT | 38.35 dB | 40.94 dB |
|
||||
| Encoder | 34.46 dB | 49.19 dB |
|
||||
| Both | 34.93 dB | 41.25 dB |
|
||||
|
||||
For the combined export, the transparent cat's alpha channel measured 32.03 dB
|
||||
PSNR and 0.81 mean absolute error on the 0–255 scale against native precision;
|
||||
individual boundary pixels can differ substantially. Online FP8 for both
|
||||
components also produced a real transparent PNG in this check. These are
|
||||
single-example comparisons, not a quality guarantee. Offline tensorwise scales
|
||||
differ from B200 online FP8's channelwise scales.
|
||||
|
||||
### GGUF components
|
||||
|
||||
Select **GGUF DiT**, **GGUF encoder**, or **GGUF DiT + encoder** under Server
|
||||
precision, then set the corresponding `.gguf` files under **Variables**.
|
||||
The picker uses `--component-weights-paths.transformer` and
|
||||
`--component-weights-paths.text_encoder`, retaining each component's architecture
|
||||
config from the base checkpoint. Each file must contain the entire component
|
||||
with native checkpoint tensor names. No online quantization flag is needed;
|
||||
the loader reads the quantization type from each GGUF tensor.
|
||||
|
||||
The tested Q4_0 export quantizes the same 224 DiT and 252 language-encoder
|
||||
matrices listed above. Other tensors retain native precision, including the
|
||||
vision tower, embeddings, output head, and VAE. Its DiT and encoder files are
|
||||
3.91 and 7.03 GiB respectively. All 476 loaded packed matrices matched the
|
||||
exported bytes; sampled CUDA dequantization matched the GGUF CPU reference
|
||||
after conversion to BF16.
|
||||
|
||||
At revision `7e0d4e9185`, DiT-only, encoder-only, and combined Q4_0 exports
|
||||
passed 1024px/40-step HTTP generation, editing, and transparent PNG output on
|
||||
B200 with FlashAttention, seed 42, and CFG 1. These are private validation
|
||||
exports, not published download targets. Use a compatible export of weights
|
||||
you are authorized to access.
|
||||
|
||||
The combined export also passed TP2 with encoder folding. On one GPU,
|
||||
all-component layerwise offload and whole-encoder CPU offload each matched
|
||||
resident generation, editing, and transparent RGBA pixels exactly. TP2 changed
|
||||
numerical results. Quantization itself is lossy:
|
||||
|
||||
| Q4_0 scope | Generation RGBA PSNR vs native | Edit RGBA PSNR vs native |
|
||||
| --- | --- | --- |
|
||||
| DiT | 24.99 dB | 33.66 dB |
|
||||
| Encoder | 28.97 dB | 43.26 dB |
|
||||
| Both | 23.86 dB | 33.46 dB |
|
||||
|
||||
The combined export's transparent cat retained alpha values from 0 to 255,
|
||||
with 66.8% of pixels at alpha 5 or below. Against native precision, its alpha
|
||||
PSNR was 21.20 dB and mean absolute error was 3.29/255; individual boundary
|
||||
pixels differed by up to 255. These single-example comparisons do not establish
|
||||
general image or cutout quality. Keep native precision when exact output is
|
||||
required.
|
||||
|
||||
GGUF reduces weight storage; it is not a promise of lower latency. The runtime
|
||||
dequantizes packed linears before BF16 matrix multiplication. Other GGUF tensor
|
||||
types, exports, and hardware need separate validation.
|
||||
See the shared [GGUF guide](/docs/sglang-diffusion/quantization#gguf)
|
||||
for loader and parallelism constraints.
|
||||
|
||||
### NVFP4 components
|
||||
|
||||
Select **NVFP4 DiT**, **NVFP4 encoder**, or **NVFP4 DiT + encoder** in the
|
||||
picker, then set the component directories under **Variables**. These options
|
||||
require Blackwell; H200 and RTX 4090 cannot run this native FP4 path. B200 has
|
||||
completed the checks below. RTX PRO 6000 and RTX 5090 remain unverified for this
|
||||
model's NVFP4 exports; their FlashInfer backend defaults to `auto`, because
|
||||
TensorRT-LLM FP4 GEMM does not support SM120. Keep that default on these GPUs.
|
||||
|
||||
Each exported directory contains its architecture config, weight shards, and
|
||||
index. The config declares `quant_method: modelopt`, `quant_algo: NVFP4`, and
|
||||
block size 16, with exclusions for native-precision layers. Use
|
||||
`--component-paths.transformer` and/or `--component-paths.text_encoder` to load
|
||||
the exported directories. Omit online quantization flags; metadata alone does
|
||||
not convert native weights into an NVFP4 checkpoint.
|
||||
|
||||
The private validation export quantizes the same 224 DiT and 252 language
|
||||
matrices as the FP8 example. Vision, embeddings, the output head, other DiT
|
||||
projections, and VAE retain native precision. Weight quantization uses ModelOpt
|
||||
0.46.1 with max calibration; static activation scales come from six separate
|
||||
1024px/40-step requests, including two edits and one transparent generation.
|
||||
This small calibration set does not establish general quality. It does not
|
||||
use SVDQuant or AWQ. All 476 loaded packed weights, block scales, and global
|
||||
scales matched the export after the runtime's layout transforms.
|
||||
|
||||
At revision `57b625d3e3`, each component and both together passed 1024px/40-step
|
||||
HTTP generation, editing, and transparent PNG output on B200 with
|
||||
FlashAttention, seed 42, CFG 1, and FlashInfer TensorRT-LLM FP4 GEMM. The combined
|
||||
export also passed TP2 with encoder folding. Single-GPU all-component layerwise
|
||||
offload and whole-encoder CPU offload each matched the combined resident RGBA
|
||||
pixels exactly. TP2 changed numerical results.
|
||||
|
||||
| NVFP4 scope | Generation RGBA PSNR vs native | Edit RGBA PSNR vs native |
|
||||
| --- | --- | --- |
|
||||
| DiT | 24.97 dB | 31.56 dB |
|
||||
| Encoder | 26.48 dB | 36.63 dB |
|
||||
| Both | 19.36 dB | 29.96 dB |
|
||||
|
||||
The combined export's transparent cat retained alpha from 0 to 255, with
|
||||
67.8% of pixels at alpha 5 or below. Against native precision, alpha PSNR was
|
||||
23.81 dB and mean absolute error was 2.22/255; some boundary pixels differed
|
||||
by 255. These are single-example comparisons of private exports, not download
|
||||
targets or quality guarantees. Native precision remains the default. See the
|
||||
shared [NVFP4 guide](/docs/sglang-diffusion/quantization#modelopt-nvfp4) for loader
|
||||
details.
|
||||
|
||||
### LoRA and execution options
|
||||
|
||||
LoRA uses the shared `--lora-path` and `--lora-merge-mode dynamic|merge` options
|
||||
and runtime adapter APIs. Diffusers keys prefixed with `transformer.` map to
|
||||
the native DiT. A synthetic adapter covering attention and MLP projections
|
||||
passed dynamic loading, merging, and removal on one B200 and TP2 with encoder
|
||||
folding. Both removal paths restored the base image exactly. This verifies
|
||||
adapter application and lifecycle, not the quality of a trained LoRA.
|
||||
|
||||
Cache-DiT hooks operate on target-image transformer blocks. Breakable CUDA
|
||||
Graph execution fills each request's prefix caches eagerly, then replays
|
||||
matching warmup graphs with those cache tensors as inputs. Warmup and request
|
||||
condition-prefix lengths must match, in addition to the output resolution;
|
||||
unseen shapes run eagerly. Text buckets alone cannot pad condition KV without
|
||||
changing attention semantics. FlashAttention, Sage
|
||||
attention and Torch SDPA are wired through the native attention layers;
|
||||
causal text runs use exact masked SDPA. Sage and Cache-DiT can change numerical
|
||||
results and require application-specific quality checks.
|
||||
|
||||
See the [compatibility inventory](/docs/sglang-diffusion/compatibility_matrix)
|
||||
for tested configurations and remaining validation boundaries. These checks
|
||||
are functional and numerical comparisons. The platform measurements above cover
|
||||
their stated HTTP workload; broader image quality is not evaluated.
|
||||
@@ -28,7 +28,7 @@ Image models generate one image request as a bounded denoising job, usually with
|
||||
<Card
|
||||
title="Qwen-Image"
|
||||
mode="card"
|
||||
href="/cookbook/diffusion/Qwen-Image/Qwen-Image"
|
||||
href="/cookbook/diffusion/Qwen-Image/Qwen-Image-2.1"
|
||||
img="/cards/logos/qwen.png"
|
||||
/>
|
||||
<Card
|
||||
|
||||
@@ -1509,6 +1509,7 @@
|
||||
{
|
||||
"group": "Qwen-Image",
|
||||
"pages": [
|
||||
"cookbook/diffusion/Qwen-Image/Qwen-Image-2.1",
|
||||
"cookbook/diffusion/Qwen-Image/Qwen-Image",
|
||||
"cookbook/diffusion/Qwen-Image/Qwen-Image-Edit"
|
||||
]
|
||||
|
||||
@@ -5,6 +5,122 @@ description: "Browse model families and public checkpoints supported by SGLang D
|
||||
|
||||
import { DiffusionModelCatalog } from '/src/snippets/diffusion/model-catalog.jsx';
|
||||
|
||||
## Qwen-Image 2.1 integration status
|
||||
|
||||
Qwen-Image 2.1 has a separate native pipeline for text-to-image and optional
|
||||
reference-image conditioning. Full-checkpoint validation on H200 includes
|
||||
1024×1024, 40-step generation and editing. Reference comparisons use
|
||||
Transformers 4.57.3, BF16 and matching Torch SDPA settings with cuDNN SDPA
|
||||
disabled. The native encoder explicitly preserves that version's pre-final-norm
|
||||
hidden state and vision position interpolation semantics.
|
||||
|
||||
Full-checkpoint functional checks on up to two H200 GPUs cover DiT TP, Ulysses,
|
||||
Ring + FA, CFG parallel, DiT layerwise offload, FA/Sage, parallel
|
||||
tiled VAE decode, Cache-DiT, breakable CUDA graphs, and multiple reference
|
||||
images and outputs. One H200 with resident weights also passed HTTP generation
|
||||
and editing, including multiple outputs and reference images. These are
|
||||
functional checks, not a broad quality evaluation of lossy optimizations.
|
||||
|
||||
Additional full-checkpoint checks on B200 cover native encoder TP, combined
|
||||
TP2 × Ulysses2 and TP2 × Ring2 + FA on four GPUs, and
|
||||
`--layerwise-offload-components all`. Two-GPU VAE `spatial_shard` decode passed
|
||||
with TP2, CFG parallelism, and all-component offload. These use 512px/4-step
|
||||
editing, plus a 1024px/4-step comparison of spatial and full VAE decode.
|
||||
Single-GPU 1024px/40-step generation, editing, and transparent PNG output also
|
||||
passed at source revision `d629d234f1`.
|
||||
|
||||
Two-B200 component regressions assert actual encoder weight sharding and compare
|
||||
text and image conditioning with a single-rank reference. VAE regressions cover
|
||||
both decoder structures and even/odd latent heights in FP32 and FP64. FP64
|
||||
spatial decode matches the full decoder; BF16 full-checkpoint output is not
|
||||
bit-exact. In the tested image, spatial versus full decode gave RGBA PSNR
|
||||
60.69 dB; this is one sample, not a general quality guarantee.
|
||||
|
||||
Request-local prefix KV reuse and new-request graph replay have CUDA regression
|
||||
tests. The deployment picker marks only its exact tested HTTP combinations as
|
||||
verified, including H200, B200, RTX PRO 6000 96GB, RTX 5090, and RTX 4090.
|
||||
CLI-only combinations remain Unverified in the picker.
|
||||
|
||||
On `1eab5de5990`, one RTX PRO 6000 Blackwell Server Edition passed repeated
|
||||
1024px/40-step HTTP generation, editing, transparent generation, and transparent
|
||||
input editing with resident weights or DiT layerwise offload and Torch SDPA.
|
||||
Repeated requests and corresponding outputs across placements matched RGBA
|
||||
pixels; transparent outputs retained alpha from 0 to 255. Its SM120
|
||||
RoPE, normalization, fusion-gate, and model regressions also passed. This check
|
||||
does not establish multi-GPU or quantized-checkpoint coverage on that platform.
|
||||
|
||||
Disaggregated encoder, denoiser, and decoder roles passed full-checkpoint HTTP
|
||||
generation, variable-length prompts, image editing, and CFG on three B200 GPUs,
|
||||
using Mooncake's same-host TCP transport. At 512px/4 steps, their RGBA pixels
|
||||
matched the monolithic server exactly. Multi-host RDMA and multi-rank roles
|
||||
are not covered by that check.
|
||||
|
||||
Online FP8 for the DiT, encoder, and both together passed 1024px/40-step HTTP
|
||||
generation and editing on one B200; both together also passed transparent PNG
|
||||
generation. Serialized E4M3FN FP8 component folders with scalar weight scales
|
||||
and dynamic activations passed generation, editing, and transparent output for
|
||||
each component and both together. The combined export also passed separate
|
||||
checks with TP2 plus encoder folding and with single-GPU all-component
|
||||
layerwise offload. The 476 loaded quantized matrices and
|
||||
scales matched their checkpoint values. Quantization and TP change numerical
|
||||
results. Revision `f1f3366c7c` fixes vision rotary initialization rounding during
|
||||
CPU/GPU transfer. In 1024px/40-step B200 checks, all-component layerwise offload
|
||||
matched resident generation, editing, and transparent RGBA pixels exactly
|
||||
for both native precision and the combined serialized export. Revision
|
||||
`81c8c550fa` additionally preserves FP8 weights and FP32 buffers during
|
||||
whole-encoder CPU offload; separate checks of that mode also matched resident
|
||||
pixels. Resident output was unchanged by these fixes. See the cookbook for
|
||||
sample quality metrics.
|
||||
|
||||
Native-name Q4_0 GGUF files passed 1024px/40-step B200 HTTP generation,
|
||||
editing, and transparent output for DiT-only, encoder-only, and both together.
|
||||
The 476 packed matrices matched their exported bytes. The combined export also
|
||||
passed TP2 with encoder folding; separate all-component layerwise and
|
||||
whole-encoder CPU offload checks matched resident RGBA pixels exactly.
|
||||
Quantization and TP change output pixels, including alpha. These checks cover
|
||||
the documented private exports, not arbitrary community checkpoints. Other
|
||||
GGUF types and hardware remain unverified for this model.
|
||||
|
||||
Calibrated ModelOpt-format NVFP4 component directories also passed the same
|
||||
six-configuration B200 HTTP matrix: each component, both together, two offload
|
||||
modes, and TP2 with encoder folding. All 476 packed matrices and their scales
|
||||
matched the exports after runtime layout transforms. Both offload modes
|
||||
matched resident RGBA pixels exactly. NVFP4 and TP2 change image and alpha
|
||||
values; this private export uses max calibration on only six requests, without
|
||||
SVDQuant or AWQ. See the cookbook for measured errors. Native FP4 requires
|
||||
Blackwell, and this model's NVFP4 exports remain unverified on RTX PRO 6000 and
|
||||
RTX 5090.
|
||||
|
||||
Synthetic Diffusers-format LoRA passed dynamic loading, merging,
|
||||
and removal on one B200 and TP2 with encoder folding; both removal paths
|
||||
restored the base image exactly. Trained-adapter quality remains unverified.
|
||||
Platform latency measurements
|
||||
and their exact workloads are in the cookbook. Breakable graphs
|
||||
replay only matching warmup signatures, including condition-prefix length;
|
||||
other shapes run eagerly. VAE tiling is opt-in and can change numerical results.
|
||||
Do not infer compatibility from the older Qwen-Image row.
|
||||
|
||||
Use an authorized local checkpoint with `--model-id Qwen-Image-2.1`.
|
||||
See the [Qwen-Image 2.1 cookbook](/cookbook/diffusion/Qwen-Image/Qwen-Image-2.1)
|
||||
for checkpoint layout and usage. This entry does not assert public weight
|
||||
availability. The standard two-GPU E2E suite includes `qwen_image21_t2i_tp2`
|
||||
with TP2, 1024px/40-step generation, two requests, and image consistency checks.
|
||||
Its runners need access to `Qwen/Qwen-Image-2.1`.
|
||||
|
||||
The additional opt-in HTTP cases check repeated generation, editing, and real
|
||||
RGBA alpha output from a local checkpoint:
|
||||
|
||||
```bash
|
||||
export SGLANG_QWEN_IMAGE21_TEST_MODEL=/models/Qwen-Image-2.1
|
||||
export SGLANG_QWEN_IMAGE21_TEST_IMAGE=/path/to/reference.png
|
||||
python -m pytest -q python/sglang/multimodal_gen/test/server/test_server_qwen_image21.py
|
||||
```
|
||||
|
||||
The module skips without the model variable; only editing skips when the image
|
||||
variable is absent. Credentials stay in the runner environment.
|
||||
|
||||
## Supported model inventory
|
||||
|
||||
Use a listed checkpoint as `--model-path` with `sglang generate` or
|
||||
`sglang serve`. This registry-backed list contains known public entry points;
|
||||
family detection may also support compatible local directories. Open the linked
|
||||
|
||||
@@ -774,6 +774,14 @@ sglang generate \
|
||||
--save-output
|
||||
```
|
||||
|
||||
Qwen-Image 2.1 also supports independent NVFP4 DiT and native language-encoder
|
||||
component directories through `--component-paths.transformer` and
|
||||
`--component-paths.text_encoder`. Private max-calibrated exports passed B200
|
||||
generation, editing, transparent RGBA, offload, and TP2 checks. These checks
|
||||
do not cover arbitrary exports or RTX 5090. See its
|
||||
[cookbook](/cookbook/diffusion/Qwen-Image/Qwen-Image-2.1#nvfp4-components) for
|
||||
calibration scope and image/alpha error measurements.
|
||||
|
||||
### Notes
|
||||
|
||||
- Use `--transformer-path` for mixed ModelOpt NVFP4 transformer repos or local
|
||||
@@ -784,8 +792,9 @@ sglang generate \
|
||||
primary `--transformer-path` override targets only `transformer`. Use a
|
||||
per-component override such as `--transformer-2-path` only when you
|
||||
intentionally want a non-default `transformer_2`.
|
||||
- On Blackwell, the diffusion ModelOpt NVFP4 path defaults to FlashInfer
|
||||
TensorRT-LLM FP4 GEMM (`flashinfer_trtllm`).
|
||||
- On B200, the diffusion ModelOpt NVFP4 path defaults to FlashInfer
|
||||
TensorRT-LLM FP4 GEMM (`flashinfer_trtllm`). On SM120, including RTX 5090,
|
||||
the default is `auto`; TensorRT-LLM FP4 GEMM does not support that architecture.
|
||||
- The published Qwen Image NVFP4 exports keep the `img_mod`/`txt_mod`
|
||||
modulation projections and first/last transformer blocks in BF16.
|
||||
- Qwen-Image NVFP4 does not always improve latency at 1024x1024. On B200, the
|
||||
@@ -916,13 +925,21 @@ startup rejects a degree that cuts a row-parallel matrix inside a GGML block.
|
||||
| MiniMax-H3 `fl2va`, pruned AdaLN curve | [`unsloth/MiniMax-H3-GGUF`](https://huggingface.co/unsloth/MiniMax-H3-GGUF) `minimax_h3_fl2va_pruned-Q4_K.gguf` (10.7 GiB loaded DiT) | 1x H200 (141 GiB) | 2-step t2va 1344x768, 107 frames, H.264 + AAC |
|
||||
| MiniMax-H3 `fl2va`, pruned AdaLN curve | `minimax_h3_fl2va_pruned-Q4_K.gguf` | 1x GB300 (CUDA 13, PyTorch 2.13) | 50-step t2va 1344x768, 107 frames, H.264 + AAC; 105.38 s and 80.88 GB peak |
|
||||
| MiniMax-H3 `fl2va`, pruned AdaLN curve | `minimax_h3_fl2va_pruned-Q4_K.gguf` | 2x GB300, TP2 (CUDA 13, PyTorch 2.13) | 2-step t2va 1344x768, 107 frames, H.264 + AAC; 7.55 s and 51.90 GB peak per rank |
|
||||
| Qwen-Image 2.1 | Private native-name Q4_0 exports; DiT 3.91 GiB, encoder 7.03 GiB | 1x B200; separate TP2 check | 1024px/40-step generation, editing, and transparent RGBA; each component and both together; combined offload matched resident pixels |
|
||||
|
||||
The DiT loads at 17.5 GiB against 61.7 GiB for the BF16 checkpoint. Weight
|
||||
The unpruned MiniMax-H3 DiT loads at 17.5 GiB against 61.7 GiB for the BF16 checkpoint. Weight
|
||||
fidelity was checked tensor-by-tensor against the BF16 reference: cosine
|
||||
1.00000 for the F32/BF16 tensors and 0.9973 for Q4_K/Q4_0.
|
||||
|
||||
Not validated in the measurements above: any other quantization type, the
|
||||
`ref2va` partition, and a BF16-vs-GGUF output comparison.
|
||||
The MiniMax-H3 measurements do not validate other quantization types, the
|
||||
`ref2va` partition, or a BF16-vs-GGUF output comparison.
|
||||
|
||||
Qwen-Image 2.1's Q4_0 exports use the base component configs and native tensor
|
||||
names. Its [cookbook](/cookbook/diffusion/Qwen-Image/Qwen-Image-2.1#gguf-components)
|
||||
documents the tested matrix selection, offload modes, TP2, and image/alpha
|
||||
differences against native precision. These private exports establish loading
|
||||
and execution compatibility, not general output quality or compatibility with
|
||||
other GGUF exports.
|
||||
|
||||
## Nunchaku (SVDQuant)
|
||||
|
||||
|
||||
@@ -1277,6 +1277,7 @@ export const Deployment = ({ config, benchmarks }) => {
|
||||
};
|
||||
|
||||
const [sel, setSel] = useState(() => initialSelectionFromCells());
|
||||
const [selectionHydrated, setSelectionHydrated] = useState(false);
|
||||
const INTERNAL_HASH_STATE_KEY = "__sglangDeployInternalHash";
|
||||
const DEPLOYMENT_COMPONENT_ID = "deployment-configurator";
|
||||
useEffect(() => {
|
||||
@@ -1310,12 +1311,14 @@ export const Deployment = ({ config, benchmarks }) => {
|
||||
if (el) el.scrollIntoView({ behavior: "smooth", block: "start" });
|
||||
};
|
||||
hydrate();
|
||||
setSelectionHydrated(true);
|
||||
window.addEventListener("hashchange", hydrate);
|
||||
return () => window.removeEventListener("hashchange", hydrate);
|
||||
}, []);
|
||||
// history.replaceState does NOT fire hashchange — dispatch a custom event so
|
||||
// the Playground hears chip-click selection changes.
|
||||
useEffect(() => {
|
||||
if (!selectionHydrated) return;
|
||||
const target = "#" + new URLSearchParams(sel).toString();
|
||||
if (window.location.hash !== target) {
|
||||
const historyState =
|
||||
@@ -1329,7 +1332,7 @@ export const Deployment = ({ config, benchmarks }) => {
|
||||
);
|
||||
}
|
||||
window.dispatchEvent(new CustomEvent("sglang-deploy-sel", { detail: sel }));
|
||||
}, [sel]);
|
||||
}, [sel, selectionHydrated]);
|
||||
|
||||
const [modal, setModal] = useState(null); // 'curl' | 'env' | 'bench' | null
|
||||
useEffect(() => {
|
||||
|
||||
@@ -0,0 +1,386 @@
|
||||
export const config = (() => {
|
||||
const sm120Hardware = ["rtx5090", "rtxpro6000"];
|
||||
const platformAttention = (s) => sm120Hardware.includes(s.hw) ? "sdpa" : "fa";
|
||||
const effectiveAttention = (s) => s.attention === "platform" || (sm120Hardware.includes(s.hw) && s.attention === "fa") ? platformAttention(s) : s.attention;
|
||||
|
||||
const config = {
|
||||
modelName: "Qwen-Image 2.1",
|
||||
supportedHardware: ["h200", "b200", "rtxpro6000", "rtx5090", "rtx4090"],
|
||||
hardware: [
|
||||
{ id: "rtxpro6000", label: "RTX PRO 6000", vram: "96GB", vendor: "consumer" },
|
||||
{ id: "rtx5090", label: "RTX 5090", vram: "32GB", vendor: "consumer" },
|
||||
{ id: "rtx4090", label: "RTX 4090", vram: "24GB", vendor: "consumer" },
|
||||
],
|
||||
groupHardware: false,
|
||||
matchDims: [],
|
||||
|
||||
overlayDims: [
|
||||
{
|
||||
id: "weights",
|
||||
title: "Checkpoint weights",
|
||||
scope: "base",
|
||||
description: "One checkpoint serves generation and editing. Set its authorized local path under Variables.",
|
||||
default: "default",
|
||||
options: [{ id: "default", label: "Qwen-Image 2.1", flags: [] }],
|
||||
},
|
||||
{
|
||||
id: "mode",
|
||||
title: "Request mode",
|
||||
scope: "base",
|
||||
description: "Switch between JSON generation and PNG uploads to the image-edit endpoint.",
|
||||
default: "text",
|
||||
options: [
|
||||
{ id: "text", label: "Text to image" },
|
||||
{ id: "edit", label: "Image edit", description: "Upload one reference PNG, preserving its alpha channel." },
|
||||
{ id: "multi", label: "Multi-image edit", description: "Upload two ordered references; Picture 1 and Picture 2 follow this order." },
|
||||
],
|
||||
},
|
||||
{
|
||||
id: "placement",
|
||||
title: "Placement",
|
||||
scope: "serve",
|
||||
description: "Hardware selection applies its recommended placement. Stream DiT layers when the full pipeline exceeds device memory.",
|
||||
learnMore: "#5-runtime-features",
|
||||
default: "resident",
|
||||
options: [
|
||||
{
|
||||
id: "resident", label: "Resident",
|
||||
recommendedWhen: (s) => ["h200", "b200", "rtxpro6000"].includes(s.hw),
|
||||
disabled: (s) => ["rtx5090", "rtx4090"].includes(s.hw) && Number(s.gpus_per_node) === 1,
|
||||
disableReason: "The full resident pipeline exceeds one consumer GPU's memory. Select CPU offload.",
|
||||
flags: (s) => [Number(s.gpus_per_node) === 1 ? "--performance-mode speed" : "--performance-mode manual"],
|
||||
description: "Keep all components on the GPU. Recommended for H200, B200, and RTX PRO 6000 96GB. RTX 5090 and RTX 4090 need offload.",
|
||||
},
|
||||
{
|
||||
id: "offload", label: "CPU offload",
|
||||
flags: (s) => ["--performance-mode manual", "--dit-layerwise-offload true", ...(s.hw === "rtx4090" ? ["--text-encoder-cpu-offload true"] : [])],
|
||||
recommendedWhen: (s) => ["rtx5090", "rtx4090"].includes(s.hw),
|
||||
soft: (s) => !["rtxpro6000", "rtx5090", "rtx4090"].includes(s.hw) || Number(s.gpus_per_node) !== 1,
|
||||
softReason: "This offload topology has not completed an HTTP verification run.",
|
||||
description: "Streams DiT layers. RTX 4090 also offloads the encoder between requests to leave room for image editing. Requires sufficient host RAM.",
|
||||
},
|
||||
{
|
||||
id: "all_offload", label: "All components layerwise",
|
||||
flags: ["--performance-mode manual", "--layerwise-offload-components all"],
|
||||
soft: true, softReason: "Full-checkpoint 512px editing passed on B200, including TP2 with spatial VAE decode; this HTTP recipe is unverified.",
|
||||
description: "Streams repeated blocks in the DiT, Qwen3-VL, and VAE. Uses more host-device transfers to reduce device memory.",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
id: "attention",
|
||||
title: "Attention",
|
||||
scope: "serve",
|
||||
description: "Choose the target-image attention kernel. Text attention retains its causal mask.",
|
||||
learnMore: "#5-runtime-features",
|
||||
default: "platform",
|
||||
options: [
|
||||
{
|
||||
id: "platform", label: "Automatic", recommended: true,
|
||||
flags: (s) => [`--attention-backend ${platformAttention(s) === "sdpa" ? "torch_sdpa" : "fa"}`],
|
||||
description: "Uses SDPA on RTX PRO 6000 and RTX 5090, and FlashAttention on the other listed GPUs.",
|
||||
},
|
||||
{ id: "fa", label: "FlashAttention", flags: ["--attention-backend fa"], description: "Exact attention with a fused kernel. This runtime falls back to Torch SDPA on RTX PRO 6000 and RTX 5090." },
|
||||
{
|
||||
id: "sdpa", label: "Torch SDPA", flags: ["--attention-backend torch_sdpa"],
|
||||
soft: (s) => !config.commandBuilder.resource.verifiedRecipes.some((r) => r.hw === s.hw && r.placement === s.placement && r.attentions.includes("sdpa") && Number(s.gpus_per_node) === r.gpus_per_node),
|
||||
softReason: "This hardware and placement combination has not completed HTTP verification with SDPA.",
|
||||
description: "Use for reference comparisons. Floating-point reduction order can differ from FlashAttention.",
|
||||
},
|
||||
{
|
||||
id: "sage", label: "SageAttention", flags: ["--attention-backend sage_attn"],
|
||||
soft: true, softReason: "CLI smoke test passed; image and alpha quality need workload-specific validation.",
|
||||
description: "Approximate attention; requires the SageAttention dependency.",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
id: "precision",
|
||||
title: "Precision",
|
||||
scope: "serve",
|
||||
description: "Native precision is the default. Quantization changes image and alpha values. Set compatible FP8/NVFP4 directories or GGUF files under Variables.",
|
||||
default: "native",
|
||||
options: [
|
||||
{ id: "native", label: "Native BF16 / FP32", recommended: true },
|
||||
{
|
||||
id: "fp8_dit", label: "Online FP8 DiT", flags: ["--component-quantizations.transformer fp8"],
|
||||
soft: true, softReason: "Online FP8 passed 1024px/40-step generation and editing on one resident B200. Other hardware, alpha, and feature combinations remain unverified.",
|
||||
},
|
||||
{
|
||||
id: "fp8_encoder", label: "Online FP8 encoder", flags: ["--component-quantizations.text_encoder fp8"],
|
||||
soft: true, softReason: "Online encoder FP8 passed 1024px/40-step generation and editing on one resident B200. It changes conditioning and output pixels.",
|
||||
},
|
||||
{
|
||||
id: "fp8_both", label: "Online FP8 DiT + encoder", flags: ["--component-quantizations.transformer fp8", "--component-quantizations.text_encoder fp8"],
|
||||
soft: true, softReason: "Online FP8 for both components passed generation, editing, and transparent output on one resident B200. Quality depends on the workload.",
|
||||
},
|
||||
{
|
||||
id: "serialized_fp8_dit", label: "Serialized FP8 DiT", flags: ['--component-paths.transformer "{{FP8_DIT_PATH}}"'],
|
||||
soft: true, softReason: "A tensorwise E4M3FN component export passed 1024px/40-step generation, editing, and transparent output on B200. Validate your exported checkpoint's quality.",
|
||||
},
|
||||
{
|
||||
id: "serialized_fp8_encoder", label: "Serialized FP8 encoder", flags: ['--component-paths.text_encoder "{{FP8_ENCODER_PATH}}"'],
|
||||
soft: true, softReason: "A tensorwise E4M3FN language encoder export passed generation, editing, and transparent output on B200; vision weights retain native precision.",
|
||||
},
|
||||
{
|
||||
id: "serialized_fp8_both", label: "Serialized FP8 DiT + encoder", flags: ['--component-paths.transformer "{{FP8_DIT_PATH}}"', '--component-paths.text_encoder "{{FP8_ENCODER_PATH}}"'],
|
||||
soft: true, softReason: "Exported components passed 1024px/40-step generation, editing, and transparent output on B200. All-component offload matched resident pixels after the vision RoPE fix; TP2 changes numerical results. Validate your exported checkpoint's quality.",
|
||||
},
|
||||
{
|
||||
id: "gguf_dit", label: "GGUF DiT", flags: ['--component-weights-paths.transformer "{{GGUF_DIT_PATH}}"'],
|
||||
soft: true, softReason: "A Q4_0 DiT export passed 1024px/40-step generation, editing, and transparent output on B200. Other exports and hardware need validation.",
|
||||
},
|
||||
{
|
||||
id: "gguf_encoder", label: "GGUF encoder", flags: ['--component-weights-paths.text_encoder "{{GGUF_ENCODER_PATH}}"'],
|
||||
soft: true, softReason: "A native-name Q4_0 language encoder export passed generation, editing, and transparent output on B200; vision weights retain native precision.",
|
||||
},
|
||||
{
|
||||
id: "gguf_both", label: "GGUF DiT + encoder", flags: ['--component-weights-paths.transformer "{{GGUF_DIT_PATH}}"', '--component-weights-paths.text_encoder "{{GGUF_ENCODER_PATH}}"'],
|
||||
soft: true, softReason: "Combined Q4_0 exports passed 1024px/40-step generation, editing, and transparent output on B200. GGUF reduces weight memory; output quality and speed depend on the export and workload.",
|
||||
},
|
||||
{
|
||||
id: "nvfp4_dit", label: "NVFP4 DiT", flags: ['--component-paths.transformer "{{NVFP4_DIT_PATH}}"'],
|
||||
disabled: (s) => !["b200", "rtxpro6000", "rtx5090"].includes(s.hw),
|
||||
disableReason: "Native NVFP4 requires a Blackwell GPU (compute capability 10.0 or newer).",
|
||||
soft: true, softReason: "A calibrated ModelOpt-format DiT export passed 1024px/40-step generation, editing, and transparent output on B200. Other exports, RTX PRO 6000, and RTX 5090 need validation.",
|
||||
},
|
||||
{
|
||||
id: "nvfp4_encoder", label: "NVFP4 encoder", flags: ['--component-paths.text_encoder "{{NVFP4_ENCODER_PATH}}"'],
|
||||
disabled: (s) => !["b200", "rtxpro6000", "rtx5090"].includes(s.hw),
|
||||
disableReason: "Native NVFP4 requires a Blackwell GPU (compute capability 10.0 or newer).",
|
||||
soft: true, softReason: "A calibrated language-encoder export passed generation, editing, and transparent output on B200; vision weights retain native precision. Output quality requires validation.",
|
||||
},
|
||||
{
|
||||
id: "nvfp4_both", label: "NVFP4 DiT + encoder", flags: ['--component-paths.transformer "{{NVFP4_DIT_PATH}}"', '--component-paths.text_encoder "{{NVFP4_ENCODER_PATH}}"'],
|
||||
disabled: (s) => !["b200", "rtxpro6000", "rtx5090"].includes(s.hw),
|
||||
disableReason: "Native NVFP4 requires a Blackwell GPU (compute capability 10.0 or newer).",
|
||||
soft: true, softReason: "Combined exports passed generation, editing, transparent output, offload, and TP2 on B200. The small max-calibration sample changes image and alpha values; validate your exported checkpoint.",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
id: "encoder",
|
||||
title: "Encoder",
|
||||
scope: "serve",
|
||||
description: "Schedule Qwen3-VL independently of target-image attention.",
|
||||
learnMore: "#5-runtime-features",
|
||||
default: "auto",
|
||||
options: [
|
||||
{ id: "auto", label: "Auto", flags: ["--encoder-parallel auto"], recommended: true },
|
||||
{ id: "replicate", label: "Replicate", flags: ["--encoder-parallel replicate"], soft: true, softReason: "Explicit replication has not been verified for this server recipe." },
|
||||
{ id: "fold", label: "Fold", flags: ["--encoder-parallel fold"], soft: true, softReason: "Native encoder TP and full-checkpoint TP2 × SP2 editing passed on B200. Requires node-local P2P; this HTTP recipe is unverified." },
|
||||
],
|
||||
},
|
||||
{
|
||||
id: "vae",
|
||||
title: "VAE decoding",
|
||||
scope: "serve",
|
||||
description: "Decode RGBA in full, in tiles, or with spatial work distributed across GPUs.",
|
||||
learnMore: "#5-runtime-features",
|
||||
default: "full",
|
||||
options: [
|
||||
{ id: "full", label: "Full image", recommended: true, description: "Default for generation and condition-image encoding." },
|
||||
{ id: "tiled", label: "Tiled", flags: ["--vae-tiling true"], soft: true, softReason: "Repeated 512px HTTP edits passed; other tiled workloads remain unverified.", description: "Reduces activation memory; can change pixels near tile boundaries." },
|
||||
{
|
||||
id: "parallel", label: "Parallel tiles", flags: ["--vae-tiling true", "--vae-sp true"],
|
||||
disabled: (s) => Number(s.gpus_per_node) < 2,
|
||||
disableReason: "Select two GPUs before distributing VAE tiles.",
|
||||
soft: true, softReason: "Two-H200 CLI decoding passed; this HTTP recipe is unverified.",
|
||||
},
|
||||
{
|
||||
id: "spatial", label: "Spatial shard", flags: ["--vae-config.parallel-decode-mode spatial_shard"],
|
||||
disabled: (s) => Number(s.gpus_per_node) < 2,
|
||||
disableReason: "Select at least two GPUs for spatial VAE decode.",
|
||||
soft: true, softReason: "Two-B200 full-checkpoint decoding passed with TP, CFG parallelism, and all-component offload; this HTTP recipe is unverified.",
|
||||
description: "Splits feature-map height and exchanges convolution halos. Preserves full-image attention; floating-point rounding can change pixels.",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
id: "execution",
|
||||
title: "Execution",
|
||||
scope: "serve",
|
||||
description: "Graph replay requires matching resolution and condition-prefix length.",
|
||||
learnMore: "#5-runtime-features",
|
||||
default: "eager",
|
||||
options: [
|
||||
{ id: "eager", label: "Eager", recommended: true },
|
||||
{
|
||||
id: "bcg", label: "Breakable CUDA Graph",
|
||||
flags: ["--enable-breakable-cuda-graph true", "--warmup-resolutions 512x512", "--bcg-text-buckets 64"],
|
||||
soft: true, softReason: "Only a matching 512px CLI warmup was verified. Other prompts or image prefixes can fall back to eager.",
|
||||
description: "Captures a 512px warmup. Text buckets do not pad condition KV; this is not a guaranteed replay recipe.",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
id: "background",
|
||||
title: "Background",
|
||||
scope: "request",
|
||||
description: "Both choices save PNG. Transparency is requested in the prompt, not imposed by postprocessing.",
|
||||
learnMore: "#transparent-png-output",
|
||||
default: "scene",
|
||||
options: [
|
||||
{ id: "scene", label: "Scene", recommended: true },
|
||||
{ id: "transparent", label: "Transparent / alpha", description: "Generate an isolated subject, or preserve the reference image's transparent background." },
|
||||
],
|
||||
},
|
||||
{
|
||||
id: "resolution",
|
||||
title: "Resolution",
|
||||
scope: "request",
|
||||
description: "Square output canvas; reference images keep their own aspect ratios.",
|
||||
default: "1024",
|
||||
options: [{ id: "512", label: "512 × 512" }, { id: "1024", label: "1024 × 1024", recommended: true }],
|
||||
},
|
||||
{
|
||||
id: "steps",
|
||||
title: "Denoising steps",
|
||||
scope: "request",
|
||||
description: "40 is the checkpoint default. Fewer steps trade detail for latency.",
|
||||
kind: "number", min: 1, max: 100, unit: "steps", default: 40, options: [],
|
||||
},
|
||||
{
|
||||
id: "outputs",
|
||||
title: "Outputs",
|
||||
scope: "request",
|
||||
description: "Generate independent images for the same prompt.",
|
||||
kind: "number", min: 1, max: 10, unit: "outputs per prompt", default: 1, options: [],
|
||||
},
|
||||
],
|
||||
|
||||
commandBuilder: {
|
||||
defaultSelection: {
|
||||
hw: "h200", nodes: 1, gpus_per_node: 1, topology_mode: "auto",
|
||||
tp_size: 1, ulysses_degree: 1, ring_degree: 1,
|
||||
},
|
||||
resource: {
|
||||
limits: { nodes: { min: 1, max: 1 }, gpus_per_node: { min: 1, max: 4 } },
|
||||
verifiedRecipes: [
|
||||
{ id: "h200-1-resident", hw: "h200", nodes: 1, gpus_per_node: 1, placement: "resident", tp_size: 1, ulysses_degree: 1, ring_degree: 1, encoder: "auto", attentions: ["fa"], default: true },
|
||||
{ id: "b200-1-resident", hw: "b200", nodes: 1, gpus_per_node: 1, placement: "resident", tp_size: 1, ulysses_degree: 1, ring_degree: 1, encoder: "auto", attentions: ["fa", "sdpa"], default: true },
|
||||
{ id: "rtxpro6000-1-resident", hw: "rtxpro6000", nodes: 1, gpus_per_node: 1, placement: "resident", tp_size: 1, ulysses_degree: 1, ring_degree: 1, encoder: "auto", attentions: ["sdpa"], default: true },
|
||||
{ id: "rtxpro6000-1-offload", hw: "rtxpro6000", nodes: 1, gpus_per_node: 1, placement: "offload", tp_size: 1, ulysses_degree: 1, ring_degree: 1, encoder: "auto", attentions: ["sdpa"] },
|
||||
{ id: "rtx5090-1-offload", hw: "rtx5090", nodes: 1, gpus_per_node: 1, placement: "offload", tp_size: 1, ulysses_degree: 1, ring_degree: 1, encoder: "auto", attentions: ["sdpa"], default: true },
|
||||
{ id: "rtx4090-1-offload", hw: "rtx4090", nodes: 1, gpus_per_node: 1, placement: "offload", tp_size: 1, ulysses_degree: 1, ring_degree: 1, encoder: "auto", attentions: ["fa"], default: true },
|
||||
],
|
||||
autoTopology: (s) => ({ tp_size: 1, ulysses_degree: Number(s.gpus_per_node), ring_degree: 1 }),
|
||||
validateTopology: (s, topology) => {
|
||||
const errors = [];
|
||||
const nodes = Number(s.nodes);
|
||||
const perNode = Number(s.gpus_per_node);
|
||||
const { tp_size: tp, ulysses_degree: ulysses, ring_degree: ring } = topology;
|
||||
if (nodes !== 1) errors.push("This picker covers single-node deployment only.");
|
||||
if (![1, 2, 4].includes(perNode)) errors.push("Select one, two, or four GPUs per node.");
|
||||
if (![tp, ulysses, ring].every((n) => [1, 2, 4].includes(n))) errors.push("TP, Ulysses and Ring must each be 1, 2, or 4.");
|
||||
if (nodes * perNode !== tp * ulysses * ring) errors.push(`World size ${nodes * perNode} must equal TP × Ulysses × Ring (${tp * ulysses * ring}).`);
|
||||
if (32 % (tp * ulysses) !== 0) errors.push("32 attention heads must be divisible by TP × Ulysses.");
|
||||
if (ring > 1 && effectiveAttention(s) === "sdpa") errors.push("Ring requires FlashAttention or SageAttention; Torch SDPA is unsupported.");
|
||||
if (s.precision?.startsWith("nvfp4_") && !["b200", "rtxpro6000", "rtx5090"].includes(s.hw)) errors.push("Native NVFP4 requires a Blackwell GPU. Select B200, RTX PRO 6000, or RTX 5090.");
|
||||
if (perNode === 1 && ["rtx5090", "rtx4090"].includes(s.hw) && s.placement === "resident") errors.push("The full resident pipeline exceeds this GPU's memory. Select CPU offload.");
|
||||
return errors;
|
||||
},
|
||||
},
|
||||
resolveDeployment: (s) => {
|
||||
const resource = config.commandBuilder.resource;
|
||||
const topology = s.topology_mode === "manual"
|
||||
? { tp_size: Number(s.tp_size), ulysses_degree: Number(s.ulysses_degree), ring_degree: Number(s.ring_degree) }
|
||||
: resource.autoTopology(s);
|
||||
const errors = resource.validateTopology(s, topology);
|
||||
const recipe = resource.verifiedRecipes.find((entry) => entry.hw === s.hw
|
||||
&& entry.nodes === Number(s.nodes) && entry.gpus_per_node === Number(s.gpus_per_node)
|
||||
&& entry.placement === s.placement && entry.tp_size === topology.tp_size
|
||||
&& entry.ulysses_degree === topology.ulysses_degree && entry.ring_degree === topology.ring_degree);
|
||||
const serveVerified = !!recipe && errors.length === 0 && s.encoder === "auto"
|
||||
&& recipe.attentions.includes(effectiveAttention(s)) && s.precision === "native"
|
||||
&& s.execution === "eager" && s.vae === "full";
|
||||
// Exact HTTP workloads from the validation matrix, not blanket quality coverage.
|
||||
const requestVerified = serveVerified
|
||||
&& ((["text", "edit"].includes(s.mode) && s.resolution === "1024" && Number(s.steps) === 40 && Number(s.outputs) === 1
|
||||
&& (["h200", "rtxpro6000"].includes(s.hw) || s.mode === "text" || s.background === "scene"))
|
||||
|| (s.hw === "h200" && s.background === "scene" && s.mode === "text" && s.resolution === "512" && Number(s.steps) === 4 && Number(s.outputs) === 2)
|
||||
|| (s.hw === "h200" && s.background === "scene" && s.mode === "multi" && s.resolution === "512" && Number(s.steps) === 4 && Number(s.outputs) === 1));
|
||||
const world = Number(s.nodes) * Number(s.gpus_per_node);
|
||||
const flags = ['--model-path "{{MODEL_PATH}}"', "--model-id Qwen-Image-2.1", `--num-gpus ${world}`];
|
||||
if (topology.tp_size > 1) flags.push(`--tp-size ${topology.tp_size}`);
|
||||
flags.push(`--ulysses-degree ${topology.ulysses_degree}`);
|
||||
if (topology.ring_degree > 1) flags.push(`--ring-degree ${topology.ring_degree}`);
|
||||
flags.push("--host {{HOST_IP}}", "--port {{PORT}}");
|
||||
const warnings = [];
|
||||
if (!serveVerified && !errors.length) warnings.push("This server combination has not completed an exact HTTP verification run.");
|
||||
if (!requestVerified && !errors.length) warnings.push("This request shape is outside the verified HTTP matrix.");
|
||||
return {
|
||||
match: { hw: s.hw }, nnodes: Number(s.nodes), verified: serveVerified, flags,
|
||||
builder: {
|
||||
topology,
|
||||
topologySummary: `TP ${topology.tp_size} · Ulysses ${topology.ulysses_degree} · Ring ${topology.ring_degree}`,
|
||||
errors, warnings,
|
||||
verification: {
|
||||
serve: errors.length ? "error" : serveVerified ? "verified" : "unverified",
|
||||
request: errors.length ? "error" : requestVerified ? "verified" : "unverified",
|
||||
},
|
||||
resolvedSettings: {
|
||||
attention: s.attention === "platform" ? `${platformAttention(s) === "sdpa" ? "Torch SDPA" : "FlashAttention"} (auto)`
|
||||
: sm120Hardware.includes(s.hw) && s.attention === "fa" ? "Torch SDPA (FA fallback)" : undefined,
|
||||
encoder: s.encoder === "auto" && world === 1 ? "Single GPU (auto)" : undefined,
|
||||
},
|
||||
},
|
||||
};
|
||||
},
|
||||
},
|
||||
|
||||
modelNames: { default: "Qwen-Image-2.1" },
|
||||
placeholders: {
|
||||
MODEL_PATH: { target: "command", label: "Authorized checkpoint directory", default: "/models/qwen-image-2.1" },
|
||||
FP8_DIT_PATH: { target: "command", label: "Serialized FP8 DiT directory", default: "/models/qwen-image-2.1-fp8/transformer" },
|
||||
FP8_ENCODER_PATH: { target: "command", label: "Serialized FP8 encoder directory", default: "/models/qwen-image-2.1-fp8/text_encoder" },
|
||||
GGUF_DIT_PATH: { target: "command", label: "GGUF DiT file", default: "/models/qwen-image-2.1-gguf/transformer-Q4_0.gguf" },
|
||||
GGUF_ENCODER_PATH: { target: "command", label: "GGUF encoder file", default: "/models/qwen-image-2.1-gguf/text_encoder-Q4_0.gguf" },
|
||||
NVFP4_DIT_PATH: { target: "command", label: "NVFP4 DiT directory", default: "/models/qwen-image-2.1-nvfp4/transformer" },
|
||||
NVFP4_ENCODER_PATH: { target: "command", label: "NVFP4 encoder directory", default: "/models/qwen-image-2.1-nvfp4/text_encoder" },
|
||||
HOST_IP: { target: "command", label: "Bind host", default: "0.0.0.0" },
|
||||
PORT: { target: "command", label: "Bind port", default: "30010" },
|
||||
CURL_HOST: { target: "curl", label: "Server host", default: "localhost" },
|
||||
CURL_PORT: { target: "curl", label: "Server port", default: "30010" },
|
||||
INPUT_IMAGE: { target: "curl", label: "First reference PNG (client path)", default: "/path/to/input.png" },
|
||||
SECOND_IMAGE: { target: "curl", label: "Second reference PNG (client path)", default: "/path/to/reference.png" },
|
||||
},
|
||||
curl: (s) => {
|
||||
const transparent = s.background === "transparent";
|
||||
const prompts = {
|
||||
text: transparent
|
||||
? "A single fluffy orange cat sitting, full body, isolated on a transparent background. A clean cutout with an alpha channel, transparent outside the cat, no floor, no shadow, no background."
|
||||
: "A capybara reading a book by candlelight",
|
||||
edit: transparent
|
||||
? "Change the orange fur of the cat to gray, keeping its pose, shape and fur detail unchanged. Preserve the transparent background and alpha channel. No floor, no shadow, no background."
|
||||
: "Change the red teapot to blue, keeping its shape, table, window, and lighting unchanged.",
|
||||
multi: transparent
|
||||
? "Combine the subjects from Picture 1 and Picture 2 into one composition on a transparent background. Preserve an alpha channel outside the subjects."
|
||||
: "Combine the subjects from Picture 1 and Picture 2 into one coherent scene, preserving their appearance.",
|
||||
};
|
||||
const request = {
|
||||
model: "{{MODEL_NAME}}", prompt: prompts[s.mode], n: Number(s.outputs),
|
||||
size: `${s.resolution}x${s.resolution}`, num_inference_steps: Number(s.steps),
|
||||
guidance_scale: 1, seed: 42, generator_device: "cpu",
|
||||
output_format: "png", response_format: "b64_json",
|
||||
background: transparent ? "transparent" : "auto",
|
||||
};
|
||||
if (s.mode === "text") {
|
||||
return `curl -sS --fail-with-body http://{{CURL_HOST}}:{{CURL_PORT}}/v1/images/generations \\
|
||||
-H 'Content-Type: application/json' \\
|
||||
-d '${JSON.stringify({ ...request, enable_cache_dit: false }, null, 2)}'`;
|
||||
}
|
||||
const fields = Object.entries(request).map(([key, value]) => ` --form-string '${key}=${value}'`);
|
||||
fields.push(' -F "image[]=@{{INPUT_IMAGE}};type=image/png"');
|
||||
if (s.mode === "multi") fields.push(' -F "image[]=@{{SECOND_IMAGE}};type=image/png"');
|
||||
return `curl -sS --fail-with-body http://{{CURL_HOST}}:{{CURL_PORT}}/v1/images/edits \\
|
||||
${fields.join(" \\\n")}`;
|
||||
},
|
||||
// The integration is installed from source; no published Docker image is verified.
|
||||
runModes: () => ["python"],
|
||||
showPlaygroundLink: false,
|
||||
cells: [],
|
||||
};
|
||||
|
||||
return config;
|
||||
})();
|
||||
@@ -25,6 +25,11 @@ export const DiffusionModelCatalog = ({ category }) => {
|
||||
],
|
||||
cookbook: "/cookbook/diffusion/Qwen-Image/Qwen-Image",
|
||||
},
|
||||
{
|
||||
name: "Qwen-Image 2.1",
|
||||
modelIds: ["Qwen/Qwen-Image-2.1"],
|
||||
cookbook: "/cookbook/diffusion/Qwen-Image/Qwen-Image-2.1",
|
||||
},
|
||||
{
|
||||
name: "Qwen-Image Edit / Layered",
|
||||
modelIds: [
|
||||
|
||||
Reference in New Issue
Block a user