[doc] standardize diffusion cookbook model pages (#34247)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -32,6 +32,7 @@ lacks, and replace the EXAMPLE cells with verified recipes. DeepSeek-V4 is a pop
|
||||
- [references/mintlify-authoring.md](references/mintlify-authoring.md) — MDX rules (forbidden syntax, JSX tables, labeled fences) + invocation-example patterns. Read before writing §1–§3 prose.
|
||||
- [references/engine-axis.md](references/engine-axis.md) — adding a new Playground feature axis (rare engine work).
|
||||
- [references/vendor-logo.md](references/vendor-logo.md) — new-vendor card logo: ask the user for the brand logo, then generate the icon-only 940×525 RGBA PNG (spec + Pillow recipe + `git add -f`).
|
||||
- [references/diffusion-authoring.md](references/diffusion-authoring.md) — required opening, tag, command-picker, and feature-overlay contract for diffusion model pages. Read before editing any `docs/cookbook/diffusion/<Vendor>/<Model>.mdx` page.
|
||||
|
||||
## Architecture at a glance
|
||||
|
||||
@@ -68,9 +69,12 @@ playground reads it), the **`sglang-deploy-sel` custom event** (deploy dispatche
|
||||
every change; playground listens — `replaceState` doesn't fire `hashchange`), and the
|
||||
shared **`sglang-deploy-env` localStorage key** (HOST/PORT placeholders).
|
||||
|
||||
> The template is **autoregressive**. Diffusion / omni pages follow their own category
|
||||
> structure — don't force the config-driven template on them; still obey the Mintlify /
|
||||
> NEW-tag / docs.json / category-card / validation rules below.
|
||||
> The main template is **autoregressive**. Diffusion pages use
|
||||
> `templates/diffusion-page.mdx.tmpl` plus
|
||||
> [references/diffusion-authoring.md](references/diffusion-authoring.md); do not force the
|
||||
> autoregressive deployment matrix on them. Omni pages follow their own category structure.
|
||||
> All categories still obey the Mintlify / NEW-tag / docs.json / category-card / validation
|
||||
> rules below.
|
||||
|
||||
---
|
||||
|
||||
@@ -131,6 +135,15 @@ this table (RTX PRO 6000, GH200, future chips) goes in the model's own `config.h
|
||||
|
||||
## Phase 2 — Instantiate the template
|
||||
|
||||
For a diffusion model, instantiate `templates/diffusion-page.mdx.tmpl` and keep the
|
||||
shared `DiffusionModelTags` component, plus `templates/diffusion-config.jsx.tmpl` for the
|
||||
opt-in scoped command builder. Put the compact install command and builder in §1 Quick
|
||||
start. The first two paragraphs in §2 Model capabilities are not generic filler: they must
|
||||
state the model's capability range, strongest differentiator, when to choose it, and at
|
||||
least one real deployment or capability boundary. Put orthogonal runtime features in
|
||||
`scope: "serve"` or `scope: "request"`, not in the base recipe; use the schema from
|
||||
`references/diffusion-authoring.md`.
|
||||
|
||||
1. **Copy** the three template files to their target paths (above). Note the two
|
||||
vendor-folder conventions: under `configs/` the folder is the **HuggingFace org**
|
||||
(`deepseek-ai`); under `cookbook/` it's the **display vendor** (`DeepSeek`).
|
||||
@@ -206,6 +219,10 @@ user notes: §1 Model Introduction (description, links, params, license, variant
|
||||
Tool-Calling / HiCache — keep only what applies; match the reasoning example to the
|
||||
parser's output shape; each runnable block gets an `**Output Example:**`).
|
||||
|
||||
For diffusion pages, follow the category-specific Quick start and capability contract in
|
||||
`references/diffusion-authoring.md`. Run `node docs/scripts/check_cookbook_configs.mjs` to
|
||||
verify the tag widget and introduction structure before rendering the page.
|
||||
|
||||
## Phase 6 — Review
|
||||
|
||||
```
|
||||
|
||||
@@ -0,0 +1,99 @@
|
||||
# Diffusion cookbook authoring contract
|
||||
|
||||
Use this contract for every model page under
|
||||
`docs/cookbook/diffusion/<Vendor>/<Model>.mdx`.
|
||||
|
||||
## Opening structure
|
||||
|
||||
Import and render the shared tag widget immediately after frontmatter:
|
||||
|
||||
```mdx
|
||||
import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';
|
||||
|
||||
<DiffusionModelTags tags={["video + audio", "text-to-video", "reference control", "4–15 seconds"]} />
|
||||
```
|
||||
|
||||
Use 4–6 tags, ordered from broad to specific:
|
||||
|
||||
1. output modality or product class;
|
||||
2. primary request modes;
|
||||
3. the capability that differentiates the model;
|
||||
4. an important scale, latency, duration, resolution, or architecture fact.
|
||||
|
||||
Do not spend tags on generic claims such as `native`, `fast`, `high quality`, or
|
||||
`SGLang`. Attention backends, Cache-DiT, and online quantization are feature overlays,
|
||||
not model identity, unless a published checkpoint is intrinsically tied to that format.
|
||||
|
||||
The page must start with `## 1. Quick start`: keep installation to one runnable
|
||||
command, add at most one short orientation paragraph, and render the scoped command
|
||||
builder before detailed capability tables or deployment commentary. A reader should
|
||||
reach a generated Serve command without scrolling through model background first.
|
||||
|
||||
Follow it with `## 2. Model capabilities` and 2–3 short paragraphs:
|
||||
|
||||
- paragraph 1: what the model does and where it is genuinely strong;
|
||||
- paragraph 2: when to choose it and the most important limitation or tradeoff;
|
||||
- optional paragraph/table: checkpoint or mode routing that a user must understand before serving.
|
||||
|
||||
Keep the capability lead between roughly 45 and 180 words. Replace marketing superlatives with
|
||||
concrete capabilities. A reader should learn more than “this is a powerful image/video
|
||||
model,” but should not have to read the architecture section to choose the right model.
|
||||
|
||||
## Scoped command builder
|
||||
|
||||
New diffusion pages use `templates/diffusion-config.jsx.tmpl` and opt into the shared
|
||||
`commandBuilder` renderer. Setup, Server, and Request choices share one semantic selection and
|
||||
one command composer; do not create a second command engine or assemble fragments in MDX.
|
||||
|
||||
Classify every dimension by lifecycle:
|
||||
|
||||
- `scope: "base"`: the visible Setup tab—hardware-independent required decisions such as
|
||||
checkpoint weights and request mode. Hardware, Nodes × GPUs/node, and the recommended
|
||||
verified deployment are supplied by the shared builder.
|
||||
- `scope: "serve"`: startup flags such as placement, attention, precision, encoder scheduling,
|
||||
and graph execution. They modify only the complete Serve command.
|
||||
- `scope: "request"`: sampling fields such as quality and outputs. They modify only the complete
|
||||
Request command.
|
||||
|
||||
Keep the topology registry small and honest. `verifiedRecipes` contains only exact end-to-end
|
||||
runs on that hardware and resource shape. `autoTopology(selection)` may construct a legal custom
|
||||
shape, but `resolveDeployment` must mark it `unverified` unless it exactly matches a recipe.
|
||||
`validateTopology` returns static errors for impossible world-size, head, partition, or placement
|
||||
combinations; errors disable both Copy actions. Never silently label a nearby GPU or topology as
|
||||
verified.
|
||||
|
||||
Expose topology-dependent best values explicitly. A Server row may remain `Auto`, but its summary
|
||||
and generated flag must show the effective policy—for example, encoder `auto` resolving to
|
||||
`replicate` across nodes. Reserve `disabled` for combinations that cannot work (a kernel the
|
||||
platform does not ship, a mode the checkpoint cannot serve); an option that runs but has not
|
||||
been through a verification round should declare `soft` instead, which keeps it selectable and
|
||||
labels the pick as unverified. Keep `torch.compile` and similarly narrow experiments in the
|
||||
detailed prose until they have a broadly compatible recipe.
|
||||
|
||||
Each dimension should provide a concise `description`, an optional quality label, and a
|
||||
`learnMore` anchor. Each option should provide at most two lines of decision-relevant explanation,
|
||||
its exact `flags`/`stripPrefixes`/`env`/`hints`, and a `disabled` (with `disableReason`), `soft`
|
||||
(with `softReason`), or `verifiedWhen` predicate when support is conditional — the reason strings
|
||||
are user-facing: blocked options flash theirs under the row on tap and expose it as a tooltip.
|
||||
The builder stores all semantic choices in the URL hash; active scope, expanded state, head
|
||||
address, and node rank stay local.
|
||||
|
||||
Legacy configs without `commandBuilder` continue to use the old matrix renderer. Do not migrate
|
||||
an existing page opportunistically; use the new schema for new diffusion models and deliberate
|
||||
model-by-model migrations.
|
||||
|
||||
## Review checklist
|
||||
|
||||
- tags render before section 1 and describe the model rather than the runtime;
|
||||
- Quick start puts installation and the generated commands before detailed model background;
|
||||
- the capability section explains strength and boundary without marketing filler;
|
||||
- checkpoint variants and request modes are unambiguous;
|
||||
- Setup keeps both commands visible as a deployment overview; Server and Request each show only
|
||||
the command controlled by that scope;
|
||||
- command bodies grow naturally up to their collapsed limit and do not reserve empty height;
|
||||
- topology-dependent recommended defaults are explicit in the setting summary and generated command;
|
||||
- legal custom topologies are Unverified and copyable; statically illegal combinations block Copy;
|
||||
- attention, quantization, caching, compile, and similar orthogonal features have explicit
|
||||
quality contracts and verified scopes;
|
||||
- unverified hardware or performance claims are absent;
|
||||
- `node docs/scripts/check_cookbook_configs.mjs` and Mintlify validation pass.
|
||||
@@ -0,0 +1,173 @@
|
||||
// Diffusion command-builder config template. Replace every __TOKEN__, keep only
|
||||
// verified hardware/features, and adapt the topology divisibility contract to
|
||||
// the model architecture.
|
||||
|
||||
export const config = {
|
||||
modelName: "__MODEL_DISPLAY__",
|
||||
supportedHardware: ["__DEFAULT_HW__"],
|
||||
groupHardware: false,
|
||||
matchDims: [],
|
||||
|
||||
overlayDims: [
|
||||
{
|
||||
id: "weights",
|
||||
title: "Checkpoint weights",
|
||||
scope: "base",
|
||||
description: "Choose the checkpoint partition required by this request mode.",
|
||||
default: "default",
|
||||
options: [{ id: "default", label: "Default", flags: [] }],
|
||||
},
|
||||
{
|
||||
id: "mode",
|
||||
title: "Request mode",
|
||||
scope: "base",
|
||||
default: "text",
|
||||
options: [{ id: "text", label: "Text" }],
|
||||
},
|
||||
{
|
||||
id: "placement",
|
||||
title: "Placement",
|
||||
scope: "serve",
|
||||
description: "Keep weights resident unless a verified capacity path requires sharding or offload.",
|
||||
default: "resident",
|
||||
options: [
|
||||
{ id: "resident", label: "Resident", flags: ["--performance-mode speed"], recommended: true },
|
||||
{ id: "fsdp", label: "FSDP", flags: ["--use-fsdp-inference true"], disabled: true },
|
||||
{ id: "offload", label: "Layerwise offload", disabled: true },
|
||||
],
|
||||
},
|
||||
{
|
||||
id: "attention",
|
||||
title: "Attention",
|
||||
scope: "serve",
|
||||
description: "Use the platform default unless another backend was measured end to end.",
|
||||
default: "platform",
|
||||
options: [{ id: "platform", label: "Platform default", recommended: true }],
|
||||
},
|
||||
{
|
||||
id: "precision",
|
||||
title: "Precision",
|
||||
scope: "serve",
|
||||
default: "native",
|
||||
options: [{ id: "native", label: "Native mixed precision", recommended: true }],
|
||||
},
|
||||
{
|
||||
id: "encoder",
|
||||
title: "Encoder",
|
||||
scope: "serve",
|
||||
default: "auto",
|
||||
options: [{ id: "auto", label: "Auto", flags: ["--encoder-parallel auto"], recommended: true }],
|
||||
},
|
||||
{
|
||||
id: "execution",
|
||||
title: "Execution",
|
||||
scope: "serve",
|
||||
default: "eager",
|
||||
options: [{ id: "eager", label: "Eager", recommended: true }],
|
||||
},
|
||||
{
|
||||
id: "quality",
|
||||
title: "Quality",
|
||||
scope: "request",
|
||||
default: "lossless",
|
||||
options: [{ id: "lossless", label: "Lossless", recommended: true }],
|
||||
},
|
||||
{
|
||||
id: "outputs",
|
||||
title: "Outputs",
|
||||
scope: "request",
|
||||
kind: "number",
|
||||
min: 1,
|
||||
max: 10,
|
||||
unit: "outputs per prompt",
|
||||
default: 1,
|
||||
options: [],
|
||||
},
|
||||
],
|
||||
|
||||
commandBuilder: {
|
||||
defaultSelection: {
|
||||
hw: "__DEFAULT_HW__",
|
||||
nodes: 1,
|
||||
gpus_per_node: __DEFAULT_GPUS__,
|
||||
topology_mode: "auto",
|
||||
tp_size: 1,
|
||||
ulysses_degree: __DEFAULT_GPUS__,
|
||||
ring_degree: 1,
|
||||
},
|
||||
resource: {
|
||||
limits: {
|
||||
nodes: { min: 1, max: __MAX_NODES__ },
|
||||
gpus_per_node: { min: 1, max: __MAX_GPUS_PER_NODE__ },
|
||||
},
|
||||
verifiedRecipes: [{
|
||||
id: "__DEFAULT_HW__-default",
|
||||
hw: "__DEFAULT_HW__",
|
||||
nodes: 1,
|
||||
gpus_per_node: __DEFAULT_GPUS__,
|
||||
placement: "resident",
|
||||
tp_size: 1,
|
||||
ulysses_degree: __DEFAULT_GPUS__,
|
||||
ring_degree: 1,
|
||||
encoder: "auto",
|
||||
default: true,
|
||||
}],
|
||||
autoTopology: (s) => ({
|
||||
tp_size: 1,
|
||||
ulysses_degree: Number(s.gpus_per_node),
|
||||
ring_degree: Number(s.nodes),
|
||||
}),
|
||||
validateTopology: (s, topology) => {
|
||||
const world = Number(s.nodes) * Number(s.gpus_per_node);
|
||||
const product = Number(topology.tp_size) * Number(topology.ulysses_degree) * Number(topology.ring_degree);
|
||||
return world === product ? [] : [`World size ${world} must equal TP × Ulysses × Ring (${product}).`];
|
||||
},
|
||||
},
|
||||
resolveDeployment: (s) => {
|
||||
const resource = config.commandBuilder.resource;
|
||||
const topology = s.topology_mode === "manual"
|
||||
? { tp_size: Number(s.tp_size), ulysses_degree: Number(s.ulysses_degree), ring_degree: Number(s.ring_degree) }
|
||||
: resource.autoTopology(s);
|
||||
const errors = resource.validateTopology(s, topology);
|
||||
const recipe = resource.verifiedRecipes.find((entry) => entry.hw === s.hw
|
||||
&& entry.nodes === Number(s.nodes)
|
||||
&& entry.gpus_per_node === Number(s.gpus_per_node)
|
||||
&& entry.placement === s.placement
|
||||
&& entry.tp_size === topology.tp_size
|
||||
&& entry.ulysses_degree === topology.ulysses_degree
|
||||
&& entry.ring_degree === topology.ring_degree);
|
||||
const world = Number(s.nodes) * Number(s.gpus_per_node);
|
||||
const flags = ["--model-path {{MODEL_NAME}}", `--num-gpus ${world}`];
|
||||
if (topology.tp_size > 1) flags.push(`--tp-size ${topology.tp_size}`);
|
||||
flags.push(`--ulysses-degree ${topology.ulysses_degree}`);
|
||||
if (topology.ring_degree > 1) flags.push(`--ring-degree ${topology.ring_degree}`);
|
||||
flags.push("--host {{HOST_IP}}", "--port {{PORT}}");
|
||||
const verified = !!recipe && errors.length === 0;
|
||||
return {
|
||||
match: { hw: s.hw },
|
||||
nnodes: Number(s.nodes),
|
||||
verified,
|
||||
flags,
|
||||
builder: {
|
||||
topology,
|
||||
topologySummary: `TP ${topology.tp_size} · Ulysses ${topology.ulysses_degree} · Ring ${topology.ring_degree}`,
|
||||
errors,
|
||||
warnings: verified ? [] : ["Valid custom topology; exact end-to-end verification is pending."],
|
||||
verification: { serve: verified ? "verified" : "unverified", request: verified ? "verified" : "unverified" },
|
||||
},
|
||||
};
|
||||
},
|
||||
},
|
||||
|
||||
modelNames: { default: "__MODEL_ID__" },
|
||||
placeholders: {
|
||||
HOST_IP: { target: "command", label: "Bind host", default: "0.0.0.0" },
|
||||
PORT: { target: "command", label: "Bind port", default: "30010" },
|
||||
CURL_HOST: { target: "curl", label: "Server host", default: "localhost" },
|
||||
CURL_PORT: { target: "curl", label: "Server port", default: "30010" },
|
||||
},
|
||||
curl: (s) => `curl -sS -X POST http://{{CURL_HOST}}:{{CURL_PORT}}/__REQUEST_PATH__ \\
|
||||
-H 'Content-Type: application/json' \\
|
||||
-d '${JSON.stringify({ model: "{{MODEL_NAME}}", prompt: "__PROMPT__", quality: s.quality, num_outputs_per_prompt: Number(s.outputs) }, null, 2)}'`,
|
||||
cells: [],
|
||||
};
|
||||
@@ -0,0 +1,44 @@
|
||||
---
|
||||
title: __MODEL_DISPLAY__
|
||||
description: "Deploy __MODEL_DISPLAY__ with SGLang Diffusion."
|
||||
metatags:
|
||||
description: "Serve __MODEL_DISPLAY__ for __PRIMARY_CAPABILITY__ with SGLang Diffusion."
|
||||
---
|
||||
|
||||
import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';
|
||||
import { Deployment } from '/src/snippets/_deployment.jsx';
|
||||
import { config } from '/src/snippets/configs/__HF_ORG__/__MODEL_SLUG__.jsx';
|
||||
|
||||
<DiffusionModelTags tags={["__MODALITY__", "__PRIMARY_TASK__", "__DIFFERENTIATOR__", "__IMPORTANT_BOUNDARY__"]} />
|
||||
|
||||
## 1. Quick start
|
||||
|
||||
Install with `uv pip install "sglang[diffusion]" --prerelease=allow`, then choose
|
||||
a verified recipe below.
|
||||
|
||||
<Deployment config={config} />
|
||||
|
||||
## 2. Model capabilities
|
||||
|
||||
[__MODEL_DISPLAY__](__MODEL_URL__) is __CONCRETE_MODEL_DESCRIPTION__. Its strongest use case is __MEASURED_OR_DOCUMENTED_STRENGTH__.
|
||||
|
||||
Choose it when __SELECTION_GUIDANCE__. The main tradeoff is __CAPABILITY_OR_DEPLOYMENT_BOUNDARY__.
|
||||
|
||||
| Checkpoint or mode | Best fit | Important boundary |
|
||||
| --- | --- | --- |
|
||||
| `__MODEL_ID__` | __USE_CASE__ | __LIMIT__ |
|
||||
|
||||
## 3. Deployment details
|
||||
|
||||
Document topology-dependent defaults, platform-specific installation details,
|
||||
and only the hardware recipes that need explanation beyond the builder.
|
||||
|
||||
## 4. Request examples
|
||||
|
||||
__REQUEST_EXAMPLES__
|
||||
|
||||
## 5. Feature details
|
||||
|
||||
Document quality contracts, installation requirements, and benchmark scope for
|
||||
the Server and Request choices exposed in the builder. Keep `torch.compile` and
|
||||
other experimental controls here until they have a broadly compatible recipe.
|
||||
@@ -80,7 +80,7 @@ jobs:
|
||||
with:
|
||||
node-version: "20"
|
||||
|
||||
- name: Check cookbook configs
|
||||
- name: Check cookbook authoring contracts
|
||||
run: node docs/scripts/check_cookbook_configs.mjs
|
||||
|
||||
# Hard gate: docs internal links/anchors/redirects must resolve.
|
||||
|
||||
@@ -6,11 +6,13 @@ metatags:
|
||||
|
||||
import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';
|
||||
|
||||
<DiffusionModelTags tags={["image", "video", "sound/action", "world model", "policy"]} />
|
||||
<DiffusionModelTags tags={["omnimodal", "image + video", "sound + action", "world model", "robot policy"]} />
|
||||
|
||||
## 1. Model Introduction
|
||||
|
||||
[NVIDIA Cosmos3](https://huggingface.co/collections/nvidia/cosmos3) is an omnimodal world-model family for image, video, sound, and action generation. SGLang Diffusion serves the public checkpoints with its native Cosmos3 pipeline.
|
||||
[NVIDIA Cosmos3](https://huggingface.co/collections/nvidia/cosmos3) is an omnimodal world-model family spanning text/image/video generation, optional synchronized sound, and robot action prediction. Its main advantage is breadth: the same native SGLang pipeline can serve media-generation checkpoints and the DROID policy checkpoint without routing through an LLM sampler.
|
||||
|
||||
Choose Nano for the broadest modality coverage and lower deployment cost, Super for the larger 64B image/video model, and a specialized checkpoint when only T2I or I2V is needed. Sound and action are checkpoint-specific heads, so they are not available from every Cosmos3 repository.
|
||||
|
||||
| Model | Status | Notes |
|
||||
| --- | --- | --- |
|
||||
|
||||
@@ -6,11 +6,13 @@ metatags:
|
||||
|
||||
import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';
|
||||
|
||||
<DiffusionModelTags tags={["image", "text-to-image", "turbo"]} />
|
||||
<DiffusionModelTags tags={["image", "text-to-image", "standard + Turbo", "single GPU"]} />
|
||||
|
||||
## 1. Model introduction
|
||||
## 1. Model Introduction
|
||||
|
||||
[ERNIE-Image](https://huggingface.co/baidu/ERNIE-Image) is Baidu's text-to-image diffusion model family. SGLang Diffusion supports both the regular and Turbo checkpoints with the native `ErnieImagePipeline`.
|
||||
[ERNIE-Image](https://huggingface.co/baidu/ERNIE-Image) is Baidu's text-to-image family with separate standard and Turbo checkpoints. The standard model is the quality-oriented path; Turbo is the lower-latency choice, and both load through SGLang's native `ErnieImagePipeline`.
|
||||
|
||||
This integration currently targets text-only image generation rather than image editing or reference conditioning. Choose it for a straightforward single-GPU T2I deployment; use a dedicated editing model when preserving source-image structure is part of the task.
|
||||
|
||||
| Model | Hugging Face model ID | Notes |
|
||||
| --- | --- | --- |
|
||||
|
||||
@@ -7,31 +7,18 @@ metatags:
|
||||
import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';
|
||||
import { FluxDeployment } from '/src/snippets/diffusion/flux-deployment.jsx';
|
||||
|
||||
<DiffusionModelTags tags={["image", "text-to-image", "image editing", "multi-reference"]} />
|
||||
<DiffusionModelTags tags={["image", "text-to-image", "image editing", "multi-reference", "rectified flow"]} />
|
||||
|
||||
## 1. Model Introduction
|
||||
|
||||
[FLUX](https://blackforestlabs.ai/) is a family of rectified flow transformer models developed by Black Forest Labs for high-quality image generation from text descriptions.
|
||||
[FLUX](https://blackforestlabs.ai/) is Black Forest Labs' rectified-flow image model family. [FLUX.1-dev](https://huggingface.co/black-forest-labs/FLUX.1-dev) is the smaller 12B text-to-image checkpoint; [FLUX.2-dev](https://huggingface.co/black-forest-labs/FLUX.2-dev) is a 32B model that adds instruction-based editing plus single- and multi-reference composition.
|
||||
|
||||
[FLUX.1-dev](https://huggingface.co/black-forest-labs/FLUX.1-dev) is a 12 billion parameter rectified flow transformer capable of generating images from text descriptions.
|
||||
FLUX is a strong default when prompt adherence, polished image quality, or reference consistency matters. The tradeoff is deployment weight: FLUX.2 needs substantially more memory than FLUX.1, and the dev checkpoints use the FLUX non-commercial license, so review the model license before production use.
|
||||
|
||||
**Key Features:**
|
||||
|
||||
- **Cutting-edge Output Quality**: Second only to the state-of-the-art FLUX.1 [pro] model
|
||||
- **Competitive Prompt Following**: Matches the performance of closed-source alternatives
|
||||
- **Guidance Distillation**: Trained using guidance distillation for improved efficiency
|
||||
- **Open Weights**: Available for personal, scientific, and commercial purposes under the FLUX [dev] Non-Commercial License
|
||||
|
||||
[FLUX.2-dev](https://huggingface.co/black-forest-labs/FLUX.2-dev) is a 32 billion parameter rectified flow transformer capable of generating, editing, and combining images based on text instructions.
|
||||
|
||||
**Key Features:**
|
||||
|
||||
- **State-of-the-art Performance**: Leading open model in text-to-image generation, single-reference editing, and multi-reference editing
|
||||
- **No Finetuning Required**: Character, object, and style reference without additional training in one model
|
||||
- **Guidance Distillation**: Trained using guidance distillation for improved efficiency
|
||||
- **Open Weights**: Available for personal, scientific, and commercial purposes under the FLUX [dev] Non-Commercial License
|
||||
|
||||
For more details, please refer to the [FLUX.1-dev HuggingFace page](https://huggingface.co/black-forest-labs/FLUX.1-dev), [FLUX.2-dev HuggingFace page](https://huggingface.co/black-forest-labs/FLUX.2-dev), and the [official blog post](https://blackforestlabs.ai/announcing-black-forest-labs/).
|
||||
| Checkpoint | Best fit | Main limitation |
|
||||
| --- | --- | --- |
|
||||
| `black-forest-labs/FLUX.1-dev` | High-quality text-to-image with the lighter FLUX deployment | No native multi-reference editing path |
|
||||
| `black-forest-labs/FLUX.2-dev` | Text-to-image, editing, and reference-guided composition in one model | 32B model with a larger memory footprint |
|
||||
|
||||
## 2. SGLang-diffusion Installation
|
||||
|
||||
|
||||
@@ -6,13 +6,13 @@ metatags:
|
||||
|
||||
import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';
|
||||
|
||||
<DiffusionModelTags tags={["image", "text-to-image", "typography", "NF4/FP8/NVFP4"]} />
|
||||
<DiffusionModelTags tags={["image", "text-to-image", "typography", "8/20-step distilled", "NF4 / FP8 / NVFP4"]} />
|
||||
|
||||
## 1. Model introduction
|
||||
## 1. Model Introduction
|
||||
|
||||
[Ideogram 4](https://huggingface.co/ideogram-ai/ideogram-4-nf4) is Ideogram's text-to-image diffusion model. SGLang Diffusion supports the official NF4 and FP8 checkpoints, the Comfy-Org NVFP4 transformer checkpoint, and fal's single-branch Fast and Instant variants.
|
||||
[Ideogram 4](https://huggingface.co/ideogram-ai/ideogram-4-nf4) is a text-to-image family focused on polished composition and typography-aware generation. SGLang supports the official NF4 and FP8 checkpoints, Comfy-Org's NVFP4 transformer, and fal's distilled Fast and Instant variants.
|
||||
|
||||
Compared with previous open-source image models, Ideogram 4 provides a significant aesthetic lift, with stronger composition, more polished visual style, and better typography-aware generation.
|
||||
Choose NF4 when memory is the primary constraint, FP8/NVFP4 for supported accelerator deployments, Fast for a 20-step balance, and Instant for the 8-step latency target. The public repositories are gated, and the available releases are already quantized or distilled rather than a single full-precision baseline.
|
||||
|
||||
| Variant | Hugging Face model ID | Notes |
|
||||
| --- | --- | --- |
|
||||
|
||||
@@ -7,13 +7,15 @@ metatags:
|
||||
|
||||
import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';
|
||||
|
||||
<DiffusionModelTags tags={["video", "audio-video", "multi-shot", "memory bank"]} />
|
||||
<DiffusionModelTags tags={["video + audio", "multi-shot", "minute-scale", "memory bank", "8-step"]} />
|
||||
|
||||
## 1. Model Introduction
|
||||
|
||||
[JoyAI-Echo](https://huggingface.co/jdopensource/JoyAI-Echo) (JoyEcho) is a long-form audio–video generation model built on the LTX-2 backbone. Its core idea is a **paired audio–video memory bank**: each shot commits decoded frames and audio latents into a rolling bank, and subsequent shots condition on that memory prefix. This enables **multi-shot, minute-scale generation** with visual and audio continuity across prompts.
|
||||
[JoyAI-Echo](https://huggingface.co/jdopensource/JoyAI-Echo) is an 8-step long-form audio-video model built on LTX-2. Its paired memory bank carries decoded visual context and audio latents across prompt changes, making it strongest for multi-shot, minute-scale sequences that need continuity in both picture and soundtrack.
|
||||
|
||||
Use `jdopensource/JoyAI-Echo` as `--model-path`. SGLang loads the monolithic release through the built-in [JoyAI-Echo-overlay](https://huggingface.co/Niehen6174/JoyAI-Echo-overlay) materialization path, similar to LTX-2.3-overlay.
|
||||
Choose JoyEcho over a standard LTX pipeline when shots must share audiovisual memory. Its distilled 832×480 path prioritizes long-form continuity and throughput rather than the higher-resolution two-stage quality modes offered by LTX-2.3.
|
||||
|
||||
Use `jdopensource/JoyAI-Echo` as `--model-path`; SGLang materializes the monolithic release through the built-in [JoyAI-Echo overlay](https://huggingface.co/Niehen6174/JoyAI-Echo-overlay).
|
||||
|
||||
| Aspect | Standard LTX-2.3 | JoyEcho |
|
||||
| --- | --- | --- |
|
||||
|
||||
@@ -6,24 +6,18 @@ metatags:
|
||||
|
||||
import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';
|
||||
|
||||
<DiffusionModelTags tags={["image", "text-to-image", "turbo", "raw"]} />
|
||||
<DiffusionModelTags tags={["image", "text-to-image", "8-step Turbo", "high-fidelity Raw", "Qwen3-VL"]} />
|
||||
|
||||
## 1. Model Introduction
|
||||
|
||||
[Krea-2](https://huggingface.co/krea/Krea-2-Turbo) is a high-quality text-to-image diffusion model from [Krea](https://www.krea.ai/). It ships in two variants that share the same backbone and differ only in their sampling recipe:
|
||||
[Krea-2](https://huggingface.co/krea/Krea-2-Turbo) is Krea's photorealistic text-to-image family, built as a single-stream MMDiT with a Qwen3-VL text encoder and Qwen-Image VAE. Both public variants use the same native SGLang pipeline and differ mainly in their sampling target.
|
||||
|
||||
- **[Krea-2-Turbo](https://huggingface.co/krea/Krea-2-Turbo)** - a distilled, few-step model that produces photorealistic images in only **8 inference steps** with no classifier-free guidance (`guidance_scale = 1.0`), ideal for fast and interactive generation.
|
||||
- **[Krea-2-Raw](https://huggingface.co/krea/Krea-2-Raw)** - the base (non-distilled) model that trades speed for maximum fidelity, using a longer schedule (~52 steps) with classifier-free guidance (`guidance_scale ≈ 4.5`).
|
||||
Choose Turbo for interactive generation: it is distilled to 8 steps with `guidance_scale=1.0`. Choose Raw when maximum fidelity matters more than latency: it uses roughly 52 steps with classifier-free guidance. Neither checkpoint is an image-editing model; use the Qwen-Image-Edit or FLUX.2 path when an input image must be preserved or transformed.
|
||||
|
||||
Both variants are built on a single-stream MMDiT with a Qwen3-VL text encoder and the Qwen-Image VAE, and are distributed in the standard diffusers layout (a `model_index.json` plus sharded `transformer/`, `text_encoder/`, `vae/`, `tokenizer/`, and `scheduler/` folders). SGLang loads them **natively** - just point `--model-path` at the repo, no conversion step required.
|
||||
|
||||
**Key Features:**
|
||||
|
||||
- **Two variants, one pipeline**: switch between fast (Turbo) and high-fidelity (Raw) by changing only the model path and the sampling settings.
|
||||
- **Photorealistic generation** at 1024x1024 and other resolutions.
|
||||
- **Native diffusers loading**: components (DiT, text encoder, VAE, scheduler) are read straight from the repo's `model_index.json`.
|
||||
|
||||
For more details, see the [Krea-2-Turbo](https://huggingface.co/krea/Krea-2-Turbo) and [Krea-2-Raw](https://huggingface.co/krea/Krea-2-Raw) HuggingFace pages.
|
||||
| Variant | Model ID | Sampling profile |
|
||||
| --- | --- | --- |
|
||||
| Turbo | `krea/Krea-2-Turbo` | 8 steps, no CFG; fastest path |
|
||||
| Raw | `krea/Krea-2-Raw` | About 52 steps with CFG; higher-fidelity path |
|
||||
|
||||
## 2. SGLang-diffusion Installation
|
||||
|
||||
|
||||
@@ -8,13 +8,15 @@ metatags:
|
||||
import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';
|
||||
import { LTXDeployment } from '/src/snippets/diffusion/ltx-deployment.jsx';
|
||||
|
||||
<DiffusionModelTags tags={["video", "text-to-video", "image-to-video", "two-stage"]} />
|
||||
<DiffusionModelTags tags={["video", "text-to-video", "image-to-video", "one/two-stage", "1088p HQ"]} />
|
||||
|
||||
## 1. Model Introduction
|
||||
|
||||
[LTX-2](https://huggingface.co/Lightricks/LTX-2) and [LTX-2.3](https://huggingface.co/Lightricks/LTX-2.3) are video generation models from Lightricks. SGLang Diffusion supports the LTX series through native one-stage and two-stage pipelines for text-to-video and image-conditioned video generation.
|
||||
[LTX-2](https://huggingface.co/Lightricks/LTX-2) and [LTX-2.3](https://huggingface.co/Lightricks/LTX-2.3) are Lightricks video models for text-to-video and image-conditioned generation. Their defining deployment choice is the pipeline: one-stage minimizes latency, standard two-stage refines quality, and LTX-2.3 adds a 1920×1088 HQ path.
|
||||
|
||||
Use `Lightricks/LTX-2` or `Lightricks/LTX-2.3` as `--model-path`. For two-stage generation, SGLang uses the spatial upsampler and distilled LoRA components from the model snapshot by default. LTX-2.3 also supports the HQ two-stage variant.
|
||||
Choose LTX when one model family must span fast generation, higher-quality refinement, and LoRA adaptation. Two-stage quality costs substantially more memory and startup work because both DiTs and the spatial upsampler participate; use the one-stage path when latency or capacity matters more than final detail.
|
||||
|
||||
Use `Lightricks/LTX-2` or `Lightricks/LTX-2.3` as `--model-path`. SGLang loads the two-stage upsampler and distilled components from the selected snapshot.
|
||||
|
||||
<Warning>
|
||||
**License notice:** LTX-2 and LTX-2.3 are released under the LTX-2 Community License Agreement, not Apache 2.0. The license includes commercial-use restrictions for some entities. Review the [official Lightricks license](https://huggingface.co/Lightricks/LTX-2.3/blob/main/LICENSE) before production or commercial use; SGLang support does not grant additional model usage rights.
|
||||
|
||||
@@ -8,7 +8,7 @@ metatags:
|
||||
import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';
|
||||
import { LTX25Deployment } from '/src/snippets/diffusion/ltx25-deployment.jsx';
|
||||
|
||||
<DiffusionModelTags tags={["video", "audio", "text-to-video", "image-to-video", "two-stage", "auto-duration", "diffusion decoder"]} />
|
||||
<DiffusionModelTags tags={["video", "audio", "text-to-video", "image-to-video", "two-stage", "auto-duration"]} />
|
||||
|
||||
## 1. Model Introduction
|
||||
|
||||
|
||||
@@ -7,13 +7,22 @@ metatags:
|
||||
|
||||
import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';
|
||||
|
||||
<DiffusionModelTags tags={["video", "text-to-video", "mixture-of-experts"]} />
|
||||
<DiffusionModelTags tags={["video", "text-to-video", "mixture-of-experts", "30B-A3B"]} />
|
||||
|
||||
## 1. Model introduction
|
||||
## 1. Model Introduction
|
||||
|
||||
[LingBot Video MoE 30B-A3B](https://huggingface.co/robbyant/lingbot-video-moe-30b-a3b)
|
||||
is a text-to-video mixture-of-experts model. SGLang Diffusion provides a native
|
||||
pipeline for the public checkpoint:
|
||||
is a text-to-video mixture-of-experts model served through a native SGLang
|
||||
Diffusion pipeline. Its main advantage is the MoE economics: 30B total
|
||||
parameters with 3B active per token, so a plain `sglang serve` invocation
|
||||
covers the released 480×480, 81-frame profile without parallelism flags.
|
||||
|
||||
Prompting is the distinctive part of this checkpoint: it expects a structured
|
||||
JSON caption rather than an unexpanded natural-language prompt, passed as the
|
||||
request's `prompt` string. A compact 17-frame, 12-step smoke-test profile is
|
||||
available for quick validation next to the released 40-step profile.
|
||||
|
||||
The public checkpoint served by the pipeline:
|
||||
|
||||
| Model ID | Task | Default output |
|
||||
| --- | --- | --- |
|
||||
|
||||
@@ -7,13 +7,13 @@ tag: REALTIME
|
||||
|
||||
import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';
|
||||
|
||||
<DiffusionModelTags tags={["realtime", "world model", "causal DiT", "camera control"]} />
|
||||
<DiffusionModelTags tags={["realtime", "world model", "14B causal DiT", "camera control", "interactive KV window"]} />
|
||||
|
||||
## 1. Model Introduction
|
||||
|
||||
lingbot-world-v2-14b-causal-fast-diffusers is a realtime camera-controlled video world model. In SGLang-diffusion, it belongs to the realtime causal path: the server keeps a live session, samples control signals per chunk, reuses causal DiT state, and decodes video frames incrementally.
|
||||
[LingBot World 2.0](https://huggingface.co/robbyant/lingbot-world-v2-14b-causal-fast-diffusers) is the 14B realtime camera-controlled world model in the LingBot family. It generates continuously from a live causal session, accepts control updates between chunks, and reuses DiT and VAE state for incremental frame delivery.
|
||||
|
||||
This is different from offline diffusion video models such as Wan or LTX. Offline models denoise a bounded latent sequence for one request. Realtime world models generate a continuing stream, so the runtime must manage session state, control events, causal attention cache, and VAE decode cache.
|
||||
Choose 2.0 when the larger model and interactive KV-window path justify an 8-GPU deployment. Like the earlier LingBot release, it prioritizes responsive control and bounded causal state over the full-clip global context of offline video diffusion.
|
||||
|
||||
## 2. Deployment
|
||||
|
||||
|
||||
@@ -7,13 +7,13 @@ tag: REALTIME
|
||||
|
||||
import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';
|
||||
|
||||
<DiffusionModelTags tags={["realtime", "world model", "causal DiT", "camera control"]} />
|
||||
<DiffusionModelTags tags={["realtime", "world model", "continuous video", "camera control", "causal KV cache"]} />
|
||||
|
||||
## 1. Model Introduction
|
||||
|
||||
[LingBot World](https://huggingface.co/robbyant/lingbot-world-fast-diffusers) is a realtime camera-controlled video world model. In SGLang-diffusion, it belongs to the realtime causal path: the server keeps a live session, samples control signals per chunk, reuses causal DiT state, and decodes video frames incrementally.
|
||||
[LingBot World](https://huggingface.co/robbyant/lingbot-world-fast-diffusers) is a realtime camera-controlled video world model. It keeps a live causal session, applies prompt and camera events between chunks, reuses DiT/VAE state, and streams decoded frames instead of finishing a bounded clip before returning.
|
||||
|
||||
This is different from offline diffusion video models such as Wan or LTX. Offline models denoise a bounded latent sequence for one request. Realtime world models generate a continuing stream, so the runtime must manage session state, control events, causal attention cache, and VAE decode cache.
|
||||
Choose it for interactive exploration and continuous control, not one-shot cinematic generation. The causal window makes long sessions practical but gives up the global bidirectional context available to offline Wan or LTX pipelines; session state and cache policy are therefore part of the serving contract.
|
||||
|
||||
## 2. Deployment
|
||||
|
||||
|
||||
@@ -5,13 +5,15 @@ description: "Serve LongLive 2.0 distilled text-to-video and image-to-video mode
|
||||
|
||||
import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';
|
||||
|
||||
<DiffusionModelTags tags={["video", "text-to-video", "image-to-video", "few-step", "multi-shot"]} />
|
||||
<DiffusionModelTags tags={["video", "text/image-to-video", "4-step", "multi-shot", "Wan2.2 5B"]} />
|
||||
|
||||
## 1. Model Introduction
|
||||
|
||||
[LongLive 2.0](https://nvlabs.github.io/LongLive/LongLive2/) is a distilled few-step text-to-video and image-to-video model from NVIDIA, built on Wan2.2-TI2V-5B. SGLang serves the Diffusers-format conversion for single-prompt and multi-shot video generation.
|
||||
[LongLive 2.0](https://nvlabs.github.io/LongLive/LongLive2/) is NVIDIA's 4-step text/image-to-video model distilled from Wan2.2-TI2V-5B. Its main strength is extending few-step causal generation across prompt changes, so a single request can produce multi-shot sequences without paying a full diffusion schedule for every shot.
|
||||
|
||||
For more details, check the [LongLive 2.0 paper](https://arxiv.org/abs/2605.18739) and [LongLive 2.0 GitHub](https://github.com/NVlabs/LongLive). The model weights are released under the NVIDIA Open Model License.
|
||||
Choose it for low-step long or multi-shot generation rather than maximum one-shot fidelity. The SGLang path uses a Diffusers conversion of the official weights, and scene continuity still depends on prompt-block and sink settings; validate transitions on the target storyboard.
|
||||
|
||||
The model weights use the NVIDIA Open Model License. See the [paper](https://arxiv.org/abs/2605.18739) and [GitHub repository](https://github.com/NVlabs/LongLive) for training details.
|
||||
|
||||
## 2. SGLang-diffusion Installation
|
||||
|
||||
|
||||
@@ -6,22 +6,18 @@ metatags:
|
||||
|
||||
import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';
|
||||
|
||||
<DiffusionModelTags tags={["video", "audio-video", "lip-sync", "environment audio"]} />
|
||||
<DiffusionModelTags tags={["video + audio", "joint generation", "lip-sync", "environment sound", "up to 8 seconds"]} />
|
||||
|
||||
## 1. Model Introduction
|
||||
|
||||
[MOVA](https://github.com/OpenMOSS/MOVA) (MOSS Video and Audio) is a foundation model developed by the SII-OpenMOSS Team, designed to break the "silent era" of open-source video generation. Unlike cascaded pipelines that generate sound as an afterthought, MOVA synthesizes video and audio simultaneously in a single inference pass for perfect alignment. It adopts an Asymmetric Dual-Tower Architecture, fusing pre-trained video and audio towers through a bidirectional cross-attention mechanism to maintain tight synchronization between video and audio during generation.
|
||||
[MOVA](https://github.com/OpenMOSS/MOVA) generates video and audio together with an asymmetric dual-tower model connected by bidirectional cross-attention. Its strongest use cases are speaking subjects, visible sound-producing events, and scenes where ambient audio must track the picture rather than be synthesized by a later cascade.
|
||||
|
||||
[MOVA-360p](https://huggingface.co/OpenMOSS-Team/MOVA-360p) is suitable for fast inference and resource-constrained environments. [MOVA-720p](https://huggingface.co/OpenMOSS-Team/MOVA-720p) provides higher resolution video generation. Both versions support generating up to 8 seconds of video-audio content.
|
||||
The public 360p and 720p checkpoints both generate up to 8 seconds. Choose 360p for the lighter deployment and 720p for output resolution; MOVA is less suitable when the task needs long-form continuity or the richer image/video/audio reference conditioning provided by H3.
|
||||
|
||||
**Key Features:**
|
||||
|
||||
- **Native Bimodal Generation**: Generates high-fidelity video and synchronized audio in a single inference pass, eliminating error accumulation from cascaded pipelines
|
||||
- **Precise Lip-Sync**: Achieves state-of-the-art performance in multilingual lip-synchronization (LSE-D: 7.094, LSE-C: 7.452 with Dual CFG on Verse-Bench Set3)
|
||||
- **Environment-Aware Sound Effects**: Generates corresponding environmental sound effects including physical interaction sounds, ambient sounds, and spatial/textural sound feedback
|
||||
- **Fully Open-Source**: Model weights, inference code, training pipelines, and LoRA fine-tuning scripts are all open-sourced
|
||||
|
||||
For more details, please refer to the [MOVA-360p HuggingFace page](https://huggingface.co/OpenMOSS-Team/MOVA-360p), the [MOVA-720p HuggingFace page](https://huggingface.co/OpenMOSS-Team/MOVA-720p), the [GitHub repository](https://github.com/OpenMOSS/MOVA), and the [technical report (arXiv)](https://arxiv.org/abs/2602.08794).
|
||||
| Checkpoint | Best fit | Output limit |
|
||||
| --- | --- | --- |
|
||||
| `OpenMOSS-Team/MOVA-360p` | Faster, lower-memory joint audiovisual generation | Up to 8 seconds at 360p |
|
||||
| `OpenMOSS-Team/MOVA-720p` | Higher-resolution lip-sync and environment audio | Up to 8 seconds at 720p |
|
||||
|
||||
## 2. SGLang-diffusion Installation
|
||||
|
||||
|
||||
@@ -5,9 +5,43 @@ metatags:
|
||||
description: "Serve MiniMax-H3 with SGLang Diffusion for text-to-video-and-audio, first/last-frame conditioning, video-to-video, and multimodal reference conditioning."
|
||||
---
|
||||
|
||||
## 1. Model introduction
|
||||
import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';
|
||||
import { Deployment } from "/src/snippets/_deployment.jsx";
|
||||
import { config } from "/src/snippets/configs/MiniMaxAI/minimax-h3.jsx";
|
||||
|
||||
[MiniMax-H3](https://huggingface.co/MiniMaxAI/MiniMax-H3) generates a video and a synchronized stereo audio track in one request. SGLang Diffusion provides a native pipeline for the three public task profiles, split across the released FL2VA (First-and-Last-Frame-to-Video-and-Audio) and Ref2VA (Reference-to-Video-and-Audio) checkpoint partitions:
|
||||
<DiffusionModelTags tags={["video + audio", "T2VA / FL2VA / Ref2VA", "multimodal references", "4–15 seconds", "768p"]} />
|
||||
|
||||
## 1. Quick start
|
||||
|
||||
Install with `uv pip install "sglang[diffusion]" --prerelease=allow`, then choose
|
||||
a verified recipe below. Setup changes the deployment; Server and Request expose
|
||||
orthogonal startup and sampling choices.
|
||||
|
||||
<Deployment config={config} />
|
||||
|
||||
<Note>
|
||||
The generated Server command already includes the recommended encoder policy.
|
||||
Change a Server option only for a deliberate trade-off; Request options do not
|
||||
reload the model.
|
||||
</Note>
|
||||
|
||||
The Docker form installs the platform-specific diffusion extra from the source
|
||||
bundled in the image. For conditioned requests, set **Host media directory**
|
||||
under **Variables**; the builder mounts it read-only at `/data/minimax-h3`.
|
||||
AMD currently offers the Python form, while NVIDIA also offers Docker.
|
||||
|
||||
To use ModelScope through the same normal `sglang serve` path, prefix the copied
|
||||
command with `SGLANG_USE_MODELSCOPE=true` and replace the model path with
|
||||
`MiniMax/MiniMax-H3`. Keep the selected variant and topology flags unchanged.
|
||||
|
||||
For platform-specific installation details, see the
|
||||
[SGLang Diffusion installation guide](/docs/sglang-diffusion/installation).
|
||||
|
||||
## 2. Model capabilities
|
||||
|
||||
[MiniMax-H3](https://huggingface.co/MiniMaxAI/MiniMax-H3) is a native joint video-and-audio model for text-to-video-and-audio, first/last-frame control, and multimodal reference conditioning. Its main strength is producing the picture and stereo soundtrack together, so speech, music, ambient sound, and visible events can stay aligned without a separate audio-generation pass.
|
||||
|
||||
Choose H3 when synchronized audiovisual output or reference-driven generation matters more than a lightweight deployment. The released recipe targets a 768-pixel short edge at 24 fps for 4–15 seconds, and its capabilities are split across two checkpoint partitions; serving every mode therefore requires separate FL2VA and Ref2VA deployments.
|
||||
|
||||
| Task | `task` value | Conditioning |
|
||||
| --- | --- | --- |
|
||||
@@ -29,64 +63,12 @@ mapping; do not point `--model-path` at a manually downloaded subdirectory.
|
||||
Review the license and usage terms in the MiniMax-H3 model card before production or commercial use. SGLang support does not grant additional model usage rights.
|
||||
</Warning>
|
||||
|
||||
## 2. Installation
|
||||
## 3. Deployment details
|
||||
|
||||
Install SGLang with the diffusion dependencies:
|
||||
|
||||
```bash Command
|
||||
uv pip install "sglang[diffusion]" --prerelease=allow
|
||||
```
|
||||
|
||||
For platform-specific setup, see the [SGLang Diffusion installation guide](/docs/sglang-diffusion/installation).
|
||||
|
||||
## 3. Serve MiniMax-H3
|
||||
|
||||
Use the interactive selector to choose a hardware platform, deployment profile,
|
||||
one of the two checkpoint partitions, a request mode, and deployment features.
|
||||
It generates Python and, where available, Docker launch forms. AMD selections
|
||||
use the Python form until an H3-capable ROCm image is validated. The **$ cURL**
|
||||
button follows the selected request mode and switches the payload across
|
||||
text-only, all three first/last-frame signatures, and the image/audio/video
|
||||
reference combinations listed below.
|
||||
Set **Outputs per prompt** in the picker’s **Env** panel to generate more than
|
||||
one output without mixing request sampling controls into the deployment
|
||||
matrix.
|
||||
|
||||
The Docker form does not assume the base SGLang image contains optional
|
||||
diffusion dependencies. It installs the platform-specific diffusion extra from
|
||||
the source bundled in the image before starting the server. Set **Host media
|
||||
directory** in the **Env** panel for FL2VA, V2V, or Ref2VA; the picker mounts
|
||||
that directory read-only at `/data/minimax-h3` inside the container.
|
||||
|
||||
Every hardware/topology cell in this picker has completed a real request on
|
||||
that exact GPU model. Approximate load-time features such as online
|
||||
quantization are called out separately in the generated command. Sampling
|
||||
behavior such as Cache-DiT is documented separately below.
|
||||
|
||||
**Deployment Profile** exposes resident and FSDP placement on B200, B300,
|
||||
H200, and H100. Resident is the latency-oriented default; FSDP reduces DiT
|
||||
weight residency at the cost of per-block parameter collectives. On H200 it
|
||||
also selects the verified 2-node cross-node topology. **Online
|
||||
Quantization** appears only on B200 and B300. AMD keeps its resident AITER
|
||||
recipe, while RTX 5090 uses its dedicated layerwise-offload profile. A
|
||||
single 24 GB card (RTX 4090) uses the same offload knobs plus online
|
||||
`kitchen_int8`; that recipe is documented below rather than in the picker.
|
||||
|
||||
import { Deployment } from "/src/snippets/_deployment.jsx";
|
||||
import { config } from "/src/snippets/configs/MiniMaxAI/minimax-h3.jsx";
|
||||
|
||||
<Deployment config={config} />
|
||||
|
||||
<Note>
|
||||
The ready-to-run request template lives behind the **$ cURL** button in the
|
||||
picker above. It regenerates as you change the selection, so the payload it
|
||||
shows always matches the serve command next to it.
|
||||
</Note>
|
||||
|
||||
The selector uses the verified Hugging Face ID. To use ModelScope through the
|
||||
same normal `sglang serve` path, prefix the copied command with
|
||||
`SGLANG_USE_MODELSCOPE=true` and replace the model path with
|
||||
`MiniMax/MiniMax-H3`; keep its selected variant and topology flags unchanged.
|
||||
The builder accepts legal custom GPU counts and topologies, marking them
|
||||
**Unverified** until the exact recipe has completed end-to-end validation.
|
||||
Static H3 head or partition violations disable Copy before they reach
|
||||
`sglang serve`.
|
||||
|
||||
For a four-card H200 host, keep the full BF16/FP32 model resident by default.
|
||||
The model fits without FSDP, so this path avoids the per-block parameter
|
||||
@@ -98,6 +80,7 @@ sglang serve \
|
||||
--model-variant fl2va \
|
||||
--num-gpus 4 \
|
||||
--ulysses-degree 4 \
|
||||
--encoder-parallel auto \
|
||||
--performance-mode speed \
|
||||
--port 30010
|
||||
```
|
||||
@@ -121,6 +104,7 @@ sglang serve \
|
||||
--num-gpus 4 \
|
||||
--tp-size 2 \
|
||||
--ulysses-degree 2 \
|
||||
--encoder-parallel auto \
|
||||
--performance-mode speed \
|
||||
--port 30010
|
||||
```
|
||||
@@ -136,6 +120,7 @@ sglang serve \
|
||||
--model-variant fl2va \
|
||||
--num-gpus 4 \
|
||||
--ulysses-degree 4 \
|
||||
--encoder-parallel auto \
|
||||
--performance-mode speed \
|
||||
--use-fsdp-inference true \
|
||||
--port 30010
|
||||
@@ -153,6 +138,7 @@ sglang serve \
|
||||
--num-gpus 2 \
|
||||
--tp-size 2 \
|
||||
--ulysses-degree 1 \
|
||||
--encoder-parallel auto \
|
||||
--performance-mode memory \
|
||||
--layerwise-offload-components dit,text_encoder,vae \
|
||||
--dit-offload-prefetch-size 1 \
|
||||
@@ -680,6 +666,7 @@ sglang serve \
|
||||
--sp-degree 4 \
|
||||
--ulysses-degree 4 \
|
||||
--ring-degree 1 \
|
||||
--encoder-parallel auto \
|
||||
--performance-mode speed \
|
||||
--use-fsdp-inference false \
|
||||
--enable-torch-compile false \
|
||||
@@ -764,6 +751,7 @@ sglang serve \
|
||||
--model-variant ref2va \
|
||||
--num-gpus 8 \
|
||||
--ulysses-degree 8 \
|
||||
--encoder-parallel auto \
|
||||
--performance-mode speed \
|
||||
--port 30010
|
||||
```
|
||||
@@ -777,7 +765,11 @@ task profile. A real B200 request has completed, but the `quality: "high"`
|
||||
path above remains fail-closed to the audited 4×H200 workload.
|
||||
</Warning>
|
||||
|
||||
## 7. Runtime feature recipes
|
||||
## 7. Feature contracts and advanced recipes
|
||||
|
||||
The generated command already contains the recommended topology and encoder
|
||||
setting. Use the detailed reference below only when applying an optional
|
||||
override or checking its installation, topology limits, and validation evidence.
|
||||
|
||||
<Tabs>
|
||||
|
||||
@@ -791,7 +783,6 @@ listed hardware and topology; it is not inherited by a similar GPU family.
|
||||
| --- | --- | --- |
|
||||
| Ulysses sequence parallelism | Verified: 8× B200, 4× H200, 4× H100, and Ulysses1/2/4/8 on MI300X and MI355X | Use `--ulysses-degree`. Combine with Ring for cross-node scaling; see the next row. |
|
||||
| Ring sequence parallelism (cross-node) | Verified: 2 nodes of 8× H200 each (Ulysses8 × Ring2) | Use `--ring-degree` together with `--nnodes`/`--node-rank`/`--dist-init-addr`. Ring shards the sequence across nodes while Ulysses shards heads within a node; H3's packed multi-segment attention only supports Ring across the node boundary, not within a single node's Ulysses group. Requires `--encoder-parallel replicate` — `auto`'s fold decision is not node-boundary aware. See the benchmark section below. |
|
||||
| SageAttention | Supported | Use `--attention-backend sage_attn` to select the native packed varlen path; install the SageAttention dependency first. |
|
||||
| Tensor parallelism | Verified: B200 TP2 + Ulysses4; H100 TP2 + Ulysses2 and TP4 + Ulysses1 | `--tp-size` may be combined with Ulysses when the TP-local head count remains divisible by the Ulysses degree. On 4×H100, TP2 + Ulysses2 is the measured speed default. |
|
||||
| FSDP inference | Verified: 4× B200 and 4× H100 + Ulysses4 | Preserves H3's mixed BF16/FP32 parameter policy. B200 completed the exact eager comparison; H100 completed consecutive real requests at about 57 GB peak memory per GPU. |
|
||||
| Resident components | Verified: B200, H200, 4×H100 with TP, and 1/2/4/8× MI300X and MI355X | This is the recommended single-request latency path when the complete workload fits. |
|
||||
@@ -814,6 +805,7 @@ sglang serve \
|
||||
--model-variant ref2va \
|
||||
--num-gpus 8 \
|
||||
--ulysses-degree 8 \
|
||||
--encoder-parallel auto \
|
||||
--performance-mode speed \
|
||||
--enable-breakable-cuda-graph true \
|
||||
--warmup-resolutions 1344x768 \
|
||||
@@ -827,6 +819,29 @@ different task profile, reference set, resolution, or prompt template.
|
||||
|
||||
</Tab>
|
||||
|
||||
<Tab title="Attention backends">
|
||||
|
||||
Leave `--attention-backend` unset for the platform default. Use
|
||||
`--attention-backend fa` only for an explicit FlashAttention comparison.
|
||||
|
||||
SageAttention uses quantized attention math and is not a consistency mode. To
|
||||
select H3's native packed-varlen Sage path, install the dependency and add
|
||||
`--attention-backend sage_attn`. On Hopper, install the upstream SM90 binding
|
||||
fix rather than the PyPI 2.2.0 build:
|
||||
|
||||
```bash Command
|
||||
pip install --force-reinstall \
|
||||
git+https://github.com/thu-ml/SageAttention.git@d9704247a5139ab4c03bf7fc6b35cc0e2cbb5ea4 \
|
||||
--no-build-isolation
|
||||
```
|
||||
|
||||
The backend is a server-wide default. Use
|
||||
`--component-attention-backends` only when a measured component needs a
|
||||
different kernel, and keep the platform default for every component not named
|
||||
in the override.
|
||||
|
||||
</Tab>
|
||||
|
||||
<Tab title="Online quantization">
|
||||
|
||||
On the verified 8× B200 topology, quantize the BF16 transformer at server load:
|
||||
@@ -837,6 +852,7 @@ sglang serve \
|
||||
--model-variant ref2va \
|
||||
--num-gpus 8 \
|
||||
--ulysses-degree 8 \
|
||||
--encoder-parallel auto \
|
||||
--performance-mode speed \
|
||||
--quantization fp8 \
|
||||
--port 30010
|
||||
@@ -852,6 +868,7 @@ sglang serve \
|
||||
--model-variant ref2va \
|
||||
--num-gpus 8 \
|
||||
--ulysses-degree 8 \
|
||||
--encoder-parallel auto \
|
||||
--quantization fp8 \
|
||||
--quantization-ignored-layers blocks.0.attn token_refiner \
|
||||
--port 30010
|
||||
@@ -861,8 +878,8 @@ sglang serve \
|
||||
Online FP8 is approximate and is not a consistency ground-truth mode. It can
|
||||
be combined with Cache-DiT, but the two approximations compound. Validate
|
||||
visual quality, audio quality, memory use, and latency on the target workload.
|
||||
The picker exposes this option only on the B200 and B300 topologies used for
|
||||
real H3 validation runs.
|
||||
This recipe is limited to the resident B200 and B300 topologies used for real
|
||||
H3 validation runs.
|
||||
</Warning>
|
||||
|
||||
On a single 24 GB card, use `kitchen_int8` instead of FP8. It quantizes the
|
||||
@@ -924,6 +941,33 @@ the strict `quality="high"` deployment contract.
|
||||
|
||||
</Tab>
|
||||
|
||||
<Tab title="Encoder scheduling">
|
||||
|
||||
The picker explicitly writes `--encoder-parallel auto` in every single-node
|
||||
recipe. At the default request batch size of one, H100/H200/B200/B300 servers
|
||||
with peer-to-peer access fold the Qwen encoder across otherwise idle Ulysses
|
||||
ranks. A pure-TP recipe keeps the encoder inside its TP group, while a
|
||||
PCIe-only host can avoid an expensive world fold. Keep `auto` unless one of
|
||||
the cases below applies.
|
||||
|
||||
Encoder DP is a throughput policy for compatible request batches. It requires
|
||||
TP1 and DiT DP1, replicates the encoder weights, and does not improve a batch
|
||||
of one:
|
||||
|
||||
```bash Overlay
|
||||
--encoder-parallel dp \
|
||||
--batching-max-size 2
|
||||
```
|
||||
|
||||
The cross-node picker recipe already uses replication because the automatic
|
||||
fold decision is not node-boundary aware:
|
||||
|
||||
```bash Overlay
|
||||
--encoder-parallel replicate
|
||||
```
|
||||
|
||||
</Tab>
|
||||
|
||||
</Tabs>
|
||||
|
||||
## 8. Configuration notes
|
||||
@@ -995,7 +1039,7 @@ The swept axes are `--model-variant` (`fl2va` / `ref2va`), `--quantization`
|
||||
|
||||
This is a single-request latency sweep (`batching_max_size: 1`), so encoder DP
|
||||
is intentionally excluded: it cannot distribute a batch of one. Use the
|
||||
picker’s **DP (batched throughput)** option for a multi-request throughput
|
||||
**DP for a request batch** setting above for a compatible multi-request
|
||||
deployment; the table below does not claim a measured H3 DP speedup.
|
||||
|
||||
**Driver.**
|
||||
|
||||
@@ -7,22 +7,13 @@ metatags:
|
||||
import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';
|
||||
import { QwenImageEditDeployment } from '/src/snippets/diffusion/qwen-image-edit-deployment.jsx';
|
||||
|
||||
<DiffusionModelTags tags={["image", "image editing", "text rendering", "character consistency"]} />
|
||||
<DiffusionModelTags tags={["image editing", "text replacement", "character consistency", "multi-person", "20B"]} />
|
||||
|
||||
## 1. Model Introduction
|
||||
|
||||
[Qwen-Image-Edit-2511](https://huggingface.co/Qwen/Qwen-Image-Edit-2511) is an enhanced version over Qwen-Image-Edit-2509, featuring multiple improvements—including notably better consistency. Built upon the 20B Qwen-Image model, Qwen-Image-Edit-2511 successfully extends Qwen-Image's unique text rendering capabilities to image editing tasks, enabling precise text editing.
|
||||
[Qwen-Image-Edit-2511](https://huggingface.co/Qwen/Qwen-Image-Edit-2511) is the 20B editing counterpart to Qwen-Image. It is strongest at changing text, materials, lighting, viewpoint, or composition while reducing drift in regions that were not requested to change.
|
||||
|
||||
Key Enhancements in Qwen-Image-Edit-2511:
|
||||
|
||||
- **Mitigate Image Drift**: Reduces unwanted changes in non-edited regions of the image.
|
||||
- **Improved Character Consistency**: The model can perform imaginative edits based on an input portrait while preserving the identity and visual characteristics of the subject.
|
||||
- **Multi-Person Consistency**: Enhanced consistency in multi-person group photos, enabling high-fidelity fusion of two separate person images into a coherent group shot.
|
||||
- **Integrated LoRA Capabilities**: Selected popular community-created LoRAs are integrated directly into the base model, unlocking their effects without extra tuning (e.g., lighting enhancement, viewpoint generation).
|
||||
- **Enhanced Industrial Design Generation**: Special attention to practical engineering scenarios, including batch industrial product design and material replacement for industrial components.
|
||||
- **Strengthened Geometric Reasoning**: Stronger geometric reasoning capability for generating auxiliary construction lines for design or annotation purposes.
|
||||
|
||||
For more details, please refer to the [official Qwen-Image-Edit-2511 HuggingFace page](https://huggingface.co/Qwen/Qwen-Image-Edit-2511), the [Blog](https://qwenlm.github.io/blog/qwen-image-edit-2511/), and the [Tech Report](https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-Image/Qwen_Image.pdf).
|
||||
Choose it for identity-sensitive portrait edits, multi-person composition, typography replacement, and geometry-aware design work. It is substantially heavier than small specialist editors, and consistency is improved rather than guaranteed; evaluate untouched-region drift on the actual editing workload.
|
||||
|
||||
## 2. SGLang-diffusion Installation
|
||||
|
||||
|
||||
@@ -7,13 +7,13 @@ metatags:
|
||||
import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';
|
||||
import { QwenImageDeployment } from '/src/snippets/diffusion/qwen-image-deployment.jsx';
|
||||
|
||||
<DiffusionModelTags tags={["image", "text-to-image", "text rendering", "NVFP4"]} />
|
||||
<DiffusionModelTags tags={["image", "text-to-image", "bilingual typography", "complex layouts", "NVFP4"]} />
|
||||
|
||||
## 1. Model Introduction
|
||||
|
||||
[Qwen-Image](https://huggingface.co/Qwen/Qwen-Image) is a text-to-image diffusion model developed by the Qwen team.
|
||||
[Qwen-Image](https://huggingface.co/Qwen/Qwen-Image) is a 20B text-to-image model built for strong prompt following and precise rendering of English and Chinese text. It is especially useful for posters, signs, diagrams, and dense layouts where typography and spatial relationships matter as much as general image quality.
|
||||
|
||||
For more details, please refer to the [official Qwen-Image HuggingFace page](https://huggingface.co/Qwen/Qwen-Image), the [Blog](https://qwenlm.github.io/blog/qwen-image/), and the [Tech Report](https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-Image/Qwen_Image.pdf).
|
||||
This page covers generation rather than editing: use Qwen-Image-Edit when an existing image, subject identity, or untouched region must be preserved. The full checkpoint is memory-heavy, while the validated ModelOpt NVFP4 release provides a supported low-precision deployment option with an expected quality tradeoff.
|
||||
|
||||
## 2. SGLang-diffusion Installation
|
||||
|
||||
|
||||
@@ -1,36 +1,44 @@
|
||||
---
|
||||
title: "Diffusion Cookbook"
|
||||
description: "Cookbook recipes for running diffusion models with SGLang"
|
||||
description: "Author and review deployment-focused diffusion model recipes for SGLang."
|
||||
metatags:
|
||||
description: "Explore SGLang diffusion cookbook structure, categories, and contribution guidance for image and video generation recipes."
|
||||
description: "SGLang diffusion cookbook structure, model-page standards, and contribution guidance."
|
||||
---
|
||||
|
||||
# SGLang Diffusion Cookbook
|
||||
|
||||
<div style={{display: 'flex', gap: '8px'}}>
|
||||
<a href="https://opensource.org/licenses/Apache-2.0"><img src="https://img.shields.io/badge/License-Apache_2.0-blue.svg" alt="License" /></a>
|
||||
<a href="https://github.com/sgl-project/sglang/pulls"><img src="https://img.shields.io/badge/PRs-welcome-brightgreen.svg" alt="PRs Welcome" /></a>
|
||||
</div>
|
||||
The diffusion cookbook is a set of model-specific, validated deployment guides. Each page
|
||||
should let a reader answer three questions quickly: what the model is good at, which
|
||||
checkpoint and base topology to choose, and which optional features change performance,
|
||||
memory, or quality.
|
||||
|
||||
Create a comprehensive cookbook for diffusion models in SGLang, demonstrating SGLang's performance advantages for image and video generation workloads.
|
||||
## What every model page includes
|
||||
|
||||
## 🎯 What You'll Find Here
|
||||
1. A compact capability tag row using the shared `DiffusionModelTags` component.
|
||||
2. A short introduction that states the model's capability range, strongest use case, and
|
||||
an important limitation or deployment tradeoff.
|
||||
3. Verified base launch recipes for the hardware actually tested.
|
||||
4. Matching request examples for every public task or checkpoint partition.
|
||||
5. Optional attention, quantization, caching, compile, and offload recipes with explicit
|
||||
quality contracts and validation scope.
|
||||
|
||||
This cookbook aggregates battle-tested SGLang recipes covering:
|
||||
Tags describe the model, not every runtime flag. Prefer `video + audio`, `multi-reference`,
|
||||
or `4–15 seconds` over generic labels such as `native`, `fast`, or `high quality`.
|
||||
|
||||
- **Models**: Mainstream Image and Video generation Models
|
||||
- **Use Cases**: Inference serving, deployment strategies
|
||||
- **Hardware**: GPU and CPU configurations, optimization for different accelerators
|
||||
- **Best Practices**: Configuration templates, performance tuning, troubleshooting guides
|
||||
## Keep the command picker small
|
||||
|
||||
Each recipe provides step-by-step instructions to help you quickly implement SGLang solutions for your specific requirements.
|
||||
The command picker selects a runnable base recipe: hardware topology, checkpoint partition,
|
||||
request mode, and a validated placement profile. Independent feature knobs do not need to
|
||||
become picker dimensions.
|
||||
|
||||
## 🚀 Quick Start
|
||||
Document orthogonal controls as feature overlays instead:
|
||||
|
||||
1. Browse the recipe index above to find your model
|
||||
2. Follow the step-by-step instructions in each guide
|
||||
3. Adapt configurations to your specific hardware and requirements
|
||||
4. Join our community to share feedback and improvements
|
||||
| Feature class | Where it belongs | Required context |
|
||||
| --- | --- | --- |
|
||||
| Attention backend | Optional feature overlays | Default backend, precision/approximation contract, installation, measured hardware |
|
||||
| Online or pre-quantized weights | Optional feature overlays | Weight source, protected precision-sensitive layers, quality and memory tradeoff |
|
||||
| Cache-DiT and sampling controls | Request/sampling recipes | Quality level, incompatible placement modes, task-specific validation |
|
||||
| TP, Ulysses, Ring, FSDP, residency | Base picker only for verified cells | Exact GPU count, topology, memory, and latency evidence |
|
||||
|
||||
For memory placement, prefer the unified
|
||||
[`--component-residency`](/docs/sglang-diffusion/api/cli#component-residency)
|
||||
@@ -40,62 +48,25 @@ selector. Each component resolves to exactly one of `resident`,
|
||||
`--image-encoder-cpu-offload`, and `--vae-cpu-offload` remain supported by all
|
||||
recipes that already use them.
|
||||
|
||||
The sglang diffusion cookbook directory structure are shown below:
|
||||
## Contributing a model
|
||||
|
||||
```text Example
|
||||
docs/cookbook/diffusion/
|
||||
├── README.mdx # Main cookbook (this file)
|
||||
├── Qwen-Image/ # Qwen-Image series docs
|
||||
│ ├── Qwen-Image.mdx
|
||||
│ └── Qwen-Image-Edit.mdx
|
||||
├── Wan/ # Wan series docs
|
||||
│ ├── Wan2.1.mdx
|
||||
│ └── Wan2.2.mdx
|
||||
├── Z-Image/ # Z-Image series docs
|
||||
│ └── Z-Image-Turbo.mdx
|
||||
├── Ernie-Image/ # ERNIE-Image series docs
|
||||
│ └── Ernie-Image.mdx
|
||||
└── ...
|
||||
Use the repo-local `cookbook-add-model` skill and its
|
||||
`templates/diffusion-page.mdx.tmpl` template. The full category contract lives in
|
||||
`.claude/skills/cookbook-add-model/references/diffusion-authoring.md`.
|
||||
|
||||
Before opening a PR, run:
|
||||
|
||||
```bash Command
|
||||
node docs/scripts/check_cookbook_configs.mjs
|
||||
cd docs
|
||||
mint validate
|
||||
mint broken-links --check-anchors --check-redirects
|
||||
```
|
||||
|
||||
## 🤝 Contributing
|
||||
The static cookbook check requires every diffusion model page to use the shared tag widget,
|
||||
the standard introduction heading, 4–6 tags, and a substantive two-paragraph lead. Mintlify
|
||||
validation then catches MDX, navigation, and internal-link problems.
|
||||
|
||||
We believe the best documentation comes from practitioners. Whether you've optimized SGLang for a specific model, solved a tricky deployment challenge, or discovered performance improvements, we encourage you to contribute your recipes!
|
||||
|
||||
**💪How to Contribute**
|
||||
|
||||
- Comment below if interested (mention which role)
|
||||
- Join discussion on implementation details
|
||||
- Fork repo and work on assigned section
|
||||
- Submit PR following SGLang cookbook standards
|
||||
- Iterate based on review feedback
|
||||
|
||||
**To contribute:**
|
||||
|
||||
```shell Command
|
||||
# Fork the repo and clone locally
|
||||
git clone https://github.com/YOUR_USERNAME/sglang.git
|
||||
cd sglang
|
||||
|
||||
# Create a new branch
|
||||
git checkout -b add-my-recipe
|
||||
|
||||
# Add your recipe under docs/cookbook/diffusion/
|
||||
# Submit a PR!
|
||||
```
|
||||
|
||||
## 📖 Resources
|
||||
|
||||
- [SGLang GitHub](https://github.com/sgl-project/sglang)
|
||||
- [SGLang Documentation](/)
|
||||
- [SGLang Diffusion Documentation](/docs/sglang-diffusion/index)
|
||||
- [SLACK Channel](https://sgl-fru7574.slack.com/archives/C07GLLLESNR)
|
||||
- [Community Slack/Discord](https://discord.gg/MpEEuAeb)
|
||||
|
||||
## 📄 License
|
||||
|
||||
This project is licensed under the Apache License 2.0 - see the [LICENSE](https://github.com/sgl-project/sglang/blob/main/LICENSE) file for details.
|
||||
|
||||
---
|
||||
|
||||
**Let's build this resource together!** 🚀 Star the repo and contribute your recipes to help the SGLang community grow.
|
||||
For runtime concepts and platform support, see the
|
||||
[SGLang Diffusion documentation](/docs/sglang-diffusion/index) and
|
||||
[compatibility matrix](/docs/sglang-diffusion/compatibility_matrix).
|
||||
|
||||
@@ -7,12 +7,20 @@ metatags:
|
||||
|
||||
import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';
|
||||
|
||||
<DiffusionModelTags tags={["video", "text-to-video"]} />
|
||||
<DiffusionModelTags tags={["video", "text-to-video", "480p", "2B lightweight"]} />
|
||||
|
||||
## 1. Model introduction
|
||||
## 1. Model Introduction
|
||||
|
||||
[SANA-Video 2B 480p](https://huggingface.co/Efficient-Large-Model/SANA-Video_2B_480p_diffusers)
|
||||
is a text-to-video model with a native SGLang Diffusion pipeline.
|
||||
is a lightweight text-to-video model served through a native SGLang Diffusion
|
||||
pipeline. Its main advantage is deployment cost: at 2B parameters it serves
|
||||
480p video from a single GPU with a plain `sglang serve` invocation, no
|
||||
parallelism flags required.
|
||||
|
||||
The released generation profile produces 832×480 output at 81 frames and
|
||||
16 FPS over 50 inference steps; a compact 17-frame, 8-step profile is covered
|
||||
by server CI for quick validation. Motion strength can be steered directly
|
||||
from the prompt with an optional `motion score: N.` suffix.
|
||||
|
||||
| Model ID | Task | Default output |
|
||||
| --- | --- | --- |
|
||||
|
||||
@@ -6,32 +6,19 @@ metatags:
|
||||
|
||||
import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';
|
||||
|
||||
<DiffusionModelTags tags={["video", "realtime", "world model", "camera control", "two-stage"]} />
|
||||
<DiffusionModelTags tags={["world model", "720p video", "minute-scale", "6-DoF camera control", "realtime"]} />
|
||||
|
||||
## 1. Model Introduction
|
||||
|
||||
[SANA-WM](https://huggingface.co/Efficient-Large-Model/SANA-WM_bidirectional) is an efficient open-source **world model** from NVLabs, trained natively for one-minute video generation. It is a **2.6B-parameter text+image-to-video (TI2V) diffusion transformer** that synthesizes **720p, minute-scale videos with precise 6-DoF camera control**, paired with an **LTX-2 refiner** for high-fidelity decoding. It builds on the [SANA](https://github.com/NVlabs/Sana) family — efficient high-resolution synthesis with a linear diffusion transformer.
|
||||
[SANA-WM](https://huggingface.co/Efficient-Large-Model/SANA-WM_bidirectional) is NVLabs' 2.6B text-and-image-to-video world model for 720p, minute-scale generation with explicit per-frame 6-DoF camera control. Its hybrid recurrent/softmax attention keeps long causal histories bounded, while an LTX-2 refiner supplies the detail that the fast Stage-1 world model does not produce on its own.
|
||||
|
||||
SANA-WM ships in two checkpoints: a **bidirectional** checkpoint (dense, one-shot) and a **streaming** checkpoint (chunk-causal, autoregressive — generated chunk-by-chunk, reusing causal DiT state across chunks for bounded memory → long, even endless, clips). From a single first frame, a text prompt, and a camera trajectory, this cookbook covers **all three serving modes** SGLang exposes:
|
||||
Choose the dense checkpoint for the best bounded-clip quality and the streaming checkpoint for long-running or interactive control. Streaming and realtime trade global bidirectional context for bounded state and lower response latency; the realtime WebSocket path is not bit-identical to offline batch streaming.
|
||||
|
||||
- **(A) Dense bidirectional** (§4) — the `SANA-WM_bidirectional` checkpoint generated in one shot (no chunking) via **`SanaWMTwoStagePipeline`** over the standard **`/v1/videos`** HTTP API. Highest single-clip quality (full bidirectional attention + dense LTX-2 refiner); matches the NVlabs dense reference.
|
||||
- **(B) Batch streaming** (§5) — the `SANA-WM_streaming` checkpoint generated chunk-by-chunk in one request via the same **`SanaWMTwoStagePipeline`** + `--streaming` over **`/v1/videos`**. This is SGLang's offline chunk-causal streaming path: the whole clip is produced chunk-by-chunk internally, then returned.
|
||||
- **(C) Live realtime** (§6–7) — the streaming pipeline exposed as **`SanaWMRealtimePipeline`** over a **WebSocket API** at `/v1/realtime_video/generate`, so a browser/client streams camera-action events frame-by-frame and receives video chunks back in real time. Realtime uses the same streaming checkpoint, but the incremental session path is not bit-identical to offline batch streaming.
|
||||
|
||||
All three modes share the camera action DSL (§8) and the configuration knobs (§9). Modes (B) and (C) share the streaming checkpoint and the chunk-causal pipeline.
|
||||
|
||||
**Key features** (per the official model):
|
||||
|
||||
- **Hybrid Linear Attention** — frame-wise Gated DeltaNet (GDN) recurrent blocks combined with softmax attention (every 4th layer, block indices {3,7,11,15,19}) for memory-efficient long-context modeling.
|
||||
- **Dual-Branch Camera Control** — independent main and camera branches (UCPE + PRoPE) for precise per-frame 6-DoF trajectory adherence.
|
||||
- **Two-Stage Pipeline** — an LTX-2 long-video refiner on top of Stage-1 latents for quality and temporal consistency.
|
||||
|
||||
In the **streaming / realtime** configuration this becomes a low-latency, interactive pipeline:
|
||||
|
||||
- **Stage-1 chunk-causal DiT** — the streaming path carries a **per-block KV cache** (recurrent GDN state + a softmax K/V window) across chunks; bounded memory means it scales to long / endless sequences. Stage-1 is intentionally coarse.
|
||||
- **LTX-2 streaming refiner** — refines each Stage-1 latent chunk block-by-block with a **sink + sliding-history KV cache** (required for sharp output).
|
||||
- **Causal LTX-2 VAE** — decodes latents chunk-by-chunk with a carried conv-cache for seam-free frames.
|
||||
- **Camera control** — drive the camera with a compact **WASD/IJKL** action DSL (move with WASD, look with IJKL; see §8) — supplied at request time on the `/v1/videos` paths, or pushed over the WebSocket at init / as live per-chunk events on the realtime path (see §7).
|
||||
| Serving mode | Checkpoint and API | Best fit | Tradeoff |
|
||||
| --- | --- | --- | --- |
|
||||
| Dense bidirectional | `SANA-WM_bidirectional` via `/v1/videos` | Highest single-clip quality | Bounded one-shot generation; full attention cost |
|
||||
| Batch streaming | `SANA-WM_streaming` via `/v1/videos` | Long clips returned as one job | Chunk-causal context rather than global bidirectional context |
|
||||
| Live realtime | `SANA-WM_streaming` via WebSocket | Interactive camera events and incremental output | Stateful session; not bit-identical to batch streaming |
|
||||
|
||||
**Architecture & components**
|
||||
|
||||
@@ -44,7 +31,7 @@ In the **streaming / realtime** configuration this becomes a low-latency, intera
|
||||
| Refiner | LTX-2 Stage-2 distilled; ~41 GB |
|
||||
| Output | up to 720p (704×1280) @ 16 fps, minute-scale |
|
||||
|
||||
For more details, see the [SANA-WM paper (arXiv)](https://arxiv.org/abs/2605.15178), the [SANA project page](https://nvlabs.github.io/Sana/), the [NVlabs/Sana GitHub](https://github.com/NVlabs/Sana), and the [SANA-WM_bidirectional model card](https://huggingface.co/Efficient-Large-Model/SANA-WM_bidirectional) (Apache-2.0).
|
||||
For architecture and training details, see the [SANA-WM paper](https://arxiv.org/abs/2605.15178) and [model card](https://huggingface.co/Efficient-Large-Model/SANA-WM_bidirectional) (Apache-2.0).
|
||||
|
||||
## 2. Installation
|
||||
|
||||
|
||||
@@ -7,24 +7,13 @@ metatags:
|
||||
import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';
|
||||
import { Wan21Deployment } from '/src/snippets/diffusion/wan21-deployment.jsx';
|
||||
|
||||
<DiffusionModelTags tags={["video", "text-to-video", "image-to-video", "LoRA", "text rendering"]} />
|
||||
<DiffusionModelTags tags={["video", "text-to-video", "image-to-video", "1.3B / 14B", "bilingual text"]} />
|
||||
|
||||
## 1. Model Introduction
|
||||
|
||||
[Wan2.1 series](https://github.com/Wan-Video/Wan2.1) is an open and advanced suite of large-scale video generative models from Wan-AI.
|
||||
[Wan2.1](https://github.com/Wan-Video/Wan2.1) is a broad open video family covering text-to-video and image-to-video across 1.3B and 14B checkpoints. Its practical strengths are motion-rich generation, temporal consistency, and readable Chinese/English text, with 480p and 720p variants for different quality and memory targets.
|
||||
|
||||
Key characteristics:
|
||||
|
||||
- **State-of-the-art video quality**: Consistently outperforms many open-source and commercial video models on internal and public benchmarks, especially for motion richness and temporal consistency.
|
||||
- **Consumer GPU friendly**: The T2V-1.3B variant can generate 5-second 480P videos on consumer GPUs with modest VRAM requirements.
|
||||
- **Multi-capability suite**: Supports Text-to-Video (T2V), Image-to-Video (I2V), video editing, text-to-image, and video-to-audio generation.
|
||||
- **Robust text rendering**: First-generation Wan model capable of generating both Chinese and English text in videos with strong readability.
|
||||
- **Powerful Wan-VAE**: A 3D causal VAE that encodes/decodes long 1080P videos while preserving temporal information, enabling efficient high-resolution video generation.
|
||||
|
||||
For more details, refer to the official Wan2.1 resources:
|
||||
|
||||
- **GitHub**: [Wan-Video/Wan2.1](https://github.com/Wan-Video/Wan2.1)
|
||||
- **Hugging Face collection**: [Wan-AI Wan2.1](https://huggingface.co/Wan-AI/Wan2.1-T2V-14B)
|
||||
Choose the 1.3B T2V model for consumer-GPU experiments and the 14B models when quality matters more than footprint. Wan2.1 is a dense DiT family; for timestep-specialized MoE capacity or the unified 5B TI2V path, use Wan2.2 instead.
|
||||
|
||||
## 2. SGLang-diffusion Installation
|
||||
|
||||
|
||||
@@ -7,20 +7,13 @@ metatags:
|
||||
import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';
|
||||
import { Wan22Deployment } from '/src/snippets/diffusion/wan22-deployment.jsx';
|
||||
|
||||
<DiffusionModelTags tags={["video", "text-to-video", "image-to-video", "TI2V", "MoE"]} />
|
||||
<DiffusionModelTags tags={["video", "T2V / I2V / TI2V", "MoE", "720p @ 24 fps", "cinematic motion"]} />
|
||||
|
||||
## 1. Model Introduction
|
||||
|
||||
[Wan2.2 series](https://github.com/Wan-Video/Wan2.2) are the most popular and open and advanced large-scale video generative models.
|
||||
[Wan2.2](https://github.com/Wan-Video/Wan2.2) extends the Wan family with timestep-specialized Mixture-of-Experts models and a unified 5B TI2V checkpoint. It targets cinematic composition, lighting, and complex motion while supporting text-to-video, image-to-video, and hybrid text/image conditioning.
|
||||
|
||||
This generation delivers comprehensive upgrades across the board:
|
||||
|
||||
- **Effective MoE Architecture**: Introduces a Mixture-of-Experts (MoE) architecture into video diffusion models. By separating the denoising process cross timesteps with specialized powerful expert models, this enlarges the overall model capacity while maintaining the same computational cost.
|
||||
- **Cinematic-level Aesthetics**: Incorporates meticulously curated aesthetic data, complete with detailed labels for lighting, composition, contrast, color tone, and more. This allows for more precise and controllable cinematic style generation, facilitating the creation of videos with customizable aesthetic preferences.
|
||||
- **Complex Motion Generation**: Trained on a significantly larger data, with +65.6% more images and +83.2% more videos. This expansion notably enhances the model's generalization across multiple dimensions such as motions, semantics, and aesthetics, achieving TOP performance among all open-sourced and closed-sourced models.
|
||||
- **Efficient High-Definition Hybrid TI2V**: Open-sources a 5B model built with our advanced Wan2.2-VAE that achieves a compression ratio of 16×16×4. This model supports both text-to-video and image-to-video generation at 720P resolution with 24fps and can also run on consumer-grade graphics cards like 4090. It is one of the fastest 720P@24fps models currently available, capable of serving both the industrial and academic sectors simultaneously.
|
||||
|
||||
For more details, please refer to the [official Wan2.2 GitHub Repository](https://github.com/Wan-Video/Wan2.2).
|
||||
Choose the A14B MoE checkpoints for maximum T2V or I2V capacity and the 5B TI2V model for a smaller unified 720p-at-24-fps path. MoE reduces active compute relative to total capacity but does not remove the memory cost of loading expert weights, so hardware selection still matters.
|
||||
|
||||
## 2. SGLang-diffusion Installation
|
||||
|
||||
|
||||
@@ -7,23 +7,13 @@ metatags:
|
||||
import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';
|
||||
import { ZImageTurboDeployment } from '/src/snippets/diffusion/zimage-turbo-deployment.jsx';
|
||||
|
||||
<DiffusionModelTags tags={["image", "text-to-image", "turbo", "8-step"]} />
|
||||
<DiffusionModelTags tags={["image", "text-to-image", "6B S3-DiT", "8-step", "bilingual text"]} />
|
||||
|
||||
## 1. Model Introduction
|
||||
|
||||
[Z-Image](https://github.com/Tongyi-MAI/Z-Image) is a powerful and highly efficient image generation model family with 6B parameters, developed by Tongyi-MAI. It adopts a Scalable Single-Stream DiT (S3-DiT) architecture, where text, visual semantic tokens, and image VAE tokens are concatenated at the sequence level to serve as a unified input stream, maximizing parameter efficiency compared to dual-stream approaches.
|
||||
[Z-Image-Turbo](https://huggingface.co/Tongyi-MAI/Z-Image-Turbo) is a distilled 6B single-stream DiT for fast text-to-image generation. It reaches its intended operating point in 8 function evaluations and is particularly strong at photorealistic scenes, prompt adherence, and English/Chinese text rendering.
|
||||
|
||||
[Z-Image-Turbo](https://huggingface.co/Tongyi-MAI/Z-Image-Turbo) is a distilled version of Z-Image that matches or exceeds leading competitors with only 8 NFEs (Number of Function Evaluations). It is powered by two core techniques: **Decoupled-DMD** (few-step distillation) and **DMDR** (fusing DMD with Reinforcement Learning).
|
||||
|
||||
**Key Features:**
|
||||
|
||||
- **Sub-second Inference Latency**: Achieves sub-second inference on enterprise-grade H800 GPUs and fits comfortably within 16GB VRAM consumer devices
|
||||
- **Photorealistic Image Generation**: Excels in high-quality photorealistic image generation with rich aesthetics
|
||||
- **Bilingual Text Rendering**: Supports accurate bilingual text rendering in both English and Chinese
|
||||
- **Robust Instruction Adherence**: Strong prompt following and instruction adherence capabilities
|
||||
- **#1 Open-Source Model**: Ranked 8th overall and #1 among open-source models on the [Artificial Analysis Text-to-Image Leaderboard](https://artificialanalysis.ai/image/leaderboard/text-to-image)
|
||||
|
||||
For more details, please refer to the [Z-Image-Turbo HuggingFace page](https://huggingface.co/Tongyi-MAI/Z-Image-Turbo), the [GitHub repository](https://github.com/Tongyi-MAI/Z-Image), and the [technical report (arXiv)](https://arxiv.org/abs/2511.22699).
|
||||
Choose it when latency and a relatively small deployment footprint matter more than the editability or maximum capacity of larger image models. It is a generation-only checkpoint; use Qwen-Image-Edit or FLUX.2 when the request includes source images or identity-preserving edits.
|
||||
|
||||
## 2. SGLang-diffusion Installation
|
||||
|
||||
|
||||
+666
-2
@@ -168,15 +168,22 @@ html.dark table tbody tr:nth-child(even) td,
|
||||
flex-wrap: wrap;
|
||||
align-items: center;
|
||||
gap: 6px;
|
||||
margin: -1.5rem 0 0;
|
||||
margin: 8px 0 30px;
|
||||
}
|
||||
|
||||
.sgd-model-tags .sgd-chip {
|
||||
margin: 0;
|
||||
padding: 4px 9px;
|
||||
border: 0;
|
||||
background: #eef2ff;
|
||||
color: #3730a3;
|
||||
font-size: 12px;
|
||||
font-weight: 750;
|
||||
line-height: 1.4;
|
||||
}
|
||||
|
||||
.sgd-model-tags + h2 {
|
||||
margin-top: 0.75rem;
|
||||
margin-top: 0;
|
||||
}
|
||||
|
||||
.prose:has(.sgd-model-tags) {
|
||||
@@ -184,6 +191,656 @@ html.dark table tbody tr:nth-child(even) td,
|
||||
overflow-y: visible;
|
||||
}
|
||||
|
||||
/* Legacy command visualizer controls. New diffusion pages can opt into the
|
||||
scoped editorial builder below without changing older configs. */
|
||||
.sg-command-visualizer:not(.sgd-command-builder) :where(button:not(:disabled), .sg-command-visualizer-choice[aria-disabled="false"], .sg-command-visualizer-tab) {
|
||||
transition: transform 160ms ease-out, border-color 160ms ease-out,
|
||||
background-color 160ms ease-out, color 160ms ease-out;
|
||||
}
|
||||
|
||||
@media (hover: hover) and (pointer: fine) {
|
||||
.sg-command-visualizer:not(.sgd-command-builder) :where(button:not(:disabled), .sg-command-visualizer-choice[aria-disabled="false"], .sg-command-visualizer-tab):hover {
|
||||
transform: translateY(-1px) scale(1.015);
|
||||
}
|
||||
}
|
||||
|
||||
.sg-command-visualizer:not(.sgd-command-builder) :where(button:not(:disabled), .sg-command-visualizer-choice[aria-disabled="false"], .sg-command-visualizer-tab):active {
|
||||
transform: translateY(0) scale(0.985);
|
||||
}
|
||||
|
||||
.sg-command-visualizer:not(.sgd-command-builder) :where(button, .sg-command-visualizer-choice, .sg-command-visualizer-tab):focus-visible {
|
||||
outline: 2px solid #e2775f;
|
||||
outline-offset: 2px;
|
||||
}
|
||||
|
||||
@media (prefers-reduced-motion: reduce) {
|
||||
.sg-command-visualizer:not(.sgd-command-builder) :where(button, .sg-command-visualizer-choice, .sg-command-visualizer-tab) {
|
||||
transition: none;
|
||||
transform: none !important;
|
||||
}
|
||||
}
|
||||
|
||||
/* Premium responsive diffusion command builder. The component is its own
|
||||
query container because Mintlify's content column is narrower than the
|
||||
browser viewport whenever either navigation rail is present. */
|
||||
.sgd-command-builder {
|
||||
container: command-builder / inline-size;
|
||||
--sgdb-surface: white;
|
||||
--sgdb-raised: white;
|
||||
--sgdb-ink: oklch(22% 0.02 258);
|
||||
--sgdb-muted: oklch(50% 0.02 258);
|
||||
--sgdb-line: oklch(90% 0.008 258);
|
||||
--sgdb-brand: oklch(57% 0.15 35);
|
||||
/* Accents stay in the brand's warm family — the materials are cool
|
||||
neutrals, so identity comes from where the orange lands, not from
|
||||
tinting the surfaces. Serve = brand orange, Request = amber. */
|
||||
--sgdb-serve: oklch(56% 0.13 40);
|
||||
--sgdb-request: oklch(60% 0.12 62);
|
||||
--sgdb-ok: oklch(52% 0.11 153);
|
||||
--sgdb-warn: oklch(59% 0.12 78);
|
||||
--sgdb-error: oklch(53% 0.16 25);
|
||||
/* Glass material system. A recessed well is a cool translucent tint (warm
|
||||
greys read muddy); the raised thumb is a glass slab: gradient fill, a
|
||||
specular top edge, a hairline ring, and a soft cast. The pair is
|
||||
redeclared for dark, where the glass is built from white overlays. */
|
||||
--sgdb-well: oklch(45% 0.03 258 / 0.08);
|
||||
--sgdb-well-inset: inset 0 1px 2px oklch(30% 0.03 258 / 0.07);
|
||||
--sgdb-thumb-bg: linear-gradient(180deg, white, oklch(98.5% 0.003 258));
|
||||
--sgdb-thumb-shadow: 0 0 0 0.5px oklch(30% 0.02 258 / 0.1),
|
||||
inset 0 1px 0 rgba(255, 255, 255, 0.9),
|
||||
0 1px 1px oklch(30% 0.02 258 / 0.07),
|
||||
0 4px 12px oklch(30% 0.02 258 / 0.1);
|
||||
/* Hover tone for anything that sits inside a tinted well. */
|
||||
--sgdb-pick-hover: oklch(45% 0.03 258 / 0.05);
|
||||
--sgdb-code-bg: oklch(97.5% 0.004 258);
|
||||
--sgdb-code-inset: oklch(30% 0.03 258 / 0.06);
|
||||
/* Colours settle on a decelerating curve; things that move get a spring —
|
||||
a touch of overshoot is what reads as alive rather than mechanical. */
|
||||
--sgdb-ease: cubic-bezier(0.32, 0.72, 0, 1);
|
||||
--sgdb-ease-dur: 200ms;
|
||||
--sgdb-spring: cubic-bezier(0.3, 1.35, 0.45, 1);
|
||||
--sgdb-spring-dur: 460ms;
|
||||
max-width: 100%;
|
||||
margin: 24px 0 40px;
|
||||
overflow: clip;
|
||||
border: 1px solid var(--sgdb-line);
|
||||
border-radius: 16px;
|
||||
background: var(--sgdb-surface);
|
||||
box-shadow: 0 1px 2px oklch(30% 0.03 258 / 0.04),
|
||||
0 12px 32px -12px oklch(30% 0.03 258 / 0.12);
|
||||
color: var(--sgdb-ink);
|
||||
}
|
||||
|
||||
.sgd-command-builder *, .sgd-command-builder *::before, .sgd-command-builder *::after { box-sizing: border-box; }
|
||||
/* Form controls inherit the page typeface instead of the browser's. Wrapped in
|
||||
:where() so the rule weighs (0,1,0) and every single-class control rule below
|
||||
still wins on order. Written as `.sgd-command-builder button` it weighed
|
||||
(0,1,1) and, being a `font` shorthand, silently reset the font-size and
|
||||
font-weight of anything selected by one class — `.sgd-builder-copy` and
|
||||
`.sgd-builder-text-action` asked for 11px/680 and rendered 16px/400. */
|
||||
.sgd-command-builder :where(button, input, select) { font: inherit; }
|
||||
|
||||
/* Segmented control with one glass pill that glides between scopes — the
|
||||
pill is a single ::before positioned off the active button via :has(),
|
||||
so the slide is pure CSS and springs into place. */
|
||||
.sgd-builder-scope-tabs {
|
||||
position: relative;
|
||||
display: grid;
|
||||
grid-template-columns: repeat(3, minmax(0, 1fr));
|
||||
margin: 12px 12px 0;
|
||||
padding: 3px;
|
||||
border-radius: 11px;
|
||||
background: var(--sgdb-well);
|
||||
box-shadow: var(--sgdb-well-inset);
|
||||
}
|
||||
|
||||
.sgd-builder-scope-tabs::before {
|
||||
position: absolute;
|
||||
top: 3px;
|
||||
bottom: 3px;
|
||||
left: 3px;
|
||||
width: calc((100% - 6px) / 3);
|
||||
border-radius: 8px;
|
||||
background: var(--sgdb-thumb-bg);
|
||||
box-shadow: var(--sgdb-thumb-shadow);
|
||||
content: "";
|
||||
/* Glide and settle, no overshoot: a scope switch is a frequent functional
|
||||
action, so the slide itself carries the life and a bounce would be noise. */
|
||||
transition: transform 300ms var(--sgdb-ease);
|
||||
}
|
||||
|
||||
.sgd-builder-scope-tabs:has(> button:nth-child(2)[data-active="true"])::before { transform: translateX(100%); }
|
||||
.sgd-builder-scope-tabs:has(> button:nth-child(3)[data-active="true"])::before { transform: translateX(200%); }
|
||||
|
||||
.sgd-builder-scope-tabs button {
|
||||
position: relative;
|
||||
z-index: 1;
|
||||
min-width: 0;
|
||||
min-height: 34px;
|
||||
padding: 0 4px;
|
||||
border: 0;
|
||||
background: transparent;
|
||||
color: var(--sgdb-muted);
|
||||
cursor: pointer;
|
||||
font-size: 12px;
|
||||
font-weight: 590;
|
||||
white-space: nowrap;
|
||||
transition: color var(--sgdb-ease-dur) var(--sgdb-ease);
|
||||
}
|
||||
|
||||
.sgd-builder-scope-tabs button[data-active="true"] { color: var(--sgdb-ink); }
|
||||
.sgd-builder-main { display: flex; min-width: 0; flex-direction: column; }
|
||||
.sgd-builder-controls, .sgd-builder-output-rail { min-width: 0; }
|
||||
.sgd-builder-controls { container: builder-controls / inline-size; padding: 4px 16px 20px; }
|
||||
|
||||
.sgd-builder-scope-panel { display: grid; gap: 0; }
|
||||
.sgd-builder-recipe {
|
||||
display: flex;
|
||||
min-width: 0;
|
||||
padding: 11px 0;
|
||||
gap: 12px;
|
||||
align-items: center;
|
||||
justify-content: space-between;
|
||||
border-bottom: 1px solid var(--sgdb-line);
|
||||
}
|
||||
.sgd-builder-recipe > div:first-child { display: grid; min-width: 0; gap: 3px; }
|
||||
.sgd-builder-recipe > div:first-child span { color: var(--sgdb-muted); font-size: 10.5px; }
|
||||
.sgd-builder-recipe > div:first-child strong { font-size: 12px; font-weight: 590; line-height: 1.4; }
|
||||
.sgd-builder-recipe > div:last-child { display: flex; flex: none; gap: 10px; align-items: center; }
|
||||
.sgd-builder-recipe > div:last-child > small { color: var(--sgdb-muted); font-size: 10px; }
|
||||
.sgd-builder-section { min-width: 0; padding: 13px 0; border-bottom: 1px solid var(--sgdb-line); }
|
||||
.sgd-builder-section:last-child { border-bottom: 0; }
|
||||
.sgd-builder-section-heading { display: flex; gap: 8px; align-items: baseline; justify-content: space-between; margin-bottom: 8px; }
|
||||
.sgd-builder-section-heading > span { font-size: 12px; font-weight: 590; }
|
||||
.sgd-builder-section-heading > small { color: var(--sgdb-muted); font-size: 11px; line-height: 1.35; text-align: right; }
|
||||
|
||||
/* One tinted well per group instead of a border on every option. Thirteen
|
||||
outlined boxes competed with each other and with the panel frame; the well
|
||||
carries the grouping so each option only has to carry its own state. */
|
||||
.sgd-builder-hardware-grid, .sgd-builder-choice-grid {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(2, minmax(0, 1fr));
|
||||
gap: 2px;
|
||||
padding: 3px;
|
||||
border-radius: 11px;
|
||||
background: var(--sgdb-well);
|
||||
box-shadow: var(--sgdb-well-inset);
|
||||
}
|
||||
.sgd-builder-hardware, .sgd-builder-choice {
|
||||
position: relative;
|
||||
display: grid;
|
||||
min-width: 0;
|
||||
min-height: 38px;
|
||||
padding: 7px 10px;
|
||||
overflow: hidden;
|
||||
/* Rows in the same grid track stretch to the tallest sibling; center the
|
||||
label stack so a single-line option does not ride the top of its cell. */
|
||||
align-content: center;
|
||||
border: 0;
|
||||
border-radius: 8px;
|
||||
background: transparent;
|
||||
/* Unselected stays legible — a deselected option is still a real choice, so
|
||||
it sits just below the ink, not down at the muted caption level. */
|
||||
color: color-mix(in oklch, var(--sgdb-ink) 72%, transparent);
|
||||
cursor: pointer;
|
||||
font-size: 12px;
|
||||
line-height: 1.25;
|
||||
text-align: left;
|
||||
transition: background-color var(--sgdb-ease-dur) var(--sgdb-ease),
|
||||
color var(--sgdb-ease-dur) var(--sgdb-ease);
|
||||
}
|
||||
|
||||
/* Hardware label and its VRAM share one line — the second text row bought
|
||||
no clarity, only cell height. Choice subtitles (checkpoint names) keep
|
||||
their own line: they are long enough to need it. */
|
||||
.sgd-builder-hardware { display: flex; min-height: 34px; flex-flow: row wrap; align-items: baseline; gap: 5px; }
|
||||
.sgd-builder-hardware strong { font-size: 12.5px; font-weight: 590; letter-spacing: -0.01em; }
|
||||
.sgd-builder-hardware small, .sgd-builder-choice small { margin-top: 1px; color: var(--sgdb-muted); font-size: 10.5px; }
|
||||
.sgd-builder-hardware small { margin-top: 0; }
|
||||
|
||||
/* The radio ring is gone — the fill is the affordance. Kept in the DOM (the
|
||||
JSX renders it) but collapsed, so no markup change is needed. */
|
||||
.sgd-builder-choice-dot { display: none; }
|
||||
|
||||
.sgd-builder-hardware[data-selected="true"], .sgd-builder-choice[data-selected="true"] {
|
||||
background: var(--sgdb-thumb-bg);
|
||||
box-shadow: var(--sgdb-thumb-shadow);
|
||||
color: var(--sgdb-ink);
|
||||
}
|
||||
|
||||
.sgd-builder-hardware:disabled,
|
||||
.sgd-builder-choice:disabled,
|
||||
.sgd-builder-choice[data-blocked="true"] { cursor: not-allowed; opacity: 0.42; }
|
||||
|
||||
/* Tap/click feedback for a blocked option — the reason the tooltip carries,
|
||||
surfaced where touch readers can see it. */
|
||||
.sgd-builder-blocked-note {
|
||||
margin: 8px 2px 0;
|
||||
color: var(--sgdb-warn);
|
||||
font-size: 11px;
|
||||
line-height: 1.45;
|
||||
animation: sgdb-note-in 220ms var(--sgdb-ease);
|
||||
}
|
||||
@keyframes sgdb-note-in {
|
||||
from { opacity: 0; transform: translateY(-2px); }
|
||||
to { opacity: 1; transform: translateY(0); }
|
||||
}
|
||||
|
||||
/* A soft (selectable-but-unverified) pick declares itself in place. */
|
||||
.sgd-builder-choice[data-soft="true"][data-selected="true"]::after {
|
||||
content: "Not verified yet";
|
||||
color: var(--sgdb-warn);
|
||||
font-size: 10px;
|
||||
line-height: 1.3;
|
||||
margin-top: 1px;
|
||||
}
|
||||
.sgd-builder-resource-grid { display: grid; gap: 8px; }
|
||||
.sgd-builder-stepper-field {
|
||||
display: flex;
|
||||
min-height: 42px;
|
||||
padding: 5px 6px 5px 12px;
|
||||
align-items: center;
|
||||
justify-content: space-between;
|
||||
border: 0;
|
||||
border-radius: 10px;
|
||||
background: var(--sgdb-well);
|
||||
box-shadow: var(--sgdb-well-inset);
|
||||
}
|
||||
.sgd-builder-stepper-field > div:first-child { display: grid; gap: 2px; }
|
||||
.sgd-builder-stepper-field span { font-size: 12px; font-weight: 500; }
|
||||
.sgd-builder-stepper-field small { color: var(--sgdb-muted); font-size: 10px; }
|
||||
/* One integrated control: a single raised track with a hairline between the
|
||||
two halves, rather than two separately outlined 34px blocks. */
|
||||
.sgd-builder-stepper {
|
||||
display: grid;
|
||||
grid-template-columns: 28px 40px 28px;
|
||||
align-items: center;
|
||||
border-radius: 8px;
|
||||
background: var(--sgdb-thumb-bg);
|
||||
box-shadow: var(--sgdb-thumb-shadow);
|
||||
}
|
||||
.sgd-builder-stepper button, .sgd-builder-request-stepper button {
|
||||
min-width: 28px;
|
||||
min-height: 30px;
|
||||
padding: 0;
|
||||
border: 0;
|
||||
border-radius: 8px;
|
||||
background: transparent;
|
||||
color: var(--sgdb-ink);
|
||||
cursor: pointer;
|
||||
font-size: 14px;
|
||||
transition: background-color var(--sgdb-ease-dur) var(--sgdb-ease);
|
||||
}
|
||||
.sgd-builder-stepper input {
|
||||
width: 40px;
|
||||
height: 30px;
|
||||
padding: 0;
|
||||
border-right: 0.5px solid color-mix(in oklch, var(--sgdb-ink) 10%, transparent);
|
||||
border-left: 0.5px solid color-mix(in oklch, var(--sgdb-ink) 10%, transparent);
|
||||
border-top: 0;
|
||||
border-bottom: 0;
|
||||
border-radius: 0;
|
||||
appearance: textfield;
|
||||
background: transparent;
|
||||
color: var(--sgdb-ink);
|
||||
font-size: 12px;
|
||||
font-variant-numeric: tabular-nums;
|
||||
font-weight: 590;
|
||||
line-height: 30px;
|
||||
text-align: center;
|
||||
}
|
||||
.sgd-builder-stepper input::-webkit-inner-spin-button,
|
||||
.sgd-builder-stepper input::-webkit-outer-spin-button { margin: 0; appearance: none; }
|
||||
.sgd-builder-stepper button:disabled, .sgd-builder-request-stepper button:disabled { opacity: 0.32; cursor: not-allowed; }
|
||||
.sgd-builder-resource-summary { margin: 8px 0 0; color: var(--sgdb-muted); font-size: 11px; line-height: 1.4; }
|
||||
|
||||
.sgd-builder-topology-toggle { margin-top: 10px; }
|
||||
.sgd-builder-text-action {
|
||||
justify-self: start;
|
||||
min-height: 32px;
|
||||
padding: 0;
|
||||
border: 0;
|
||||
background: transparent;
|
||||
color: var(--sgdb-muted);
|
||||
cursor: pointer;
|
||||
font-size: 11px;
|
||||
font-weight: 500;
|
||||
}
|
||||
.sgd-builder-text-action:disabled { cursor: not-allowed; opacity: 0.4; }
|
||||
.sgd-builder-advanced { margin-top: 12px; padding-top: 12px; border-top: 1px solid var(--sgdb-line); }
|
||||
.sgd-builder-advanced > p { margin: 0 0 12px; color: var(--sgdb-muted); font-size: 11px; line-height: 1.5; }
|
||||
.sgd-builder-topology-inputs { display: grid; gap: 8px; }
|
||||
.sgd-builder-topology-inputs label { display: grid; gap: 5px; color: var(--sgdb-muted); font-size: 10.5px; }
|
||||
.sgd-builder-topology-inputs select {
|
||||
width: 100%;
|
||||
min-height: 38px;
|
||||
padding: 0 8px;
|
||||
border: 1px solid var(--sgdb-line);
|
||||
border-radius: 6px;
|
||||
background: var(--sgdb-raised);
|
||||
color: var(--sgdb-ink);
|
||||
font-size: 12px;
|
||||
}
|
||||
.sgd-builder-messages { margin-top: 12px; padding: 8px 10px; border-radius: 6px; background: color-mix(in oklch, var(--sgdb-warn) 9%, transparent); color: color-mix(in oklch, var(--sgdb-warn) 75%, var(--sgdb-ink)); }
|
||||
.sgd-builder-messages[data-state="error"] { background: color-mix(in oklch, var(--sgdb-error) 9%, transparent); color: color-mix(in oklch, var(--sgdb-error) 78%, var(--sgdb-ink)); }
|
||||
.sgd-builder-messages p { margin: 0; font-size: 11px; line-height: 1.45; }
|
||||
.sgd-builder-messages p + p { margin-top: 5px; }
|
||||
|
||||
.sgd-builder-setting-layout, .sgd-builder-setting-list, .sgd-builder-setting-item { min-width: 0; }
|
||||
.sgd-builder-setting-item { border-bottom: 1px solid var(--sgdb-line); }
|
||||
.sgd-builder-setting-item:last-child { border-bottom: 0; }
|
||||
.sgd-builder-setting-row {
|
||||
display: grid;
|
||||
grid-template-columns: minmax(76px, auto) minmax(0, 1fr) auto auto;
|
||||
width: 100%;
|
||||
min-height: 44px;
|
||||
padding: 8px;
|
||||
gap: 8px;
|
||||
align-items: center;
|
||||
border: 0;
|
||||
border-bottom: 0;
|
||||
border-radius: 8px;
|
||||
background: transparent;
|
||||
transition: background-color var(--sgdb-ease-dur) var(--sgdb-ease),
|
||||
color var(--sgdb-ease-dur) var(--sgdb-ease);
|
||||
color: var(--sgdb-ink);
|
||||
cursor: pointer;
|
||||
text-align: left;
|
||||
}
|
||||
.sgd-builder-setting-row > span:first-child { color: var(--sgdb-muted); font-size: 11px; }
|
||||
.sgd-builder-setting-row strong { overflow: hidden; font-size: 12px; font-weight: 500; text-overflow: ellipsis; white-space: nowrap; }
|
||||
.sgd-builder-setting-row small { color: var(--sgdb-ok); font-size: 10px; font-weight: 500; }
|
||||
.sgd-builder-setting-row > span:last-child { color: var(--sgdb-muted); font-size: 17px; }
|
||||
.sgd-builder-setting-row[data-active="true"] { background: color-mix(in oklch, var(--sgdb-serve) 4%, transparent); color: var(--sgdb-serve); }
|
||||
|
||||
.sgd-builder-context { padding: 16px 0 4px; }
|
||||
.sgd-builder-context--inline { padding: 12px 4px 16px; border-top: 1px solid var(--sgdb-line); }
|
||||
.sgd-builder-context--rail { display: none; }
|
||||
.sgd-builder-context-heading { display: flex; align-items: start; justify-content: space-between; gap: 12px; }
|
||||
.sgd-builder-context-heading > div > span { font-size: 13px; font-weight: 590; }
|
||||
.sgd-builder-context-heading p { margin: 4px 0 0; color: var(--sgdb-muted); font-size: 11px; line-height: 1.45; }
|
||||
.sgd-builder-context-heading > small { flex: none; color: var(--sgdb-muted); font-size: 10px; }
|
||||
.sgd-builder-context-options { display: grid; grid-template-columns: repeat(2, minmax(0, 1fr)); gap: 8px; margin-top: 12px; }
|
||||
.sgd-builder-context-note { margin-top: 12px; padding-top: 10px; border-top: 1px solid var(--sgdb-line); }
|
||||
.sgd-builder-context-note p { margin: 0; color: var(--sgdb-muted); font-size: 11px; line-height: 1.5; }
|
||||
.sgd-builder-context-note a {
|
||||
display: inline-flex;
|
||||
margin-top: 8px;
|
||||
min-height: 24px;
|
||||
align-items: center;
|
||||
gap: 5px;
|
||||
color: var(--sgdb-serve);
|
||||
font-size: 11px;
|
||||
font-weight: 500;
|
||||
text-decoration: none;
|
||||
transition: color var(--sgdb-ease-dur) var(--sgdb-ease);
|
||||
}
|
||||
.sgd-builder-context-note a + a { margin-left: 16px; }
|
||||
.sgd-builder-context-note a svg { flex: none; transition: transform var(--sgdb-ease-dur) var(--sgdb-ease); }
|
||||
/* Hover answers: the label underlines and the glyph leans in. */
|
||||
.sgd-builder-context-note a:hover {
|
||||
text-decoration: underline;
|
||||
text-decoration-thickness: 1px;
|
||||
text-underline-offset: 3px;
|
||||
}
|
||||
.sgd-builder-context-note a:hover svg { transform: translateX(1px) scale(1.08); }
|
||||
.sgd-builder-scope-panel[data-scope="request"] .sgd-builder-context-note a { color: var(--sgdb-request); }
|
||||
.sgd-builder-request-stepper { display: grid; grid-template-columns: 40px 52px 40px auto; gap: 6px; align-items: center; margin-top: 14px; }
|
||||
.sgd-builder-request-stepper input {
|
||||
width: 52px;
|
||||
min-width: 0;
|
||||
height: 34px;
|
||||
padding: 0 4px;
|
||||
border: 1px solid var(--sgdb-line);
|
||||
border-radius: 6px;
|
||||
appearance: textfield;
|
||||
background: var(--sgdb-surface);
|
||||
color: var(--sgdb-ink);
|
||||
font-size: 16px;
|
||||
font-variant-numeric: tabular-nums;
|
||||
font-weight: 590;
|
||||
text-align: center;
|
||||
}
|
||||
.sgd-builder-request-stepper input::-webkit-inner-spin-button,
|
||||
.sgd-builder-request-stepper input::-webkit-outer-spin-button { margin: 0; appearance: none; }
|
||||
.sgd-builder-request-stepper > span { margin-left: 4px; color: var(--sgdb-muted); font-size: 11px; }
|
||||
.sgd-builder-request-setting { border-bottom: 1px solid var(--sgdb-line); }
|
||||
.sgd-builder-request-setting:last-child { border-bottom: 0; }
|
||||
.sgd-builder-request-setting .sgd-builder-context { padding: 16px 0; }
|
||||
|
||||
/* stretch + the code area flexing below keep side-by-side cards the same
|
||||
height, so their footers sit on one line instead of ragged bottoms. */
|
||||
.sgd-builder-output-rail { display: grid; gap: 12px; padding: 0 12px 16px; align-items: stretch; }
|
||||
.sgd-builder-output {
|
||||
--output-accent: var(--sgdb-serve);
|
||||
display: flex;
|
||||
min-width: 0;
|
||||
flex-direction: column;
|
||||
overflow: hidden;
|
||||
border: 1px solid var(--sgdb-line);
|
||||
border-radius: 12px;
|
||||
background: var(--sgdb-raised);
|
||||
box-shadow: 0 1px 2px oklch(30% 0.03 258 / 0.05);
|
||||
transition: border-color 160ms ease-out, transform 160ms ease-out;
|
||||
}
|
||||
.sgd-builder-output[data-output="request"] { --output-accent: var(--sgdb-request); }
|
||||
.sgd-builder-output[data-emphasized="true"] { border-color: color-mix(in oklch, var(--output-accent) 24%, var(--sgdb-line)); }
|
||||
.sgd-builder-output > header { display: grid; grid-template-columns: 24px minmax(0, 1fr) auto; gap: 8px; align-items: center; min-height: 50px; padding: 8px 10px; border-bottom: 1px solid var(--sgdb-line); }
|
||||
.sgd-builder-output[data-emphasized="true"] > header {
|
||||
background: linear-gradient(180deg,
|
||||
color-mix(in oklch, var(--output-accent) 7%, transparent),
|
||||
color-mix(in oklch, var(--output-accent) 2%, transparent));
|
||||
}
|
||||
/* Neutral step disc — the accents stay reserved for state (active tab, status),
|
||||
so a static ordinal does not compete with them. */
|
||||
.sgd-builder-output-index { display: grid; width: 20px; height: 20px; place-items: center; border: 0; border-radius: 50%; background: var(--sgdb-well); color: var(--sgdb-muted); font-size: 10px; font-weight: 590; font-variant-numeric: tabular-nums; }
|
||||
.sgd-builder-output-title { display: flex; min-width: 0; gap: 7px; align-items: baseline; }
|
||||
.sgd-builder-output-title strong { font-size: 13px; font-weight: 590; }
|
||||
.sgd-builder-output-title span { color: var(--sgdb-muted); font-size: 10.5px; }
|
||||
.sgd-builder-status { display: inline-flex; gap: 5px; align-items: center; color: var(--sgdb-muted); font-size: 10px; white-space: nowrap; }
|
||||
.sgd-builder-status > span {
|
||||
width: 6px;
|
||||
height: 6px;
|
||||
border-radius: 50%;
|
||||
background: var(--sgdb-warn);
|
||||
box-shadow: 0 0 6px color-mix(in oklch, var(--sgdb-warn) 60%, transparent);
|
||||
}
|
||||
.sgd-builder-status[data-status="verified"] > span { background: var(--sgdb-ok); }
|
||||
.sgd-builder-status[data-status="error"] > span { background: var(--sgdb-error); }
|
||||
.sgd-builder-output-tabs { display: flex; min-height: 38px; padding: 5px 10px; gap: 14px; align-items: center; border-bottom: 1px solid var(--sgdb-line); }
|
||||
.sgd-builder-output-tabs button { min-height: 28px; padding: 0; border: 0; border-bottom: 1px solid transparent; background: transparent; color: var(--sgdb-muted); cursor: pointer; font-size: 10.5px; font-weight: 500; }
|
||||
.sgd-builder-output-tabs button[data-selected="true"] { border-bottom-color: var(--output-accent); color: var(--sgdb-ink); }
|
||||
.sgd-builder-node-fields { display: grid; grid-template-columns: minmax(0, 1fr) 84px; gap: 8px; padding: 8px 10px; border-bottom: 1px solid var(--sgdb-line); }
|
||||
.sgd-builder-node-fields label { display: grid; min-width: 0; gap: 4px; color: var(--sgdb-muted); font-size: 10px; }
|
||||
.sgd-builder-node-fields input { width: 100%; min-width: 0; min-height: 34px; padding: 0 7px; border: 1px solid var(--sgdb-line); border-radius: 6px; background: var(--sgdb-surface); color: var(--sgdb-ink); font-size: 11px; }
|
||||
/* Positioning context for the floating copy control; grid so the pre keeps
|
||||
min-width:0 scrolling. flex:1 lets the code area absorb the height
|
||||
difference between side-by-side cards. */
|
||||
.sgd-builder-code { position: relative; display: grid; min-width: 0; flex: 1 1 auto; }
|
||||
.sgd-builder-code > pre {
|
||||
height: 100%;
|
||||
max-height: calc(13.95em + 24px);
|
||||
min-width: 0;
|
||||
margin: 0;
|
||||
padding: 12px;
|
||||
overflow: auto;
|
||||
border: 0;
|
||||
border-radius: 0;
|
||||
background: var(--sgdb-code-bg);
|
||||
box-shadow: inset 0 1px 3px var(--sgdb-code-inset);
|
||||
color: var(--sgdb-ink);
|
||||
font-size: clamp(11.5px, 1.2cqi, 12.5px);
|
||||
line-height: 1.55;
|
||||
white-space: pre;
|
||||
}
|
||||
.sgd-builder-code > pre::-webkit-scrollbar-corner { background: transparent; }
|
||||
.sgd-builder-code > pre.is-expanded { height: auto; max-height: 70vh; }
|
||||
.sgd-builder-output code { background: transparent; color: inherit; font: inherit; font-variant-ligatures: none; }
|
||||
.sgd-builder-output-error { padding: 8px 10px; border-top: 1px solid color-mix(in oklch, var(--sgdb-error) 35%, var(--sgdb-line)); background: color-mix(in oklch, var(--sgdb-error) 8%, transparent); color: var(--sgdb-error); font-size: 10.5px; line-height: 1.4; }
|
||||
.sgd-builder-output > footer { display: flex; min-height: 44px; padding: 5px 10px; align-items: center; justify-content: space-between; border-top: 1px solid var(--sgdb-line); }
|
||||
.sgd-builder-output > footer > div { display: flex; margin-left: auto; gap: 12px; align-items: center; }
|
||||
/* Copy floats inside the code area (top-right), revealed on hover — the
|
||||
standard code-block pattern. A frosted chip keeps the glyph legible over
|
||||
scrolled code without drawing a box when idle. */
|
||||
.sgd-builder-copy {
|
||||
position: absolute;
|
||||
top: 7px;
|
||||
right: 7px;
|
||||
display: grid;
|
||||
width: 28px;
|
||||
height: 28px;
|
||||
padding: 0;
|
||||
border: 0;
|
||||
border-radius: 7px;
|
||||
background: var(--sgdb-thumb-bg);
|
||||
-webkit-backdrop-filter: blur(8px) saturate(1.4);
|
||||
backdrop-filter: blur(8px) saturate(1.4);
|
||||
box-shadow: var(--sgdb-thumb-shadow);
|
||||
color: var(--sgdb-muted);
|
||||
cursor: pointer;
|
||||
opacity: 0;
|
||||
place-items: center;
|
||||
transform: translateY(2px) scale(0.9);
|
||||
transition: opacity var(--sgdb-ease-dur) var(--sgdb-ease),
|
||||
color var(--sgdb-ease-dur) var(--sgdb-ease),
|
||||
transform var(--sgdb-spring-dur) var(--sgdb-spring);
|
||||
}
|
||||
.sgd-builder-code:hover > .sgd-builder-copy,
|
||||
.sgd-builder-copy:focus-visible,
|
||||
.sgd-builder-copy[data-copied="true"] { opacity: 1; transform: translateY(0) scale(1); }
|
||||
/* No hover on touch — the control must stay visible. */
|
||||
@media (hover: none) {
|
||||
.sgd-builder-copy { opacity: 1; transform: none; }
|
||||
}
|
||||
.sgd-builder-copy:disabled { cursor: not-allowed; color: color-mix(in oklch, var(--sgdb-muted) 45%, transparent); }
|
||||
/* Both glyphs share the cell; the swap is a settle-in scale rather than a
|
||||
hard display switch. */
|
||||
.sgd-builder-copy svg {
|
||||
grid-area: 1 / 1;
|
||||
pointer-events: none;
|
||||
transition: opacity 140ms var(--sgdb-ease), transform 220ms var(--sgdb-ease);
|
||||
}
|
||||
.sgd-builder-copy .sgd-builder-copy-check { opacity: 0; transform: scale(0.5); }
|
||||
.sgd-builder-copy[data-copied="true"] { color: var(--sgdb-ok); }
|
||||
.sgd-builder-copy[data-copied="true"] .sgd-builder-copy-glyph { opacity: 0; transform: scale(0.5); }
|
||||
.sgd-builder-copy[data-copied="true"] .sgd-builder-copy-check { opacity: 1; transform: scale(1); }
|
||||
|
||||
/* Hover only changes tone. Lifting the card on hover (translateY) made the
|
||||
whole grid twitch as the pointer crossed it. */
|
||||
@media (hover: hover) and (pointer: fine) {
|
||||
.sgd-command-builder button:not(:disabled):not([data-blocked="true"]):hover { color: var(--sgdb-ink); }
|
||||
.sgd-builder-scope-tabs button:not(:disabled):hover { color: var(--sgdb-ink); }
|
||||
.sgd-builder-hardware:not(:disabled):not([data-selected="true"]):hover,
|
||||
.sgd-builder-choice:not(:disabled):not([data-blocked="true"]):not([data-selected="true"]):hover {
|
||||
background: var(--sgdb-pick-hover);
|
||||
}
|
||||
.sgd-builder-setting-row:not(:disabled):hover { background: color-mix(in oklch, var(--sgdb-serve) 5%, transparent); }
|
||||
.sgd-builder-stepper button:not(:disabled):hover,
|
||||
.sgd-builder-request-stepper button:not(:disabled):hover { background: var(--sgdb-pick-hover); }
|
||||
.sgd-builder-copy:not(:disabled):hover { color: var(--sgdb-ink); }
|
||||
.sgd-builder-copy[data-copied="true"]:hover { color: var(--sgdb-ok); }
|
||||
.sgd-builder-text-action:not(:disabled):hover { color: var(--output-accent, var(--sgdb-brand)); }
|
||||
}
|
||||
|
||||
.sgd-command-builder :where(button, input, select, a):focus-visible { outline: 2px solid color-mix(in oklch, var(--sgdb-brand) 76%, white); outline-offset: 2px; }
|
||||
/* Press reads as the control yielding under the finger, not jumping. */
|
||||
.sgd-command-builder button:not(:disabled):not([data-blocked="true"]):active { transform: scale(0.97); transition-duration: 60ms; }
|
||||
|
||||
@container builder-controls (min-width: 560px) {
|
||||
.sgd-builder-scope-panel[data-scope="base"] { grid-template-columns: repeat(2, minmax(0, 1fr)); column-gap: 28px; }
|
||||
.sgd-builder-scope-panel[data-scope="base"] > .sgd-builder-recipe { grid-column: 1 / -1; }
|
||||
/* Half a column cannot host two steppers side by side — stack them. */
|
||||
.sgd-builder-scope-panel[data-scope="base"] .sgd-builder-resource-grid { grid-template-columns: minmax(0, 1fr); }
|
||||
.sgd-builder-setting-layout { display: grid; grid-template-columns: minmax(240px, 0.9fr) minmax(0, 1.1fr); }
|
||||
.sgd-builder-setting-list { padding-right: 16px; border-right: 1px solid var(--sgdb-line); }
|
||||
.sgd-builder-context--inline { display: none; }
|
||||
.sgd-builder-context--rail { display: block; padding: 12px 0 4px 16px; }
|
||||
.sgd-builder-request-direct { grid-template-columns: repeat(2, minmax(0, 1fr)); }
|
||||
.sgd-builder-request-setting { padding-right: 16px; border-right: 1px solid var(--sgdb-line); border-bottom: 0; }
|
||||
.sgd-builder-request-setting + .sgd-builder-request-setting { padding-right: 0; padding-left: 16px; border-right: 0; }
|
||||
}
|
||||
|
||||
/* Container queries measure the content box; subtract the two 1px borders so
|
||||
the documented 420 / 720 / 960px outer widths switch at the intended point. */
|
||||
@container command-builder (max-width: 717px) {
|
||||
.sgd-builder-output-rail { display: contents; }
|
||||
.sgd-builder-controls { order: 0; }
|
||||
.sgd-builder-output { width: calc(100% - 24px); margin: 12px 12px 16px; order: 1; }
|
||||
.sgd-builder-main[data-scope="serve"] .sgd-builder-output[data-output="serve"],
|
||||
.sgd-builder-main[data-scope="request"] .sgd-builder-output[data-output="request"] { order: -1; }
|
||||
}
|
||||
|
||||
@container command-builder (min-width: 718px) and (max-width: 957px) {
|
||||
.sgd-builder-output-rail { order: -1; padding-top: 12px; }
|
||||
.sgd-builder-main[data-scope="base"] .sgd-builder-output-rail { order: 1; padding-top: 0; }
|
||||
.sgd-builder-output[data-emphasized="false"] { order: 1; }
|
||||
}
|
||||
|
||||
@container command-builder (min-width: 418px) {
|
||||
.sgd-builder-scope-tabs { margin-inline: 16px; }
|
||||
.sgd-builder-scope-tabs button { font-size: 13px; }
|
||||
.sgd-builder-hardware-grid { grid-template-columns: repeat(3, minmax(0, 1fr)); }
|
||||
.sgd-builder-resource-grid { grid-template-columns: repeat(2, minmax(0, 1fr)); }
|
||||
.sgd-builder-topology-inputs { grid-template-columns: repeat(3, minmax(0, 1fr)); }
|
||||
}
|
||||
|
||||
@container command-builder (min-width: 718px) {
|
||||
.sgd-builder-scope-panel[data-scope="base"] { grid-template-columns: repeat(12, minmax(0, 1fr)); column-gap: 16px; }
|
||||
.sgd-builder-scope-panel[data-scope="base"] > :where(.sgd-builder-recipe, .sgd-builder-section) { grid-column: 1 / -1; }
|
||||
.sgd-builder-scope-panel[data-scope="base"] > .sgd-builder-section:nth-last-child(-n + 2) { grid-column: span 6; }
|
||||
.sgd-builder-output-rail { grid-template-columns: minmax(0, 1fr); }
|
||||
.sgd-builder-main[data-scope="base"] .sgd-builder-output-rail { grid-template-columns: repeat(2, minmax(0, 1fr)); }
|
||||
.sgd-builder-choice-grid[data-density="compact"] { grid-template-columns: repeat(3, minmax(0, 1fr)); }
|
||||
}
|
||||
|
||||
@container command-builder (min-width: 958px) {
|
||||
.sgd-builder-scope-tabs { margin-inline: 20px; }
|
||||
.sgd-builder-scope-tabs button { font-size: 14px; }
|
||||
.sgd-builder-main { display: grid; grid-template-columns: minmax(0, 38fr) minmax(0, 62fr); }
|
||||
.sgd-builder-controls { padding: 8px 20px 24px; border-right: 1px solid var(--sgdb-line); }
|
||||
.sgd-builder-output-rail { grid-template-columns: minmax(0, 1fr); padding: 16px; }
|
||||
.sgd-builder-scope-panel[data-scope="base"] { display: grid; grid-template-columns: minmax(0, 1fr); }
|
||||
.sgd-builder-scope-panel[data-scope="base"] > .sgd-builder-recipe,
|
||||
.sgd-builder-scope-panel[data-scope="base"] > .sgd-builder-section,
|
||||
.sgd-builder-scope-panel[data-scope="base"] > .sgd-builder-section:nth-last-child(-n + 2) { grid-column: 1; }
|
||||
.sgd-builder-hardware-grid { grid-template-columns: repeat(2, minmax(0, 1fr)); }
|
||||
.sgd-builder-output { box-shadow: 0 1px 2px color-mix(in oklch, var(--sgdb-ink) 5%, transparent); }
|
||||
}
|
||||
|
||||
html.dark .sgd-command-builder, [data-theme="dark"] .sgd-command-builder {
|
||||
--sgdb-surface: oklch(24% 0.015 258);
|
||||
--sgdb-raised: oklch(27% 0.015 258);
|
||||
--sgdb-ink: oklch(94% 0.006 80);
|
||||
--sgdb-muted: oklch(71% 0.012 258);
|
||||
--sgdb-line: rgba(255, 255, 255, 0.09);
|
||||
--sgdb-brand: oklch(70% 0.14 40);
|
||||
--sgdb-serve: oklch(74% 0.11 45);
|
||||
--sgdb-request: oklch(77% 0.1 64);
|
||||
--sgdb-ok: oklch(74% 0.11 153);
|
||||
--sgdb-warn: oklch(76% 0.11 78);
|
||||
--sgdb-error: oklch(73% 0.14 25);
|
||||
/* Dark glass is built from white overlays: the thumb is a translucent
|
||||
gradient with a specular top edge, wells recess into real black, and
|
||||
the panel carries its own top highlight so it reads as a lit slab
|
||||
rather than a flat grey rectangle on a near-identical page. */
|
||||
--sgdb-well: rgba(0, 0, 0, 0.32);
|
||||
--sgdb-well-inset: inset 0 1px 2px rgba(0, 0, 0, 0.4);
|
||||
--sgdb-thumb-bg: linear-gradient(180deg, rgba(255, 255, 255, 0.14), rgba(255, 255, 255, 0.07));
|
||||
--sgdb-thumb-shadow: 0 0 0 0.5px rgba(255, 255, 255, 0.16),
|
||||
inset 0 1px 0 rgba(255, 255, 255, 0.13),
|
||||
0 2px 8px rgba(0, 0, 0, 0.35);
|
||||
--sgdb-pick-hover: rgba(255, 255, 255, 0.06);
|
||||
--sgdb-code-bg: oklch(18% 0.012 258);
|
||||
--sgdb-code-inset: rgba(0, 0, 0, 0.5);
|
||||
box-shadow: inset 0 1px 0 rgba(255, 255, 255, 0.06),
|
||||
0 16px 40px -16px rgba(0, 0, 0, 0.5);
|
||||
}
|
||||
|
||||
@media (prefers-reduced-motion: reduce) {
|
||||
.sgd-command-builder *, .sgd-command-builder *::before, .sgd-command-builder *::after { scroll-behavior: auto !important; transition: none !important; animation: none !important; }
|
||||
}
|
||||
|
||||
|
||||
.sgd-muted {
|
||||
color: rgb(107, 114, 128);
|
||||
font-size: 0.85rem;
|
||||
@@ -219,6 +876,13 @@ html.dark .sgd-chip,
|
||||
color: rgb(253, 186, 116);
|
||||
}
|
||||
|
||||
html.dark .sgd-model-tags .sgd-chip,
|
||||
[data-theme="dark"] .sgd-model-tags .sgd-chip {
|
||||
border: 0;
|
||||
background: rgba(129, 140, 248, 0.18);
|
||||
color: rgb(199, 210, 254);
|
||||
}
|
||||
|
||||
html.dark .sgd-muted,
|
||||
[data-theme="dark"] .sgd-muted {
|
||||
color: rgb(156, 163, 175);
|
||||
|
||||
@@ -284,7 +284,7 @@ sglang generate \
|
||||
|
||||
MiniMax-H3 supports this path while preserving its required FP32 patch,
|
||||
timestep, and output projections. See the
|
||||
[MiniMax-H3 cookbook](/cookbook/diffusion/MiniMax/MiniMax-H3#7-runtime-feature-recipes)
|
||||
[MiniMax-H3 cookbook](/cookbook/diffusion/MiniMax/MiniMax-H3#7-feature-contracts-and-advanced-recipes)
|
||||
for its distributed serving recipe.
|
||||
|
||||
### MXFP4 Online Quantization
|
||||
@@ -334,7 +334,7 @@ projections take that path.
|
||||
<Warning>
|
||||
`kitchen_int8` is approximate and is not a consistency ground-truth mode.
|
||||
The BF16 path is unchanged when `comfy-kitchen` is not installed. See the
|
||||
[MiniMax-H3 cookbook](/cookbook/diffusion/MiniMax/MiniMax-H3#7-runtime-feature-recipes)
|
||||
[MiniMax-H3 cookbook](/cookbook/diffusion/MiniMax/MiniMax-H3#7-feature-contracts-and-advanced-recipes)
|
||||
for the 24 GB offload recipe, including why `vae` must stay out of
|
||||
`--layerwise-offload-components`.
|
||||
</Warning>
|
||||
|
||||
@@ -21,11 +21,12 @@
|
||||
// role can remove that algorithm from the generated command.
|
||||
|
||||
import { readFileSync, readdirSync } from "node:fs";
|
||||
import { dirname, join, relative } from "node:path";
|
||||
import { basename, dirname, join, relative } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
const SNIPPETS = join(dirname(fileURLToPath(import.meta.url)), "..", "src", "snippets");
|
||||
const CONFIGS = join(SNIPPETS, "configs");
|
||||
const DIFFUSION_COOKBOOK = join(SNIPPETS, "..", "..", "cookbook", "diffusion");
|
||||
const LEGACY_DIMS = ["variants", "quantizations", "strategies", "nodesOptions"];
|
||||
|
||||
const failures = [];
|
||||
@@ -88,11 +89,14 @@ const selectionSpace = (config) => {
|
||||
...(config.matchDims || []),
|
||||
...(config.overlayDims || []),
|
||||
];
|
||||
let space = [{}];
|
||||
let space = [{ ...(config.commandBuilder?.defaultSelection || {}) }];
|
||||
for (const d of dims) {
|
||||
const next = [];
|
||||
const options = d.kind === "number"
|
||||
? [...new Set([d.default, d.min, d.max])].map((id) => ({ id }))
|
||||
: (d.options || []);
|
||||
for (const partial of space) {
|
||||
for (const opt of (d.options || [])) next.push({ ...partial, [d.id]: opt.id });
|
||||
for (const opt of options) next.push({ ...partial, [d.id]: opt.id });
|
||||
}
|
||||
space = next.length ? next : space;
|
||||
if (space.length > 20000) return space.slice(0, 20000); // cheap blow-up guard
|
||||
@@ -154,11 +158,144 @@ for (const path of walk(CONFIGS)) {
|
||||
|
||||
for (const dim of (config.overlayDims || [])) {
|
||||
const ids = (dim.options || []).map((o) => o.id);
|
||||
if (dim.default !== undefined && !ids.includes(dim.default)) {
|
||||
if (dim.kind === "number") {
|
||||
if (!Number.isInteger(dim.min) || !Number.isInteger(dim.max) || dim.min > dim.max) {
|
||||
fail(where, `overlayDims.${dim.id} has invalid numeric bounds`);
|
||||
}
|
||||
if (!Number.isInteger(dim.default) || dim.default < dim.min || dim.default > dim.max) {
|
||||
fail(where, `overlayDims.${dim.id}.default is outside [${dim.min}, ${dim.max}]`);
|
||||
}
|
||||
} else if (dim.default !== undefined && !ids.includes(dim.default)) {
|
||||
fail(where, `overlayDims.${dim.id}.default="${dim.default}" is not one of [${ids}]`);
|
||||
}
|
||||
}
|
||||
|
||||
const builder = config.commandBuilder;
|
||||
if (builder) {
|
||||
const scopes = new Set(["base", "serve", "request"]);
|
||||
for (const dim of (config.overlayDims || [])) {
|
||||
if (!scopes.has(dim.scope)) {
|
||||
fail(where, `builder dimension ${dim.id} has invalid scope "${dim.scope}"`);
|
||||
}
|
||||
}
|
||||
if (!builder.defaultSelection || typeof builder.defaultSelection !== "object") {
|
||||
fail(where, "commandBuilder.defaultSelection is required");
|
||||
}
|
||||
if (typeof builder.resource?.autoTopology !== "function") {
|
||||
fail(where, "commandBuilder.resource.autoTopology must be a function");
|
||||
}
|
||||
if (typeof builder.resource?.validateTopology !== "function") {
|
||||
fail(where, "commandBuilder.resource.validateTopology must be a function");
|
||||
}
|
||||
if (typeof builder.resolveDeployment !== "function") {
|
||||
fail(where, "commandBuilder.resolveDeployment must be a function");
|
||||
}
|
||||
for (const [key, bounds] of Object.entries(builder.resource?.limits || {})) {
|
||||
if (!Number.isInteger(bounds.min) || !Number.isInteger(bounds.max) || bounds.min > bounds.max) {
|
||||
fail(where, `commandBuilder.resource.limits.${key} is invalid`);
|
||||
}
|
||||
const value = builder.defaultSelection?.[key];
|
||||
if (!Number.isInteger(value) || value < bounds.min || value > bounds.max) {
|
||||
fail(where, `commandBuilder.defaultSelection.${key} is outside its bounds`);
|
||||
}
|
||||
}
|
||||
|
||||
const selectionOf = (extra = {}) => {
|
||||
const defaults = { ...(builder.defaultSelection || {}) };
|
||||
for (const dim of (config.overlayDims || [])) defaults[dim.id] = dim.default;
|
||||
return { ...defaults, ...extra };
|
||||
};
|
||||
const validateResolved = (selection, label, expectVerified = false) => {
|
||||
let resolved;
|
||||
try {
|
||||
resolved = builder.resolveDeployment(selection);
|
||||
} catch (e) {
|
||||
fail(where, `${label} resolver throws: ${e.message}`);
|
||||
return null;
|
||||
}
|
||||
if (!resolved || !Array.isArray(resolved.flags) || !resolved.builder) {
|
||||
fail(where, `${label} resolver must return a cell with flags and builder metadata`);
|
||||
return resolved;
|
||||
}
|
||||
if (!Array.isArray(resolved.builder.errors) || !Array.isArray(resolved.builder.warnings)) {
|
||||
fail(where, `${label} resolver errors/warnings must be arrays`);
|
||||
}
|
||||
if (expectVerified && (resolved.builder.errors?.length || resolved.builder.verification?.serve !== "verified")) {
|
||||
fail(where, `${label} is declared verified but resolved as ${resolved.builder.verification?.serve || "invalid"}`);
|
||||
}
|
||||
|
||||
let flags = [...resolved.flags];
|
||||
for (const dim of (config.overlayDims || [])) {
|
||||
const option = (dim.options || []).find((entry) => entry.id === selection[dim.id]);
|
||||
if (!option) continue;
|
||||
const strip = typeof option.stripPrefixes === "function"
|
||||
? option.stripPrefixes(selection) : (option.stripPrefixes || []);
|
||||
if (strip.length) flags = flags.filter((flag) => !strip.includes(flag.split(/[\s=]/)[0]));
|
||||
const extra = typeof option.flags === "function" ? option.flags(selection) : option.flags;
|
||||
flags.push(...(extra || []));
|
||||
}
|
||||
const families = flags.map((flag) => flag.split(/[\s=]/)[0]);
|
||||
const duplicate = families.find((family, index) => families.indexOf(family) !== index);
|
||||
if (duplicate) fail(where, `${label} emits duplicate flag family ${duplicate}`);
|
||||
return resolved;
|
||||
};
|
||||
|
||||
validateResolved(selectionOf(), "commandBuilder default");
|
||||
const recipeIds = new Set();
|
||||
const recipeSignatures = new Set();
|
||||
for (const [index, recipe] of (builder.resource?.verifiedRecipes || []).entries()) {
|
||||
if (!recipe.id || recipeIds.has(recipe.id)) fail(where, `verifiedRecipes[${index}] has a duplicate/missing id`);
|
||||
recipeIds.add(recipe.id);
|
||||
const signature = [recipe.hw, recipe.nodes, recipe.gpus_per_node, recipe.placement,
|
||||
recipe.tp_size, recipe.ulysses_degree, recipe.ring_degree].join("|");
|
||||
if (recipeSignatures.has(signature)) fail(where, `verifiedRecipes[${index}] duplicates ${signature}`);
|
||||
recipeSignatures.add(signature);
|
||||
const selection = selectionOf({ ...recipe, topology_mode: "auto" });
|
||||
const topology = builder.resource.autoTopology(selection);
|
||||
const errors = builder.resource.validateTopology(selection, topology);
|
||||
if (!Array.isArray(errors) || errors.length) {
|
||||
fail(where, `verifiedRecipes[${index}] fails topology validation: ${(errors || []).join("; ")}`);
|
||||
}
|
||||
validateResolved(selection, `verifiedRecipes[${index}]`, true);
|
||||
}
|
||||
|
||||
// H3's architectural contract is important enough to pin directly: exact
|
||||
// platform recipes, legal custom admission, and each invalidity family.
|
||||
if (config.modelName === "MiniMax-H3") {
|
||||
const checkH3 = (label, extra, expected, verified = true) => {
|
||||
const selection = selectionOf(extra);
|
||||
const resolved = validateResolved(selection, `H3 ${label}`, verified);
|
||||
if (!resolved) return;
|
||||
for (const [key, value] of Object.entries(expected)) {
|
||||
if (resolved.builder.topology?.[key] !== value) {
|
||||
fail(where, `H3 ${label} topology.${key}=${resolved.builder.topology?.[key]}, expected ${value}`);
|
||||
}
|
||||
}
|
||||
};
|
||||
checkH3("B200 1x8", { hw: "b200", nodes: 1, gpus_per_node: 8, placement: "resident" }, { tp_size: 1, ulysses_degree: 8, ring_degree: 1 });
|
||||
checkH3("H100 1x4", { hw: "h100", nodes: 1, gpus_per_node: 4, placement: "resident" }, { tp_size: 2, ulysses_degree: 2, ring_degree: 1 });
|
||||
checkH3("H200 2x8", { hw: "h200", nodes: 2, gpus_per_node: 8, placement: "resident" }, { tp_size: 1, ulysses_degree: 8, ring_degree: 2 });
|
||||
for (const hw of ["mi300x", "mi355x"]) {
|
||||
for (const count of [1, 2, 4, 8]) {
|
||||
checkH3(`${hw} 1x${count}`, { hw, nodes: 1, gpus_per_node: count, placement: "resident" }, { tp_size: 1, ulysses_degree: count, ring_degree: 1 });
|
||||
}
|
||||
}
|
||||
const custom = validateResolved(selectionOf({ hw: "b200", nodes: 1, gpus_per_node: 2, placement: "resident" }), "H3 legal custom");
|
||||
if (custom?.builder.errors?.length || custom?.builder.verification?.serve !== "unverified") {
|
||||
fail(where, "H3 legal custom topology must be copyable and Unverified");
|
||||
}
|
||||
for (const [label, extra] of [
|
||||
["3 GPU", { hw: "h200", nodes: 1, gpus_per_node: 3, placement: "resident" }],
|
||||
["TP3", { hw: "h100", nodes: 1, gpus_per_node: 4, placement: "resident", topology_mode: "manual", tp_size: 3, ulysses_degree: 1, ring_degree: 1 }],
|
||||
["head divisibility", { hw: "h200", nodes: 2, gpus_per_node: 8, placement: "resident", topology_mode: "manual", tp_size: 2, ulysses_degree: 8, ring_degree: 1 }],
|
||||
["sequence alignment", { hw: "mi300x", nodes: 2, gpus_per_node: 3, placement: "resident", topology_mode: "manual", tp_size: 1, ulysses_degree: 2, ring_degree: 3 }],
|
||||
]) {
|
||||
const resolved = validateResolved(selectionOf(extra), `H3 invalid ${label}`);
|
||||
if (!resolved?.builder.errors?.length) fail(where, `H3 invalid ${label} was not rejected`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Predicates and flag builders must survive every reachable selection.
|
||||
const space = selectionSpace(config);
|
||||
const probe = (fn, label) => {
|
||||
@@ -185,10 +322,13 @@ for (const path of walk(CONFIGS)) {
|
||||
}
|
||||
for (const dim of [...(config.matchDims || []), ...(config.overlayDims || [])]) {
|
||||
if (typeof dim.showWhen === "function") probe(dim.showWhen, `${dim.id}.showWhen`);
|
||||
if (typeof dim.verifiedWhen === "function") probe(dim.verifiedWhen, `${dim.id}.verifiedWhen`);
|
||||
for (const opt of (dim.options || [])) {
|
||||
const tag = `${dim.id}.${opt.id}`;
|
||||
if (typeof opt.showWhen === "function") probe(opt.showWhen, `${tag}.showWhen`);
|
||||
if (typeof opt.disabled === "function") probe(opt.disabled, `${tag}.disabled`);
|
||||
if (typeof opt.soft === "function") probe(opt.soft, `${tag}.soft`);
|
||||
if (typeof opt.verifiedWhen === "function") probe(opt.verifiedWhen, `${tag}.verifiedWhen`);
|
||||
for (const key of ["flags", "env", "hints"]) {
|
||||
if (typeof opt[key] !== "function") continue;
|
||||
probe((sel) => {
|
||||
@@ -231,6 +371,81 @@ for (const path of walk(CONFIGS)) {
|
||||
}
|
||||
}
|
||||
|
||||
// ----------------------------------------------------- Diffusion page opening
|
||||
// Keep the first screen consistent across model pages. This check intentionally
|
||||
// guards structure, not editorial judgment; the authoring skill carries the
|
||||
// capability/strength/boundary rubric that cannot be reduced to keywords.
|
||||
const walkMdx = (dir) => readdirSync(dir, { withFileTypes: true }).flatMap((e) =>
|
||||
e.isDirectory() ? walkMdx(join(dir, e.name))
|
||||
: (e.name.endsWith(".mdx") ? [join(dir, e.name)] : []));
|
||||
|
||||
for (const path of walkMdx(DIFFUSION_COOKBOOK)) {
|
||||
if (["README.mdx", "intro.mdx"].includes(basename(path))) continue;
|
||||
|
||||
const where = relative(join(SNIPPETS, "..", ".."), path);
|
||||
const src = readFileSync(path, "utf8");
|
||||
const quickStart = /^## 1\. Quick start\s*$/m.exec(src);
|
||||
const legacyIntroduction = /^## 1\. Model Introduction\s*$/m.exec(src);
|
||||
const sectionOne = quickStart || legacyIntroduction;
|
||||
if (!sectionOne) {
|
||||
fail(where, "missing `## 1. Quick start` or legacy `## 1. Model Introduction` heading");
|
||||
continue;
|
||||
}
|
||||
|
||||
const capabilityHeading = quickStart
|
||||
? /^## 2\. Model capabilities\s*$/m.exec(src)
|
||||
: legacyIntroduction;
|
||||
if (!capabilityHeading) {
|
||||
fail(where, "Quick start pages need an exact `## 2. Model capabilities` heading");
|
||||
continue;
|
||||
}
|
||||
|
||||
const importPattern = /import\s+\{\s*DiffusionModelTags\s*\}\s+from\s+['"]\/src\/snippets\/diffusion\/model-tags\.jsx['"]/;
|
||||
if (!importPattern.test(src.slice(0, sectionOne.index))) {
|
||||
fail(where, "missing the shared DiffusionModelTags import before section 1");
|
||||
}
|
||||
|
||||
const tagMatch = /<DiffusionModelTags\s+tags=\{\[([^\]]+)]}\s*\/>/.exec(src.slice(0, sectionOne.index));
|
||||
if (!tagMatch) {
|
||||
fail(where, "missing `<DiffusionModelTags tags={[...]} />` before section 1");
|
||||
} else {
|
||||
const tags = [...tagMatch[1].matchAll(/["']([^"']+)["']/g)].map((m) => m[1].trim());
|
||||
if (tags.length < 4 || tags.length > 6) {
|
||||
fail(where, `tag widget has ${tags.length} tags; expected 4–6`);
|
||||
}
|
||||
if (tags.some((tag) => !tag)) fail(where, "tag widget contains an empty tag");
|
||||
}
|
||||
|
||||
if (quickStart) {
|
||||
const quickStartBody = src.slice(quickStart.index + quickStart[0].length, capabilityHeading.index);
|
||||
if (!quickStartBody.includes("<Deployment config={config} />")) {
|
||||
fail(where, "Quick start must render the command builder before model capabilities");
|
||||
}
|
||||
if (!quickStartBody.includes('uv pip install "sglang[diffusion]"')) {
|
||||
fail(where, "Quick start must include the diffusion installation command");
|
||||
}
|
||||
}
|
||||
|
||||
const introStart = capabilityHeading.index + capabilityHeading[0].length;
|
||||
const rest = src.slice(introStart);
|
||||
const boundaries = ["\n|", "\n<Warning", "\n<Note", quickStart ? "\n## 3" : "\n## 2"]
|
||||
.map((marker) => rest.indexOf(marker))
|
||||
.filter((i) => i >= 0);
|
||||
const lead = rest.slice(0, boundaries.length ? Math.min(...boundaries) : rest.length).trim();
|
||||
const paragraphs = lead.split(/\n\s*\n/).map((p) => p.trim()).filter(Boolean);
|
||||
if (paragraphs.length < 2) {
|
||||
fail(where, "model capability introduction needs at least two lead paragraphs");
|
||||
}
|
||||
const prose = lead
|
||||
.replace(/\[([^\]]+)]\([^)]+\)/g, "$1")
|
||||
.replace(/`[^`]+`/g, "value")
|
||||
.replace(/[*_#]/g, " ");
|
||||
const words = prose.match(/[A-Za-z0-9][A-Za-z0-9+./@–—-]*/g) || [];
|
||||
if (words.length < 45 || words.length > 180) {
|
||||
fail(where, `lead introduction has ${words.length} words; expected 45–180`);
|
||||
}
|
||||
}
|
||||
|
||||
if (failures.length) {
|
||||
console.error(`FAIL (${failures.length})`);
|
||||
for (const f of failures) console.error(" - " + f);
|
||||
|
||||
@@ -18,14 +18,33 @@
|
||||
// `single` or `multi-N` → --nnodes N)
|
||||
// matchDims optional — replaces the legacy four. {id, title, options}[]
|
||||
// where each option is {id, label, showWhen?(sel), disabled?,
|
||||
// disableReason?}. `hw` is always the implicit first dim.
|
||||
// Cells are then keyed on (hw × <these ids>).
|
||||
// disableReason?, soft?, softReason?}. `hw` is always the
|
||||
// implicit first dim. Cells are then keyed on (hw × <these
|
||||
// ids>). `disabled` is for combinations that cannot work;
|
||||
// an option that runs but sits outside the verified matrix
|
||||
// should declare `soft` instead — it stays selectable and
|
||||
// announces itself as unverified (tooltip + in-cell note).
|
||||
// Blocked options flash their disableReason under the row
|
||||
// when tapped, so the reason also reaches touch readers.
|
||||
// overlayDims optional — rows that do NOT participate in cell lookup; the
|
||||
// picked option layers onto the matched cell, so an orthogonal
|
||||
// knob does not multiply the cell count. Same option shape plus
|
||||
// `flags` / `env` / `hints` (each a literal array or a function
|
||||
// of the whole selection), and a row-level `default` / `showWhen`.
|
||||
// `hints` render as `# ...` lines above the command.
|
||||
// Builder-aware dimensions may additionally declare
|
||||
// `scope: "base" | "serve" | "request"`, `description`,
|
||||
// `quality`, `verifiedWhen`, and `learnMore`. Legacy configs
|
||||
// omit these fields and keep the original renderer.
|
||||
// commandBuilder optional — opts this config into the responsive diffusion
|
||||
// builder while reusing this engine's overlay composition and
|
||||
// command rendering. Shape:
|
||||
// {defaultSelection, resource: {limits, verifiedRecipes,
|
||||
// autoTopology(sel), validateTopology(sel)},
|
||||
// resolveDeployment(sel)}. The resolver returns a cell plus
|
||||
// `builder` metadata (topologySummary, errors, warnings,
|
||||
// verification, resolvedSettings). UI-only scope/expand and
|
||||
// local head-address/rank state never enter the URL hash.
|
||||
// cells {match, verified?, verificationStatus?, nnodes?, warn?, redirect?,
|
||||
// env, flags}[] — one per
|
||||
// (hw × match dims); env/flags are flat literals, only
|
||||
@@ -474,6 +493,7 @@ export const Deployment = ({ config, benchmarks }) => {
|
||||
options: d.options || config[d.optionsKey] || [],
|
||||
}));
|
||||
const overlayDimSpecs = config.overlayDims || [];
|
||||
const commandBuilder = config.commandBuilder || null;
|
||||
// DIMENSIONS is ordered by priority — higher-index dims adapt to lower-index
|
||||
// picks, never the reverse. Drives the grey-out/snap logic below.
|
||||
const DIMENSIONS = ["hw", ...matchDimSpecs.map((d) => d.id)];
|
||||
@@ -545,6 +565,11 @@ export const Deployment = ({ config, benchmarks }) => {
|
||||
return out;
|
||||
};
|
||||
// ==== end MIRROR ====
|
||||
// `soft` marks an option that is plausible but outside the verified matrix:
|
||||
// it stays selectable (the status badge reports verification separately),
|
||||
// where `disabled` is reserved for combinations that cannot work at all.
|
||||
const optionSoft = (opt, sel) =>
|
||||
typeof opt.soft === "function" ? opt.soft(sel) : !!opt.soft;
|
||||
const findCell = (cells, sel) =>
|
||||
cells.find((c) => DIMENSIONS.every((d) => c.match[d] === sel[d]));
|
||||
|
||||
@@ -680,6 +705,8 @@ export const Deployment = ({ config, benchmarks }) => {
|
||||
// dim it is a property of the cell itself (`nnodes`), since the deployment
|
||||
// shape is then fixed by the hardware rather than picked by the reader.
|
||||
const parseNnodes = (id) => {
|
||||
if (Number.isInteger(id)) return id;
|
||||
if (/^\d+$/.test(id || "")) return parseInt(id, 10);
|
||||
if (id === "single") return 1;
|
||||
const m = /^multi-(\d+)$/.exec(id || "");
|
||||
return m ? parseInt(m[1], 10) : 1;
|
||||
@@ -709,12 +736,14 @@ export const Deployment = ({ config, benchmarks }) => {
|
||||
if (multinode) {
|
||||
// Insert the multi-node trio after the last parallelism flag,
|
||||
// falling back to right after --model-path.
|
||||
const PARALLELISM_ANCHORS = ["--enable-dp-attention", "--dp", "--tp-size", "--tp"];
|
||||
let i = -1;
|
||||
for (const anchor of PARALLELISM_ANCHORS) {
|
||||
i = flags.findIndex((f) => f.split(/[\s=]/)[0] === anchor);
|
||||
if (i !== -1) break;
|
||||
}
|
||||
const PARALLELISM_ANCHORS = new Set([
|
||||
"--enable-dp-attention", "--dp", "--tp-size", "--tp",
|
||||
"--sp-degree", "--ulysses-degree", "--ring-degree",
|
||||
]);
|
||||
let i = flags.reduce(
|
||||
(last, flag, index) => PARALLELISM_ANCHORS.has(flag.split(/[\s=]/)[0]) ? index : last,
|
||||
-1,
|
||||
);
|
||||
if (i === -1) i = flags.findIndex((f) => f.startsWith("--model-path"));
|
||||
flags.splice(i + 1, 0,
|
||||
`--nnodes ${nnodes}`,
|
||||
@@ -1096,7 +1125,7 @@ export const Deployment = ({ config, benchmarks }) => {
|
||||
// verified cell first). Overlay dims seed from their own `default`, or the
|
||||
// first option, since no cell carries them.
|
||||
const initialSelectionFromCells = () => {
|
||||
const first = config.cells[0];
|
||||
const first = (config.cells || [])[0];
|
||||
const sel = Object.fromEntries(
|
||||
DIMENSIONS.map((d) => [d, first ? first.match[d] : ""]),
|
||||
);
|
||||
@@ -1104,7 +1133,48 @@ export const Deployment = ({ config, benchmarks }) => {
|
||||
const opts = spec.options || [];
|
||||
sel[spec.id] = spec.default ?? (opts[0] && opts[0].id) ?? "";
|
||||
}
|
||||
return sel;
|
||||
if (!commandBuilder) return sel;
|
||||
return {
|
||||
...sel,
|
||||
hw: commandBuilder.defaultSelection?.hw || config.supportedHardware?.[0] || "",
|
||||
...(commandBuilder.defaultSelection || {}),
|
||||
};
|
||||
};
|
||||
|
||||
// Builder hashes are semantic rather than cell ids. Keep known dimension
|
||||
// values, clamp bounded resources, and discard stale topology overrides.
|
||||
// UI state (active scope, expanded cards, head IP, node rank) is deliberately
|
||||
// absent from this object, so shared links stay portable and credential-free.
|
||||
const normalizeBuilderSelection = (parsed) => {
|
||||
const out = { ...initialSelectionFromCells(), ...parsed };
|
||||
if (!(config.supportedHardware || []).includes(out.hw)) {
|
||||
out.hw = commandBuilder.defaultSelection?.hw || config.supportedHardware?.[0] || "";
|
||||
}
|
||||
for (const spec of overlayDimSpecs) {
|
||||
if (spec.kind === "number") {
|
||||
const value = Number.parseInt(out[spec.id], 10);
|
||||
out[spec.id] = Math.min(
|
||||
spec.max,
|
||||
Math.max(spec.min, Number.isFinite(value) ? value : Number(spec.default ?? spec.min)),
|
||||
);
|
||||
continue;
|
||||
}
|
||||
const options = spec.options || [];
|
||||
if (!options.some((option) => option.id === out[spec.id])) {
|
||||
out[spec.id] = spec.default ?? options[0]?.id ?? "";
|
||||
}
|
||||
}
|
||||
for (const [key, bounds] of Object.entries(commandBuilder.resource?.limits || {})) {
|
||||
const fallback = Number(commandBuilder.defaultSelection?.[key] ?? bounds.min ?? 1);
|
||||
const value = Number.parseInt(out[key], 10);
|
||||
out[key] = Math.min(bounds.max, Math.max(bounds.min, Number.isFinite(value) ? value : fallback));
|
||||
}
|
||||
for (const key of ["tp_size", "ulysses_degree", "ring_degree"]) {
|
||||
const value = Number.parseInt(out[key], 10);
|
||||
out[key] = Number.isFinite(value) && value > 0 ? value : 1;
|
||||
}
|
||||
out.topology_mode = out.topology_mode === "manual" ? "manual" : "auto";
|
||||
return out;
|
||||
};
|
||||
|
||||
const placeholderDefaults = (schema) => {
|
||||
@@ -1164,8 +1234,13 @@ export const Deployment = ({ config, benchmarks }) => {
|
||||
if (key in parsed) { parsed[key] = value; touched = true; }
|
||||
});
|
||||
if (!touched) return;
|
||||
// Snap to a real cell if the hash named an impossible combo (stale link).
|
||||
setSel(validateSelection(config.cells, parsed));
|
||||
// Cell configs snap to a real recipe; builders normalize their semantic
|
||||
// resource state without forcing a custom-but-valid topology to a preset.
|
||||
setSel(
|
||||
commandBuilder
|
||||
? normalizeBuilderSelection(parsed)
|
||||
: validateSelection(config.cells, parsed),
|
||||
);
|
||||
const historyState = window.history.state;
|
||||
const isInternalHash =
|
||||
historyState &&
|
||||
@@ -1225,6 +1300,25 @@ export const Deployment = ({ config, benchmarks }) => {
|
||||
: config.runModes;
|
||||
const runModes = configuredRunModes || ["python", "docker"];
|
||||
const [runMode, setRunMode] = useState(runModes[0]); // "python" | "docker"
|
||||
const [builderScope, setBuilderScope] = useState("base");
|
||||
const [builderServerSetting, setBuilderServerSetting] = useState(null);
|
||||
const [builderAdvanced, setBuilderAdvanced] = useState(false);
|
||||
const [serveExpanded, setServeExpanded] = useState(false);
|
||||
const [requestExpanded, setRequestExpanded] = useState(false);
|
||||
const [builderHeadAddress, setBuilderHeadAddress] = useState("<head-node-ip>");
|
||||
const [builderNodeRank, setBuilderNodeRank] = useState(0);
|
||||
// Tapping a disabled option surfaces its reason under the row — hover-only
|
||||
// tooltips never reach touch readers. {dim, reason}; each note clears only
|
||||
// itself, so a newer note is never cut short by an older timer.
|
||||
const [blockedNote, setBlockedNote] = useState(null);
|
||||
const flashBlockedNote = (dim, reason) => {
|
||||
const note = { dim, reason };
|
||||
setBlockedNote(note);
|
||||
setTimeout(() => setBlockedNote((cur) => (cur === note ? null : cur)), 4000);
|
||||
};
|
||||
useEffect(() => {
|
||||
if (builderNodeRank >= Number(sel.nodes || 1)) setBuilderNodeRank(0);
|
||||
}, [sel.nodes, builderNodeRank]);
|
||||
const hasRunMode = runModes.includes(runMode);
|
||||
const fallbackRunMode = runModes[0];
|
||||
const activeRunMode = hasRunMode ? runMode : fallbackRunMode;
|
||||
@@ -1248,7 +1342,10 @@ export const Deployment = ({ config, benchmarks }) => {
|
||||
|
||||
// ==== 5. Derived values ====
|
||||
const s = makeStyles(isDark);
|
||||
const cell = findCell(config.cells, sel);
|
||||
const cell = commandBuilder
|
||||
? commandBuilder.resolveDeployment(sel)
|
||||
: findCell(config.cells, sel);
|
||||
const builderMeta = (cell && cell.builder) || {};
|
||||
const verifyStatus = cellVerifyStatus(cell, sel);
|
||||
// Pin the calculator-computed ratio into the rendered command (before the
|
||||
// host/port tail); cells themselves stay ratio-free.
|
||||
@@ -1262,7 +1359,14 @@ export const Deployment = ({ config, benchmarks }) => {
|
||||
else flags.push(line);
|
||||
return { ...cell, flags };
|
||||
})();
|
||||
const command = renderCommand(cellWithRatio, sel, env, activeRunMode);
|
||||
const commandEnv = commandBuilder
|
||||
? {
|
||||
...env,
|
||||
NODE_RANK: String(builderNodeRank),
|
||||
NODE0_IP: builderHeadAddress || "<head-node-ip>",
|
||||
}
|
||||
: env;
|
||||
const command = renderCommand(cellWithRatio, sel, commandEnv, activeRunMode);
|
||||
// Speculative-decoding hint on the EFFECTIVE flags — speculation can arrive via
|
||||
// the Spec Decode overlay as well as the cell. SGLang resets
|
||||
// --max-running-requests to 48 when spec is on and it's unset; verified for both
|
||||
@@ -1347,7 +1451,8 @@ export const Deployment = ({ config, benchmarks }) => {
|
||||
const isEnabled = (dim, value) => {
|
||||
const opt = findOption(dim, value);
|
||||
if (opt && optionDisabled(opt, sel)) return false;
|
||||
return isOverlayDim(dim) || isOptionAvailable(config.cells, sel, dim, value);
|
||||
if (commandBuilder && dim === "hw") return true;
|
||||
return isOverlayDim(dim) || isOptionAvailable(config.cells || [], sel, dim, value);
|
||||
};
|
||||
|
||||
// Switching a match dim can hide the option a dependent row currently holds
|
||||
@@ -1365,7 +1470,46 @@ export const Deployment = ({ config, benchmarks }) => {
|
||||
return out;
|
||||
};
|
||||
|
||||
const recommendedBuilderRecipe = (hw) => {
|
||||
const recipes = commandBuilder.resource?.verifiedRecipes || [];
|
||||
return recipes.find((entry) => entry.hw === hw && entry.default)
|
||||
|| recipes.find((entry) => entry.hw === hw);
|
||||
};
|
||||
|
||||
const handleSelect = (dim, value) => {
|
||||
if (commandBuilder) {
|
||||
setSel((prev) => {
|
||||
let next = { ...prev, [dim]: value };
|
||||
if (dim === "hw") {
|
||||
const currentRecipe = recommendedBuilderRecipe(prev.hw);
|
||||
const nextRecipe = recommendedBuilderRecipe(value);
|
||||
const resourcesFollowPlatformDefault = !!currentRecipe
|
||||
&& Number(prev.nodes) === Number(currentRecipe.nodes)
|
||||
&& Number(prev.gpus_per_node) === Number(currentRecipe.gpus_per_node);
|
||||
next = {
|
||||
...next,
|
||||
nodes: resourcesFollowPlatformDefault
|
||||
? (nextRecipe?.nodes ?? next.nodes)
|
||||
: next.nodes,
|
||||
gpus_per_node: resourcesFollowPlatformDefault
|
||||
? (nextRecipe?.gpus_per_node ?? next.gpus_per_node)
|
||||
: next.gpus_per_node,
|
||||
topology_mode: "auto",
|
||||
tp_size: resourcesFollowPlatformDefault
|
||||
? (nextRecipe?.tp_size ?? 1)
|
||||
: next.tp_size,
|
||||
ulysses_degree: resourcesFollowPlatformDefault
|
||||
? (nextRecipe?.ulysses_degree ?? 1)
|
||||
: next.ulysses_degree,
|
||||
ring_degree: resourcesFollowPlatformDefault
|
||||
? (nextRecipe?.ring_degree ?? 1)
|
||||
: next.ring_degree,
|
||||
};
|
||||
}
|
||||
return reseatHiddenPicks(normalizeBuilderSelection(next));
|
||||
});
|
||||
return;
|
||||
}
|
||||
setSel((prev) =>
|
||||
reseatHiddenPicks(
|
||||
isOverlayDim(dim)
|
||||
@@ -1375,6 +1519,70 @@ export const Deployment = ({ config, benchmarks }) => {
|
||||
);
|
||||
};
|
||||
|
||||
const commitBuilderNumber = (event, currentValue, bounds, commit) => {
|
||||
const parsed = Number(event.currentTarget.value);
|
||||
if (!Number.isInteger(parsed)) {
|
||||
event.currentTarget.value = String(currentValue);
|
||||
return;
|
||||
}
|
||||
const value = Math.min(bounds.max, Math.max(bounds.min, parsed));
|
||||
event.currentTarget.value = String(value);
|
||||
commit(value);
|
||||
};
|
||||
|
||||
const renderBuilderNumberInput = ({ identity, value, min, max, label, onCommit }) => (
|
||||
<input
|
||||
key={identity}
|
||||
type="number"
|
||||
inputMode="numeric"
|
||||
min={min}
|
||||
max={max}
|
||||
step="1"
|
||||
defaultValue={value}
|
||||
aria-label={label}
|
||||
onFocus={(event) => event.currentTarget.select()}
|
||||
onBlur={(event) => commitBuilderNumber(
|
||||
event,
|
||||
value,
|
||||
{ min, max },
|
||||
onCommit,
|
||||
)}
|
||||
onKeyDown={(event) => {
|
||||
if (event.key === "Enter") event.currentTarget.blur();
|
||||
}}
|
||||
/>
|
||||
);
|
||||
|
||||
const updateBuilderResource = (key, delta) => {
|
||||
if (!commandBuilder) return;
|
||||
const bounds = commandBuilder.resource?.limits?.[key] || { min: 1, max: 8 };
|
||||
setSel((prev) => {
|
||||
const value = Math.min(bounds.max, Math.max(bounds.min, Number(prev[key]) + delta));
|
||||
return normalizeBuilderSelection({ ...prev, [key]: value, topology_mode: "auto" });
|
||||
});
|
||||
};
|
||||
|
||||
const setBuilderResource = (key, rawValue) => {
|
||||
if (!commandBuilder) return;
|
||||
const value = Number.parseInt(rawValue, 10);
|
||||
if (!Number.isFinite(value)) return;
|
||||
const bounds = commandBuilder.resource?.limits?.[key] || { min: 1, max: 8 };
|
||||
setSel((prev) => normalizeBuilderSelection({
|
||||
...prev,
|
||||
[key]: Math.min(bounds.max, Math.max(bounds.min, value)),
|
||||
topology_mode: "auto",
|
||||
}));
|
||||
};
|
||||
|
||||
const editBuilderTopology = (key, value) => {
|
||||
if (!commandBuilder) return;
|
||||
setSel((prev) => normalizeBuilderSelection({
|
||||
...prev,
|
||||
topology_mode: "manual",
|
||||
[key]: Number.parseInt(value, 10) || 1,
|
||||
}));
|
||||
};
|
||||
|
||||
const handleCopy = () => {
|
||||
navigator.clipboard.writeText(command);
|
||||
setCopied(true);
|
||||
@@ -1408,6 +1616,11 @@ export const Deployment = ({ config, benchmarks }) => {
|
||||
return (
|
||||
<label
|
||||
key={item.id}
|
||||
className="sg-command-visualizer-choice"
|
||||
role="radio"
|
||||
aria-checked={checked}
|
||||
aria-disabled={disabled}
|
||||
tabIndex={disabled ? -1 : 0}
|
||||
style={{
|
||||
...s.labelBase,
|
||||
...(checked ? s.checked : {}),
|
||||
@@ -1422,6 +1635,11 @@ export const Deployment = ({ config, benchmarks }) => {
|
||||
if (disabled) { e.preventDefault(); return; }
|
||||
handleSelect(dim, item.id);
|
||||
}}
|
||||
onKeyDown={(e) => {
|
||||
if (disabled || (e.key !== "Enter" && e.key !== " ")) return;
|
||||
e.preventDefault();
|
||||
handleSelect(dim, item.id);
|
||||
}}
|
||||
>
|
||||
<input type="radio" checked={checked} disabled={disabled} readOnly style={{ display: "none" }} />
|
||||
<span>{item.label}</span>
|
||||
@@ -1445,11 +1663,573 @@ export const Deployment = ({ config, benchmarks }) => {
|
||||
|
||||
const maxHwCols = Math.max(...hwGroups.map((x) => x.items.length));
|
||||
|
||||
if (commandBuilder) {
|
||||
const scopeLabel = { base: "Setup", serve: "Server", request: "Request" };
|
||||
const scopedDims = (scope) => overlayDimSpecs.filter((dim) => {
|
||||
if ((dim.scope || "base") !== scope) return false;
|
||||
if (dim.kind === "number") {
|
||||
return typeof dim.showWhen !== "function" || dim.showWhen(sel);
|
||||
}
|
||||
return rowVisible(dim, sel);
|
||||
});
|
||||
const baseDims = scopedDims("base");
|
||||
const serveDims = scopedDims("serve");
|
||||
const requestDims = scopedDims("request");
|
||||
const errors = builderMeta.errors || [];
|
||||
const warnings = builderMeta.warnings || [];
|
||||
const invalid = errors.length > 0;
|
||||
const totalGpus = Number(sel.nodes) * Number(sel.gpus_per_node);
|
||||
const topology = builderMeta.topology || {};
|
||||
const verification = builderMeta.verification || {};
|
||||
const scopeIsVerified = (scope) => scopedDims(scope).every((dim) => {
|
||||
const option = (dim.options || []).find((entry) => entry.id === sel[dim.id]);
|
||||
// A pick whose `soft` predicate is active is by definition outside the
|
||||
// verified matrix — the scope badge must not keep reading Verified.
|
||||
if (option && optionSoft(option, sel)) return false;
|
||||
const predicate = option?.verifiedWhen ?? dim.verifiedWhen;
|
||||
return typeof predicate === "function" ? !!predicate(sel) : predicate !== false;
|
||||
});
|
||||
const serveStatus = invalid
|
||||
? "error"
|
||||
: (scopeIsVerified("serve") ? (verification.serve || verifyStatus) : "unverified");
|
||||
const requestStatus = invalid
|
||||
? "error"
|
||||
: (scopeIsVerified("request") ? (verification.request || verifyStatus) : "unverified");
|
||||
const statusText = (status) => ({
|
||||
verified: "Verified",
|
||||
unverified: "Unverified",
|
||||
"in-progress": "Verification in progress",
|
||||
error: "Invalid configuration",
|
||||
}[status] || "Unverified");
|
||||
const activeServerSetting = serveDims.find((dim) => dim.id === builderServerSetting) || serveDims[0];
|
||||
const selectedOption = (dim) => (dim.options || []).find((option) => option.id === sel[dim.id]);
|
||||
const effectiveSetting = (dim) =>
|
||||
builderMeta.resolvedSettings?.[dim.id]
|
||||
|| selectedOption(dim)?.label
|
||||
|| sel[dim.id]
|
||||
|| "—";
|
||||
const recommendedRecipe = recommendedBuilderRecipe(sel.hw);
|
||||
const recommendedInUse = !!recommendedRecipe
|
||||
&& Number(sel.nodes) === recommendedRecipe.nodes
|
||||
&& Number(sel.gpus_per_node) === recommendedRecipe.gpus_per_node
|
||||
&& sel.topology_mode === "auto"
|
||||
&& ["auto", recommendedRecipe.placement].includes(sel.placement)
|
||||
&& sel.attention === "platform"
|
||||
&& sel.precision === "native"
|
||||
&& ["auto", recommendedRecipe.encoder].includes(sel.encoder)
|
||||
&& sel.execution === "eager";
|
||||
|
||||
const restoreRecommendedRecipe = () => {
|
||||
if (!recommendedRecipe) return;
|
||||
setSel((prev) => reseatHiddenPicks(normalizeBuilderSelection({
|
||||
...prev,
|
||||
nodes: recommendedRecipe.nodes,
|
||||
gpus_per_node: recommendedRecipe.gpus_per_node,
|
||||
topology_mode: "auto",
|
||||
tp_size: recommendedRecipe.tp_size,
|
||||
ulysses_degree: recommendedRecipe.ulysses_degree,
|
||||
ring_degree: recommendedRecipe.ring_degree,
|
||||
placement: recommendedRecipe.placement || "auto",
|
||||
attention: "platform",
|
||||
precision: "native",
|
||||
encoder: recommendedRecipe.encoder || "auto",
|
||||
execution: "eager",
|
||||
})));
|
||||
};
|
||||
|
||||
const renderBuilderChoice = (item, dim) => {
|
||||
const checked = sel[dim.id] === item.id;
|
||||
const disabled = !isEnabled(dim.id, item.id);
|
||||
const soft = !disabled && optionSoft(item, sel);
|
||||
const reason = disabled
|
||||
? item.disableReason || "Not available for this configuration"
|
||||
: soft
|
||||
? item.softReason || "Runs, but this combination is not a verified recipe yet."
|
||||
: "";
|
||||
// aria-disabled instead of the disabled attribute: the control stays
|
||||
// focusable and hoverable, so the reason is reachable by tooltip, by
|
||||
// keyboard, and by the tap-feedback note below the row.
|
||||
return (
|
||||
<button
|
||||
key={item.id}
|
||||
type="button"
|
||||
className="sgd-builder-choice"
|
||||
data-selected={checked ? "true" : "false"}
|
||||
data-blocked={disabled ? "true" : undefined}
|
||||
data-soft={soft ? "true" : undefined}
|
||||
aria-disabled={disabled}
|
||||
aria-pressed={checked}
|
||||
title={reason}
|
||||
onClick={() => {
|
||||
if (disabled) { flashBlockedNote(dim.id, reason); return; }
|
||||
handleSelect(dim.id, item.id);
|
||||
}}
|
||||
>
|
||||
<span className="sgd-builder-choice-dot" aria-hidden="true" />
|
||||
<span>{item.label}</span>
|
||||
{item.subtitle && <small>{item.subtitle}</small>}
|
||||
</button>
|
||||
);
|
||||
};
|
||||
|
||||
const renderBuilderDimension = (dim) => (
|
||||
<section className="sgd-builder-section" key={dim.id}>
|
||||
<div className="sgd-builder-section-heading">
|
||||
<span>{dim.title}</span>
|
||||
{dim.description && <small>{dim.description}</small>}
|
||||
</div>
|
||||
<div className="sgd-builder-choice-grid" data-density={(dim.options || []).length > 5 ? "compact" : "normal"}>
|
||||
{visibleOptions(dim, sel).map((option) => renderBuilderChoice(option, dim))}
|
||||
</div>
|
||||
{blockedNote && blockedNote.dim === dim.id && (
|
||||
<p className="sgd-builder-blocked-note" role="status">{blockedNote.reason}</p>
|
||||
)}
|
||||
</section>
|
||||
);
|
||||
|
||||
const renderStepper = (key, label, detail) => {
|
||||
const bounds = commandBuilder.resource?.limits?.[key] || { min: 1, max: 8 };
|
||||
return (
|
||||
<div className="sgd-builder-stepper-field">
|
||||
<div>
|
||||
<span>{label}</span>
|
||||
{detail && <small>{detail}</small>}
|
||||
</div>
|
||||
<div className="sgd-builder-stepper" aria-label={label}>
|
||||
<button
|
||||
type="button"
|
||||
aria-label={`Decrease ${label}`}
|
||||
disabled={Number(sel[key]) <= bounds.min}
|
||||
onClick={() => updateBuilderResource(key, -1)}
|
||||
>−</button>
|
||||
{renderBuilderNumberInput({
|
||||
identity: `${key}-${sel[key]}`,
|
||||
value: sel[key],
|
||||
min: bounds.min,
|
||||
max: bounds.max,
|
||||
label,
|
||||
onCommit: (value) => setBuilderResource(key, value),
|
||||
})}
|
||||
<button
|
||||
type="button"
|
||||
aria-label={`Increase ${label}`}
|
||||
disabled={Number(sel[key]) >= bounds.max}
|
||||
onClick={() => updateBuilderResource(key, 1)}
|
||||
>+</button>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
};
|
||||
|
||||
const renderBaseScope = () => (
|
||||
<div className="sgd-builder-scope-panel" data-scope="base">
|
||||
{recommendedRecipe && (
|
||||
<section className="sgd-builder-recipe">
|
||||
<div>
|
||||
{/* This is the verified operating point, not sizing advice — a
|
||||
hardware whose validation ran on 8 GPUs is not "recommending"
|
||||
8 over a smaller deployment. */}
|
||||
<span>Verified recipe · {sel.hw.toUpperCase()}</span>
|
||||
<strong>
|
||||
{[
|
||||
`${recommendedRecipe.nodes * recommendedRecipe.gpus_per_node} GPUs`,
|
||||
recommendedRecipe.tp_size > 1 && `TP ${recommendedRecipe.tp_size}`,
|
||||
`Ulysses ${recommendedRecipe.ulysses_degree}`,
|
||||
recommendedRecipe.ring_degree > 1 && `Ring ${recommendedRecipe.ring_degree}`,
|
||||
{ resident: "Resident", fsdp: "FSDP", offload: "Layerwise offload" }[recommendedRecipe.placement],
|
||||
].filter(Boolean).join(" · ")}
|
||||
</strong>
|
||||
</div>
|
||||
<div>
|
||||
{renderStatus("verified")}
|
||||
{recommendedInUse
|
||||
? <small>In use</small>
|
||||
: <button type="button" className="sgd-builder-text-action" onClick={restoreRecommendedRecipe}>Use verified recipe</button>}
|
||||
</div>
|
||||
</section>
|
||||
)}
|
||||
<section className="sgd-builder-section">
|
||||
<div className="sgd-builder-section-heading"><span>Hardware</span></div>
|
||||
<div className="sgd-builder-hardware-grid">
|
||||
{hwGroups.flatMap((group) => group.items).map((item) => {
|
||||
const selected = sel.hw === item.id;
|
||||
return (
|
||||
<button
|
||||
key={item.id}
|
||||
type="button"
|
||||
className="sgd-builder-hardware"
|
||||
data-selected={selected ? "true" : "false"}
|
||||
aria-pressed={selected}
|
||||
onClick={() => handleSelect("hw", item.id)}
|
||||
>
|
||||
<span className="sgd-builder-choice-dot" aria-hidden="true" />
|
||||
<strong>{item.label}</strong>
|
||||
<small>{item.subtitle}</small>
|
||||
</button>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
</section>
|
||||
|
||||
{/* Topology lives with Resources: its summary is the heading's detail
|
||||
line, so the two rows that used to repeat each other are one
|
||||
section. The advanced editor and its messages stay here too. */}
|
||||
<section className="sgd-builder-section">
|
||||
<div className="sgd-builder-section-heading">
|
||||
<span>Resources</span>
|
||||
<small>{builderMeta.topologySummary || "No valid topology"}</small>
|
||||
</div>
|
||||
<div className="sgd-builder-resource-grid">
|
||||
{renderStepper("nodes", "Nodes")}
|
||||
{renderStepper("gpus_per_node", "GPUs / node")}
|
||||
</div>
|
||||
{Number(sel.nodes) > 1 && (
|
||||
<p className="sgd-builder-resource-summary">
|
||||
{sel.nodes} nodes × {sel.gpus_per_node} {sel.hw.toUpperCase()} = {totalGpus} GPUs
|
||||
</p>
|
||||
)}
|
||||
<button
|
||||
type="button"
|
||||
className="sgd-builder-text-action sgd-builder-topology-toggle"
|
||||
aria-expanded={builderAdvanced}
|
||||
onClick={() => setBuilderAdvanced((open) => !open)}
|
||||
>
|
||||
Advanced topology <span aria-hidden="true">{builderAdvanced ? "↗" : "↘"}</span>
|
||||
</button>
|
||||
{builderAdvanced && (
|
||||
<div className="sgd-builder-advanced">
|
||||
<p>Auto uses an exact verified recipe when one exists; manual values are allowed when the model constraints remain valid.</p>
|
||||
<div className="sgd-builder-topology-inputs">
|
||||
{[
|
||||
["tp_size", "Tensor parallel", [1, 2, 4, 8]],
|
||||
["ulysses_degree", "Ulysses", [1, 2, 4, 8, 16]],
|
||||
["ring_degree", "Ring", [1, 2, 4, 8]],
|
||||
].map(([key, label, values]) => (
|
||||
<label key={key}>
|
||||
<span>{label}</span>
|
||||
<select
|
||||
value={sel.topology_mode === "manual" ? sel[key] : (topology[key] || 1)}
|
||||
onChange={(event) => editBuilderTopology(key, event.target.value)}
|
||||
>
|
||||
{values.map((value) => <option value={value} key={value}>{value}</option>)}
|
||||
</select>
|
||||
</label>
|
||||
))}
|
||||
</div>
|
||||
<button
|
||||
type="button"
|
||||
className="sgd-builder-text-action"
|
||||
disabled={sel.topology_mode === "auto"}
|
||||
onClick={() => setSel((prev) => normalizeBuilderSelection({ ...prev, topology_mode: "auto" }))}
|
||||
>Use automatic topology</button>
|
||||
</div>
|
||||
)}
|
||||
{(errors.length > 0 || warnings.length > 0) && (
|
||||
<div className="sgd-builder-messages" data-state={errors.length ? "error" : "warning"}>
|
||||
{(errors.length ? errors : warnings).map((message, index) => <p key={index}>{message}</p>)}
|
||||
</div>
|
||||
)}
|
||||
</section>
|
||||
|
||||
{baseDims.map(renderBuilderDimension)}
|
||||
</div>
|
||||
);
|
||||
|
||||
const renderSettingEditor = (dim, className = "", direct = false) => {
|
||||
if (!dim) return null;
|
||||
const options = visibleOptions(dim, sel);
|
||||
const currentOption = selectedOption(dim);
|
||||
return (
|
||||
<section className={`sgd-builder-context ${className}`} aria-live={direct ? undefined : "polite"}>
|
||||
<div className="sgd-builder-context-heading">
|
||||
<div>
|
||||
<span>{direct ? dim.title : `${dim.title} options`}</span>
|
||||
{dim.description && <p>{dim.description}</p>}
|
||||
</div>
|
||||
{dim.quality && <small>{dim.quality}</small>}
|
||||
</div>
|
||||
{dim.kind === "number" ? (
|
||||
<div className="sgd-builder-request-stepper">
|
||||
<button
|
||||
type="button"
|
||||
aria-label={`Decrease ${dim.title}`}
|
||||
disabled={Number(sel[dim.id]) <= dim.min}
|
||||
onClick={() => setSel((prev) => ({
|
||||
...prev,
|
||||
[dim.id]: Math.max(dim.min, Number(prev[dim.id]) - 1),
|
||||
}))}
|
||||
>−</button>
|
||||
{renderBuilderNumberInput({
|
||||
identity: `${dim.id}-${sel[dim.id]}`,
|
||||
value: sel[dim.id],
|
||||
min: dim.min,
|
||||
max: dim.max,
|
||||
label: dim.title,
|
||||
onCommit: (value) => setSel((prev) => ({ ...prev, [dim.id]: value })),
|
||||
})}
|
||||
<button
|
||||
type="button"
|
||||
aria-label={`Increase ${dim.title}`}
|
||||
disabled={Number(sel[dim.id]) >= dim.max}
|
||||
onClick={() => setSel((prev) => ({
|
||||
...prev,
|
||||
[dim.id]: Math.min(dim.max, Number(prev[dim.id]) + 1),
|
||||
}))}
|
||||
>+</button>
|
||||
<span>{dim.unit || "outputs"}</span>
|
||||
</div>
|
||||
) : (
|
||||
<div className="sgd-builder-context-options">
|
||||
{options.map((option) => renderBuilderChoice(option, dim))}
|
||||
</div>
|
||||
)}
|
||||
{blockedNote && blockedNote.dim === dim.id && (
|
||||
<p className="sgd-builder-blocked-note" role="status">{blockedNote.reason}</p>
|
||||
)}
|
||||
{(currentOption?.description || dim.learnMore) && (
|
||||
<div className="sgd-builder-context-note">
|
||||
{currentOption?.description && <p>{currentOption.description}</p>}
|
||||
{/* Icons name the destination: section lines = an anchor on
|
||||
this page, book = the runtime documentation. */}
|
||||
{dim.learnMore && (
|
||||
<a href={dim.learnMore}>
|
||||
<svg width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" strokeLinecap="round" aria-hidden="true"><line x1="4" y1="6" x2="20" y2="6" /><line x1="4" y1="12" x2="16" y2="12" /><line x1="4" y1="18" x2="11" y2="18" /></svg>
|
||||
Learn more
|
||||
</a>
|
||||
)}
|
||||
{dim.docsHref && (
|
||||
<a href={dim.docsHref}>
|
||||
<svg width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" strokeLinecap="round" strokeLinejoin="round" aria-hidden="true"><path d="M4 19.5A2.5 2.5 0 0 1 6.5 17H20" /><path d="M6.5 2H20v20H6.5A2.5 2.5 0 0 1 4 19.5v-15A2.5 2.5 0 0 1 6.5 2z" /></svg>
|
||||
SGLang docs
|
||||
</a>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
</section>
|
||||
);
|
||||
};
|
||||
|
||||
const renderServerScope = () => (
|
||||
<div className="sgd-builder-scope-panel" data-scope="serve">
|
||||
<div className="sgd-builder-setting-layout">
|
||||
<div className="sgd-builder-setting-list">
|
||||
{serveDims.map((dim) => {
|
||||
const isActive = dim.id === activeServerSetting?.id;
|
||||
const option = selectedOption(dim);
|
||||
const recommended = typeof option?.recommendedWhen === "function"
|
||||
? option.recommendedWhen(sel)
|
||||
: !!option?.recommended;
|
||||
return (
|
||||
<div className="sgd-builder-setting-item" key={dim.id}>
|
||||
<button
|
||||
type="button"
|
||||
className="sgd-builder-setting-row"
|
||||
data-active={isActive ? "true" : "false"}
|
||||
aria-expanded={isActive}
|
||||
onClick={() => setBuilderServerSetting(dim.id)}
|
||||
>
|
||||
<span>{dim.title}</span>
|
||||
<strong>{effectiveSetting(dim)}</strong>
|
||||
{recommended && <small>Recommended</small>}
|
||||
<span aria-hidden="true">{isActive ? "⌄" : "›"}</span>
|
||||
</button>
|
||||
{isActive && renderSettingEditor(dim, "sgd-builder-context--inline")}
|
||||
</div>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
{renderSettingEditor(activeServerSetting, "sgd-builder-context--rail")}
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
|
||||
const renderRequestScope = () => (
|
||||
<div className="sgd-builder-scope-panel sgd-builder-request-direct" data-scope="request">
|
||||
{requestDims.map((dim) => (
|
||||
<div className="sgd-builder-request-setting" key={dim.id}>
|
||||
{renderSettingEditor(dim, "", true)}
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
);
|
||||
|
||||
const renderScopeControls = () => {
|
||||
if (builderScope === "base") return renderBaseScope();
|
||||
if (builderScope === "serve") return renderServerScope();
|
||||
return renderRequestScope();
|
||||
};
|
||||
|
||||
const renderStatus = (status) => (
|
||||
<span className="sgd-builder-status" data-status={status}>
|
||||
<span aria-hidden="true" />{statusText(status)}
|
||||
</span>
|
||||
);
|
||||
|
||||
const renderOutputCard = (type) => {
|
||||
const serve = type === "serve";
|
||||
const text = serve ? command : curlText;
|
||||
const canExpand = text.split("\n").length > 9;
|
||||
const expanded = serve ? serveExpanded : requestExpanded;
|
||||
const setExpanded = serve ? setServeExpanded : setRequestExpanded;
|
||||
const status = serve ? serveStatus : requestStatus;
|
||||
const emphasized = builderScope === "base" || builderScope === type;
|
||||
return (
|
||||
<section
|
||||
className="sgd-builder-output"
|
||||
data-output={type}
|
||||
data-emphasized={emphasized ? "true" : "false"}
|
||||
>
|
||||
<header>
|
||||
<div className="sgd-builder-output-index">{serve ? "1" : "2"}</div>
|
||||
<div className="sgd-builder-output-title">
|
||||
<strong>{serve ? "Serve" : "Request"}</strong>
|
||||
<span>
|
||||
{serve
|
||||
? `${sel.hw.toUpperCase()} · ${activeRunMode === "docker" ? "Docker" : "Python"}`
|
||||
: "cURL"}
|
||||
</span>
|
||||
</div>
|
||||
{renderStatus(status)}
|
||||
</header>
|
||||
{serve && runModes.length > 1 && (
|
||||
<div className="sgd-builder-output-tabs" role="tablist" aria-label="Serve command format">
|
||||
{runModes.map((mode) => (
|
||||
<button
|
||||
type="button"
|
||||
role="tab"
|
||||
aria-selected={activeRunMode === mode}
|
||||
data-selected={activeRunMode === mode ? "true" : "false"}
|
||||
key={mode}
|
||||
onClick={() => setRunMode(mode)}
|
||||
>{mode === "docker" ? "Docker" : "Python"}</button>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
{serve && Number(sel.nodes) > 1 && (
|
||||
<div className="sgd-builder-node-fields">
|
||||
<label>
|
||||
<span>Head address</span>
|
||||
<input value={builderHeadAddress} onChange={(event) => setBuilderHeadAddress(event.target.value)} />
|
||||
</label>
|
||||
<label>
|
||||
<span>Node rank</span>
|
||||
{renderBuilderNumberInput({
|
||||
identity: `node-rank-${builderNodeRank}-${sel.nodes}`,
|
||||
value: builderNodeRank,
|
||||
min: 0,
|
||||
max: Number(sel.nodes) - 1,
|
||||
label: "Node rank",
|
||||
onCommit: setBuilderNodeRank,
|
||||
})}
|
||||
</label>
|
||||
</div>
|
||||
)}
|
||||
<div className="sgd-builder-code">
|
||||
<pre className={expanded ? "is-expanded" : ""}><code>{text}</code></pre>
|
||||
<button
|
||||
type="button"
|
||||
className="sgd-builder-copy"
|
||||
disabled={invalid}
|
||||
aria-label={(serve ? copied : curlCopied) ? "Copied" : "Copy command"}
|
||||
data-copied={(serve ? copied : curlCopied) ? "true" : undefined}
|
||||
onClick={serve ? handleCopy : copyCurl}
|
||||
>
|
||||
<svg className="sgd-builder-copy-glyph" width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="1.9" strokeLinecap="round" strokeLinejoin="round" aria-hidden="true"><rect x="9" y="9" width="12" height="12" rx="2.5" /><path d="M15 5v-.25A2.75 2.75 0 0 0 12.25 2h-7.5A2.75 2.75 0 0 0 2 4.75v7.5A2.75 2.75 0 0 0 4.75 15H5" /></svg>
|
||||
<svg className="sgd-builder-copy-check" width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2.2" strokeLinecap="round" strokeLinejoin="round" aria-hidden="true"><path d="M20 6 9 17l-5-5" /></svg>
|
||||
</button>
|
||||
</div>
|
||||
{invalid && <div className="sgd-builder-output-error">{errors[0]}</div>}
|
||||
<footer>
|
||||
{canExpand && (
|
||||
<button type="button" className="sgd-builder-text-action" onClick={() => setExpanded(!expanded)}>
|
||||
{expanded ? "Collapse" : "Expand"}
|
||||
</button>
|
||||
)}
|
||||
<div>
|
||||
<button type="button" className="sgd-builder-text-action" onClick={() => setModal("env")}>Variables</button>
|
||||
</div>
|
||||
</footer>
|
||||
</section>
|
||||
);
|
||||
};
|
||||
|
||||
return (
|
||||
<section
|
||||
id={DEPLOYMENT_COMPONENT_ID}
|
||||
className="not-prose sg-command-visualizer sgd-command-builder"
|
||||
style={{ scrollMarginTop: "104px" }}
|
||||
aria-label={`${config.modelName} command builder`}
|
||||
>
|
||||
<nav className="sgd-builder-scope-tabs" role="tablist" aria-label="Command builder scope">
|
||||
{["base", "serve", "request"].map((scope) => (
|
||||
<button
|
||||
type="button"
|
||||
role="tab"
|
||||
key={scope}
|
||||
aria-label={scopeLabel[scope]}
|
||||
aria-selected={builderScope === scope}
|
||||
aria-controls={`${DEPLOYMENT_COMPONENT_ID}-controls`}
|
||||
data-active={builderScope === scope ? "true" : "false"}
|
||||
onClick={() => setBuilderScope(scope)}
|
||||
>
|
||||
{scopeLabel[scope]}
|
||||
</button>
|
||||
))}
|
||||
</nav>
|
||||
|
||||
<div className="sgd-builder-main" data-scope={builderScope}>
|
||||
<div
|
||||
id={`${DEPLOYMENT_COMPONENT_ID}-controls`}
|
||||
className="sgd-builder-controls"
|
||||
role="tabpanel"
|
||||
aria-label={`${scopeLabel[builderScope]} settings`}
|
||||
>
|
||||
{renderScopeControls()}
|
||||
</div>
|
||||
<div className="sgd-builder-output-rail">
|
||||
{builderScope !== "request" && renderOutputCard("serve")}
|
||||
{builderScope !== "serve" && renderOutputCard("request")}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{modal === "env" && (
|
||||
<div style={s.modalBackdrop} onClick={() => setModal(null)}>
|
||||
<div style={s.modalBox} onClick={(event) => event.stopPropagation()}>
|
||||
<div style={s.modalHeader}>
|
||||
<div style={s.modalTitle}>Command variables</div>
|
||||
<button style={s.modalCloseBtn} onClick={() => setModal(null)} aria-label="Close">×</button>
|
||||
</div>
|
||||
{["command", "curl"].map((target) => placeholderGroups[target].length > 0 && (
|
||||
<div key={target}>
|
||||
<div style={s.sectionHeading}>{target === "command" ? "Serve" : "Request"}</div>
|
||||
{placeholderGroups[target].map(({ key, label }) => (
|
||||
<div key={key} style={s.formField}>
|
||||
<label style={s.formLabel}>{label}</label>
|
||||
<input
|
||||
style={s.formInput}
|
||||
value={envDraft[key] ?? ""}
|
||||
onChange={(event) => setEnvDraft({ ...envDraft, [key]: event.target.value })}
|
||||
/>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
))}
|
||||
<div style={{ display: "flex", justifyContent: "flex-end", gap: 8, marginTop: 16 }}>
|
||||
<button style={{ ...s.iconButton, padding: "6px 14px" }} onClick={() => setModal(null)}>Cancel</button>
|
||||
<button style={s.primaryBtn} onClick={() => { saveEnv(envDraft); setModal(null); }}>Save</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
</section>
|
||||
);
|
||||
}
|
||||
|
||||
return (
|
||||
<div
|
||||
id={DEPLOYMENT_COMPONENT_ID}
|
||||
style={{ ...s.container, scrollMarginTop: "104px" }}
|
||||
className="not-prose"
|
||||
className="not-prose sg-command-visualizer"
|
||||
>
|
||||
{/* Hardware section (2 vendor rows in one card, equal-width grid) */}
|
||||
<div style={s.cardColumn}>
|
||||
@@ -1499,6 +2279,7 @@ export const Deployment = ({ config, benchmarks }) => {
|
||||
{runModes.map((mode, index) => (
|
||||
<span
|
||||
key={mode}
|
||||
className="sg-command-visualizer-tab"
|
||||
style={{
|
||||
...(index === runModes.length - 1
|
||||
? s.runModeChipLast(activeRunMode === mode)
|
||||
@@ -1506,7 +2287,13 @@ export const Deployment = ({ config, benchmarks }) => {
|
||||
...(runModes.length === 1 ? { borderRadius: 7 } : {}),
|
||||
}}
|
||||
onClick={() => setRunMode(mode)}
|
||||
onKeyDown={(e) => {
|
||||
if (e.key !== "Enter" && e.key !== " ") return;
|
||||
e.preventDefault();
|
||||
setRunMode(mode);
|
||||
}}
|
||||
role="tab"
|
||||
tabIndex={0}
|
||||
aria-selected={activeRunMode === mode}
|
||||
>
|
||||
{mode === "docker" ? "Docker" : "Python"}
|
||||
|
||||
@@ -1,12 +1,9 @@
|
||||
// MiniMax-H3 diffusion deployment matrix. Consumed by _deployment.jsx.
|
||||
//
|
||||
// The mode, quantization, and encoder choices are deployment overlays because
|
||||
// they do not change which base hardware topology fits. Request sampling
|
||||
// controls remain in the generated cURL instead of being mixed into this
|
||||
// deployment matrix.
|
||||
// Hardware/profile cells remain deliberately small and carry an honest
|
||||
// verification state for the exact platform, rather than inheriting a result
|
||||
// measured on a different GPU.
|
||||
// H3 is the first opt-in user of the scoped commandBuilder renderer. Topology,
|
||||
// checkpoint, server overlays, and request fields share one semantic selection,
|
||||
// while the UI presents them by lifecycle and composes them through the existing
|
||||
// deployment command engine.
|
||||
|
||||
|
||||
export const config = {
|
||||
@@ -26,47 +23,26 @@ export const config = {
|
||||
],
|
||||
groupHardware: false,
|
||||
|
||||
matchDims: [
|
||||
{
|
||||
id: "profile",
|
||||
title: "Deployment Profile",
|
||||
showWhen: (s) => ["b200", "b300", "h200", "h100"].includes(s.hw),
|
||||
options: [
|
||||
{ id: "resident", label: "Resident" },
|
||||
{
|
||||
id: "fsdp",
|
||||
label: "FSDP sharded",
|
||||
showWhen: (s) =>
|
||||
["b200", "b300", "h200", "h100"].includes(s.hw),
|
||||
},
|
||||
{
|
||||
id: "offload",
|
||||
label: "Layerwise offload",
|
||||
showWhen: (s) => s.hw === "rtx5090",
|
||||
},
|
||||
{
|
||||
id: "cross_node",
|
||||
label: "Cross-node (2 nodes)",
|
||||
showWhen: (s) => s.hw === "h200",
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
matchDims: [],
|
||||
|
||||
overlayDims: [
|
||||
{
|
||||
id: "weights",
|
||||
title: "Checkpoint Weights",
|
||||
scope: "base",
|
||||
description: "Choose the checkpoint partition required by the request mode.",
|
||||
default: "fl2va",
|
||||
options: [
|
||||
{
|
||||
id: "fl2va",
|
||||
label: "FL2VA (First-and-Last-Frame-to-Video-and-Audio)",
|
||||
label: "FL2VA",
|
||||
subtitle: "(First-and-Last-Frame-to-Video-and-Audio)",
|
||||
flags: ["--model-variant fl2va"],
|
||||
},
|
||||
{
|
||||
id: "ref2va",
|
||||
label: "Ref2VA (Reference-to-Video-and-Audio)",
|
||||
label: "Ref2VA",
|
||||
subtitle: "(Reference-to-Video-and-Audio)",
|
||||
flags: ["--model-variant ref2va"],
|
||||
},
|
||||
],
|
||||
@@ -74,6 +50,8 @@ export const config = {
|
||||
{
|
||||
id: "mode",
|
||||
title: "Request Mode",
|
||||
scope: "base",
|
||||
description: "The visible modes follow the selected checkpoint.",
|
||||
default: "t2va",
|
||||
options: [
|
||||
{
|
||||
@@ -129,75 +107,442 @@ export const config = {
|
||||
],
|
||||
},
|
||||
{
|
||||
id: "quant",
|
||||
title: "Online Quantization",
|
||||
default: "bf16",
|
||||
showWhen: (s) => ["b200", "b300"].includes(s.hw),
|
||||
id: "placement",
|
||||
title: "Placement",
|
||||
scope: "serve",
|
||||
docsHref: "/docs/sglang-diffusion/api/cli#component-residency",
|
||||
description: "Keep weights resident for latency; shard or offload only when capacity requires it.",
|
||||
quality: "Memory policy",
|
||||
learnMore: "#7-feature-contracts-and-advanced-recipes",
|
||||
default: "resident",
|
||||
options: [
|
||||
{ id: "bf16", label: "Off — Native BF16/FP32" },
|
||||
{
|
||||
id: "auto",
|
||||
label: "Auto",
|
||||
flags: (s) => {
|
||||
const recipe = config.commandBuilder.resource.verifiedRecipes.find((entry) =>
|
||||
entry.hw === s.hw && entry.nodes === Number(s.nodes)
|
||||
&& entry.gpus_per_node === Number(s.gpus_per_node));
|
||||
const placement = recipe?.placement || (s.hw === "rtx5090" ? "offload" : "resident");
|
||||
if (placement === "fsdp") return ["--performance-mode speed", "--use-fsdp-inference true"];
|
||||
return placement === "offload" ? [
|
||||
"--performance-mode memory",
|
||||
"--layerwise-offload-components dit,text_encoder,vae",
|
||||
"--dit-offload-prefetch-size 1",
|
||||
"--dit-layerwise-resident-layers 20",
|
||||
"--enable-torch-compile false",
|
||||
] : ["--performance-mode speed"];
|
||||
},
|
||||
description: "Use the recommended placement for the selected hardware and resource shape.",
|
||||
},
|
||||
{
|
||||
id: "resident",
|
||||
label: "Resident",
|
||||
flags: ["--performance-mode speed"],
|
||||
recommendedWhen: (s) => s.hw !== "rtx5090",
|
||||
description: "Lowest-latency path when the full pipeline fits in aggregate GPU memory.",
|
||||
},
|
||||
{
|
||||
id: "fsdp",
|
||||
label: "FSDP",
|
||||
flags: ["--performance-mode speed", "--use-fsdp-inference true"],
|
||||
soft: (s) => !["b200", "b300", "h200", "h100"].includes(s.hw) || s.nodes > 1,
|
||||
softReason: "Verified on single-node B200/B300/H200/H100. Other hardware and multi-node runs take the same flags but have not been through a verification round.",
|
||||
description: "Reduces resident DiT memory but adds parameter collectives on every block.",
|
||||
},
|
||||
{
|
||||
id: "offload",
|
||||
label: "Layerwise offload",
|
||||
flags: [
|
||||
"--performance-mode memory",
|
||||
"--layerwise-offload-components dit,text_encoder,vae",
|
||||
"--dit-offload-prefetch-size 1",
|
||||
"--dit-layerwise-resident-layers 20",
|
||||
"--enable-torch-compile false",
|
||||
],
|
||||
soft: (s) => s.hw !== "rtx5090",
|
||||
softReason: "Tuned and verified on RTX 5090. It runs on the datacenter GPUs too, where a resident recipe is simply faster.",
|
||||
recommendedWhen: (s) => s.hw === "rtx5090",
|
||||
description: "Capacity-first PCIe path. It is substantially slower than a resident datacenter recipe.",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
id: "attention",
|
||||
title: "Attention",
|
||||
scope: "serve",
|
||||
docsHref: "/docs/sglang-diffusion/attention_backends",
|
||||
description: "Select the packed-attention kernel used by H3 transformer modules.",
|
||||
quality: "Kernel policy",
|
||||
learnMore: "#7-feature-contracts-and-advanced-recipes",
|
||||
default: "platform",
|
||||
options: [
|
||||
{
|
||||
id: "platform",
|
||||
label: "Automatic",
|
||||
flags: (s) => ["mi300x", "mi355x"].includes(s.hw) ? ["--attention-backend aiter"] : [],
|
||||
env: (s) => ["mi300x", "mi355x"].includes(s.hw) ? ["SGLANG_USE_AITER=1"] : [],
|
||||
recommended: true,
|
||||
description: "Applies the verified backend policy for the selected hardware.",
|
||||
},
|
||||
{
|
||||
id: "fa",
|
||||
label: "FlashAttention",
|
||||
flags: ["--attention-backend fa"],
|
||||
disabled: (s) => ["mi300x", "mi355x"].includes(s.hw),
|
||||
disableReason: "Use the verified AITER platform default on AMD.",
|
||||
description: "An explicit native-dtype CUDA comparison path; reduction ordering may still differ.",
|
||||
},
|
||||
{
|
||||
id: "sage",
|
||||
label: "SageAttention",
|
||||
flags: ["--attention-backend sage_attn"],
|
||||
disabled: (s) => ["mi300x", "mi355x"].includes(s.hw),
|
||||
disableReason: "SageAttention is not exposed for the AMD recipes.",
|
||||
description: "Approximate attention math. Install its packed-varlen dependency and inspect video and audio quality.",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
id: "precision",
|
||||
title: "Precision",
|
||||
scope: "serve",
|
||||
docsHref: "/docs/sglang-diffusion/quantization",
|
||||
description: "Choose native mixed precision or a validated online weight quantization path.",
|
||||
quality: "Weight precision",
|
||||
learnMore: "#7-feature-contracts-and-advanced-recipes",
|
||||
default: "native",
|
||||
options: [
|
||||
{
|
||||
id: "native",
|
||||
label: "BF16 / FP32",
|
||||
recommended: true,
|
||||
description: "Reference mixed precision: BF16 transformer weights with required projections retained in FP32.",
|
||||
},
|
||||
{
|
||||
id: "fp8",
|
||||
label: "FP8 — Approximate",
|
||||
showWhen: (s) => ["b200", "b300"].includes(s.hw),
|
||||
disabled: (s) => s.profile !== "resident",
|
||||
disableReason:
|
||||
"The documented FP8 operating point keeps the transformer resident; FSDP combinations have not been validated.",
|
||||
label: "Online FP8",
|
||||
flags: ["--quantization fp8"],
|
||||
hints: [
|
||||
"Online FP8 is approximate. Validate both video and audio quality;",
|
||||
"verified B200 and B300 runs reduced memory; re-benchmark latency on the target workload.",
|
||||
],
|
||||
soft: (s) => !["b200", "b300"].includes(s.hw)
|
||||
|| !["auto", "resident"].includes(s.placement)
|
||||
|| s.nodes !== 1,
|
||||
softReason: "Verified for resident single-node B200/B300. Other hardware and placements take the same flag, but those recipes have not been verified yet.",
|
||||
description: "Approximate transformer weight quantization with the required H3 projections protected.",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
id: "encoder",
|
||||
title: "Text Encoder Parallel",
|
||||
title: "Encoder",
|
||||
scope: "serve",
|
||||
docsHref: "/docs/sglang-diffusion/encoder_parallel",
|
||||
description: "Control text-encoder work placement independently from DiT topology.",
|
||||
quality: "Parallel policy",
|
||||
learnMore: "#7-feature-contracts-and-advanced-recipes",
|
||||
default: "auto",
|
||||
options: [
|
||||
{
|
||||
id: "auto",
|
||||
label: "Auto (recommended)",
|
||||
hints: [
|
||||
"Auto uses folding for the single-request recipes below and can",
|
||||
"select data parallel encoding for a compatible TP1 request batch.",
|
||||
],
|
||||
},
|
||||
{
|
||||
id: "fold",
|
||||
label: "Fold (single-request)",
|
||||
flags: ["--encoder-parallel fold"],
|
||||
hints: [
|
||||
"Fold shards the resident Qwen3-VL encoder across the replica and is",
|
||||
"best suited to single-node GPUs with fast peer-to-peer links.",
|
||||
],
|
||||
label: "Auto",
|
||||
flags: (s) => [`--encoder-parallel ${s.nodes > 1 ? "replicate" : "auto"}`],
|
||||
recommended: true,
|
||||
description: "Folds on verified single-host P2P systems and resolves to replicate across nodes.",
|
||||
},
|
||||
{
|
||||
id: "dp",
|
||||
label: "DP (batched throughput)",
|
||||
disabled: (s) =>
|
||||
s.hw === "rtx5090" ||
|
||||
(s.hw === "h100" && s.profile === "resident"),
|
||||
disableReason:
|
||||
"Encoder DP requires TP1 and DiT DP1; this verified recipe uses TP2.",
|
||||
flags: [
|
||||
"--encoder-parallel dp",
|
||||
"--batching-max-size {{BATCHING_MAX_SIZE}}",
|
||||
],
|
||||
hints: [
|
||||
"DP distributes a compatible multi-request text batch across ranks;",
|
||||
"it does not improve a batch of one and replicates encoder weights.",
|
||||
],
|
||||
label: "Data parallel",
|
||||
flags: ["--encoder-parallel dp"],
|
||||
disabled: (s) => (s.topology_mode === "manual"
|
||||
? Number(s.tp_size)
|
||||
: config.commandBuilder.resource.autoTopology(s).tp_size) > 1,
|
||||
disableReason: "The server rejects encoder DP with TP > 1 (encoder_parallel=dp requires tp_size=1).",
|
||||
soft: (s) => s.nodes > 1,
|
||||
softReason: "Runs across nodes, but the measured 1.9× encode speedup comes from a single-node 2× H100 run; cross-node encoder DP is unverified.",
|
||||
description: "Useful for a real request batch; it is not bitwise-identical to fold scheduling.",
|
||||
},
|
||||
{
|
||||
id: "fold",
|
||||
label: "Fold",
|
||||
flags: ["--encoder-parallel fold"],
|
||||
disabled: (s) => s.nodes > 1,
|
||||
disableReason: "Fold assumes fast node-local peer-to-peer access.",
|
||||
description: "Uses one folded encoder copy across a node-local group and preserves native weights.",
|
||||
},
|
||||
{
|
||||
id: "replicate",
|
||||
label: "Replicate (compatibility)",
|
||||
label: "Replicate",
|
||||
flags: ["--encoder-parallel replicate"],
|
||||
recommendedWhen: (s) => s.nodes > 1,
|
||||
description: "The safe cross-node default because encoder auto is not node-boundary aware.",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
id: "execution",
|
||||
title: "Execution",
|
||||
scope: "serve",
|
||||
description: "Choose eager execution or the measured breakable CUDA graph path.",
|
||||
quality: "Graph policy",
|
||||
learnMore: "#7-feature-contracts-and-advanced-recipes",
|
||||
default: "eager",
|
||||
options: [
|
||||
{
|
||||
id: "eager",
|
||||
label: "Eager",
|
||||
recommended: true,
|
||||
description: "Reference execution and the consistency baseline.",
|
||||
},
|
||||
{
|
||||
id: "bcg",
|
||||
label: "Compatible BCG",
|
||||
flags: [
|
||||
"--enable-breakable-cuda-graph true",
|
||||
"--warmup-resolutions 1344x768",
|
||||
"--bcg-text-buckets 5504",
|
||||
],
|
||||
soft: (s) => !(["b200", "h200"].includes(s.hw) && s.weights === "ref2va"),
|
||||
softReason: "Verified for B200/H200 Ref2VA; BCG runs on the other recipes but they have not been through a verification round yet.",
|
||||
description: "Reuses matching execution signatures and reserves capture memory; it takes precedence over Cache-DiT.",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
id: "quality",
|
||||
title: "Quality",
|
||||
scope: "request",
|
||||
docsHref: "/docs/sglang-diffusion/cache_dit",
|
||||
description: "Reference execution or the audited Cache-DiT acceleration preset.",
|
||||
quality: "Sampling policy",
|
||||
learnMore: "#choose-the-quality-level",
|
||||
default: "lossless",
|
||||
options: [
|
||||
{
|
||||
id: "lossless",
|
||||
label: "Lossless",
|
||||
recommended: true,
|
||||
description: "Reference-exact denoising without Cache-DiT approximation.",
|
||||
},
|
||||
{
|
||||
id: "high",
|
||||
label: "Audited high",
|
||||
disabled: (s) => s.execution !== "eager",
|
||||
disableReason: "BCG supersedes Cache-DiT, so the preset would have no effect — switch Execution to Eager to use it.",
|
||||
soft: (s) => !(s.hw === "h200" && s.nodes === 1 && s.gpus_per_node === 4
|
||||
&& ["auto", "resident"].includes(s.placement)),
|
||||
softReason: "The 1.40× / SSIM 0.931 audit covers the resident eager 4× H200 workload; elsewhere the preset runs but its quality figures are unaudited.",
|
||||
description: "Measured 1.40× with SSIM 0.931 and PSNR 28.16 dB on the audited workload.",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
id: "outputs",
|
||||
title: "Outputs",
|
||||
scope: "request",
|
||||
description: "Generate independent variants from one request.",
|
||||
quality: "1–10",
|
||||
kind: "number",
|
||||
min: 1,
|
||||
max: 10,
|
||||
unit: "outputs per prompt",
|
||||
default: 1,
|
||||
options: [],
|
||||
},
|
||||
],
|
||||
|
||||
commandBuilder: {
|
||||
defaultSelection: {
|
||||
hw: "b200",
|
||||
nodes: 1,
|
||||
gpus_per_node: 8,
|
||||
topology_mode: "auto",
|
||||
tp_size: 1,
|
||||
ulysses_degree: 8,
|
||||
ring_degree: 1,
|
||||
},
|
||||
resource: {
|
||||
limits: {
|
||||
nodes: { min: 1, max: 8 },
|
||||
gpus_per_node: { min: 1, max: 8 },
|
||||
},
|
||||
verifiedRecipes: [
|
||||
{ id: "b200-resident-8", hw: "b200", nodes: 1, gpus_per_node: 8, placement: "resident", tp_size: 1, ulysses_degree: 8, ring_degree: 1, encoder: "auto", default: true },
|
||||
{ id: "b200-fsdp-4", hw: "b200", nodes: 1, gpus_per_node: 4, placement: "fsdp", tp_size: 1, ulysses_degree: 4, ring_degree: 1, encoder: "auto" },
|
||||
{ id: "b300-resident-8", hw: "b300", nodes: 1, gpus_per_node: 8, placement: "resident", tp_size: 1, ulysses_degree: 8, ring_degree: 1, encoder: "auto", default: true },
|
||||
{ id: "b300-fsdp-8", hw: "b300", nodes: 1, gpus_per_node: 8, placement: "fsdp", tp_size: 1, ulysses_degree: 8, ring_degree: 1, encoder: "auto" },
|
||||
{ id: "h200-resident-4", hw: "h200", nodes: 1, gpus_per_node: 4, placement: "resident", tp_size: 1, ulysses_degree: 4, ring_degree: 1, encoder: "auto", default: true },
|
||||
{ id: "h200-fsdp-4", hw: "h200", nodes: 1, gpus_per_node: 4, placement: "fsdp", tp_size: 1, ulysses_degree: 4, ring_degree: 1, encoder: "auto" },
|
||||
{ id: "h200-cross-node-16", hw: "h200", nodes: 2, gpus_per_node: 8, placement: "resident", tp_size: 1, ulysses_degree: 8, ring_degree: 2, encoder: "replicate" },
|
||||
{ id: "h100-resident-4", hw: "h100", nodes: 1, gpus_per_node: 4, placement: "resident", tp_size: 2, ulysses_degree: 2, ring_degree: 1, encoder: "auto", default: true },
|
||||
{ id: "h100-fsdp-4", hw: "h100", nodes: 1, gpus_per_node: 4, placement: "fsdp", tp_size: 1, ulysses_degree: 4, ring_degree: 1, encoder: "auto" },
|
||||
{ id: "mi300x-resident-1", hw: "mi300x", nodes: 1, gpus_per_node: 1, placement: "resident", tp_size: 1, ulysses_degree: 1, ring_degree: 1, encoder: "auto" },
|
||||
{ id: "mi300x-resident-2", hw: "mi300x", nodes: 1, gpus_per_node: 2, placement: "resident", tp_size: 1, ulysses_degree: 2, ring_degree: 1, encoder: "auto" },
|
||||
{ id: "mi300x-resident-4", hw: "mi300x", nodes: 1, gpus_per_node: 4, placement: "resident", tp_size: 1, ulysses_degree: 4, ring_degree: 1, encoder: "auto" },
|
||||
{ id: "mi300x-resident-8", hw: "mi300x", nodes: 1, gpus_per_node: 8, placement: "resident", tp_size: 1, ulysses_degree: 8, ring_degree: 1, encoder: "auto", default: true },
|
||||
{ id: "mi355x-resident-1", hw: "mi355x", nodes: 1, gpus_per_node: 1, placement: "resident", tp_size: 1, ulysses_degree: 1, ring_degree: 1, encoder: "auto" },
|
||||
{ id: "mi355x-resident-2", hw: "mi355x", nodes: 1, gpus_per_node: 2, placement: "resident", tp_size: 1, ulysses_degree: 2, ring_degree: 1, encoder: "auto" },
|
||||
{ id: "mi355x-resident-4", hw: "mi355x", nodes: 1, gpus_per_node: 4, placement: "resident", tp_size: 1, ulysses_degree: 4, ring_degree: 1, encoder: "auto" },
|
||||
{ id: "mi355x-resident-8", hw: "mi355x", nodes: 1, gpus_per_node: 8, placement: "resident", tp_size: 1, ulysses_degree: 8, ring_degree: 1, encoder: "auto", default: true },
|
||||
{ id: "rtx5090-offload-2", hw: "rtx5090", nodes: 1, gpus_per_node: 2, placement: "offload", tp_size: 2, ulysses_degree: 1, ring_degree: 1, encoder: "auto", default: true },
|
||||
],
|
||||
autoTopology: (s) => {
|
||||
const recipes = config.commandBuilder.resource.verifiedRecipes;
|
||||
const exact = recipes.find((recipe) => recipe.hw === s.hw
|
||||
&& recipe.nodes === Number(s.nodes)
|
||||
&& recipe.gpus_per_node === Number(s.gpus_per_node)
|
||||
&& (s.placement === "auto" || recipe.placement === s.placement));
|
||||
if (exact) {
|
||||
return {
|
||||
tp_size: exact.tp_size,
|
||||
ulysses_degree: exact.ulysses_degree,
|
||||
ring_degree: exact.ring_degree,
|
||||
};
|
||||
}
|
||||
return {
|
||||
tp_size: 1,
|
||||
ulysses_degree: Number(s.gpus_per_node),
|
||||
ring_degree: Number(s.nodes),
|
||||
};
|
||||
},
|
||||
validateTopology: (s, topology) => {
|
||||
const errors = [];
|
||||
const nodes = Number(s.nodes);
|
||||
const perNode = Number(s.gpus_per_node);
|
||||
const world = nodes * perNode;
|
||||
const tp = Number(topology.tp_size);
|
||||
const ulysses = Number(topology.ulysses_degree);
|
||||
const ring = Number(topology.ring_degree);
|
||||
if (!Number.isInteger(nodes) || nodes < 1 || nodes > 8) errors.push("H3 supports 1–8 nodes.");
|
||||
if (!Number.isInteger(perNode) || perNode < 1 || perNode > 8) errors.push("H3 supports 1–8 GPUs per node.");
|
||||
if (![1, 2, 4, 8].includes(tp)) errors.push("Tensor parallel size must be one of 1, 2, 4, or 8.");
|
||||
if (world !== tp * ulysses * ring) errors.push(`World size ${world} must equal TP × Ulysses × Ring (${tp * ulysses * ring}).`);
|
||||
if (56 % tp !== 0 || (56 / tp) % ulysses !== 0) errors.push("H3's 56 attention heads must divide evenly across TP and Ulysses.");
|
||||
if (64 % (ulysses * ring) !== 0) errors.push("Ulysses × Ring must divide the 64 packed sequence partitions.");
|
||||
return errors;
|
||||
},
|
||||
},
|
||||
resolveDeployment: (s) => {
|
||||
const resource = config.commandBuilder.resource;
|
||||
const topology = s.topology_mode === "manual"
|
||||
? {
|
||||
tp_size: Number(s.tp_size),
|
||||
ulysses_degree: Number(s.ulysses_degree),
|
||||
ring_degree: Number(s.ring_degree),
|
||||
}
|
||||
: resource.autoTopology(s);
|
||||
const errors = resource.validateTopology(s, topology);
|
||||
const automaticRecipe = resource.verifiedRecipes.find((entry) => entry.hw === s.hw
|
||||
&& entry.nodes === Number(s.nodes)
|
||||
&& entry.gpus_per_node === Number(s.gpus_per_node)
|
||||
&& entry.tp_size === topology.tp_size
|
||||
&& entry.ulysses_degree === topology.ulysses_degree
|
||||
&& entry.ring_degree === topology.ring_degree);
|
||||
const resolvedPlacement = s.placement === "auto"
|
||||
? (automaticRecipe?.placement || (s.hw === "rtx5090" ? "offload" : "resident"))
|
||||
: s.placement;
|
||||
const coverageWarnings = [];
|
||||
if (resolvedPlacement === "offload" && s.hw !== "rtx5090") {
|
||||
coverageWarnings.push("Layerwise offload is tuned and verified on RTX 5090; on this hardware it runs unverified and a resident recipe is faster.");
|
||||
}
|
||||
if (resolvedPlacement === "fsdp" && (s.nodes !== 1 || !["b200", "b300", "h200", "h100"].includes(s.hw))) {
|
||||
coverageWarnings.push("FSDP outside the single-node NVIDIA recipes runs unverified.");
|
||||
}
|
||||
if (s.precision === "fp8" && (! ["b200", "b300"].includes(s.hw)
|
||||
|| resolvedPlacement !== "resident" || s.nodes !== 1)) {
|
||||
coverageWarnings.push("Online FP8 outside resident single-node B200/B300 runs unverified.");
|
||||
}
|
||||
const highAudited = s.hw === "h200" && s.nodes === 1
|
||||
&& s.gpus_per_node === 4 && resolvedPlacement === "resident";
|
||||
if (s.quality === "high" && s.execution !== "eager") {
|
||||
coverageWarnings.push("BCG supersedes Cache-DiT, so the high preset has no effect under this execution mode.");
|
||||
} else if (s.quality === "high" && !highAudited) {
|
||||
coverageWarnings.push("The high preset's 1.40× / SSIM 0.931 figures were audited on resident eager 4× H200; this workload is unaudited.");
|
||||
}
|
||||
|
||||
const recipe = resource.verifiedRecipes.find((entry) => entry.hw === s.hw
|
||||
&& entry.nodes === Number(s.nodes)
|
||||
&& entry.gpus_per_node === Number(s.gpus_per_node)
|
||||
&& entry.placement === resolvedPlacement
|
||||
&& entry.tp_size === topology.tp_size
|
||||
&& entry.ulysses_degree === topology.ulysses_degree
|
||||
&& entry.ring_degree === topology.ring_degree);
|
||||
const topologyVerified = !!recipe && errors.length === 0;
|
||||
const encoderVerified = s.encoder === "auto"
|
||||
|| s.encoder === recipe?.encoder
|
||||
|| (s.nodes > 1 && s.encoder === "replicate");
|
||||
const attentionVerified = s.attention === "platform";
|
||||
const precisionVerified = s.precision === "native"
|
||||
|| (s.precision === "fp8" && ["b200", "b300"].includes(s.hw));
|
||||
const executionVerified = s.execution === "eager"
|
||||
|| (s.execution === "bcg" && ["b200", "h200"].includes(s.hw) && s.weights === "ref2va");
|
||||
const serveVerified = topologyVerified && encoderVerified && attentionVerified
|
||||
&& precisionVerified && executionVerified;
|
||||
const requestVerified = topologyVerified && (s.quality === "lossless"
|
||||
|| (s.quality === "high" && highAudited && s.execution === "eager"));
|
||||
|
||||
const topologyParts = [];
|
||||
if (topology.tp_size > 1) topologyParts.push(`TP ${topology.tp_size}`);
|
||||
if (Number(s.nodes) > 1) {
|
||||
topologyParts.push(`Ulysses ${topology.ulysses_degree} inside each node`);
|
||||
topologyParts.push(`Ring ${topology.ring_degree} across nodes`);
|
||||
} else {
|
||||
topologyParts.push(`Ulysses ${topology.ulysses_degree}`);
|
||||
}
|
||||
topologyParts.push({ resident: "Resident", fsdp: "FSDP", offload: "Layerwise offload" }[resolvedPlacement]);
|
||||
topologyParts.push(Number(s.nodes) > 1 ? `${s.nodes} nodes` : "Single node");
|
||||
|
||||
const world = Number(s.nodes) * Number(s.gpus_per_node);
|
||||
const flags = ["--model-path {{MODEL_NAME}}", `--num-gpus ${world}`];
|
||||
if (topology.ring_degree > 1) flags.push(`--sp-degree ${world}`);
|
||||
if (topology.tp_size > 1) flags.push(`--tp-size ${topology.tp_size}`);
|
||||
flags.push(`--ulysses-degree ${topology.ulysses_degree}`);
|
||||
if (topology.ring_degree > 1) flags.push(`--ring-degree ${topology.ring_degree}`);
|
||||
flags.push("--host {{HOST_IP}}", "--port {{PORT}}");
|
||||
|
||||
const warnings = [...coverageWarnings];
|
||||
if (!topologyVerified && errors.length === 0) {
|
||||
warnings.push("This topology satisfies H3's static constraints but has not completed an exact end-to-end verification run.");
|
||||
}
|
||||
if (resolvedPlacement === "fsdp") {
|
||||
warnings.push("FSDP lowers resident DiT memory but adds per-block parameter collectives; prefer Resident when the pipeline fits.");
|
||||
}
|
||||
if (s.hw === "rtx5090") {
|
||||
warnings.push("The 2× RTX 5090 path requires a 384 GiB-class host and prioritizes capacity over latency.");
|
||||
}
|
||||
|
||||
let automaticAttention = "FlashAttention (auto)";
|
||||
if (["mi300x", "mi355x"].includes(s.hw)) {
|
||||
automaticAttention = "AITER (auto)";
|
||||
} else if (topology.ring_degree === 1 && ["b200", "b300"].includes(s.hw)) {
|
||||
automaticAttention = "Dynamic cuDNN / FA (auto)";
|
||||
} else if (topology.ring_degree === 1 && s.hw === "rtx5090") {
|
||||
automaticAttention = "Torch SDPA (auto)";
|
||||
}
|
||||
|
||||
return {
|
||||
match: { hw: s.hw },
|
||||
nnodes: Number(s.nodes),
|
||||
verified: serveVerified,
|
||||
verificationStatus: serveVerified ? "verified" : "unverified",
|
||||
flags,
|
||||
builder: {
|
||||
topology,
|
||||
topologySummary: topologyParts.filter(Boolean).join(" · "),
|
||||
errors,
|
||||
warnings,
|
||||
verification: {
|
||||
serve: errors.length ? "error" : (serveVerified ? "verified" : "unverified"),
|
||||
request: errors.length ? "error" : (requestVerified ? "verified" : "unverified"),
|
||||
},
|
||||
resolvedSettings: {
|
||||
placement: { resident: "Resident", fsdp: "FSDP", offload: "Layerwise offload" }[resolvedPlacement],
|
||||
attention: s.attention === "platform" ? automaticAttention : undefined,
|
||||
encoder: s.encoder === "auto" ? (s.nodes > 1 ? "Replicate (auto)" : "Auto") : undefined,
|
||||
},
|
||||
},
|
||||
};
|
||||
},
|
||||
},
|
||||
|
||||
modelNames: {
|
||||
default: "MiniMaxAI/MiniMax-H3",
|
||||
},
|
||||
@@ -233,16 +578,6 @@ export const config = {
|
||||
label: "Server port",
|
||||
default: "30010",
|
||||
},
|
||||
NUM_OUTPUTS: {
|
||||
target: "curl",
|
||||
label: "Outputs per prompt (1-10)",
|
||||
default: "1",
|
||||
},
|
||||
BATCHING_MAX_SIZE: {
|
||||
target: "command",
|
||||
label: "Maximum request batch size",
|
||||
default: "2",
|
||||
},
|
||||
DURATION_SECONDS: {
|
||||
target: "curl",
|
||||
label: "Duration (seconds, 4-15)",
|
||||
@@ -313,7 +648,8 @@ export const config = {
|
||||
aspect_ratio: "16:9",
|
||||
duration_seconds: "{{DURATION_SECONDS}}",
|
||||
},
|
||||
num_outputs_per_prompt: "{{NUM_OUTPUTS}}",
|
||||
quality: s.quality,
|
||||
num_outputs_per_prompt: Number(s.outputs),
|
||||
num_inference_steps: 50,
|
||||
flow_shift: 12.0,
|
||||
audio_flow_shift: 3.0,
|
||||
@@ -413,7 +749,7 @@ export const config = {
|
||||
}
|
||||
|
||||
const body = JSON.stringify(request, null, 2).replace(
|
||||
/"{{(NUM_OUTPUTS|DURATION_SECONDS|INPUT_VIDEO_START_SECONDS|SECOND_INPUT_VIDEO_START_SECONDS)}}"/g,
|
||||
/"{{(DURATION_SECONDS|INPUT_VIDEO_START_SECONDS|SECOND_INPUT_VIDEO_START_SECONDS)}}"/g,
|
||||
"{{$1}}",
|
||||
);
|
||||
return `curl -sS -X POST http://{{CURL_HOST}}:{{CURL_PORT}}/v1/videos \\
|
||||
@@ -443,199 +779,5 @@ export const config = {
|
||||
|
||||
showPlaygroundLink: false,
|
||||
|
||||
cells: [
|
||||
{
|
||||
match: { hw: "b200", profile: "resident" },
|
||||
nnodes: 1,
|
||||
verified: true,
|
||||
flags: [
|
||||
"--model-path {{MODEL_NAME}}",
|
||||
"--num-gpus 8",
|
||||
"--ulysses-degree 8",
|
||||
"--performance-mode speed",
|
||||
"--host {{HOST_IP}}",
|
||||
"--port {{PORT}}",
|
||||
],
|
||||
},
|
||||
{
|
||||
match: { hw: "b300", profile: "resident" },
|
||||
nnodes: 1,
|
||||
verified: true,
|
||||
flags: [
|
||||
"--model-path {{MODEL_NAME}}",
|
||||
"--num-gpus 8",
|
||||
"--ulysses-degree 8",
|
||||
"--performance-mode speed",
|
||||
"--host {{HOST_IP}}",
|
||||
"--port {{PORT}}",
|
||||
],
|
||||
warn:
|
||||
"This is the B300 topology used for the documented benchmark sweep, not a claimed minimum GPU count.",
|
||||
},
|
||||
{
|
||||
match: { hw: "h200", profile: "resident" },
|
||||
nnodes: 1,
|
||||
verified: true,
|
||||
flags: [
|
||||
"--model-path {{MODEL_NAME}}",
|
||||
"--num-gpus 4",
|
||||
"--ulysses-degree 4",
|
||||
"--performance-mode speed",
|
||||
"--host {{HOST_IP}}",
|
||||
"--port {{PORT}}",
|
||||
],
|
||||
},
|
||||
{
|
||||
match: { hw: "b300", profile: "fsdp" },
|
||||
nnodes: 1,
|
||||
verified: true,
|
||||
flags: [
|
||||
"--model-path {{MODEL_NAME}}",
|
||||
"--num-gpus 8",
|
||||
"--ulysses-degree 8",
|
||||
"--performance-mode speed",
|
||||
"--use-fsdp-inference true",
|
||||
"--host {{HOST_IP}}",
|
||||
"--port {{PORT}}",
|
||||
],
|
||||
warn:
|
||||
"FSDP reduces resident DiT memory but adds per-block parameter collectives. Prefer Resident when the full pipeline fits.",
|
||||
},
|
||||
{
|
||||
match: { hw: "h200", profile: "fsdp" },
|
||||
nnodes: 1,
|
||||
verified: true,
|
||||
flags: [
|
||||
"--model-path {{MODEL_NAME}}",
|
||||
"--num-gpus 4",
|
||||
"--ulysses-degree 4",
|
||||
"--performance-mode speed",
|
||||
"--use-fsdp-inference true",
|
||||
"--host {{HOST_IP}}",
|
||||
"--port {{PORT}}",
|
||||
],
|
||||
warn:
|
||||
"FSDP reduces resident DiT memory but adds per-block parameter collectives. Prefer Resident when the full pipeline fits.",
|
||||
},
|
||||
{
|
||||
match: { hw: "h200", profile: "cross_node" },
|
||||
nnodes: 2,
|
||||
verified: true,
|
||||
flags: [
|
||||
"--model-path {{MODEL_NAME}}",
|
||||
"--num-gpus 16",
|
||||
"--sp-degree 16",
|
||||
"--ulysses-degree 8",
|
||||
"--ring-degree 2",
|
||||
"--encoder-parallel replicate",
|
||||
"--performance-mode speed",
|
||||
"--host {{HOST_IP}}",
|
||||
"--port {{PORT}}",
|
||||
],
|
||||
warn:
|
||||
"Verified on 2 nodes of 8× H200 each (Ulysses8 within a node, Ring2 across nodes). Requires --encoder-parallel replicate: --encoder-parallel auto's fold decision is not yet node-boundary aware and will crash across nodes.",
|
||||
},
|
||||
{
|
||||
match: { hw: "b200", profile: "fsdp" },
|
||||
nnodes: 1,
|
||||
verified: true,
|
||||
flags: [
|
||||
"--model-path {{MODEL_NAME}}",
|
||||
"--num-gpus 4",
|
||||
"--ulysses-degree 4",
|
||||
"--performance-mode speed",
|
||||
"--use-fsdp-inference true",
|
||||
"--host {{HOST_IP}}",
|
||||
"--port {{PORT}}",
|
||||
],
|
||||
warn:
|
||||
"The 4-GPU FSDP path is lossless but slower than the 8-GPU resident recipe.",
|
||||
},
|
||||
{
|
||||
match: { hw: "h100", profile: "resident" },
|
||||
nnodes: 1,
|
||||
verified: true,
|
||||
flags: [
|
||||
"--model-path {{MODEL_NAME}}",
|
||||
"--num-gpus 4",
|
||||
"--tp-size 2",
|
||||
"--ulysses-degree 2",
|
||||
"--performance-mode speed",
|
||||
"--host {{HOST_IP}}",
|
||||
"--port {{PORT}}",
|
||||
],
|
||||
warn:
|
||||
"Fastest measured 4× H100 80 GB topology. TP4 + Ulysses1 lowers peak memory at a small latency cost.",
|
||||
},
|
||||
{
|
||||
match: { hw: "h100", profile: "fsdp" },
|
||||
nnodes: 1,
|
||||
verified: true,
|
||||
flags: [
|
||||
"--model-path {{MODEL_NAME}}",
|
||||
"--num-gpus 4",
|
||||
"--ulysses-degree 4",
|
||||
"--performance-mode speed",
|
||||
"--use-fsdp-inference true",
|
||||
"--host {{HOST_IP}}",
|
||||
"--port {{PORT}}",
|
||||
],
|
||||
warn:
|
||||
"Capacity path on 4× H100 80 GB. Prefer the resident TP2 + Ulysses2 profile for latency.",
|
||||
},
|
||||
{
|
||||
match: { hw: "mi300x", profile: "resident" },
|
||||
nnodes: 1,
|
||||
verified: true,
|
||||
env: ["SGLANG_USE_AITER=1"],
|
||||
flags: [
|
||||
"--model-path {{MODEL_NAME}}",
|
||||
"--num-gpus 8",
|
||||
"--ulysses-degree 8",
|
||||
"--performance-mode speed",
|
||||
"--attention-backend aiter",
|
||||
"--host {{HOST_IP}}",
|
||||
"--port {{PORT}}",
|
||||
],
|
||||
warn:
|
||||
"Validated on 1×, 2×, 4×, and 8× MI300X with BF16 and AITER packed attention. The picker emits the fastest measured 8-GPU topology; set --num-gpus and --ulysses-degree to the same lower count for a measured capacity recipe.",
|
||||
},
|
||||
{
|
||||
match: { hw: "mi355x", profile: "resident" },
|
||||
nnodes: 1,
|
||||
verified: true,
|
||||
env: ["SGLANG_USE_AITER=1"],
|
||||
flags: [
|
||||
"--model-path {{MODEL_NAME}}",
|
||||
"--num-gpus 8",
|
||||
"--ulysses-degree 8",
|
||||
"--performance-mode speed",
|
||||
"--attention-backend aiter",
|
||||
"--host {{HOST_IP}}",
|
||||
"--port {{PORT}}",
|
||||
],
|
||||
warn:
|
||||
"Validated on 1×, 2×, 4×, and 8× MI355X with BF16 and AITER packed attention. The picker emits the fastest measured 8-GPU topology; set --num-gpus and --ulysses-degree to the same lower count for a measured capacity recipe.",
|
||||
},
|
||||
{
|
||||
match: { hw: "rtx5090", profile: "offload" },
|
||||
nnodes: 1,
|
||||
verified: true,
|
||||
flags: [
|
||||
"--model-path {{MODEL_NAME}}",
|
||||
"--num-gpus 2",
|
||||
"--tp-size 2",
|
||||
"--ulysses-degree 1",
|
||||
"--performance-mode memory",
|
||||
"--layerwise-offload-components dit,text_encoder,vae",
|
||||
"--dit-offload-prefetch-size 1",
|
||||
"--dit-layerwise-resident-layers 20",
|
||||
"--enable-torch-compile false",
|
||||
"--host {{HOST_IP}}",
|
||||
"--port {{PORT}}",
|
||||
],
|
||||
warn:
|
||||
"Validated lossless BF16/FP32 recipe on 2× RTX 5090 (32 GB each) with a 384 GiB-class host. TP2 avoids the full per-rank DiT replication observed with Ulysses2 on PCIe.",
|
||||
},
|
||||
],
|
||||
cells: [],
|
||||
};
|
||||
|
||||
Reference in New Issue
Block a user