[Docs] Rename docs_new/ to docs/ (#32123)

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-03 16:51:00 -07:00
parent c949e91f18
commit b819d2fb5b
491 changed files with 122 additions and 102 deletions
@@ -0,0 +1,260 @@
+---
+title: Cosmos3
+metatags:
+    description: "Serve NVIDIA Cosmos3 image, video, sound, and action generation with SGLang Diffusion."
+---
+
+import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';
+
+<DiffusionModelTags tags={["image", "video", "sound/action", "world model", "policy"]} />
+
+## 1. Model Introduction
+
+[NVIDIA Cosmos3](https://huggingface.co/collections/nvidia/cosmos3) is an omnimodal world-model family for image, video, sound, and action generation. SGLang Diffusion serves the public checkpoints with the native `Cosmos3OmniDiffusersPipeline`.
+
+| Model | Status | Notes |
+| --- | --- | --- |
+| `nvidia/Cosmos3-Nano` | Supported | T2I, T2V, I2V, V2V, joint sound, and action |
+| `nvidia/Cosmos3-Super` | Supported | T2I, T2V, I2V, and V2V; use multi-GPU for the 64B checkpoint |
+| `nvidia/Cosmos3-Super-Text2Image` | Supported | T2I-specialized checkpoint |
+| `nvidia/Cosmos3-Super-Image2Video` | Supported | I2V-specialized checkpoint |
+| `nvidia/Cosmos3-Nano-Policy-DROID` | Supported | DROID policy action generation |
+
+Sound and action generation require the corresponding checkpoint heads. SGLang uses the flow-native `FlowUniPCMultistepScheduler` for Cosmos3 even if the checkpoint metadata names another scheduler. The default `flow_shift` is `3.0` for T2I and `10.0` for video and action modes.
+
+## 2. Installation
+
+Install SGLang with the diffusion dependencies:
+
+```bash Command
+pip install -e "python[diffusion]"
+```
+
+Cosmos3 guardrails are enabled by default when the package is available:
+
+```bash Command
+pip install "cosmos-guardrail==0.3.1"
+```
+
+`cosmos-guardrail` downloads gated NVIDIA guardrail weights, so pass a Hugging Face token if your environment needs one. If the package is not installed, SGLang skips Cosmos3 guardrails and logs a warning. To disable Cosmos3 guardrails for local experiments, set `SGLANG_DISABLE_COSMOS3_GUARDRAILS=1` before starting the server.
+
+## 3. Serve Cosmos3
+
+Serve `Cosmos3-Nano` directly from the Hugging Face model ID:
+
+```bash Command
+sglang serve \
+  --model-path nvidia/Cosmos3-Nano \
+  --num-gpus 1
+```
+
+For `Cosmos3-Super`, split the model across multiple GPUs:
+
+```bash Command
+sglang serve \
+  --model-path nvidia/Cosmos3-Super \
+  --num-gpus 4
+```
+
+The server also accepts the specialized `nvidia/Cosmos3-Super-Text2Image` and `nvidia/Cosmos3-Super-Image2Video` checkpoint IDs.
+
+## 4. OpenAI-Compatible Requests
+
+### Text to image
+
+Cosmos3 text-to-image uses `/v1/images/generations`. The default Cosmos3 image response is `b64_json`, matching vLLM-Omni's examples.
+
+```bash Command
+curl -sS -X POST http://127.0.0.1:30010/v1/images/generations \
+  -H "Content-Type: application/json" \
+  -d '{
+    "prompt": "A warehouse robot folds a blue cloth on a clean workbench.",
+    "size": "1280x720",
+    "n": 1,
+    "num_inference_steps": 35,
+    "guidance_scale": 6.0,
+    "flow_shift": 3.0,
+    "seed": 0,
+    "extra_args": {
+      "use_resolution_template": false,
+      "guardrails": true
+    }
+  }'
+```
+
+### Text to video with sound
+
+Use `/v1/videos` to create an asynchronous job, then poll the job and download the completed MP4. Set `generate_sound=true` to generate and mux a stereo 48 kHz audio track; omit it for a silent video.
+
+```bash Command
+job_id=$(curl -sS -X POST http://127.0.0.1:30010/v1/videos \
+  --form-string "prompt=A small warehouse robot moves a blue box across a clean floor." \
+  --form-string "negative_prompt=blurry, distorted, low quality" \
+  --form-string "size=1280x720" \
+  --form-string "num_frames=81" \
+  --form-string "fps=24" \
+  --form-string "num_inference_steps=35" \
+  --form-string "guidance_scale=4.0" \
+  --form-string "flow_shift=10.0" \
+  --form-string "generate_sound=true" \
+  --form-string "seed=42" \
+  --form-string 'extra_params={"guardrails":true,"use_resolution_template":false,"use_duration_template":false}' \
+  | python -c 'import json, sys; print(json.load(sys.stdin)["id"])')
+
+while true; do
+  status=$(curl -sS "http://127.0.0.1:30010/v1/videos/${job_id}" \
+    | python -c 'import json, sys; print(json.load(sys.stdin)["status"])')
+  [ "$status" = "completed" ] && break
+  [ "$status" = "failed" ] && exit 1
+  sleep 1
+done
+
+curl -sS -L "http://127.0.0.1:30010/v1/videos/${job_id}/content" \
+  -o cosmos3_t2v.mp4
+```
+
+### Image to video
+
+This mirrors the official `nvidia/Cosmos3-Nano` Hugging Face image-to-video example:
+
+```python Python
+import json
+import time
+from pathlib import Path
+
+import requests
+from huggingface_hub import snapshot_download
+
+base_url = "http://127.0.0.1:30010"
+model_dir = Path(snapshot_download("nvidia/Cosmos3-Nano"))
+asset_dir = model_dir / "assets"
+
+prompt = json.dumps(json.loads((asset_dir / "example_i2v_prompt.json").read_text()))
+negative_prompt = json.dumps(
+    json.loads((asset_dir / "negative_prompt.json").read_text())
+)
+
+data = {
+    "prompt": prompt,
+    "negative_prompt": negative_prompt,
+    "size": "1280x720",
+    "num_frames": "189",
+    "fps": "24",
+    "num_inference_steps": "35",
+    "guidance_scale": "6.0",
+    "max_sequence_length": "4096",
+    "flow_shift": "10.0",
+    "seed": "1111",
+    "extra_params": json.dumps(
+        {
+            "use_resolution_template": False,
+            "use_duration_template": False,
+            "guardrails": True,
+        }
+    ),
+}
+
+with (asset_dir / "example_i2v_input.jpg").open("rb") as image:
+    response = requests.post(
+        f"{base_url}/v1/videos",
+        data=data,
+        files={"input_reference": ("example_i2v_input.jpg", image, "image/jpeg")},
+        timeout=60,
+    )
+response.raise_for_status()
+video_id = response.json()["id"]
+
+while True:
+    job = requests.get(f"{base_url}/v1/videos/{video_id}", timeout=30).json()
+    if job["status"] == "completed":
+        break
+    if job["status"] == "failed":
+        raise RuntimeError(job.get("error") or "Video generation failed")
+    time.sleep(1)
+
+response = requests.get(f"{base_url}/v1/videos/{video_id}/content", timeout=300)
+response.raise_for_status()
+Path("cosmos3_i2v.mp4").write_bytes(response.content)
+```
+
+### Video to video
+
+Upload a source video with `video_reference`. Cosmos3 keeps latent frames `[0, 1]` by default and generates the remaining frames. Use `condition_frame_indexes` to select different latent frames, and `condition_video_keep` to take conditioning frames from the start or end of the source.
+
+```bash Command
+job_id=$(curl -sS -X POST http://127.0.0.1:30010/v1/videos \
+  --form-string "prompt=A robotic arm pours liquid into a glass on a white tabletop." \
+  --form "video_reference=@robot_pouring.mp4;type=video/mp4" \
+  --form-string "size=1280x704" \
+  --form-string "num_frames=45" \
+  --form-string "fps=24" \
+  --form-string "num_inference_steps=35" \
+  --form-string "guidance_scale=6.0" \
+  --form-string 'condition_frame_indexes=[0,1]' \
+  --form-string "condition_video_keep=first" \
+  | python -c 'import json, sys; print(json.load(sys.stdin)["id"])')
+```
+
+Poll and download this job with the same status and content endpoints used by the T2V example.
+
+### Action generation
+
+For DROID policy generation, start a single-GPU server with the policy checkpoint. Cosmos3 action generation does not currently support CFG or sequence parallelism.
+
+```bash Command
+sglang serve \
+  --model-path nvidia/Cosmos3-Nano-Policy-DROID \
+  --num-gpus 1
+```
+
+The following request predicts a 16-step action chunk from one observation. The chunk length is `num_frames - 1`, and the completed job's `action` field contains the tensor data, shape, mode, and active action dimension.
+
+```bash Command
+job_id=$(curl -sS -X POST http://127.0.0.1:30010/v1/videos \
+  --form-string "prompt=Put the pot to the left of the purple item." \
+  --form "input_reference=@observation.png;type=image/png" \
+  --form-string "size=832x480" \
+  --form-string "num_frames=17" \
+  --form-string "fps=5" \
+  --form-string "num_inference_steps=30" \
+  --form-string "guidance_scale=1.0" \
+  --form-string "action_mode=policy" \
+  --form-string "domain_name=droid_lerobot" \
+  | python -c 'import json, sys; print(json.load(sys.stdin)["id"])')
+
+# After the job reaches "completed":
+curl -sS "http://127.0.0.1:30010/v1/videos/${job_id}" \
+  | python -c 'import json, sys; print(json.dumps(json.load(sys.stdin)["action"], indent=2))'
+```
+
+The other action modes are `forward_dynamics` (condition on an observation and an `action` JSON array to generate video) and `inverse_dynamics` (condition on a full video to predict action). Select the embodiment head with `domain_name` or `domain_id`; set `raw_action_dim` explicitly when it cannot be inferred from the domain name.
+
+## 5. Cosmos3 Parameters
+
+Cosmos3 supports the standard SGLang video and image fields such as `size`, `num_frames`, `fps`, `num_inference_steps`, `guidance_scale`, `negative_prompt`, and `seed`.
+
+Top-level Cosmos3 request fields:
+
+- `max_sequence_length`: maximum text token length used by the Cosmos3 tokenizer.
+- `flow_shift`: per-request scheduler shift. If omitted, SGLang uses `--flow-shift`, then the mode default (`3.0` for T2I and `10.0` for video/action).
+
+Cosmos3 omnimodal fields are accepted as extra JSON fields or multipart form fields:
+
+- `generate_sound`: generate a sound track whose duration follows `num_frames / fps`.
+- `sound_duration`: explicit sound duration in seconds; takes precedence over the derived duration.
+- `condition_frame_indexes`: V2V latent-frame indexes to keep from the source video; defaults to `[0, 1]`.
+- `condition_video_keep`: use the `first` or `last` source frames for V2V conditioning.
+- `action_mode`: `policy`, `forward_dynamics`, or `inverse_dynamics`.
+- `domain_name` / `domain_id`: select the action embodiment head.
+- `raw_action_dim`: number of active action dimensions; inferred for known domain names.
+- `action`: action array with shape `[T, D]`, required by `forward_dynamics`.
+- `action_fps`: action-token frame rate for temporal mRoPE; defaults to the video FPS.
+- `action_view_point`: viewpoint used in the structured action caption.
+- `action_normalization`: dataset normalization mode, such as `quantile`, `meanstd`, or `minmax`.
+
+Put model-specific compatibility knobs in `extra_params` for video requests, or `extra_args` for image requests:
+
+- `use_duration_template`: whether to append SGLang's generated duration suffix to video prompts.
+- `use_resolution_template`: accepted for vLLM-Omni request compatibility.
+- `use_system_prompt`: whether to add the Cosmos3 system prompt to the chat template.
+- `guardrails` or `use_guardrails`: per-request guardrail toggle when the server started with guardrails enabled.
@@ -0,0 +1,81 @@
+---
+title: ERNIE-Image
+metatags:
+    description: "Deploy ERNIE-Image and ERNIE-Image-Turbo with SGLang Diffusion."
+---
+
+import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';
+
+<DiffusionModelTags tags={["image", "text-to-image", "turbo"]} />
+
+## 1. Model introduction
+
+[ERNIE-Image](https://huggingface.co/baidu/ERNIE-Image) is Baidu's text-to-image diffusion model family. SGLang Diffusion supports both the regular and Turbo checkpoints with the native `ErnieImagePipeline`.
+
+| Model | Hugging Face model ID | Notes |
+| --- | --- | --- |
+| ERNIE-Image | `baidu/ERNIE-Image` | Regular text-to-image checkpoint |
+| ERNIE-Image-Turbo | `baidu/ERNIE-Image-Turbo` | Turbo text-to-image checkpoint |
+
+## 2. Installation
+
+Install SGLang with the diffusion dependencies:
+
+```bash Command
+pip install -e "python[diffusion]"
+```
+
+For full installation options, see the [SGLang Diffusion installation guide](/docs/sglang-diffusion/installation).
+
+## 3. Serve the model
+
+The commands below target a single supported NVIDIA CUDA or AMD ROCm GPU. Start with `--performance-mode auto`; use `speed` only when the full pipeline fits comfortably on the selected GPU(s), and use `memory` when you need lower peak GPU memory.
+
+Serve ERNIE-Image:
+
+```bash Command
+sglang serve \
+  --model-path baidu/ERNIE-Image \
+  --num-gpus 1 \
+  --performance-mode auto \
+  --port 30010
+```
+
+Serve ERNIE-Image-Turbo:
+
+```bash Command
+sglang serve \
+  --model-path baidu/ERNIE-Image-Turbo \
+  --num-gpus 1 \
+  --performance-mode auto \
+  --port 30010
+```
+
+## 4. Generate an image
+
+Use the OpenAI-compatible image generation API after the server starts:
+
+```python Python
+import base64
+from openai import OpenAI
+
+client = OpenAI(api_key="EMPTY", base_url="http://127.0.0.1:30010/v1")
+
+response = client.images.generate(
+    model="baidu/ERNIE-Image-Turbo",
+    prompt="A cinematic photo of a quiet lakeside cabin at sunrise",
+    n=1,
+    response_format="b64_json",
+)
+
+image_bytes = base64.b64decode(response.data[0].b64_json)
+with open("ernie_image.png", "wb") as f:
+    f.write(image_bytes)
+```
+
+## 5. Configuration tips
+
+- ERNIE-Image is a text-to-image pipeline; do not pass `--image-path`.
+- `--performance-mode auto` keeps conservative defaults while preserving explicit user flags.
+- If the checkpoint includes a PE component, SGLang loads it automatically from `model_index.json`.
+- Treat FSDP, SP/Ulysses/Ring, and TP as explicit benchmark knobs. Measure the target resolution, step count, and GPU type before making them production defaults.
@@ -0,0 +1,391 @@
+---
+title: FLUX
+metatags:
+    description: "Deploy FLUX diffusion models with SGLang - 12B/32B rectified flow transformers for high-quality text-to-image generation."
+---
+
+import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';
+import { FluxDeployment } from '/src/snippets/diffusion/flux-deployment.jsx';
+
+<DiffusionModelTags tags={["image", "text-to-image", "image editing", "multi-reference"]} />
+
+## 1. Model Introduction
+
+[FLUX](https://blackforestlabs.ai/) is a family of rectified flow transformer models developed by Black Forest Labs for high-quality image generation from text descriptions.
+
+[FLUX.1-dev](https://huggingface.co/black-forest-labs/FLUX.1-dev) is a 12 billion parameter rectified flow transformer capable of generating images from text descriptions.
+
+**Key Features:**
+
+- **Cutting-edge Output Quality**: Second only to the state-of-the-art FLUX.1 [pro] model
+- **Competitive Prompt Following**: Matches the performance of closed-source alternatives
+- **Guidance Distillation**: Trained using guidance distillation for improved efficiency
+- **Open Weights**: Available for personal, scientific, and commercial purposes under the FLUX [dev] Non-Commercial License
+
+[FLUX.2-dev](https://huggingface.co/black-forest-labs/FLUX.2-dev) is a 32 billion parameter rectified flow transformer capable of generating, editing, and combining images based on text instructions.
+
+**Key Features:**
+
+- **State-of-the-art Performance**: Leading open model in text-to-image generation, single-reference editing, and multi-reference editing
+- **No Finetuning Required**: Character, object, and style reference without additional training in one model
+- **Guidance Distillation**: Trained using guidance distillation for improved efficiency
+- **Open Weights**: Available for personal, scientific, and commercial purposes under the FLUX [dev] Non-Commercial License
+
+For more details, please refer to the [FLUX.1-dev HuggingFace page](https://huggingface.co/black-forest-labs/FLUX.1-dev), [FLUX.2-dev HuggingFace page](https://huggingface.co/black-forest-labs/FLUX.2-dev), and the [official blog post](https://blackforestlabs.ai/announcing-black-forest-labs/).
+
+## 2. SGLang-diffusion Installation
+
+SGLang-diffusion offers multiple installation methods. You can choose the most suitable installation method based on your hardware platform and requirements.
+
+Please refer to the [official SGLang-diffusion installation guide](https://docs.sglang.io/docs/sglang-diffusion/installation) for installation instructions.
+
+## 3. Model Deployment
+
+This section provides deployment configurations optimized for different hardware platforms and use cases.
+
+### 3.1 Basic Configuration
+
+FLUX models are optimized for high-quality image generation. The recommended launch configurations vary by hardware and model version.
+
+**Interactive Command Generator**: Use the configuration selector below to automatically generate the appropriate deployment command for your hardware platform and model version. SGLang supports serving FLUX on NVIDIA B200, H200, H100, and AMD MI355X, MI325X, MI300X GPUs and Ascend A2, A3 NPUs.
+
+<FluxDeployment />
+
+### 3.2 Configuration Tips
+
+Currently supported optimizations are listed [here](/docs/sglang-diffusion/compatibility_matrix).
+
+- `--vae-path`: Path to a custom VAE model or HuggingFace model ID (e.g., fal/FLUX.2-Tiny-AutoEncoder). If not specified, the VAE will be loaded from the main model path.
+- `--num-gpus`: Number of GPUs to use
+- `--tp-size`: Tensor parallelism size (only for the encoder; should not be larger than 1 if text encoder offload is enabled, as layer-wise offload plus prefetch is faster)
+- `--sp-degree`: Sequence parallelism size (typically should match the number of GPUs)
+- `--ulysses-degree`: The degree of DeepSpeed-Ulysses-style SP in USP
+- `--ring-degree`: The degree of ring attention-style SP in USP
+
+## 4. API Usage
+
+For complete API documentation, please refer to the [official API usage guide](/docs/sglang-diffusion/api/openai_api).
+
+### 4.1 Generate an Image
+
+```python Example
+import base64
+from openai import OpenAI
+
+client = OpenAI(api_key="EMPTY", base_url="http://localhost:3000/v1")
+
+response = client.images.generate(
+    model="black-forest-labs/FLUX.1-dev",
+    prompt="A cat holding a sign that says hello world",
+    size="1024x1024",
+    n=1,
+    response_format="b64_json",
+)
+
+# Save the generated image
+image_bytes = base64.b64decode(response.data[0].b64_json)
+with open("output.png", "wb") as f:
+    f.write(image_bytes)
+```
+
+### 4.2 Advanced Usage
+
+#### 4.2.1 Cache-DiT Acceleration
+
+SGLang integrates [Cache-DiT](https://github.com/vipshop/cache-dit), a caching acceleration engine for Diffusion Transformers (DiT), to achieve up to 7.4x inference speedup with minimal quality loss. You can set `SGLANG_CACHE_DIT_ENABLED=True` to enable it. For more details, please refer to the SGLang Cache-DiT [documentation](/docs/sglang-diffusion/cache_dit).
+
+**Basic Usage**
+
+```bash Command
+SGLANG_CACHE_DIT_ENABLED=true sglang serve --model-path black-forest-labs/FLUX.1-dev
+```
+
+**Advanced Usage**
+
+- DBCache Parameters: DBCache controls block-level caching behavior:
+
+<table style={{width: "100%", borderCollapse: "collapse", tableLayout: "fixed"}}>
+  <colgroup>
+    <col style={{width: "25.0%"}} />
+    <col style={{width: "25.0%"}} />
+    <col style={{width: "25.0%"}} />
+    <col style={{width: "25.0%"}} />
+  </colgroup>
+  <thead>
+    <tr style={{borderBottom: "2px solid #d55816"}}>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Parameter</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>Env Variable</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Default</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>Description</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Fn</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_FN`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>1</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Number of first blocks to always compute</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Bn</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_BN`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>0</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Number of last blocks to always compute</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>W</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_WARMUP`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>4</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Warmup steps before caching starts</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>R</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_RDT`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>0.24</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Residual difference threshold</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>MC</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_MC`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>3</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Maximum continuous cached steps</td>
+    </tr>
+  </tbody>
+</table>
+- TaylorSeer Configuration: TaylorSeer improves caching accuracy using Taylor expansion:
+
+<table style={{width: "100%", borderCollapse: "collapse", tableLayout: "fixed"}}>
+  <colgroup>
+    <col style={{width: "25.0%"}} />
+    <col style={{width: "25.0%"}} />
+    <col style={{width: "25.0%"}} />
+    <col style={{width: "25.0%"}} />
+  </colgroup>
+  <thead>
+    <tr style={{borderBottom: "2px solid #d55816"}}>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Parameter</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>Env Variable</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Default</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>Description</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Enable</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_TAYLORSEER`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>false</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Enable TaylorSeer calibrator</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Order</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_TS_ORDER`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>1</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Taylor expansion order (1 or 2)</td>
+    </tr>
+  </tbody>
+</table>
+
+  Combined Configuration Example:
+
+```bash Command
+SGLANG_CACHE_DIT_ENABLED=true \
+SGLANG_CACHE_DIT_FN=2 \
+SGLANG_CACHE_DIT_BN=1 \
+SGLANG_CACHE_DIT_WARMUP=4 \
+SGLANG_CACHE_DIT_RDT=0.4 \
+SGLANG_CACHE_DIT_MC=4 \
+SGLANG_CACHE_DIT_TAYLORSEER=true \
+SGLANG_CACHE_DIT_TS_ORDER=2 \
+sglang serve --model-path black-forest-labs/FLUX.1-dev
+```
+
+#### 4.2.2 CPU Offload
+
+- `--dit-cpu-offload`: Use CPU offload for DiT inference. Enable if run out of memory.
+- `--text-encoder-cpu-offload`: Use CPU offload for text encoder inference.
+- `--vae-cpu-offload`: Use CPU offload for VAE.
+- `--pin-cpu-memory`: Pin memory for CPU offload. Only added as a temp workaround if it throws "CUDA error: invalid argument".
+
+## 5. Benchmark
+
+### 5.1 Speedup Benchmark
+
+#### 5.1.1 Generate a image
+
+Test Environment:
+
+- Hardware: NVIDIA B200 GPU (1x)
+- Model: black-forest-labs/FLUX.1-dev
+- sglang diffusion version: 0.5.6.post2
+
+<Tabs>
+  <Tab title="NVIDIA B200">
+    **Server Command**:
+
+    ```shell Command
+    sglang serve --model-path black-forest-labs/FLUX.1-dev --port 30000
+    ```
+
+    **Benchmark Command**:
+
+    ```shell Command
+    python3 -m sglang.multimodal_gen.benchmarks.bench_serving \
+        --backend sglang-video --dataset vbench --task t2v --num-prompts 1 --max-concurrency 1
+    ```
+
+    **Result**:
+
+    ```text Output
+    ================= Serving Benchmark Result =================
+    Backend:                                 sglang-image
+    Model:                                   black-forest-labs/FLUX.1-dev
+    Dataset:                                 vbench
+    Task:                                    t2v
+    --------------------------------------------------
+    Benchmark duration (s):                  50.97
+    Request rate:                            inf
+    Max request concurrency:                 1
+    Successful requests:                     1/1
+    --------------------------------------------------
+    Request throughput (req/s):              0.02
+    Latency Mean (s):                        50.9681
+    Latency Median (s):                      50.9681
+    Latency P99 (s):                         50.9681
+    --------------------------------------------------
+    Peak Memory Max (MB):                    27905.19
+    Peak Memory Mean (MB):                   27905.19
+    Peak Memory Median (MB):                 27905.19
+    ============================================================
+    ```
+  </Tab>
+  <Tab title="Ascend A3">
+    **Server Command**:
+
+    ```shell Command
+    #One A3 card has 2 npu chips
+    sglang serve --tp-size 2 --sp-degree 1 --model-path black-forest-labs/FLUX.1-dev --num-gpus 2
+    ```
+
+    **Benchmark Command**:
+
+    ```shell Command
+    python -m sglang.multimodal_gen.benchmarks.bench_serving --dataset vbench --task text-to-image --num-prompts 1 --max-concurrency 1
+    ```
+
+    **Result**:
+
+    ```text Output
+    ================= Serving Benchmark Result =================
+    Task:                                         text-to-image
+    Model:                                        black-forest-labs/FLUX.1-dev
+    Dataset:                                      vbench
+    --------------------------------------------------
+    Benchmark duration (s):                       16.30
+    Request rate:                                 inf
+    Max request concurrency:                      1
+    Successful requests:                          1/1
+    Completed outputs:                            1
+    Outputs per prompt:                           1
+    --------------------------------------------------
+    Request throughput (req/s):                   0.06
+    Output throughput (outputs/s):                0.06
+    Latency Mean (s):                             16.30
+    Latency Median (s):                           16.30
+    Latency P90 (s):                              16.30
+    Latency P95 (s):                              16.30
+    Latency P99 (s):                              16.30
+    --------------------------------------------------
+    Peak Memory Max (MB):                         19972.00
+    Peak Memory Mean (MB):                        19972.00
+    Peak Memory Median (MB):                      19972.00
+    ------------------------------------------------------------
+    ```
+  </Tab>
+</Tabs>
+
+#### 5.1.2 Generate images with high concurrency
+
+<Tabs>
+  <Tab title="NVIDIA B200">
+    **Server Command** :
+
+    ```shell Command
+    sglang serve --model-path black-forest-labs/FLUX.1-dev --port 30000
+    ```
+
+    **Benchmark Command** :
+
+    ```shell Command
+    python3 -m sglang.multimodal_gen.benchmarks.bench_serving \
+        --backend sglang-image --dataset vbench --task t2v --num-prompts 20 --max-concurrency 20
+    ```
+
+    **Result** :
+
+    ```text Output
+    ================= Serving Benchmark Result =================
+    Backend:                                 sglang-image
+    Model:                                   black-forest-labs/FLUX.1-dev
+    Dataset:                                 vbench
+    Task:                                    t2v
+    --------------------------------------------------
+    Benchmark duration (s):                  111.79
+    Request rate:                            inf
+    Max request concurrency:                 20
+    Successful requests:                     20/20
+    --------------------------------------------------
+    Request throughput (req/s):              0.18
+    Latency Mean (s):                        67.0646
+    Latency Median (s):                      66.9691
+    Latency P99 (s):                         110.8949
+    --------------------------------------------------
+    Peak Memory Max (MB):                    27917.19
+    Peak Memory Mean (MB):                   27916.59
+    Peak Memory Median (MB):                 27917.19
+    ============================================================
+    ```
+  </Tab>
+  <Tab title="Ascend A3">
+    **Server Command** :
+
+    ```shell Command
+    #One A3 card has 2 npu chips
+    sglang serve --tp-size 2 --sp-degree 1 --model-path black-forest-labs/FLUX.1-dev --num-gpus 2
+    ```
+
+    **Benchmark Command** :
+
+    ```shell Command
+    python -m sglang.multimodal_gen.benchmarks.bench_serving --dataset vbench --task text-to-image --num-prompts 20 --max-concurrency 20
+    ```
+
+    **Result** :
+
+    ```text Output
+    ================= Serving Benchmark Result =================
+    Task:                                         text-to-image
+    Model:                                        black-forest-labs/FLUX.1-dev
+    Dataset:                                      vbench
+    --------------------------------------------------
+    Benchmark duration (s):                       300.85
+    Request rate:                                 inf
+    Max request concurrency:                      20
+    Successful requests:                          18/20
+    Completed outputs:                            18
+    Outputs per prompt:                           1
+    --------------------------------------------------
+    Request throughput (req/s):                   0.06
+    Output throughput (outputs/s):                0.06
+    Latency Mean (s):                             155.16
+    Latency Median (s):                           155.11
+    Latency P90 (s):                              266.30
+    Latency P95 (s):                              280.15
+    Latency P99 (s):                              291.23
+    --------------------------------------------------
+    Peak Memory Max (MB):                         19972.00
+    Peak Memory Mean (MB):                        19972.00
+    Peak Memory Median (MB):                      19972.00
+    ------------------------------------------------------------
+    ```
+  </Tab>
+</Tabs>
@@ -0,0 +1,189 @@
+---
+title: Ideogram 4
+metatags:
+    description: "Deploy Ideogram 4 with SGLang Diffusion for high-aesthetic text-to-image generation."
+---
+
+import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';
+
+<DiffusionModelTags tags={["image", "text-to-image", "typography", "NF4/FP8/NVFP4"]} />
+
+## 1. Model introduction
+
+[Ideogram 4](https://huggingface.co/ideogram-ai/ideogram-4-nf4) is Ideogram's text-to-image diffusion model. SGLang Diffusion supports the official NF4 and FP8 checkpoints, the Comfy-Org NVFP4 transformer checkpoint, and fal's single-branch Fast and Instant variants.
+
+Compared with previous open-source image models, Ideogram 4 provides a significant aesthetic lift, with stronger composition, more polished visual style, and better typography-aware generation.
+
+| Variant | Hugging Face model ID | Notes |
+| --- | --- | --- |
+| NF4 | `ideogram-ai/ideogram-4-nf4` | Official bitsandbytes NF4 checkpoint. Use this path first for low-memory deployment. |
+| FP8 | `ideogram-ai/ideogram-4-fp8` | Official FP8 checkpoint. |
+| NVFP4 | `Comfy-Org/Ideogram-4` | Comfy-Org NVFP4 transformer weights. SGLang loads non-transformer components from `ideogram-ai/ideogram-4-fp8`. |
+| Fast | `fal/ideogram-v4-fast` | 20-step, CFG-distilled, FP4-targeted floating checkpoint. Defaults to `V4_FAST_20`. |
+| Instant | `fal/ideogram-v4-instant` | 8-step, CFG- and timestep-distilled BF16 checkpoint. Defaults to `V4_INSTANT_8`. |
+
+The fal repositories contain only the transformer component. When either model ID is passed directly to `--model-path`, SGLang loads the text encoder, tokenizer, VAE, and scheduler from the `ideogram-ai/ideogram-4-nf4-diffusers` revision referenced by fal's model cards, then runs the distilled transformer without the unconditional branch.
+
+## 2. Prerequisites
+
+- NVIDIA CUDA GPU.
+- SGLang installed with diffusion dependencies.
+- `bitsandbytes>=0.46.1` and `accelerate>=1.1.0` for the NF4 checkpoint and the fal variants' shared NF4 text encoder.
+- `HF_TOKEN` with access to the Ideogram 4 gated repositories.
+
+## 3. Serve the model
+
+NF4:
+
+```bash Command
+HF_TOKEN=$HF_TOKEN sglang serve \
+  --model-path ideogram-ai/ideogram-4-nf4 \
+  --num-gpus 1 \
+  --performance-mode auto \
+  --port 30010
+```
+
+FP8:
+
+```bash Command
+HF_TOKEN=$HF_TOKEN sglang serve \
+  --model-path ideogram-ai/ideogram-4-fp8 \
+  --num-gpus 1 \
+  --performance-mode auto \
+  --port 30010
+```
+
+Comfy-Org NVFP4:
+
+```bash Command
+HF_TOKEN=$HF_TOKEN sglang serve \
+  --model-path Comfy-Org/Ideogram-4 \
+  --num-gpus 1 \
+  --performance-mode auto \
+  --port 30010
+```
+
+Use B200 or another Blackwell GPU for NVFP4.
+
+fal Fast:
+
+```bash Command
+HF_TOKEN=$HF_TOKEN sglang serve \
+  --model-path fal/ideogram-v4-fast \
+  --num-gpus 1 \
+  --performance-mode auto \
+  --port 30010
+```
+
+fal Instant:
+
+```bash Command
+HF_TOKEN=$HF_TOKEN sglang serve \
+  --model-path fal/ideogram-v4-instant \
+  --num-gpus 1 \
+  --performance-mode auto \
+  --port 30010
+```
+
+### Tensor and sequence parallelism
+
+Both fal variants support native DiT tensor parallelism and Ulysses sequence parallelism. The following two-GPU layouts were validated at 1024×1024 on B200 GPUs for both `fal/ideogram-v4-fast` and `fal/ideogram-v4-instant`.
+
+TP2 shards the distilled DiT weights. The shared bitsandbytes NF4 text encoder is replicated because 4-bit row-parallel quantization states cannot be safely sharded:
+
+```bash Command
+HF_TOKEN=$HF_TOKEN sglang serve \
+  --model-path fal/ideogram-v4-instant \
+  --num-gpus 2 \
+  --tp-size 2 \
+  --port 30010
+```
+
+Ulysses/SP2 shards the image-token sequence while keeping the DiT weights replicated:
+
+```bash Command
+HF_TOKEN=$HF_TOKEN sglang serve \
+  --model-path fal/ideogram-v4-instant \
+  --num-gpus 2 \
+  --ulysses-degree 2 \
+  --port 30010
+```
+
+Replace the model ID with `fal/ideogram-v4-fast` to use the same layouts for Fast. TP2 and Ulysses/SP2 are the validated configurations; Ring SP is not yet validated for this pipeline. Ideogram 4 has 18 attention heads, so any other TP degree must divide 18.
+
+### Layerwise offload
+
+If the distilled DiT does not fit in GPU memory, enable transformer layerwise offload. This keeps the shared NF4 text encoder on the GPU while streaming the DiT blocks from pinned CPU memory. It reduces peak VRAM at the cost of additional latency:
+
+```bash Command
+HF_TOKEN=$HF_TOKEN sglang serve \
+  --model-path fal/ideogram-v4-instant \
+  --num-gpus 1 \
+  --dit-layerwise-offload \
+  --layerwise-offload-components transformer \
+  --port 30010
+```
+
+The same option applies to Fast. Layerwise offload was validated with a real Instant generation; TP2 and Ulysses/SP2 were validated separately without offload.
+
+The released Fast weights are stored in a floating-point pre-pack form and can be loaded directly, but fal trained them with QAD for an NVFP4 execution path. SGLang does not currently quantize dense NVIDIA DiTs to NVFP4 at load time, so direct Fast inference bypasses the intended quantization path and may be visibly degraded. Treat direct floating-point Fast inference as compatibility/testing support, not the production-quality path. The released Instant checkpoint is BF16 and is the recommended local checkpoint today.
+
+## 4. Generate an image
+
+```python Example
+import base64
+from openai import OpenAI
+
+client = OpenAI(api_key="EMPTY", base_url="http://localhost:30010/v1")
+
+response = client.images.generate(
+    model="ideogram-ai/ideogram-4-nf4",
+    prompt="A cinematic poster of a quiet bookstore at dusk with elegant hand-lettered signage",
+    size="1024x1024",
+    n=1,
+    response_format="b64_json",
+    extra_body={"preset": "V4_QUALITY_48", "seed": 0},
+)
+
+image_bytes = base64.b64decode(response.data[0].b64_json)
+with open("ideogram4.png", "wb") as f:
+    f.write(image_bytes)
+```
+
+fal's hosted endpoint expands natural-language prompts, but these local checkpoints expect Ideogram 4's structured JSON caption format. Pass a complete caption as the API `prompt`, for example:
+
+```python Example
+import json
+
+prompt = json.dumps(
+    {
+        "high_level_description": (
+            "A bold typographic poster centered on the exact words INSTANT BY FAL, "
+            "printed in black and electric orange on warm white paper."
+        ),
+        "compositional_deconstruction": {
+            "background": "Warm white textured paper with generous negative space.",
+            "elements": [
+                {
+                    "type": "text",
+                    "text": "INSTANT BY FAL",
+                    "desc": "Large uppercase geometric sans-serif lettering, precisely centered.",
+                }
+            ],
+        },
+    },
+    ensure_ascii=False,
+    separators=(",", ":"),
+)
+
+response = client.images.generate(
+    model="fal/ideogram-v4-instant",
+    prompt=prompt,
+    size="1024x1024",
+    n=1,
+    response_format="b64_json",
+    extra_body={"seed": 42},
+)
+```
+
+Base Ideogram 4 presets are `V4_DEFAULT_20`, `V4_QUALITY_48`, and `V4_TURBO_12`. The fal variants automatically select `V4_FAST_20` and `V4_INSTANT_8`, respectively. A preset controls both `num_inference_steps` and guidance, so do not set those fields directly.
@@ -0,0 +1,177 @@
+---
+title: JoyAI-Echo
+description: Run JoyAI-Echo multi-shot audio–video generation with SGLang Diffusion.
+metatags:
+    description: "Deploy and use JoyAI-Echo long-form audio–video generation with SGLang Diffusion, including single-shot and multi-shot memory-bank workflows."
+---
+
+import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';
+
+<DiffusionModelTags tags={["video", "audio-video", "multi-shot", "memory bank"]} />
+
+## 1. Model Introduction
+
+[JoyAI-Echo](https://huggingface.co/jdopensource/JoyAI-Echo) (JoyEcho) is a long-form audio–video generation model built on the LTX-2 backbone. Its core idea is a **paired audio–video memory bank**: each shot commits decoded frames and audio latents into a rolling bank, and subsequent shots condition on that memory prefix. This enables **multi-shot, minute-scale generation** with visual and audio continuity across prompts.
+
+Use `jdopensource/JoyAI-Echo` as `--model-path`. SGLang loads the monolithic release through the built-in [JoyAI-Echo-overlay](https://huggingface.co/Niehen6174/JoyAI-Echo-overlay) materialization path, similar to LTX-2.3-overlay.
+
+| Aspect | Standard LTX-2.3 | JoyEcho |
+| --- | --- | --- |
+| Pipeline | `LTX2Pipeline` / `LTX2TwoStageHQPipeline` | `JoyEchoPipeline` (default for this model) |
+| Denoising | Multi-step flow matching + CFG | LTX-2 DMD distilled path (8 steps, `guidance_scale=1.0`) |
+| Multi-shot | Not supported | Paired audio–video memory bank across shots |
+| Sequence parallelism | LTX-2 SP (video/audio sharded) | Ulysses SP (`ulysses_degree=2`): single-shot and multi-shot + memory bank |
+| Post-processing | Optional two-stage HQ upscaling | Per-shot mp4 output |
+
+<Warning>
+Review the model license on the [JoyAI-Echo Hugging Face page](https://huggingface.co/jdopensource/JoyAI-Echo) before production or commercial use. SGLang support does not grant additional model usage rights.
+</Warning>
+
+## 2. SGLang-diffusion Installation
+
+Install SGLang with diffusion dependencies:
+
+```bash
+uv pip install "sglang[diffusion]" --prerelease=allow
+```
+
+For platform-specific setup, see the [SGLang Diffusion installation guide](/docs/sglang-diffusion/installation).
+
+## 3. Model Deployment
+
+JoyEcho uses the default `JoyEchoPipeline` registered for `jdopensource/JoyAI-Echo`. A single high-VRAM GPU (for example H100 or H200) is enough for the common 832x480 / 121-frame / 8-step setting.
+
+```bash
+sglang serve \
+  --model-path jdopensource/JoyAI-Echo
+```
+
+Optional environment variable for long runs:
+
+```bash
+export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
+```
+
+For multi-GPU serving, tensor parallelism (TP) and **Ulysses sequence parallelism (SP)** are supported. JoyEcho SP uses an **asymmetric layout**: video target latents are time-sharded across ranks, while audio (including memory tokens) is **replicated** on every rank so cross-attention stays temporally aligned. Multi-shot runs with `enable_memory_bank=true` are supported on SP.
+
+```bash
+sglang serve \
+  --model-path jdopensource/JoyAI-Echo \
+  --num-gpus 2 \
+  --ulysses-degree 2
+```
+
+<Note>
+JoyEcho SP currently targets **Ulysses-only** parallelism (`ulysses_degree=2`, `ring_degree=1`). Ring SP is not validated for this pipeline. For `sglang generate`, add `--num-gpus 2 --ulysses-degree 2` to the commands in section 4.
+</Note>
+
+## 4. Model Invocation
+
+### 4.1 Default sampling
+
+| Setting | Default |
+| --- | --- |
+| Resolution | 832x480 |
+| Frames | 121 |
+| FPS | 25 |
+| Steps | 8 |
+| Guidance scale | 1.0 |
+| Seed | 12345 |
+
+### 4.2 Single-shot text-to-video
+
+```bash
+sglang generate \
+  --model-path jdopensource/JoyAI-Echo \
+  --prompt "A curious raccoon walks through a sunlit forest path" \
+  --height 480 --width 832 --num-frames 121 --fps 25 \
+  --num-inference-steps 8 --seed 42 \
+  --save-output
+```
+
+Disable the memory bank for standalone clips with a config file:
+
+```bash
+cat > /tmp/joy_echo_single.json <<'EOF'
+{
+  "model_path": "jdopensource/JoyAI-Echo",
+  "prompt": "A curious raccoon walks through a sunlit forest path",
+  "enable_memory_bank": false,
+  "seed": 42,
+  "height": 480,
+  "width": 832,
+  "num_frames": 121,
+  "fps": 25,
+  "num_inference_steps": 8
+}
+EOF
+
+sglang generate --config /tmp/joy_echo_single.json --save-output
+```
+
+### 4.3 Multi-shot generation
+
+JoyEcho does **not** generate all shots in one forward pass. Each shot is one generation request. Continuity is carried by an in-process **memory bank** on the pipeline instance.
+
+Typical workflow:
+
+1. **Shot 0** — memory bank is empty; the model generates a standalone A/V clip.
+2. **After decode** — decoded video frames and packed audio latents are committed to the memory bank (up to 7 slots by default).
+3. **Shot 1+** — prior-shot frames are re-encoded and prepended as a memory prefix before denoising.
+4. **Per-shot seeding** — official semantics use `prompt_seed = base_seed + shot_index`.
+
+Pass multiple prompts as a list in a config file:
+
+```bash
+cat > /tmp/joy_echo_4shot.json <<'EOF'
+{
+  "model_path": "jdopensource/JoyAI-Echo",
+  "prompt": [
+    "Shot 0: A raccoon wakes up in a cozy attic.",
+    "Shot 1: The raccoon climbs down and opens the back door.",
+    "Shot 2: It walks through a rainy alley under neon signs.",
+    "Shot 3: The raccoon finds a warm bakery window and stops."
+  ],
+  "enable_memory_bank": true,
+  "reset_memory_bank": true,
+  "seed": 42,
+  "height": 480,
+  "width": 832,
+  "num_frames": 121,
+  "fps": 25,
+  "num_inference_steps": 8
+}
+EOF
+
+sglang generate --config /tmp/joy_echo_4shot.json --save-output
+```
+
+You can also pass prompts from a text file (one prompt per line) with `--prompt-path`:
+
+```bash
+sglang generate \
+  --model-path jdopensource/JoyAI-Echo \
+  --prompt-path /tmp/joy_echo_shots.txt \
+  --seed 42 \
+  --height 480 --width 832 --num-frames 121 --fps 25 \
+  --num-inference-steps 8 \
+  --save-output
+```
+
+### 4.4 Memory bank controls
+
+| Parameter | Default | Meaning |
+| --- | --- | --- |
+| `enable_memory_bank` | `true` | Read/write the paired A/V memory bank between shots. |
+| `reset_memory_bank` | `true` | Clear the bank and shot counter at the start of a new session (`request_id` change or first shot). |
+
+Set `enable_memory_bank=false` when you want independent shots without cross-shot continuity.
+
+## 5. Practical Tips
+
+- Use `--num-inference-steps 8` and `--guidance-scale 1.0` to match the official JoyEcho DMD distilled path.
+- Multi-shot prompts can be passed as a `prompt` list, via `prompt_path`, or as sequential API calls on the same server instance.
+- The memory bank caps at **7 slots**; from shot 8 onward the oldest slots roll off.
+- For **2-GPU latency**, try **Ulysses SP** (`--num-gpus 2 --ulysses-degree 2`) on both single-shot and multi-shot runs. Use **TP** when you need a different sharding strategy or more than two GPUs.
+- Set `PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True` for long multi-shot SP sessions.
+- JoyEcho outputs per-shot mp4 files with synchronized audio. There is no built-in two-stage HQ upscaling path like LTX-2.3 HQ.
@@ -0,0 +1,340 @@
+---
+title: Krea-2
+metatags:
+    description: "Deploy Krea-2 with SGLang - fast, high-quality text-to-image generation."
+---
+
+import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';
+
+<DiffusionModelTags tags={["image", "text-to-image", "turbo", "raw"]} />
+
+## 1. Model Introduction
+
+[Krea-2](https://huggingface.co/krea/Krea-2-Turbo) is a high-quality text-to-image diffusion model from [Krea](https://www.krea.ai/). It ships in two variants that share the same backbone and differ only in their sampling recipe:
+
+- **[Krea-2-Turbo](https://huggingface.co/krea/Krea-2-Turbo)** - a distilled, few-step model that produces photorealistic images in only **8 inference steps** with no classifier-free guidance (`guidance_scale = 1.0`), ideal for fast and interactive generation.
+- **[Krea-2-Raw](https://huggingface.co/krea/Krea-2-Raw)** - the base (non-distilled) model that trades speed for maximum fidelity, using a longer schedule (~52 steps) with classifier-free guidance (`guidance_scale ≈ 4.5`).
+
+Both variants are built on a single-stream MMDiT with a Qwen3-VL text encoder and the Qwen-Image VAE, and are distributed in the standard diffusers layout (a `model_index.json` plus sharded `transformer/`, `text_encoder/`, `vae/`, `tokenizer/`, and `scheduler/` folders). SGLang loads them **natively** - just point `--model-path` at the repo, no conversion step required.
+
+**Key Features:**
+
+- **Two variants, one pipeline**: switch between fast (Turbo) and high-fidelity (Raw) by changing only the model path and the sampling settings.
+- **Photorealistic generation** at 1024x1024 and other resolutions.
+- **Native diffusers loading**: components (DiT, text encoder, VAE, scheduler) are read straight from the repo's `model_index.json`.
+
+For more details, see the [Krea-2-Turbo](https://huggingface.co/krea/Krea-2-Turbo) and [Krea-2-Raw](https://huggingface.co/krea/Krea-2-Raw) HuggingFace pages.
+
+## 2. SGLang-diffusion Installation
+
+SGLang-diffusion offers multiple installation methods. You can choose the most suitable installation method based on your hardware platform and requirements.
+
+Please refer to the [official SGLang-diffusion installation guide](https://docs.sglang.io/docs/sglang-diffusion/installation) for installation instructions.
+
+## 3. Model Deployment
+
+This section covers deploying Krea-2-Turbo for fast, high-quality image generation.
+
+### 3.1 Basic Configuration
+
+Krea-2-Turbo generates high-quality images in only 8 inference steps. Launch the server with:
+
+```bash Command
+sglang serve \
+  --model-path krea/Krea-2-Turbo \
+  --num-gpus 1 \
+  --port 30000
+```
+
+The step count and guidance scale are **request-time** settings (see [API Usage](#4-api-usage)); Krea-2-Turbo defaults to 8 steps with `guidance_scale = 1.0`.
+
+### 3.2 Configuration Tips
+
+Currently supported optimizations are listed [here](/docs/sglang-diffusion/compatibility_matrix).
+
+- `--num-gpus`: Number of GPUs to use.
+- Multi-GPU (tensor and/or sequence parallelism): see [Section 3.3](#3-3-multi-gpu-tensor-and-sequence-parallelism).
+
+### 3.3 Multi-GPU: tensor and sequence parallelism
+
+Krea-2 supports two multi-GPU axes that can be combined; `--num-gpus` must equal
+`tp_size × ulysses_degree`.
+
+- **Tensor parallelism (`--tp-size N`)** shards the DiT weights across GPUs, lowering
+  per-GPU VRAM. Krea-2's attention heads (48 query / 12 KV) and text heads (20) are
+  divisible by a tp size of 1, 2, or 4.
+- **Sequence parallelism / Ulysses (`--ulysses-degree N`)** shards the image-token
+  sequence across GPUs while keeping the text prefix replicated. It does **not** shard
+  weights (per-GPU VRAM is unchanged), but its output is **bitwise-identical** to
+  single-GPU. It currently requires a single prompt per request (ragged/padded
+  multi-prompt batches under SP are not supported — use `--tp-size` for those).
+
+```bash Command
+# Tensor parallel (2 GPUs) — lowest per-GPU VRAM (DiT weights sharded)
+sglang serve --model-path krea/Krea-2-Turbo --num-gpus 2 --tp-size 2 --port 30000
+
+# Sequence parallel / Ulysses (2 GPUs) — output bitwise-identical to single-GPU
+sglang serve --model-path krea/Krea-2-Turbo --num-gpus 2 --ulysses-degree 2 --port 30000
+
+# Hybrid TP × SP (4 GPUs) — composes both axes
+sglang serve --model-path krea/Krea-2-Turbo --num-gpus 4 --tp-size 2 --ulysses-degree 2 --port 30000
+```
+
+Measured on 2× H200 (Krea-2-Turbo, 8 steps, 1024×1024): `--tp-size 2` and
+`--ulysses-degree 2` each give ~1.7× denoise speedup over single-GPU; the hybrid
+TP=2 × SP=2 reaches ~2.8× on 4 GPUs. **Choosing:** on memory-constrained GPUs prefer
+`--tp-size` (it shards the ~24 GB DiT, e.g. ~38 GB → ~27 GB per GPU on 2 GPUs); on
+large-VRAM GPUs sequence parallelism is marginally faster and numerically exact, and
+the two compose for the highest throughput.
+
+## 4. API Usage
+
+For complete API documentation, please refer to the [official API usage guide](/docs/sglang-diffusion/api/openai_api).
+
+### 4.1 Generate an Image
+
+Generate an image with the OpenAI-compatible images API:
+
+```python Example
+import base64
+from openai import OpenAI
+
+client = OpenAI(api_key="EMPTY", base_url="http://localhost:30000/v1")
+
+response = client.images.generate(
+    model="krea/Krea-2-Turbo",
+    prompt="a red fox sitting in fresh snow, golden hour, photorealistic",
+    n=1,
+    response_format="b64_json",
+)
+
+# Save the generated image
+image_bytes = base64.b64decode(response.data[0].b64_json)
+with open("output.png", "wb") as f:
+    f.write(image_bytes)
+```
+
+You can also generate a single image from the command line:
+
+```bash Command
+sglang generate --model-path krea/Krea-2-Turbo \
+    --prompt "a red fox sitting in fresh snow, golden hour, photorealistic" \
+    --num-inference-steps 8 --height 1024 --width 1024 --save-output
+```
+
+### 4.2 Advanced Usage
+
+#### 4.2.1 Cache-DiT Acceleration
+
+SGLang integrates [Cache-DiT](https://github.com/vipshop/cache-dit), a caching acceleration engine for Diffusion Transformers (DiT), to speed up inference with minimal quality loss. Enable it by setting `SGLANG_CACHE_DIT_ENABLED=true`. For more details, see the SGLang Cache-DiT [documentation](/docs/sglang-diffusion/cache_dit).
+
+Cache-DiT works for **both** Krea-2 variants with no extra configuration: SGLang tracks each request's classifier-free-guidance mode, so Krea-2-Turbo (no CFG, `guidance_scale = 1.0`) and Krea-2-Raw (CFG, `guidance_scale ≈ 4.5`) both cache correctly and automatically.
+
+**Basic Usage**
+
+```bash Command
+SGLANG_CACHE_DIT_ENABLED=true sglang serve \
+  --model-path krea/Krea-2-Turbo \
+  --num-gpus 1 \
+  --port 30000
+```
+
+Measured per-image denoise speedup with the default cache settings (NVIDIA H200, 1024x1024, seed 0):
+
+| Variant | Inference steps | Denoise (no cache → cache) | Speedup |
+| :--- | :--- | :--- | :--- |
+| Krea-2-Turbo (no CFG) | 8 | 1.27s → 0.92s | ~1.4x |
+| Krea-2-Raw (CFG 4.5) | 50 | 18.0s → 6.3s | ~2.9x |
+
+Caching has the most headroom on Raw's longer schedule; the 8-step distilled Turbo has only a few cacheable steps after warmup.
+
+**Advanced Usage**
+
+- DBCache Parameters: DBCache controls block-level caching behavior:
+
+<table style={{width: "100%", borderCollapse: "collapse", tableLayout: "fixed"}}>
+  <colgroup>
+    <col style={{width: "25.0%"}} />
+    <col style={{width: "25.0%"}} />
+    <col style={{width: "25.0%"}} />
+    <col style={{width: "25.0%"}} />
+  </colgroup>
+  <thead>
+    <tr style={{borderBottom: "2px solid #d55816"}}>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Parameter</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>Env Variable</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Default</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>Description</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Fn</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_FN`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>1</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Number of first blocks to always compute</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Bn</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_BN`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>0</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Number of last blocks to always compute</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>W</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_WARMUP`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>4</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Warmup steps before caching starts</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>R</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_RDT`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>0.24</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Residual difference threshold</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>MC</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_MC`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>3</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Maximum continuous cached steps</td>
+    </tr>
+  </tbody>
+</table>
+- TaylorSeer Configuration: TaylorSeer improves caching accuracy using Taylor expansion (best suited to the longer Raw schedule; not recommended for the 8-step Turbo):
+
+<table style={{width: "100%", borderCollapse: "collapse", tableLayout: "fixed"}}>
+  <colgroup>
+    <col style={{width: "25.0%"}} />
+    <col style={{width: "25.0%"}} />
+    <col style={{width: "25.0%"}} />
+    <col style={{width: "25.0%"}} />
+  </colgroup>
+  <thead>
+    <tr style={{borderBottom: "2px solid #d55816"}}>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Parameter</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>Env Variable</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Default</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>Description</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Enable</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_TAYLORSEER`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>false</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Enable TaylorSeer calibrator</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Order</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_TS_ORDER`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>1</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Taylor expansion order (1 or 2)</td>
+    </tr>
+  </tbody>
+</table>
+
+  Combined Configuration Example (Krea-2-Raw, default cache settings shown explicitly):
+
+```bash Command
+SGLANG_CACHE_DIT_ENABLED=true \
+SGLANG_CACHE_DIT_FN=1 \
+SGLANG_CACHE_DIT_BN=0 \
+SGLANG_CACHE_DIT_WARMUP=4 \
+SGLANG_CACHE_DIT_RDT=0.24 \
+SGLANG_CACHE_DIT_MC=3 \
+sglang serve --model-path krea/Krea-2-Raw
+```
+
+#### 4.2.2 Memory & CPU Offload
+
+Krea-2's DiT is ~24 GB in bf16 (the bulk of the model). On memory-constrained GPUs you can keep less of it resident:
+
+- `--dit-layerwise-offload`: stream the DiT's transformer blocks layer-by-layer with async host-to-device prefetch overlap, so only a small working set stays on the GPU. This is the primary way to fit Krea-2 on a single consumer / 32 GB-class card, at a modest latency cost. Tune the memory/latency trade-off with `--dit-offload-prefetch-size` (`0.0` prefetches one layer for the lowest memory; larger values prefetch more layers -- faster but more memory).
+- `--dit-cpu-offload`: keep the whole DiT in host memory. Combine it with `--dit-layerwise-offload` for the lowest peak GPU memory (weights stay on host and only the layers needed for the current step are brought on-device).
+- `--text-encoder-cpu-offload`: offload the Qwen3-VL text encoder (it is idle during the denoise loop).
+- `--vae-cpu-offload`: offload the VAE.
+- `--pin-cpu-memory`: pin host memory for offload. Add only as a temporary workaround if you hit `CUDA error: invalid argument`.
+
+On large-VRAM GPUs (e.g. H200), keep everything resident (offloads off) for the fastest latency.
+
+## 5. Benchmark
+
+Test Environment:
+
+- Hardware: NVIDIA H200 GPU (1x)
+- Model: krea/Krea-2-Turbo (8 inference steps)
+- sglang diffusion version: 0.5.13
+
+**Server Command** (used for both benchmarks below):
+
+```shell Command
+sglang serve --model-path krea/Krea-2-Turbo --port 30000
+```
+
+### 5.1 Generate an image
+
+**Benchmark Command**:
+
+```shell Command
+python3 -m sglang.multimodal_gen.benchmarks.bench_serving \
+    --model krea/Krea-2-Turbo --dataset vbench --task text-to-image \
+    --num-prompts 1 --max-concurrency 1
+```
+
+**Result**:
+
+```text Output
+================= Serving Benchmark Result =================
+Task:                                    text-to-image
+Model:                                   krea/Krea-2-Turbo
+Dataset:                                 vbench
+--------------------------------------------------
+Benchmark duration (s):                  1.56
+Request rate:                            inf
+Max request concurrency:                 1
+Successful requests:                     1/1
+--------------------------------------------------
+Request throughput (req/s):              0.64
+Latency Mean (s):                        1.5600
+Latency Median (s):                      1.5600
+Latency P99 (s):                         1.5600
+--------------------------------------------------
+Peak Memory Max (MB):                    37466.00
+Peak Memory Mean (MB):                   37466.00
+Peak Memory Median (MB):                 37466.00
+============================================================
+```
+
+### 5.2 Generate images with high concurrency
+
+**Benchmark Command**:
+
+```shell Command
+python3 -m sglang.multimodal_gen.benchmarks.bench_serving \
+    --model krea/Krea-2-Turbo --dataset vbench --task text-to-image \
+    --num-prompts 20 --max-concurrency 20
+```
+
+**Result**:
+
+```text Output
+================= Serving Benchmark Result =================
+Task:                                    text-to-image
+Model:                                   krea/Krea-2-Turbo
+Dataset:                                 vbench
+--------------------------------------------------
+Benchmark duration (s):                  31.47
+Request rate:                            inf
+Max request concurrency:                 20
+Successful requests:                     20/20
+--------------------------------------------------
+Request throughput (req/s):              0.64
+Latency Mean (s):                        16.5000
+Latency Median (s):                      16.5200
+Latency P99 (s):                         31.1300
+--------------------------------------------------
+Peak Memory Max (MB):                    37468.00
+Peak Memory Mean (MB):                   37466.40
+Peak Memory Median (MB):                 37466.00
+============================================================
+```
@@ -0,0 +1,250 @@
+---
+title: LTX2 & LTX2.3
+description: Run LTX-2 and LTX-2.3 video generation pipelines with SGLang Diffusion.
+metatags:
+    description: "Deploy and use LTX-2 and LTX-2.3 video generation models with SGLang Diffusion, including one-stage, two-stage, HQ, TI2V, and LoRA examples."
+---
+
+import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';
+import { LTXDeployment } from '/src/snippets/diffusion/ltx-deployment.jsx';
+
+<DiffusionModelTags tags={["video", "text-to-video", "image-to-video", "two-stage"]} />
+
+## 1. Model Introduction
+
+[LTX-2](https://huggingface.co/Lightricks/LTX-2) and [LTX-2.3](https://huggingface.co/Lightricks/LTX-2.3) are video generation models from Lightricks. SGLang Diffusion supports the LTX series through native one-stage and two-stage pipelines for text-to-video and image-conditioned video generation.
+
+Use `Lightricks/LTX-2` or `Lightricks/LTX-2.3` as `--model-path`. For two-stage generation, SGLang uses the spatial upsampler and distilled LoRA components from the model snapshot by default. LTX-2.3 also supports the HQ two-stage variant.
+
+<Warning>
+**License notice:** LTX-2 and LTX-2.3 are released under the LTX-2 Community License Agreement, not Apache 2.0. The license includes commercial-use restrictions for some entities. Review the [official Lightricks license](https://huggingface.co/Lightricks/LTX-2.3/blob/main/LICENSE) before production or commercial use; SGLang support does not grant additional model usage rights.
+</Warning>
+
+## 2. SGLang-diffusion Installation
+
+Install SGLang with diffusion dependencies:
+
+```bash
+uv pip install "sglang[diffusion]" --prerelease=allow
+```
+
+For platform-specific setup, see the [SGLang Diffusion installation guide](/docs/sglang-diffusion/installation).
+
+## 3. Model Deployment
+
+This section provides deployment configurations optimized for different LTX pipelines and hardware targets.
+
+### 3.1 Basic Configuration
+
+The LTX series supports one-stage and two-stage pipelines. LTX-2.3 also supports the HQ two-stage pipeline. The recommended launch configuration depends on whether the target GPU can keep both two-stage DiTs resident.
+
+**Interactive Command Generator**: Use the configuration selector below to generate a deployment command. The default selection targets a single NVIDIA H200 with `resident` two-stage mode. For multi-GPU serving, start from the 2-GPU or 4-GPU presets and only change parallelism if you need more memory headroom.
+
+<LTXDeployment />
+
+### 3.2 Configuration Tips
+
+Choose the pipeline class based on the quality and latency target:
+
+| Use case | Pipeline class | Notes |
+| --- | --- | --- |
+| One-stage generation | `LTX2Pipeline` | Fastest LTX native path. Supports T2V and TI2V. |
+| Two-stage generation | `LTX2TwoStagePipeline` | Uses a base stage and a refinement stage. Supported by LTX-2 and LTX-2.3. |
+| Two-stage High Quality (HQ) generation | `LTX2TwoStageHQPipeline` | LTX-2.3 HQ path; defaults to 1920x1088 unless you override `--width` and `--height`. |
+
+Feature compatibility:
+
+| Pipeline class | T2V | TI2V (`--image-path`) | LoRA (`--lora-path`) | Notes |
+| --- | --- | --- | --- | --- |
+| `LTX2Pipeline` | Yes | Yes | Yes | One-stage path. Cannot be combined with HQ because HQ is a separate two-stage pipeline class. |
+| `LTX2TwoStagePipeline` | Yes | Yes | Yes | Standard two-stage path for LTX-2 and LTX-2.3. |
+| `LTX2TwoStageHQPipeline` | Yes | Yes | Yes | High Quality two-stage path for LTX-2.3. Use this instead of `LTX2Pipeline`; it is not a one-stage mode flag. |
+
+For two-stage pipelines, `--ltx2-two-stage-device-mode` controls transformer residency:
+
+| Mode | When to use it |
+| --- | --- |
+| `resident` | Best latency on high-VRAM GPUs because both DiTs can stay resident. |
+| `original` | Closest to the original two-stage switching semantics. |
+
+`snapshot` is kept only as a deprecated compatibility alias for `original` and may be removed after two release cycles; use `original` or `resident` in new configs.
+
+Other deployment flags:
+
+- `--lora-path`: Preload a community LoRA adapter.
+- `--lora-weight-name`: Select the exact safetensors file when the LoRA repository contains multiple weight files.
+
+<Note>
+For native LTX-2.3 two-stage serving without a user LoRA, `resident` is the fastest high-VRAM path. LTX-2 still applies the distilled LoRA during the stage switch, so `--ltx2-two-stage-device-mode` is mainly an LTX-2.3 optimization. When you pass `--lora-path`, SGLang still applies the user LoRA during the two-stage switch, so use `resident` on H200-class GPUs for enough VRAM, but do not expect the same premerged-stage2 benefit as the no-user-LoRA path.
+</Note>
+
+### 3.3 Fast multi-GPU presets
+
+For latency-oriented LTX serving, prefer CFG parallel over sequence parallelism. CFG parallel splits guidance branches across GPUs, while SP/Ulysses is mainly a memory/long-sequence tool for LTX.
+
+| Target | Recommended server flags | Notes |
+| --- | --- | --- |
+| LTX-2.3, 1 high-VRAM GPU | `--ltx2-two-stage-device-mode resident` | Fastest two-stage setup when both DiTs fit. |
+| LTX-2.3, 1 standard GPU | `--ltx2-two-stage-device-mode original` | Lower VRAM than `resident`; use this when H100-class memory is tight. |
+| LTX-2, 2 GPUs | `--num-gpus 2 --enable-cfg-parallel` | Fastest verified 2-GPU setup; keep `--dit-layerwise-offload` disabled unless memory is tight. |
+| LTX-2.3, 2 GPUs | `--num-gpus 2 --enable-cfg-parallel --ltx2-two-stage-device-mode resident` | Fastest common 2-GPU setup. |
+| LTX-2.3, 4 GPUs | `--num-gpus 4 --tp-size 2 --enable-cfg-parallel --ltx2-two-stage-device-mode resident` | Fastest common 4-GPU layout: TP2 inside each CFG branch. |
+| Official comparison | `--ltx2-two-stage-device-mode original` | Use this only when matching the original LTX-2.3 stage-switch semantics matters. |
+
+Use `--enable-cfg-parallel` for degree-2 CFG parallel. Use `--cfg-parallel-size` only when you explicitly need a different CFG branch count. If `resident` exceeds available VRAM, keep the same parallelism preset and switch only the device mode to `original`.
+
+On high-VRAM GPUs, add `--text-encoder-cpu-offload false` if text encoding latency matters and you have enough memory.
+
+#### 3.3.1 Two GPUs
+
+```bash
+sglang serve \
+  --model-path Lightricks/LTX-2.3 \
+  --pipeline-class-name LTX2TwoStagePipeline \
+  --num-gpus 2 \
+  --enable-cfg-parallel \
+  --ltx2-two-stage-device-mode resident
+```
+
+#### 3.3.2 Four GPUs
+
+```bash
+sglang serve \
+  --model-path Lightricks/LTX-2.3 \
+  --pipeline-class-name LTX2TwoStagePipeline \
+  --num-gpus 4 \
+  --tp-size 2 \
+  --enable-cfg-parallel \
+  --ltx2-two-stage-device-mode resident
+```
+
+## 4. Model Invocation
+
+### 4.1 Basic Usage
+
+The examples below spell out the current SGLang sampling defaults for reproducibility:
+
+| Model path | Default output | Default frames | Default steps |
+| --- | --- | --- | --- |
+| `Lightricks/LTX-2` | 768x512 | 121 | 40 |
+| `Lightricks/LTX-2.3` | 768x512 | 121 | 30 |
+| `Lightricks/LTX-2.3` with `LTX2TwoStageHQPipeline` | 1920x1088 | 121 | 15 |
+
+#### 4.1.1 LTX-2 one-stage text-to-video
+
+```bash
+sglang generate \
+  --model-path Lightricks/LTX-2 \
+  --pipeline-class-name LTX2Pipeline \
+  --prompt "A quiet coastal town at sunrise, fishing boats moving slowly through golden mist, cinematic camera movement" \
+  --save-output
+```
+
+#### 4.1.2 LTX-2.3 one-stage text-to-video
+
+```bash
+sglang generate \
+  --model-path Lightricks/LTX-2.3 \
+  --pipeline-class-name LTX2Pipeline \
+  --prompt "A quiet coastal town at sunrise, fishing boats moving slowly through golden mist, cinematic camera movement" \
+  --save-output
+```
+
+#### 4.1.3 LTX-2 two-stage text-to-video
+
+```bash
+sglang generate \
+  --model-path Lightricks/LTX-2 \
+  --pipeline-class-name LTX2TwoStagePipeline \
+  --prompt "A handheld shot follows a red tram crossing a rainy city square at night, reflections on the pavement, cinematic lighting" \
+  --save-output
+```
+
+#### 4.1.4 LTX-2.3 two-stage text-to-video
+
+```bash
+sglang generate \
+  --model-path Lightricks/LTX-2.3 \
+  --pipeline-class-name LTX2TwoStagePipeline \
+  --prompt "A handheld shot follows a red tram crossing a rainy city square at night, reflections on the pavement, cinematic lighting" \
+  --save-output
+```
+
+#### 4.1.5 LTX-2.3 HQ text-to-video
+
+```bash
+sglang generate \
+  --model-path Lightricks/LTX-2.3 \
+  --pipeline-class-name LTX2TwoStageHQPipeline \
+  --prompt "A wide cinematic shot of alpine clouds rolling over a mountain ridge, soft morning light, slow aerial camera movement" \
+  --save-output
+```
+
+#### 4.1.6 Image-to-video with one reference image
+
+Pass one image to `--image-path` for image-conditioned generation:
+
+```bash
+sglang generate \
+  --model-path Lightricks/LTX-2.3 \
+  --pipeline-class-name LTX2TwoStagePipeline \
+  --image-path ./inputs/start.png \
+  --prompt "The camera slowly pushes forward as the subject turns toward warm window light, subtle natural motion, cinematic" \
+  --save-output
+```
+
+#### 4.1.7 First-to-last-frame transition with two reference images
+
+Pass two images to `--image-path` for transition-style TI2V. The first image is used as the starting condition and the second image is used as the ending condition.
+
+```bash
+sglang generate \
+  --model-path Lightricks/LTX-2.3 \
+  --pipeline-class-name LTX2TwoStagePipeline \
+  --image-path ./inputs/start.png ./inputs/end.png \
+  --prompt "A smooth cinematic transition from the first scene into the final scene, dynamic camera motion, motion blur, zhuanchang" \
+  --save-output
+```
+
+### 4.2 Advanced Usage
+
+#### 4.2.1 Use community LoRAs
+
+Use `--lora-path` to load a LoRA adapter. If the Hugging Face repo contains multiple safetensors files, use `--lora-weight-name` to select the exact file. `--lora-scale` maps to the standard LoRA merge scale and defaults to `1.0`.
+
+The following example uses [`valiantcat/LTX-2.3-Transition-LORA`](https://huggingface.co/valiantcat/LTX-2.3-Transition-LORA):
+
+```bash
+sglang generate \
+  --model-path Lightricks/LTX-2.3 \
+  --pipeline-class-name LTX2TwoStagePipeline \
+  --lora-path valiantcat/LTX-2.3-Transition-LORA \
+  --lora-weight-name ltx2.3-transition.safetensors \
+  --prompt "A low-angle tracking shot moves through a foggy forest road. The camera rises above the treetops and transitions into a clear view of a snowy mountain peak under bright sunlight, zhuanchang" \
+  --save-output
+```
+
+You can combine the Transition LoRA with two reference images:
+
+```bash
+sglang generate \
+  --model-path Lightricks/LTX-2.3 \
+  --pipeline-class-name LTX2TwoStagePipeline \
+  --image-path ./inputs/start.png ./inputs/end.png \
+  --lora-path valiantcat/LTX-2.3-Transition-LORA \
+  --lora-weight-name ltx2.3-transition.safetensors \
+  --prompt "A fast cinematic transition from the first image to the second image, whip-pan motion, atmospheric lighting, zhuanchang" \
+  --save-output
+```
+
+<Note>
+Some community LoRAs only include weights for transformer blocks. In that case, SGLang logs a concise coverage summary and leaves unmatched LoRA-capable layers on the base model weights. This is expected when the adapter format intentionally omits those layers.
+</Note>
+
+## 5. Practical Tips
+
+- Use `--pipeline-class-name LTX2TwoStagePipeline` as the default LTX two-stage quality path.
+- Use `--pipeline-class-name LTX2TwoStageHQPipeline` when you want the HQ path and have enough VRAM for larger outputs.
+- Use `--ltx2-two-stage-device-mode resident` on high-VRAM GPUs if latency matters more than memory usage.
+- Use `--ltx2-two-stage-device-mode original` when comparing against official two-stage behavior.
+- Keep `--width` and `--height` aligned with the target model resolution; for LTX models, these are output video dimensions.
@@ -0,0 +1,175 @@
+---
+title: LingBot World 2.0
+metatags:
+    description: "Serve LingBot World 2.0 realtime camera-controlled video world models with SGLang-diffusion."
+tag: REALTIME
+---
+
+import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';
+
+<DiffusionModelTags tags={["realtime", "world model", "causal DiT", "camera control"]} />
+
+## 1. Model Introduction
+
+lingbot-world-v2-14b-causal-fast-diffusers is a realtime camera-controlled video world model. In SGLang-diffusion, it belongs to the realtime causal path: the server keeps a live session, samples control signals per chunk, reuses causal DiT state, and decodes video frames incrementally.
+
+This is different from offline diffusion video models such as Wan or LTX. Offline models denoise a bounded latent sequence for one request. Realtime world models generate a continuing stream, so the runtime must manage session state, control events, causal attention cache, and VAE decode cache.
+
+## 2. Deployment
+
+```bash Command
+export SGLANG_LINGBOT_LAZY_VAE_ENCODE_BLACK_FRAMES=60
+export SGLANG_LINGBOT_ENABLE_INTERACTIVE_KV_WINDOW=true
+sglang serve \
+  --model-path robbyant/lingbot-world-v2-14b-causal-fast-diffusers \
+  --pipeline-class-name LingBotWorldCausalDMDPipeline \
+  --num-gpus 8 \
+  --ulysses-degree 8 \
+  --dit-cpu-offload false \
+  --text-encoder-cpu-offload false \
+  --vae-config.use-parallel-decode true \
+  --vae-config.parallel-decode-mode spatial \
+  --enable-torch-compile false
+```
+
+## 3. Realtime WebUI
+
+The lightweight local WebUI is useful for validating latency, frame transport, and camera control behavior.
+
+```bash Command
+python -m http.server 18080 -d python/sglang/multimodal_gen/apps/realtime_webui
+```
+
+Open `http://127.0.0.1:18080` and use:
+
+```text Example
+ws://127.0.0.1:30000/v1/realtime_video/generate
+```
+
+## 4. HTTP and WebSocket API
+
+LingBot World 2.0 uses the realtime video WebSocket endpoint. The server keeps one live session, generates one chunk at a time, and accepts runtime control events while generation is running.
+
+### Endpoints
+
+| API | Method | Purpose | Notes |
+| --- | --- | --- | --- |
+| `/v1/models` | `GET` | Query the served model id before opening a session. | The WebUI uses this to fill the model field when the server exposes model metadata. |
+| `/v1/realtime_video/generate` | `WebSocket` | Create one realtime LingBot session and stream generated video chunks. | The first client message must be an `init` message encoded with MessagePack. |
+
+### `init` message
+
+Send this MessagePack map immediately after the WebSocket opens.
+
+| Parameter | Type | Required | Meaning |
+| --- | --- | --- | --- |
+| `type` | string | Yes | Must be `"init"`. |
+| `model` | string | No | Model id. Leave empty to use the served model. |
+| `prompt` | string | Yes | Text prompt for the initial scene and motion style. |
+| `first_frame` | bytes or string | Yes | Initial reference image. Send bytes from the WebUI/client, or a server-readable image path/string. |
+| `size` | string | Yes | Generation size as `WIDTHxHEIGHT`, for example `832x480`. |
+| `fps` | number | Yes | Target playback FPS for the generated stream. |
+| `num_frames` | integer | Yes | Frames per generated chunk. LingBot uses chunked causal generation, so this controls per-chunk latency and queue size. |
+| `seed` | integer | No | Random seed for deterministic sampling. |
+| `num_inference_steps` | integer | No | Denoising steps per chunk. LingBot defaults to `4` when omitted. |
+| `guidance_scale` | number | No | Classifier-free guidance scale. Realtime LingBot commonly uses `1`. |
+| `negative_prompt` | string | No | Negative prompt passed to the diffusion pipeline. |
+| `max_chunks` | integer | No | Stop after this many chunks. Omit for a continuous session. |
+| `realtime_causal_sink_size` | integer | No | Number of sink frames/tokens retained in the causal attention window. |
+| `realtime_causal_kv_cache_num_frames` | integer | No | Number of recent frames retained in the causal KV cache window. |
+| `realtime_output_format` | `"webp"`, `"jpeg"`, `"raw"` | No | Preview/output transport. `webp` and `jpeg` send encoded preview frames; `raw` sends raw RGB; omit for lossless delta-gzip RGB. |
+| `output_compression` | integer | No | Preview quality for `webp` or `jpeg`, from `1` to `100`. |
+| `enable_upscaling` | boolean | No | Enable server-side super resolution after frame decode. |
+| `upscaling_scale` | integer | No | Super-resolution scale. Current default is `4` when upscaling is enabled. |
+| `upscaling_model_path` | string | No | Optional Real-ESRGAN model path. |
+| `enable_frame_interpolation` | boolean | No | Enable frame interpolation. Keep this disabled when measuring true generated FPS. |
+| `frame_interpolation_exp` | integer | No | Interpolation multiplier exponent. `1` means 2x frames. |
+| `frame_interpolation_scale` | number | No | RIFE internal scale for interpolation. |
+| `frame_interpolation_model_path` | string | No | Optional RIFE model path. |
+| `condition_inputs.camera_actions` | `list[list[string]]` | No | Initial scripted camera actions, one action list per frame. |
+
+### Runtime `event` messages
+
+After `init`, send MessagePack event maps to update the live session.
+
+| Parameter | Type | Required | Meaning |
+| --- | --- | --- | --- |
+| `type` | string | Yes | Must be `"event"`. |
+| `kind` | `"prompt"`, `"camera_actions"`, or `"composite_input"` | Yes | Runtime event kind. |
+| `payload` | string, object, or list | Yes | For `prompt`, a non-empty string. For `camera_actions`, either scripted `list[list[string]]` or state-mode payload. For `composite_input`, a map containing `input_types` plus each named input. |
+| `event_id` | integer | No | Client sequence id. The server echoes it in chunk/frame metadata after the event is sampled. |
+
+`camera_actions` supports two payload modes:
+
+| Mode | Payload shape | Meaning |
+| --- | --- | --- |
+| Script | `list[list[string]]` | A fixed sequence of per-frame actions consumed by upcoming chunks. |
+| State | `{ "mode": "state", "transitions": [{"actions": [...], "client_ts_ms": ...}] }` | Live control state transitions from keyboard or UI controls. |
+
+Supported LingBot action tokens include `w`, `a`, `s`, `d` for camera movement and `i`, `j`, `k`, `l` for look controls.
+
+Use `composite_input` when multiple runtime inputs should be sampled together, such as updating the prompt and camera controls in one event.
+
+### Server messages
+
+| Message | Payload | Meaning |
+| --- | --- | --- |
+| `frame_batch` | MessagePack map with `payload` bytes | One batch of frames. The map includes `chunk_index`, `num_frames`, `content_type`, `encoding`, `width`, `height`, and frame-batch metadata. |
+| `chunk_stats` | MessagePack map | Per-chunk timing and transport metrics, including `scheduler_forward_ms`, `raw_payload_build_ms`, `chunk_total_ms`, `num_frames`, and `ws_payload_bytes`. |
+| `error` | MessagePack map | Server-side validation or generation error. |
+
+### Minimal client sketch
+
+```python Python
+import msgspec.msgpack
+import websocket
+
+ws = websocket.create_connection("ws://127.0.0.1:30000/v1/realtime_video/generate")
+ws.send_binary(msgspec.msgpack.encode({
+    "type": "init",
+    "prompt": "A quiet rainy London alley, stable camera motion.",
+    "first_frame": open("reference.jpg", "rb").read(),
+    "size": "832x480",
+    "fps": 25,
+    "num_frames": 9,
+    "num_inference_steps": 4,
+    "guidance_scale": 1,
+    "realtime_output_format": "webp",
+    "output_compression": 95,
+}))
+
+ws.send_binary(msgspec.msgpack.encode({
+    "type": "event",
+    "kind": "camera_actions",
+    "event_id": 1,
+    "payload": {"mode": "state", "transitions": [{"actions": ["w"], "client_ts_ms": 0}]},
+}))
+
+ws.send_binary(msgspec.msgpack.encode({
+    "type": "event",
+    "kind": "prompt",
+    "event_id": 2,
+    "payload": "A quiet snowy Tokyo alley, stable camera motion.",
+}))
+
+ws.send_binary(msgspec.msgpack.encode({
+    "type": "event",
+    "kind": "composite_input",
+    "event_id": 3,
+    "payload": {
+        "input_types": ["prompt", "camera_actions"],
+        "prompt": "A quiet neon Shanghai alley, stable forward camera motion.",
+        "camera_actions": [["w"], ["w"], []],
+    },
+}))
+```
+
+## 5. Consistency
+
+LingBot World 2.0 uses raw-frame websocket GT plus per-chunk latency guards for consistency checks.
+
+## 6. Notes
+
+- Use the realtime endpoint for interactive sessions: `/v1/realtime_video/generate`.
+- Prefer WebP preview transport for interactive testing; use raw-frame transport for consistency checks.
+- Long-running sessions should be validated with raw-frame consistency before changing causal cache, condition sampling, or VAE decode behavior.
@@ -0,0 +1,150 @@
+---
+title: LingBot World
+metatags:
+    description: "Serve LingBot World realtime camera-controlled video world models with SGLang-diffusion."
+tag: REALTIME
+---
+
+import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';
+
+<DiffusionModelTags tags={["realtime", "world model", "causal DiT", "camera control"]} />
+
+## 1. Model Introduction
+
+[LingBot World](https://huggingface.co/robbyant/lingbot-world-fast-diffusers) is a realtime camera-controlled video world model. In SGLang-diffusion, it belongs to the realtime causal path: the server keeps a live session, samples control signals per chunk, reuses causal DiT state, and decodes video frames incrementally.
+
+This is different from offline diffusion video models such as Wan or LTX. Offline models denoise a bounded latent sequence for one request. Realtime world models generate a continuing stream, so the runtime must manage session state, control events, causal attention cache, and VAE decode cache.
+
+## 2. Deployment
+
+```bash Command
+sglang serve \
+  --model-path robbyant/lingbot-world-fast-diffusers \
+  --pipeline-class-name LingBotWorldCausalDMDPipeline \
+  --num-gpus 4 \
+  --ulysses-degree 4 \
+  --dit-cpu-offload false \
+  --text-encoder-cpu-offload false
+```
+
+## 3. Realtime WebUI
+
+The lightweight local WebUI is useful for validating latency, frame transport, and camera control behavior.
+
+```bash Command
+python -m http.server 18080 -d python/sglang/multimodal_gen/apps/realtime_webui
+```
+
+Open `http://127.0.0.1:18080` and use:
+
+```text Example
+ws://127.0.0.1:30000/v1/realtime_video/generate
+```
+
+## 4. HTTP and WebSocket API
+
+LingBot World uses the realtime video WebSocket endpoint. The server keeps one live session, generates one chunk at a time, and accepts runtime control events while generation is running.
+
+### Endpoints
+
+| API | Method | Purpose | Notes |
+| --- | --- | --- | --- |
+| `/v1/models` | `GET` | Query the served model id before opening a session. | The WebUI uses this to fill the model field when the server exposes model metadata. |
+| `/v1/realtime_video/generate` | `WebSocket` | Create one realtime LingBot session and stream generated video chunks. | The first client message must be an `init` message encoded with MessagePack. |
+
+### `init` message
+
+Send this MessagePack map immediately after the WebSocket opens.
+
+| Parameter | Type | Required | Meaning |
+| --- | --- | --- | --- |
+| `type` | string | Yes | Must be `"init"`. |
+| `model` | string | No | Model id. Leave empty to use the served model. |
+| `prompt` | string | Yes | Text prompt for the initial scene and motion style. |
+| `first_frame` | bytes or string | Yes | Initial reference image. Send bytes from the WebUI/client, or a server-readable image path/string. |
+| `size` | string | Yes | Generation size as `WIDTHxHEIGHT`, for example `832x480`. |
+| `fps` | number | Yes | Target playback FPS for the generated stream. |
+| `num_frames` | integer | Yes | Frames per generated chunk. LingBot uses chunked causal generation, so this controls per-chunk latency and queue size. |
+| `seed` | integer | No | Random seed for deterministic sampling. |
+| `num_inference_steps` | integer | No | Denoising steps per chunk. LingBot defaults to `4` when omitted. |
+| `guidance_scale` | number | No | Classifier-free guidance scale. Realtime LingBot commonly uses `1`. |
+| `negative_prompt` | string | No | Negative prompt passed to the diffusion pipeline. |
+| `max_chunks` | integer | No | Stop after this many chunks. Omit for a continuous session. |
+| `realtime_causal_sink_size` | integer | No | Number of sink frames/tokens retained in the causal attention window. |
+| `realtime_causal_kv_cache_num_frames` | integer | No | Number of recent frames retained in the causal KV cache window. |
+| `realtime_output_format` | `"webp"`, `"jpeg"`, `"raw"` | No | Preview/output transport. `webp` and `jpeg` send encoded preview frames; `raw` sends raw RGB; omit for lossless delta-gzip RGB. |
+| `output_compression` | integer | No | Preview quality for `webp` or `jpeg`, from `1` to `100`. |
+| `enable_upscaling` | boolean | No | Enable server-side super resolution after frame decode. |
+| `upscaling_scale` | integer | No | Super-resolution scale. Current default is `4` when upscaling is enabled. |
+| `upscaling_model_path` | string | No | Optional Real-ESRGAN model path. |
+| `enable_frame_interpolation` | boolean | No | Enable frame interpolation. Keep this disabled when measuring true generated FPS. |
+| `frame_interpolation_exp` | integer | No | Interpolation multiplier exponent. `1` means 2x frames. |
+| `frame_interpolation_scale` | number | No | RIFE internal scale for interpolation. |
+| `frame_interpolation_model_path` | string | No | Optional RIFE model path. |
+| `condition_inputs.camera_actions` | `list[list[string]]` | No | Initial scripted camera actions, one action list per frame. |
+
+### Runtime `event` messages
+
+After `init`, send MessagePack event maps to update the live session.
+
+| Parameter | Type | Required | Meaning |
+| --- | --- | --- | --- |
+| `type` | string | Yes | Must be `"event"`. |
+| `kind` | `"prompt"` or `"camera_actions"` | Yes | Runtime event kind. |
+| `payload` | string or object/list | Yes | For `prompt`, a non-empty string. For `camera_actions`, either scripted `list[list[string]]` or state-mode payload. |
+| `event_id` | integer | No | Client sequence id. The server echoes it in chunk/frame metadata after the event is sampled. |
+
+`camera_actions` supports two payload modes:
+
+| Mode | Payload shape | Meaning |
+| --- | --- | --- |
+| Script | `list[list[string]]` | A fixed sequence of per-frame actions consumed by upcoming chunks. |
+| State | `{ "mode": "state", "transitions": [{"actions": [...], "client_ts_ms": ...}] }` | Live control state transitions from keyboard or UI controls. |
+
+Supported LingBot action tokens include `w`, `a`, `s`, `d` for camera movement and `i`, `j`, `k`, `l` for look controls.
+
+### Server messages
+
+| Message | Payload | Meaning |
+| --- | --- | --- |
+| `frame_batch` | MessagePack map with `payload` bytes | One batch of frames. The map includes `chunk_index`, `num_frames`, `content_type`, `encoding`, `width`, `height`, and frame-batch metadata. |
+| `chunk_stats` | MessagePack map | Per-chunk timing and transport metrics, including `scheduler_forward_ms`, `raw_payload_build_ms`, `chunk_total_ms`, `num_frames`, and `ws_payload_bytes`. |
+| `error` | MessagePack map | Server-side validation or generation error. |
+
+### Minimal client sketch
+
+```python Python
+import msgspec.msgpack
+import websocket
+
+ws = websocket.create_connection("ws://127.0.0.1:30000/v1/realtime_video/generate")
+ws.send_binary(msgspec.msgpack.encode({
+    "type": "init",
+    "prompt": "A quiet rainy London alley, stable camera motion.",
+    "first_frame": open("reference.jpg", "rb").read(),
+    "size": "832x480",
+    "fps": 25,
+    "num_frames": 9,
+    "num_inference_steps": 4,
+    "guidance_scale": 1,
+    "realtime_output_format": "webp",
+    "output_compression": 95,
+}))
+
+ws.send_binary(msgspec.msgpack.encode({
+    "type": "event",
+    "kind": "camera_actions",
+    "event_id": 1,
+    "payload": {"mode": "state", "transitions": [{"actions": ["w"], "client_ts_ms": 0}]},
+}))
+```
+
+## 5. Consistency
+
+LingBot World uses raw-frame websocket GT plus per-chunk latency guards for consistency checks.
+
+## 6. Notes
+
+- Use the realtime endpoint for interactive sessions: `/v1/realtime_video/generate`.
+- Prefer WebP preview transport for interactive testing; use raw-frame transport for consistency checks.
+- Long-running sessions should be validated with raw-frame consistency before changing causal cache, condition sampling, or VAE decode behavior.
@@ -0,0 +1,112 @@
+---
+title: LongLive 2.0
+description: "Serve LongLive 2.0 distilled text-to-video and image-to-video models with SGLang-diffusion."
+---
+
+import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';
+
+<DiffusionModelTags tags={["video", "text-to-video", "image-to-video", "few-step", "multi-shot"]} />
+
+## 1. Model Introduction
+
+[LongLive 2.0](https://nvlabs.github.io/LongLive/LongLive2/) is a distilled few-step text-to-video and image-to-video model from NVIDIA, built on Wan2.2-TI2V-5B. SGLang serves the Diffusers-format conversion for single-prompt and multi-shot video generation.
+
+For more details, check the [LongLive 2.0 paper](https://arxiv.org/abs/2605.18739) and [LongLive 2.0 GitHub](https://github.com/NVlabs/LongLive). The model weights are released under the NVIDIA Open Model License.
+
+## 2. SGLang-diffusion Installation
+
+Please refer to the [official SGLang-diffusion installation guide](/docs/sglang-diffusion/installation) for installation instructions.
+
+## 3. Deployment
+
+```bash Command
+sglang serve --model-path Rabinovich/LongLive-2.0-5B-Diffusers
+```
+
+If the GPU runs out of memory, move the text encoder, VAE, and DiT to CPU between stages:
+
+```bash Command
+sglang serve \
+  --model-path Rabinovich/LongLive-2.0-5B-Diffusers \
+  --dit-cpu-offload \
+  --text-encoder-cpu-offload \
+  --vae-cpu-offload
+```
+
+`Rabinovich/LongLive-2.0-5B-Diffusers` is the Diffusers-format conversion of the official `Efficient-Large-Model/LongLive-2.0-5B` weights.
+
+## 4. Generation
+
+### 4.1 Single prompt
+
+Generate one clip without starting a server:
+
+```bash Command
+sglang generate \
+  --model-path Rabinovich/LongLive-2.0-5B-Diffusers \
+  --prompt "A quiet street at dusk" \
+  --num-frames 61 \
+  --save-output \
+  --output-path outputs
+```
+
+61 frames is 16 latent frames, which is two causal blocks of 8.
+
+### 4.2 Multi-shot long video
+
+Multi-shot prompts are sampling parameters, so pass them through the Python API:
+
+```python Python
+from sglang import DiffGenerator
+
+gen = DiffGenerator.from_pretrained("Rabinovich/LongLive-2.0-5B-Diffusers")
+result = gen.generate(sampling_params_kwargs={
+    "shot_prompts": [
+        "A husky walks down a sunlit hallway.",
+        "The husky turns and looks at the camera.",
+        "Two dogs play together on a carpet.",
+    ],
+    "chunks_per_shot": 4,
+    "num_frames": 381,  # 3 shots x 4 chunks x 8 = 96 latent frames -> 381 frames
+    "scene_cut_prefix": "The scene transitions. ",
+    "multi_shot_sink": True,
+    "multi_shot_rope_offset": 8.0,
+    "save_output": True,
+    "output_path": "outputs",
+})
+```
+
+Each shot runs for `chunks_per_shot` causal blocks before the next prompt is used. The multi-shot defaults mirror the original LongLive prompt-block settings.
+
+### 4.3 Key parameters
+
+These are SGLang request parameters. Original LongLive configs use latent-frame `num_output_frames`; SGLang exposes output-video `num_frames`.
+
+- `num_frames`: 61 in the examples. This maps to 16 latent frames, while the original release config defaults to 128 latent frames.
+- `num_inference_steps`: 4, matching original `sampling_steps`.
+- `guidance_scale`: 1.0, matching the original inference config.
+- `height` / `width`: 704 / 1280 by default, matching original latent H/W 44 / 80 with 16x spatial compression.
+- `shot_prompts`, `chunks_per_shot`, `scene_cut_prefix`, `multi_shot_sink`, and `multi_shot_rope_offset`: SGLang request fields for the original prompt-block and multi-shot behavior.
+
+### 4.4 Image-to-video
+
+Pass a first frame with `--image-path` to condition the clip on an image:
+
+```bash Command
+sglang generate \
+  --model-path Rabinovich/LongLive-2.0-5B-Diffusers \
+  --prompt "A quiet street at dusk" \
+  --image-path first_frame.png \
+  --num-frames 61 \
+  --save-output \
+  --output-path outputs
+```
+
+The image is used as the first-frame condition.
+
+## 5. Notes
+
+- `num_frames` must map to a whole number of causal blocks. The latent frame count is `(num_frames - 1) / 4 + 1` and must be divisible by 8. For example, 61, 125, and 189 frames give 16, 32, and 48 latent frames.
+- SGLang supports T2V sizes 1280x704, 704x1280, 832x480, and 480x832.
+- I2V request images follow the Wan TI2V preprocessing path in SGLang. This is different from the original LongLive dataset resize path.
+- For multi-shot runs, set `num_frames` to match `len(shot_prompts) * chunks_per_shot * 8` latent frames, that is `num_frames = (len(shot_prompts) * chunks_per_shot * 8 - 1) * 4 + 1`.
@@ -0,0 +1,272 @@
+---
+title: MOVA
+metatags:
+    description: "Deploy MOVA with SGLang - simultaneous video and audio generation with asymmetric dual-tower architecture, precise lip-sync, and environment-aware sound effects."
+---
+
+import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';
+
+<DiffusionModelTags tags={["video", "audio-video", "lip-sync", "environment audio"]} />
+
+## 1. Model Introduction
+
+[MOVA](https://github.com/OpenMOSS/MOVA) (MOSS Video and Audio) is a foundation model developed by the SII-OpenMOSS Team, designed to break the "silent era" of open-source video generation. Unlike cascaded pipelines that generate sound as an afterthought, MOVA synthesizes video and audio simultaneously in a single inference pass for perfect alignment. It adopts an Asymmetric Dual-Tower Architecture, fusing pre-trained video and audio towers through a bidirectional cross-attention mechanism to maintain tight synchronization between video and audio during generation.
+
+[MOVA-360p](https://huggingface.co/OpenMOSS-Team/MOVA-360p) is suitable for fast inference and resource-constrained environments. [MOVA-720p](https://huggingface.co/OpenMOSS-Team/MOVA-720p) provides higher resolution video generation. Both versions support generating up to 8 seconds of video-audio content.
+
+**Key Features:**
+
+- **Native Bimodal Generation**: Generates high-fidelity video and synchronized audio in a single inference pass, eliminating error accumulation from cascaded pipelines
+- **Precise Lip-Sync**: Achieves state-of-the-art performance in multilingual lip-synchronization (LSE-D: 7.094, LSE-C: 7.452 with Dual CFG on Verse-Bench Set3)
+- **Environment-Aware Sound Effects**: Generates corresponding environmental sound effects including physical interaction sounds, ambient sounds, and spatial/textural sound feedback
+- **Fully Open-Source**: Model weights, inference code, training pipelines, and LoRA fine-tuning scripts are all open-sourced
+
+For more details, please refer to the [MOVA-360p HuggingFace page](https://huggingface.co/OpenMOSS-Team/MOVA-360p), the [MOVA-720p HuggingFace page](https://huggingface.co/OpenMOSS-Team/MOVA-720p), the [GitHub repository](https://github.com/OpenMOSS/MOVA), and the [technical report (arXiv)](https://arxiv.org/abs/2602.08794).
+
+## 2. SGLang-diffusion Installation
+
+SGLang-diffusion offers multiple installation methods. You can choose the most suitable installation method based on your hardware platform and requirements.
+
+Please refer to the [official SGLang-diffusion installation guide](https://docs.sglang.io/docs/sglang-diffusion/installation) for installation instructions.
+
+## 3. Model Deployment
+
+This section provides deployment configurations optimized for different hardware platforms and use cases.
+
+### 3.1 Basic Configuration
+
+MOVA supports both online serving and CLI generation modes. The recommended launch configurations vary by hardware and resolution.
+
+**Interactive Command Generator**: Use the configuration selector below to automatically generate the appropriate deployment command for your hardware platform.
+
+import { MOVADeployment } from '/src/snippets/diffusion/mova-deployment.jsx'
+
+<MOVADeployment />
+
+### 3.2 Configuration Tips
+
+Currently supported optimizations are listed [here](/docs/sglang-diffusion/compatibility_matrix).
+
+- `--num-gpus`: Number of GPUs to use
+- `--tp`: Tensor parallelism size (should not be larger than 1 if text encoder offload is enabled, as layer-wise offload plus prefetch is faster)
+- `--ring-degree`: The degree of ring attention-style SP in USP
+- `--ulysses-degree`: The degree of DeepSpeed-Ulysses-style SP in USP
+- `--adjust-frames`: Whether to adjust frames automatically (set to `false` for MOVA)
+- `--enable-torch-compile`: Enable torch.compile for faster inference
+
+## 4. API Usage
+
+For complete API documentation, please refer to the [official API usage guide](/docs/sglang-diffusion/api/openai_api).
+
+### 4.1 CLI Generation (sglang generate)
+
+```bash Command
+sglang generate \
+  --model-path OpenMOSS-Team/MOVA-720p \
+  --prompt "A man in a blue blazer and glasses speaks in a formal indoor setting, \
+  framed by wooden furniture and a filled bookshelf. \
+  Quiet room acoustics underscore his measured tone as he delivers his remarks. \
+  At one point, he says, \"I would also believe that this advance in AI recently wasn't unexpected.\"" \
+  --image-path "<YOUR-IMAGE-PATH>" \
+  --adjust-frames false \
+  --num-gpus 8 \
+  --ring-degree 2 \
+  --ulysses-degree 4 \
+  --num-frames 193 \
+  --fps 24 \
+  --seed 67 \
+  --num-inference-steps 25 \
+  --enable-torch-compile \
+  --save-output
+```
+
+### 4.2 Generate a Video
+
+```bash Command
+curl -X POST "http://0.0.0.0:30002/v1/videos" \
+  -F "prompt=A man in a blue blazer and glasses speaks in a formal indoor setting, framed by wooden furniture and a filled bookshelf. Quiet room acoustics underscore his measured tone as he delivers his remarks. At one point, he says, \"I would also believe that this advance in AI recently wasn't unexpected.\"" \
+  -F "input_reference=@<YOUR-IMAGE-PATH>" \
+  -F "size=640x352" \
+  -F "num_frames=193" \
+  -F "fps=24" \
+  -F "seed=67" \
+  -F "guidance_scale=5.0" \
+  -F "num_inference_steps=25" \
+  -o create_video.json
+```
+
+### 4.3 Advanced Usage
+
+#### 4.3.1 Cache-DiT Acceleration
+
+SGLang integrates [Cache-DiT](https://github.com/vipshop/cache-dit), a caching acceleration engine for Diffusion Transformers (DiT), to achieve up to 7.4x inference speedup with minimal quality loss. You can set `SGLANG_CACHE_DIT_ENABLED=True` to enable it. For more details, please refer to the SGLang Cache-DiT [documentation](/docs/sglang-diffusion/cache_dit).
+
+**Basic Usage**
+
+```bash Command
+SGLANG_CACHE_DIT_ENABLED=true sglang serve --model-path OpenMOSS-Team/MOVA-720p
+```
+
+**Advanced Usage**
+
+- DBCache Parameters: DBCache controls block-level caching behavior:
+
+<table style={{width: "100%", borderCollapse: "collapse", tableLayout: "fixed"}}>
+  <thead>
+    <tr style={{borderBottom: "2px solid #d55816"}}>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Parameter</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>Env Variable</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Default</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>Description</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Fn</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_FN`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>1</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Number of first blocks to always compute</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Bn</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_BN`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>0</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Number of last blocks to always compute</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>W</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_WARMUP`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>4</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Warmup steps before caching starts</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>R</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_RDT`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>0.24</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Residual difference threshold</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>MC</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_MC`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>3</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Maximum continuous cached steps</td>
+    </tr>
+  </tbody>
+</table>
+
+- TaylorSeer Configuration: TaylorSeer improves caching accuracy using Taylor expansion:
+
+<table style={{width: "100%", borderCollapse: "collapse", tableLayout: "fixed"}}>
+  <thead>
+    <tr style={{borderBottom: "2px solid #d55816"}}>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Parameter</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>Env Variable</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Default</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>Description</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Enable</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_TAYLORSEER`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>false</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Enable TaylorSeer calibrator</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Order</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_TS_ORDER`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>1</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Taylor expansion order (1 or 2)</td>
+    </tr>
+  </tbody>
+</table>
+
+  Combined Configuration Example:
+
+```bash Command
+SGLANG_CACHE_DIT_ENABLED=true \
+SGLANG_CACHE_DIT_FN=2 \
+SGLANG_CACHE_DIT_BN=1 \
+SGLANG_CACHE_DIT_WARMUP=4 \
+SGLANG_CACHE_DIT_RDT=0.4 \
+SGLANG_CACHE_DIT_MC=4 \
+SGLANG_CACHE_DIT_TAYLORSEER=true \
+SGLANG_CACHE_DIT_TS_ORDER=2 \
+sglang serve --model-path OpenMOSS-Team/MOVA-720p
+```
+
+#### 4.3.2 CPU Offload
+
+- `--dit-cpu-offload`: Use CPU offload for DiT inference. Enable if run out of memory.
+- `--text-encoder-cpu-offload`: Use CPU offload for text encoder inference.
+- `--vae-cpu-offload`: Use CPU offload for VAE.
+- `--pin-cpu-memory`: Pin memory for CPU offload. Only added as a temp workaround if it throws "CUDA error: invalid argument".
+
+## 5. Benchmark
+
+### 5.1 Speedup Benchmark
+
+#### 5.1.1 Generate a video
+
+Test Environment:
+
+- Hardware: NVIDIA H200 x 8
+- git revision: 443b1a8
+- Model: OpenMOSS-Team/MOVA-720p
+
+**Server Command**:
+
+```bash Command
+sglang serve --model-path OpenMOSS-Team/MOVA-720p --port 30002 \
+  --adjust-frames false --num-gpus 8 --ring-degree 2 --ulysses-degree 4 \
+  --tp 1 --enable-torch-compile
+```
+
+**Benchmark Command**:
+
+```bash Command
+python3 -m sglang.multimodal_gen.benchmarks.bench_serving \
+    --task image-to-video --dataset vbench --num-prompts 1 --max-concurrency 1 \
+    --port 30002
+```
+
+**Result**:
+```text Output
+================= Serving Benchmark Result =================
+Task:                                    image-to-video
+Model:                                   OpenMOSS-Team/MOVA-720p
+Dataset:                                 vbench
+--------------------------------------------------
+Benchmark duration (s):                  590.76
+Request rate:                            inf
+Max request concurrency:                 1
+Successful requests:                     1/1
+--------------------------------------------------
+Request throughput (req/s):              0.00
+Latency Mean (s):                        590.7549
+Latency Median (s):                      590.7549
+Latency P99 (s):                         590.7549
+--------------------------------------------------
+Peak Memory Max (MB):                    74996.00
+Peak Memory Mean (MB):                   74996.00
+Peak Memory Median (MB):                 74996.00
+============================================================
+```
+
+#### 5.1.2 Generate videos with high concurrency
+
+**Server Command**:
+
+```bash Command
+sglang serve --model-path OpenMOSS-Team/MOVA-720p --port 30002 \
+  --adjust-frames false --num-gpus 8 --ring-degree 2 --ulysses-degree 4 \
+  --tp 1 --enable-torch-compile
+```
+
+**Benchmark Command**:
+
+```bash Command
+python3 -m sglang.multimodal_gen.benchmarks.bench_serving \
+    --task image-to-video --dataset vbench --num-prompts 20 --max-concurrency 20 \
+    --port 30002
+```
@@ -0,0 +1,839 @@
+---
+title: MiniMax-H3
+description: Run native MiniMax-H3 video-and-audio generation with SGLang Diffusion.
+metatags:
+    description: "Serve MiniMax-H3 with SGLang Diffusion for text-to-video-and-audio, first/last-frame conditioning, video-to-video, and multimodal reference conditioning."
+---
+
+## 1. Model introduction
+
+[MiniMax-H3](https://huggingface.co/MiniMaxAI/MiniMax-H3) generates a video and a synchronized stereo audio track in one request. SGLang Diffusion provides a native pipeline for the three public task profiles, split across the released FL2VA (First-and-Last-Frame-to-Video-and-Audio) and Ref2VA (Reference-to-Video-and-Audio) checkpoint partitions:
+
+| Task | `task` value | Conditioning |
+| --- | --- | --- |
+| Text to video and audio | `t2va` | Text prompt only |
+| First/last frame to video and audio | `fl2va` | First frame, last frame, or both |
+| Reference to video and audio | `ref2va` | Image, video, and audio references |
+
+Video-to-video (V2V) is a supported `ref2va` use case, not a fourth task
+value. Run the `Ref2VA` partition and provide a video reference in
+`conditions`.
+
+Use the selected Hub's root model ID: `MiniMaxAI/MiniMax-H3` on Hugging Face
+or `MiniMax/MiniMax-H3` on ModelScope. Select the checkpoint variant with
+`--model-variant`: `fl2va` serves both `t2va` and `fl2va`, while `ref2va`
+serves reference-conditioned requests. SGLang owns the checkpoint-directory
+mapping; do not point `--model-path` at a manually downloaded subdirectory.
+
+<Warning>
+Review the license and usage terms in the MiniMax-H3 model card before production or commercial use. SGLang support does not grant additional model usage rights.
+</Warning>
+
+## 2. Installation
+
+Install SGLang with the diffusion dependencies:
+
+```bash Command
+uv pip install "sglang[diffusion]" --prerelease=allow
+```
+
+For platform-specific setup, see the [SGLang Diffusion installation guide](/docs/sglang-diffusion/installation).
+
+## 3. Serve MiniMax-H3
+
+Use the interactive selector to choose a hardware platform, deployment profile,
+one of the two checkpoint partitions, a request mode, and deployment features.
+It generates Python and, where available, Docker launch forms. AMD selections
+use the Python form until an H3-capable ROCm image is validated. The **$ cURL**
+button follows the selected request mode and switches the payload across
+text-only, all three first/last-frame signatures, and the image/audio/video
+reference combinations listed below.
+Set **Outputs per prompt** in the picker’s **Env** panel to generate more than
+one output without mixing request sampling controls into the deployment
+matrix.
+
+The Docker form does not assume the base SGLang image contains optional
+diffusion dependencies. It installs the platform-specific diffusion extra from
+the source bundled in the image before starting the server. Set **Host media
+directory** in the **Env** panel for FL2VA, V2V, or Ref2VA; the picker mounts
+that directory read-only at `/data/minimax-h3` inside the container.
+
+Every hardware/topology cell in this picker has completed a real request on
+that exact GPU model. Approximate load-time features such as online
+quantization are called out separately in the generated command. Sampling
+behavior such as Cache-DiT is documented separately below.
+
+**Deployment Profile** exposes resident and FSDP placement on B200, B300,
+H200, and H100. Resident is the latency-oriented default; FSDP reduces DiT
+weight residency at the cost of per-block parameter collectives. **Online
+Quantization** appears only on B200 and B300. AMD keeps its resident AITER
+recipe, while RTX 5090 uses its dedicated layerwise-offload profile.
+
+import { Deployment } from "/src/snippets/_deployment.jsx";
+import { config } from "/src/snippets/configs/MiniMaxAI/minimax-h3.jsx";
+
+<Deployment config={config} />
+
+<Note>
+The ready-to-run request template lives behind the **$ cURL** button in the
+picker above. It regenerates as you change the selection, so the payload it
+shows always matches the serve command next to it.
+</Note>
+
+The selector uses the verified Hugging Face ID. To use ModelScope through the
+same normal `sglang serve` path, prefix the copied command with
+`SGLANG_USE_MODELSCOPE=true` and replace the model path with
+`MiniMax/MiniMax-H3`; keep its selected variant and topology flags unchanged.
+
+For a four-card H200 host, keep the full BF16/FP32 model resident by default.
+The model fits without FSDP, so this path avoids the per-block parameter
+all-gathers of the memory-oriented FSDP profile:
+
+```bash 4×H200 resident
+sglang serve \
+  --model-path MiniMaxAI/MiniMax-H3 \
+  --model-variant fl2va \
+  --num-gpus 4 \
+  --ulysses-degree 4 \
+  --performance-mode speed \
+  --port 30010
+```
+
+Pure Ulysses4 is also the faster measured topology on H200, not just a
+capacity default. The 4×H100 TP2 + Ulysses2 recipe below fits on 141 GB H200
+cards, but it replaces the Ulysses all-to-all exchange with two per-block
+tensor-parallel all-reduces and measured slower end-to-end, at about 30 GB
+lower peak memory per GPU. See the **H200 topology comparison** in the
+Benchmarks section for the measured numbers; treat TP2 + Ulysses2 on H200 as
+a deliberate memory trade, not a latency default.
+
+For 4×H100 80 GB, balance the large packed activation with resident weight
+sharding. TP2 + Ulysses2 was the fastest measured lossless topology while the
+Qwen encoder still folds across all four GPUs:
+
+```bash 4×H100 fastest
+sglang serve \
+  --model-path MiniMaxAI/MiniMax-H3 \
+  --model-variant fl2va \
+  --num-gpus 4 \
+  --tp-size 2 \
+  --ulysses-degree 2 \
+  --performance-mode speed \
+  --port 30010
+```
+
+Pure Ulysses4 could not keep the full pipeline resident on 80 GB H100s. Use
+`--tp-size 4 --ulysses-degree 1` when lower resident memory matters more than
+the last few percent of latency. FSDP remains a verified capacity option, but
+its per-block weight all-gathers do not make it the H100 speed default:
+
+```bash 4×H100 FSDP capacity
+sglang serve \
+  --model-path MiniMaxAI/MiniMax-H3 \
+  --model-variant fl2va \
+  --num-gpus 4 \
+  --ulysses-degree 4 \
+  --performance-mode speed \
+  --use-fsdp-inference true \
+  --port 30010
+```
+
+For a two-card RTX 5090 host, use TP2 and keep 20 DiT blocks
+resident. Layerwise placement is lossless: it changes parameter placement and
+transfer scheduling, not the BF16/FP32 denoising or VAE math. This is the
+fastest measured 32 GB operating point:
+
+```bash 2×RTX 5090 fastest lossless
+sglang serve \
+  --model-path MiniMaxAI/MiniMax-H3 \
+  --model-variant fl2va \
+  --num-gpus 2 \
+  --tp-size 2 \
+  --ulysses-degree 1 \
+  --performance-mode memory \
+  --layerwise-offload-components dit,text_encoder,vae \
+  --dit-offload-prefetch-size 1 \
+  --dit-layerwise-resident-layers 20 \
+  --enable-torch-compile false \
+  --port 30010
+```
+
+The DiT residency and prefetch knobs apply only to the repeatedly executed DiT
+blocks. The text encoder and the video VAE decoder blocks use one-layer
+prefetch with zero resident layers. The video VAE encoder stays resident
+because its indexed down blocks cannot host executable layerwise hooks; the
+roughly 577 MiB audio VAE also stays resident because offloading it only adds
+transfer overhead. This exact recipe was validated on
+2× RTX 5090 (32 GB each) and a 377 GiB host; use a 384 GiB-class machine. The
+latency and memory comparison is collected in the benchmark section below.
+
+The first launch downloads the model through the selected Hub. If the Hugging
+Face repository requires authentication, export a Hugging Face token in the
+server environment.
+
+For MiniMax-H3, `--performance-mode speed` deliberately keeps the DiT eager. The current `torch.compile` path changes the model's numerical output, so it is not enabled implicitly by any recommended lossless preset. An explicit `--enable-torch-compile true` remains available for controlled experiments, but it should not be used to generate consistency ground truth.
+
+## 4. Generate video and audio
+
+MiniMax-H3 uses the asynchronous OpenAI-compatible video endpoint. Choose a
+generation mode below, submit a job, poll its status, and then download the
+completed MP4.
+
+<Tabs>
+
+<Tab title="T2VA">
+
+MiniMax-H3 supports output durations from 4 through 15 seconds, inclusive. The
+following request keeps the verified 5-second profile at a 768-pixel short
+edge. MiniMax-H3 resolves the aligned output canvas and frame count from
+`target`.
+
+```bash Command
+video_id=$(
+  curl -sS -X POST http://127.0.0.1:30010/v1/videos \
+    -H "Content-Type: application/json" \
+    -d '{
+      "model": "MiniMaxAI/MiniMax-H3",
+      "prompt": "At night, while their owner sleeps in a bedroom, three cats march in loudly playing tiny brass instruments, then abruptly file out.",
+      "seconds": 5,
+      "task": "t2va",
+      "conditions": [],
+      "target": {
+        "short_edge": 768,
+        "aspect_ratio": "16:9",
+        "duration_seconds": 5.0
+      },
+      "num_outputs_per_prompt": 1,
+      "num_inference_steps": 50,
+      "flow_shift": 12.0,
+      "audio_flow_shift": 3.0,
+      "seed": 1101
+    }' |
+    jq -r '.id'
+)
+
+while true; do
+  status=$(curl -sS "http://127.0.0.1:30010/v1/videos/${video_id}" | jq -r '.status')
+  [ "$status" = "completed" ] && break
+  [ "$status" = "failed" ] && exit 1
+  sleep 1
+done
+
+curl -sS -L "http://127.0.0.1:30010/v1/videos/${video_id}/content" \
+  -o minimax-h3-t2va.mp4
+```
+
+The output contract is an MP4 containing H.264 video at 24 fps and one AAC stereo audio stream at 32 kHz.
+
+</Tab>
+
+<Tab title="FL2VA">
+
+For `fl2va`, provide one or two image conditions with role `keyframe`. The supported frame-index sets are `[0]`, `[-1]`, and `[0, -1]`.
+
+The following request uses one server-local first frame. Use
+`frame_index: -1` for a last frame, or include both entries for first-and-last
+conditioning.
+
+Choose FL2VA when the supplied image should be the actual first or last frame
+of the generated clip. Use image-based Ref2VA instead when the image should
+guide identity, style, or composition without being preserved as an endpoint;
+Ref2VA may recompose or crop the reference.
+
+```bash Command
+curl -sS -X POST http://127.0.0.1:30010/v1/videos \
+  -H "Content-Type: application/json" \
+  -d '{
+    "model": "MiniMaxAI/MiniMax-H3",
+    "prompt": "The supplied frame continues with calm, natural motion and synchronized ambient sound.",
+    "seconds": 5,
+    "task": "fl2va",
+    "conditions": [
+      {
+        "type": "image",
+        "uri": "file:///data/minimax-h3/first-frame.png",
+        "role": "keyframe",
+        "frame_index": 0
+      }
+    ],
+    "target": {
+      "short_edge": 768,
+      "aspect_ratio": "auto",
+      "duration_seconds": 5.0
+    },
+    "num_outputs_per_prompt": 1,
+    "num_inference_steps": 50,
+    "flow_shift": 12.0,
+    "audio_flow_shift": 3.0,
+    "seed": 2101
+  }'
+```
+
+</Tab>
+
+<Tab title="V2V">
+
+V2V uses the reference-conditioning weights. Launch the server with
+`--model-variant ref2va`, keep the request `task` set to `ref2va`, and provide a video
+reference in `conditions`. There is no separate `v2v` task value.
+
+Use `type: "video"` when the input may be silent. If the file has a soundtrack,
+H3 also uses it as an audio reference. Use `type: "video_audio"` only when both
+streams are required; that form rejects an input without audio. The prompt tag
+for the visual stream is `<Video 1>`; an available soundtrack is exposed as
+`<Audio 1>`.
+
+<Note>
+Ref2VA treats the input video as reference material, not as a pixel-aligned
+edit source. It can resynthesize or reorder motion and cuts, and it does not
+expose a denoising-strength control. Do not rely on it to preserve every source
+frame or exact timing.
+</Note>
+
+Set `conditions[].start_time_seconds` to select a segment from a longer source.
+The default is `0`. SGLang seeks the visual stream and soundtrack to the same
+offset, then decodes at most the requested target duration in one pass; the
+source is not re-encoded into an intermediate clip.
+
+```bash Command
+curl -sS -X POST http://127.0.0.1:30010/v1/videos \
+  -H "Content-Type: application/json" \
+  -d '{
+    "model": "MiniMaxAI/MiniMax-H3",
+    "prompt": "Follow the motion and appearance of <Video 1>, changing the setting to a moonlit bedroom while preserving coherent timing.",
+    "seconds": 5,
+    "task": "ref2va",
+    "conditions": [
+      {
+        "type": "video",
+        "uri": "file:///data/minimax-h3/input.mp4",
+        "role": "reference",
+        "start_time_seconds": 35.0
+      }
+    ],
+    "target": {
+      "short_edge": 768,
+      "aspect_ratio": "16:9",
+      "duration_seconds": 5.0
+    },
+    "num_outputs_per_prompt": 1,
+    "num_inference_steps": 50,
+    "flow_shift": 12.0,
+    "audio_flow_shift": 3.0,
+    "seed": 4101
+  }'
+```
+
+Use `conditions[].uri` for H3 V2V. The generic top-level `video_path`,
+`video_url`, and `video_reference` upload fields are not lowered into H3
+reference conditions.
+
+</Tab>
+
+<Tab title="Multimodal Ref2VA">
+
+For `ref2va`, first launch the reference-conditioning capability with
+`--model-variant ref2va`, then provide conditions with role `reference`.
+Image, video, and audio references can be combined. Material tags in the
+prompt use the one-based order for each modality.
+
+An image condition here is semantic reference material rather than a
+pixel-aligned first frame. Use the FL2VA tab when animating a screenshot from
+that exact starting composition.
+
+```bash Command
+curl -sS -X POST http://127.0.0.1:30010/v1/videos \
+  -H "Content-Type: application/json" \
+  -d '{
+    "model": "MiniMaxAI/MiniMax-H3",
+    "prompt": "Use <Picture 1> as the visual subject and <Audio 1> as the sound reference, with coherent natural motion.",
+    "seconds": 5,
+    "task": "ref2va",
+    "conditions": [
+      {
+        "type": "image",
+        "uri": "file:///data/minimax-h3/reference.png",
+        "role": "reference"
+      },
+      {
+        "type": "audio",
+        "uri": "file:///data/minimax-h3/reference.mp3",
+        "role": "reference"
+      }
+    ],
+    "target": {
+      "short_edge": 768,
+      "aspect_ratio": "auto",
+      "duration_seconds": 5.0
+    },
+    "num_outputs_per_prompt": 1,
+    "num_inference_steps": 50,
+    "flow_shift": 12.0,
+    "audio_flow_shift": 3.0,
+    "seed": 3101
+  }'
+```
+
+</Tab>
+
+</Tabs>
+
+Poll and download any conditioned request with the same job-status and
+content endpoints used in the T2VA example. Server-local `file://` URIs must
+refer to files visible inside the SGLang server environment.
+
+## 5. Sampling and output controls
+
+MiniMax-H3 supports more than one output per prompt. The video API accepts
+`num_outputs_per_prompt` (or OpenAI-compatible `n`) from 1 through 10. Offline
+generation accepts `--num-outputs-per-prompt N`; `--num-outputs N` is the short
+alias. A scalar seed is expanded deterministically as `seed + output_index`, so
+the outputs do not reuse the same noise.
+
+Same-prompt fan-out reuses text conditioning. On the verified 2× RTX 5090
+recipe, a 5-step two-output request completed in 155.39 seconds versus 78.11
+seconds for one output, while producing two distinct valid MP4 files. The
+independent denoise and decode passes remain sequential on this 32 GB profile
+to keep peak memory bounded; the grouped path adds essentially no orchestration
+overhead. Use server replicas when lower wall-clock latency for many variants
+matters more than per-server memory efficiency.
+
+For example, set `"num_outputs_per_prompt": 2` in any request above. After the
+job completes, download both outputs by selecting each zero-based variant:
+
+```bash Command
+video_id="<completed-job-id>"
+for variant in 0 1; do
+  curl -sS -L \
+    "http://127.0.0.1:30010/v1/videos/${video_id}/content?variant=${variant}" \
+    -o "minimax-h3-${variant}.mp4"
+done
+```
+
+### Choose a quality profile
+
+`quality` is a request-scoped sampling parameter. One resident server can
+switch between all four profiles; an approximate request mounts its audited
+Cache-DiT policy at the batch boundary, and a later `lossless` request removes
+the hooks before denoising.
+
+Start the validated server once:
+
+```bash Command
+sglang serve \
+  --model-path MiniMaxAI/MiniMax-H3 \
+  --model-variant fl2va \
+  --num-gpus 4 \
+  --tp-size 1 \
+  --sp-degree 4 \
+  --ulysses-degree 4 \
+  --ring-degree 1 \
+  --performance-mode speed \
+  --use-fsdp-inference false \
+  --enable-torch-compile false \
+  --port 30010
+```
+
+Then choose a request tag:
+
+<Tabs>
+
+<Tab title="lossless">
+
+Native denoising with no feature-cache approximation. This is the default.
+
+```json Request field
+{
+  "quality": "lossless"
+}
+```
+
+</Tab>
+
+<Tab title="high">
+
+The least aggressive approximate profile. Use it when output should stay
+closest to the same-seed lossless trajectory.
+
+```json Request field
+{
+  "quality": "high"
+}
+```
+
+</Tab>
+
+<Tab title="medium">
+
+The balanced profile: substantially lower latency with a larger change from
+the same-seed lossless output.
+
+```json Request field
+{
+  "quality": "medium"
+}
+```
+
+</Tab>
+
+<Tab title="low">
+
+The fastest validated profile and the largest visual deviation. Use it for
+latency-sensitive previews and high-throughput generation.
+
+```json Request field
+{
+  "quality": "low"
+}
+```
+
+</Tab>
+
+</Tabs>
+
+The measured trade-off is:
+
+| `quality` | Mean <br />inference <br />latency | Speedup | SSIM vs <br />lossless | PSNR vs <br />lossless | Expected <br />trade-off |
+| --- | ---: | ---: | ---: | ---: | --- |
+| `lossless` | 75.10 s | 1.00× | 1.000 | exact | Native reference path |
+| `high` | 53.70 s | 1.40× | 0.931 | 28.16 dB | Smallest same-seed visual change |
+| `medium` | 30.23 s | 2.48× | 0.818 | 20.40 dB | Balanced latency and visual deviation |
+| `low` | 25.81 s | 2.91× | 0.794 | 19.25 dB | Largest deviation; fastest preview path |
+
+These numbers use 1344×768, 124-frame, 24 fps T2VA with 50 inference steps,
+video flow shift 12, audio flow shift 3, and three fixed prompt/seed pairs on
+4×H200. The prompts cover a quiet detailed scene, fast multi-subject action,
+and a moving close-up portrait. `inference_time_s` is averaged across the three
+prompts; the quiet-scene point is itself the mean of two repeats.
+
+SSIM and PSNR compare decoded, frame-aligned output with the `lossless` result
+for the same prompt and seed. They measure trajectory deviation, not absolute
+perceptual quality: an approximate profile can produce a different but still
+plausible realization. The profiles also change the joint audio-video denoise
+trajectory, while these two metrics cover video only.
+
+Approximate profiles currently accept only the exact workload and 4×H200
+deployment above; other hardware, task modes, request shapes, step counts, or
+flow shifts fail before denoising. Offline generation uses the same profile
+name, for example `sglang generate --quality medium`.
+
+<Note>
+`quality` selects a model sampling profile and can change generated content.
+`output_quality` controls only output-file compression; it is a separate field.
+</Note>
+
+For manually tuned Cache-DiT experiments outside that validated profile, omit
+the request `quality` field and set the process-wide environment controls
+directly. An explicit `quality: lossless` request overrides those controls and
+restores native denoising:
+
+```bash Command
+SGLANG_CACHE_DIT_ENABLED=true \
+SGLANG_CACHE_DIT_FN=1 \
+SGLANG_CACHE_DIT_BN=0 \
+SGLANG_CACHE_DIT_WARMUP=4 \
+SGLANG_CACHE_DIT_RDT=0.12 \
+SGLANG_CACHE_DIT_MC=2 \
+sglang serve \
+  --model-path MiniMaxAI/MiniMax-H3 \
+  --model-variant ref2va \
+  --num-gpus 8 \
+  --ulysses-degree 8 \
+  --performance-mode speed \
+  --port 30010
+```
+
+<Warning>
+Cache-DiT skips selected block computation and is approximate. It cannot be
+combined with FSDP inference or DiT layerwise offload. Breakable CUDA graph
+execution takes precedence and leaves Cache-DiT disabled. Tune the cache
+thresholds only after comparing both video and audio quality on the target
+task profile. A real B200 request has completed, but the named profiles above
+remain fail-closed to the audited 4×H200 workload.
+</Warning>
+
+## 6. Runtime feature recipes
+
+<Tabs>
+
+<Tab title="Lossless runtime">
+
+The recommended `speed` launch already combines resident components with
+Ulysses sequence parallelism. Validation status below applies only to the
+listed hardware and topology; it is not inherited by a similar GPU family.
+
+| Feature | Validation status | Notes |
+| --- | --- | --- |
+| Ulysses sequence parallelism | Verified: 8× B200, 4× H200, 4× H100, and Ulysses1/2/4/8 on MI300X and MI355X | Use `--ulysses-degree`; Ring is not compatible with H3's packed multi-segment attention. |
+| Tensor parallelism | Verified: B200 TP2 + Ulysses4; H100 TP2 + Ulysses2 and TP4 + Ulysses1 | `--tp-size` may be combined with Ulysses when the TP-local head count remains divisible by the Ulysses degree. On 4×H100, TP2 + Ulysses2 is the measured speed default. |
+| FSDP inference | Verified: 4× B200 and 4× H100 + Ulysses4 | Preserves H3's mixed BF16/FP32 parameter policy. B200 completed the exact eager comparison; H100 completed consecutive real requests at about 57 GB peak memory per GPU. |
+| Resident components | Verified: B200, H200, 4×H100 with TP, and 1/2/4/8× MI300X and MI355X | This is the recommended single-request latency path when the complete workload fits. |
+| CPU and layerwise offload | Verified: 2× RTX 5090 TP2 | The measured lossless recipe keeps 20 DiT blocks plus both VAE encoders resident, streams the remaining DiT blocks, text encoder, and video VAE decoder blocks, and leaves the small audio VAE resident. This status applies only to the listed topology. |
+| Breakable CUDA graph | Verified: B200 Ref2VA, opt-in | Matching eager output was observed for the captured signature, without a measured speedup. Re-capture for other shapes and reference sets. |
+| `torch.compile` | Measured: H200, opt-in | Steady-state benefit was below measurement noise, while startup increased and numerical output changed. Do not use it for consistency ground truth. |
+
+The verified parallel, placement, and matching-signature BCG paths keep the
+BF16/FP32 weights and denoising math. `torch.compile` is the exception called
+out above. Always use the eager BF16/FP32 launch when producing CI consistency
+ground truth.
+
+For the validated 1344×768 Ref2VA profile, use a 5504-row text bucket so both
+the server warmup and reference-conditioned requests share the captured
+signature:
+
+```bash Command
+sglang serve \
+  --model-path MiniMaxAI/MiniMax-H3 \
+  --model-variant ref2va \
+  --num-gpus 8 \
+  --ulysses-degree 8 \
+  --performance-mode speed \
+  --enable-breakable-cuda-graph true \
+  --warmup-resolutions 1344x768 \
+  --bcg-text-buckets 5504 \
+  --port 30010
+```
+
+BCG is lossless for a matching captured signature, but capture reserves extra
+GPU memory. Re-measure the live H3 text length before reusing this bucket for a
+different task profile, reference set, resolution, or prompt template.
+
+</Tab>
+
+<Tab title="Online quantization">
+
+On the verified 8× B200 topology, quantize the BF16 transformer at server load:
+
+```bash Command
+sglang serve \
+  --model-path MiniMaxAI/MiniMax-H3 \
+  --model-variant ref2va \
+  --num-gpus 8 \
+  --ulysses-degree 8 \
+  --performance-mode speed \
+  --quantization fp8 \
+  --port 30010
+```
+
+H3 automatically keeps its video/audio patch projections, timestep MLP, and
+final video/audio heads in FP32. All other linear layers have stable full
+module prefixes, so additional layers can be kept unquantized:
+
+```bash Command
+sglang serve \
+  --model-path MiniMaxAI/MiniMax-H3 \
+  --model-variant ref2va \
+  --num-gpus 8 \
+  --ulysses-degree 8 \
+  --quantization fp8 \
+  --quantization-ignored-layers blocks.0.attn token_refiner \
+  --port 30010
+```
+
+<Warning>
+Online FP8 is approximate and is not a consistency ground-truth mode. It can
+be combined with Cache-DiT, but the two approximations compound. Validate
+visual quality, audio quality, memory use, and latency on the target workload.
+The picker exposes this option only on the B200 and B300 topologies used for
+real H3 validation runs.
+</Warning>
+
+</Tab>
+
+</Tabs>
+
+## 7. Configuration notes
+
+- MiniMax-H3 produces the canonical 24 fps output; request duration is expressed through `target.duration_seconds`.
+- `target.duration_seconds` must be between 4 and 15 seconds, inclusive. The command picker defaults to the verified 5-second profile.
+- Use a 768-pixel short edge for the released quality profile. The aligned output dimensions are derived from `target.aspect_ratio`.
+- `flow_shift` controls video diffusion and `audio_flow_shift` controls audio diffusion.
+- V2V uses `task: "ref2va"` with a `video` or `video_audio` reference; it is served by the `Ref2VA` partition and is not a separate public task value.
+- `conditions[].start_time_seconds` selects a non-negative offset for a video reference. Its visual and audio streams are always sought together.
+- Ref2VA condition order is semantic and must match the one-based material tags in the prompt. For Ref2VA, `target.aspect_ratio: "auto"` resolves to the model's 16:9 fallback rather than inheriting a reference asset's geometry.
+- The distilled pipeline uses a single denoising branch, so CFG parallelism does not apply. Do not enable it: `--enable-cfg-parallel true` or `--cfg-parallel-size` greater than 1 is rejected instead of duplicating the positive branch. Explicitly disabling CFG, or setting its size to 1, remains a valid no-op.
+- The released visual VAE quality recipe uses overlapping tiled decode. SGLang keeps that recipe by default and distributes complete tiles across the decode group; this changes scheduling, not the computation inside each tile.
+- H3 rejects `--vae-config.parallel-decode-mode spatial` and `spatial_shard`: validation found output mismatches. Use the default released tiled recipe.
+- Keep the default `--encoder-parallel auto`. With the server’s default `batching_max_size` of 1, single-node H100/H200/B200/B300 recipes with peer-to-peer access fold the Qwen text encoder over otherwise idle Ulysses ranks. This is separate from DiT tensor parallelism. A pure-TP recipe already shards the encoder over its TP group and does not add a world fold.
+- For throughput-oriented serving, select **DP (batched throughput)**. The picker pairs `--encoder-parallel dp` with an editable `--batching-max-size` greater than 1; compatible requests are distributed across ranks, while every rank keeps a full encoder replica. Encoder DP requires TP1 and DiT DP1, so it is disabled for the H100 TP2 + Ulysses2 and RTX 5090 TP2 recipes. It provides no benefit for a batch of one and is not bitwise-identical to the folded deployment.
+- Use explicit **Fold** to prioritize single-request latency and encoder memory on a measured high-bandwidth single-node topology. Use **Replicate** as the compatibility path when folding or encoder DP is unsuitable.
+- `--use-fsdp-inference true` shards only the DiT. MiniMax-H3 preserves the original FP32 dtype of its patch, time, and output projections during FSDP all-gather, so this path does not trade numerical correctness for memory. On 4×H100, prefer TP2 + Ulysses2 for speed; use FSDP as an explicit capacity policy rather than assuming it is faster.
+- `speed` keeps model components resident, `auto` applies the model-aware 120 GiB residency threshold, and `memory` enables the memory-saving placement policy. Explicit `--layerwise-offload-components` overrides that placement list. DiT residency/prefetch knobs are scoped to the DiT; the text encoder and video VAE decoder use one-layer prefetch and zero residency, while the H3 video VAE encoder stays resident. When `memory` is combined with explicit FSDP, H3 instead keeps the sharded DiT on GPU and layerwise-offloads the text encoder and executable VAE decoder blocks. Use `speed` only after confirming that the complete target workload fits.
+- Breakable CUDA graph execution is an explicit opt-in, not part of the recommended `speed` preset. It requires `--enable-breakable-cuda-graph`, every served size in `--warmup-resolutions`, and `--bcg-text-buckets` that cover the live H3 condition sequence. The validated 1344×768 Ref2VA recipe uses 5504; other task profiles and reference sets may need a different value. It preserves eager output for matching captured signatures, but graph capture consumes additional GPU memory and may provide little latency benefit when Ulysses attention and collectives dominate, so benchmark it on the target topology before enabling it.
+
+## 8. Benchmarks
+
+The picker exposes resident and FSDP profiles on NVIDIA datacenter GPUs. GPU
+counts are properties of the selected recipes, not a claim that every platform
+requires that many GPUs. The detailed tables below report performance only for
+the configurations with collected measurements:
+
+| Hardware | Default resident recipe | Other profile or topology |
+| --- | --- | --- |
+| B300 | 8× Ulysses8 resident | 8× FSDP + Ulysses8; the 8-GPU sweep is not a minimum-GPU claim. |
+| B200 | 8× Ulysses8 resident | 4× FSDP + Ulysses4 |
+| H200 | 4× Ulysses4 resident | 4× FSDP + Ulysses4; 4× TP2 + Ulysses2 |
+| H100 | 4× TP2 + Ulysses2 resident | 4× TP4 + Ulysses1; 4× FSDP + Ulysses4 |
+| MI300X / MI355X | 8× Ulysses8 resident | 1×, 2×, and 4× scaling runs |
+| RTX 5090 | 2× TP2 + layerwise offload | — |
+
+### B300 precision and encoder placement
+
+A 12-configuration sweep on a single 8× B300 host, covering both checkpoint
+partitions, both transformer precisions, and all three text-encoder
+placements. It answers one question — *how long does one request take, and how
+much memory does it need*.
+
+### What was measured
+
+**Hardware.** 8× NVIDIA B300 SXM6, single node.
+
+**Model.** `MiniMaxAI/MiniMax-H3`, both released weight partitions.
+
+**Serve command.** Exactly the recipe the picker emits for B300, plus the one
+or two overlay flags under test:
+
+```bash Command
+sglang serve \
+  --model-path MiniMaxAI/MiniMax-H3 \
+  --model-variant fl2va \
+  --num-gpus 8 \
+  --ulysses-degree 8 \
+  --performance-mode speed \
+  --host 0.0.0.0 \
+  --port 30010
+```
+
+The swept axes are `--model-variant` (`fl2va` / `ref2va`), `--quantization`
+(unset for BF16 / `fp8`), and `--encoder-parallel` (`auto` / `fold` /
+`replicate`). Nothing else differs between the 12 servers.
+
+This is a single-request latency sweep (`batching_max_size: 1`), so encoder DP
+is intentionally excluded: it cannot distribute a batch of one. Use the
+picker’s **DP (batched throughput)** option for a multi-request throughput
+deployment; the table below does not claim a measured H3 DP speedup.
+
+**Driver.**
+
+```bash Command
+python3 -m sglang.multimodal_gen.benchmarks.bench_serving \
+  --host 127.0.0.1 --port 30010 \
+  --model MiniMaxAI/MiniMax-H3 \
+  --dataset vbench --task text-to-video \
+  --num-prompts 1 --max-concurrency 1 \
+  --warmup-requests 1 --warmup-inference-steps 50 \
+  --extra-body '{"task":"t2va","conditions":[],"target":{"short_edge":768,"aspect_ratio":"16:9","duration_seconds":5.0},"seconds":5,"flow_shift":12.0,"audio_flow_shift":3.0}'
+```
+
+**Workload**
+
+| Property | Value |
+| --- | --- |
+| Output duration | 5.167 s |
+| Resolution | 1344×768 |
+| Frames | 124 @ 24 fps |
+| Denoising steps | 50 |
+| `flow_shift` / `audio_flow_shift` | 12.0 / 3.0 |
+| Requests in flight | 1 (`--max-concurrency 1`, server at `batching_max_size: 1`) |
+| Requests measured | 1 per cell, after 1 warmup request |
+
+### Results
+
+| Weights | Precision | Encoder | Load | Warmup | Latency | Peak/GPU |
+| --- | --- | --- | ---: | ---: | ---: | ---: |
+| FL2VA | BF16 | auto | 118.1 s | 29.65 s | **19.04 s** | 83,578 MB |
+| FL2VA | BF16 | fold | 114.0 s | 28.72 s | **19.04 s** | 83,578 MB |
+| FL2VA | BF16 | replicate | 116.0 s | 28.33 s | **19.04 s** | 124,158 MB |
+| FL2VA | FP8 | auto | 116.0 s | 27.16 s | **18.03 s** | 51,926 MB |
+| FL2VA | FP8 | fold | 116.0 s | 25.99 s | **18.04 s** | 51,926 MB |
+| FL2VA | FP8 | replicate | 118.0 s | 27.97 s | **18.04 s** | 92,506 MB |
+| Ref2VA | BF16 | auto | 114.0 s | 38.69 s | **29.12 s** | 83,968 MB |
+| Ref2VA | BF16 | fold | 118.0 s | 36.58 s | **29.13 s** | 83,968 MB |
+| Ref2VA | BF16 | replicate | 116.0 s | 35.17 s | **29.13 s** | 124,490 MB |
+| Ref2VA | FP8 | auto | 124.0 s | 34.30 s | **27.12 s** | 52,816 MB |
+| Ref2VA | FP8 | fold | 112.0 s | 34.44 s | **27.12 s** | 52,816 MB |
+| Ref2VA | FP8 | replicate | 116.0 s | 33.42 s | **27.12 s** | 93,396 MB |
+
+### H200 topology comparison
+
+The same four-card H200 host completed both lossless resident placements with
+the standard 1344×768, 5-second, 50-step T2VA request (fixed prompt and seed,
+eager BF16/FP32, back-to-back runs on an otherwise idle host). Latency is the
+warmed-up request; the first pair uses the default warmup request, the second
+pair adds `--warmup-resolutions 1344x768` so warmup already covers the served
+resolution:
+
+| Topology | Warmup | Denoise | Decode | E2E | Peak/GPU |
+| --- | --- | ---: | ---: | ---: | ---: |
+| Ulysses4 | default | 79.04 s | 3.77 s | **84.14 s** | 94,288 MB |
+| TP2 + Ulysses2 | default | 81.17 s | 2.97 s | 85.51 s | 63,490 MB |
+| Ulysses4 | `--warmup-resolutions 1344x768` | 71.73 s | 1.32 s | **74.38 s** | 94,290 MB |
+| TP2 + Ulysses2 | `--warmup-resolutions 1344x768` | 75.52 s | 1.29 s | 78.33 s | 63,490 MB |
+
+Ulysses4 stays the H200 latency default: 5.0 % faster end-to-end than
+TP2 + Ulysses2 once warmup covers the served resolution (1.6 % with the
+default warmup, where first-request cold start masks the topology gap).
+TP2 + Ulysses2 shards the DiT weights and holds peak memory about 30 GB per
+GPU lower, which is why it remains the 80 GB H100 recipe. Matching the warmup
+request to the served resolution removes the cold first-request cost on both
+topologies (about 10 s end-to-end on this workload).
+
+### H100 topology comparison
+
+The same four-card H100 host completed three lossless placements. TP2 with
+Ulysses2 was the fastest; TP4 used the least memory:
+
+| Topology | Pipeline latency | Peak/GPU |
+| --- | ---: | ---: |
+| TP2 + Ulysses2 | 13.25 s | 66.04 GB |
+| FSDP + Ulysses4 | 13.36 s | 57.01 GB |
+| TP4 + Ulysses1 | 13.86 s | 49.80 GB |
+
+### RTX 5090 capacity run
+
+The verified two-card RTX 5090 host used TP2 with layerwise offload. The full
+50-step, 1344×768, 5-second request completed in 559.67 seconds: 525.05
+seconds of denoising and 33.61 seconds of decoding, with a 26.3 GiB sampled
+peak per GPU.
+
+| DiT settings | 5-step denoise | Inference | Peak/GPU | Result |
+| --- | ---: | ---: | ---: | --- |
+| prefetch 1, resident 20 | 43.48 s | 78.11 s | 26.3 GiB | Selected recipe |
+| prefetch 2, resident 20 | 43.37 s | 78.06 s | 27.5 GiB | No measurable gain |
+| Ulysses2, prefetch 2, resident 10 | Did not reach warmup | — | — | Rejected |
+
+### AMD Instinct task and scaling runs
+
+The AMD recipes keep the released BF16/FP32 precision policy and use AITER
+packed attention. The picker emits the fastest measured topology, 8 GPUs with
+Ulysses degree 8. All runs below completed full H.264/AAC decoding and
+representative-frame inspection.
+
+| Hardware | Task | Denoise | Decode | Peak/GPU |
+| --- | --- | ---: | ---: | ---: |
+| MI355X | T2VA | 55.2907 s | 9.5344 s | 97,444 MB |
+| MI355X | FL2VA | 53.7978 s | 9.4477 s | 96,922 MB |
+| MI355X | Ref2VA | 41.3812 s | 6.8247 s | 94,518 MB |
+| MI300X | T2VA | 167.4878 s | 25.3244 s | 97,272 MB |
+| MI300X | FL2VA | 150.2311 s | 12.5684 s | 96,750 MB |
+| MI300X | Ref2VA | 107.6232 s | 11.3768 s | 94,268 MB |
+
+The task matrix used 8 GPUs and 50 denoising steps. The scaling matrix uses
+one 1344×768, 209-frame T2VA request and changes only the GPU count and
+matching Ulysses degree:
+
+| Hardware | GPUs | Denoise | Decode | Peak/GPU |
+| --- | ---: | ---: | ---: | ---: |
+| MI355X | 8 | 55.2907 s | 9.5344 s | 97,444 MB |
+| MI355X | 4 | 104.2294 s | 11.1824 s | 103,350 MB |
+| MI355X | 2 | 223.0246 s | 15.5330 s | 115,250 MB |
+| MI355X | 1 | 288.7968 s | 24.0472 s | 137,676 MB |
+| MI300X | 8 | 167.4878 s | 25.3244 s | 97,272 MB |
+| MI300X | 4 | 297.3727 s | 26.5067 s | 103,436 MB |
+| MI300X | 2 | 585.5401 s | 29.4909 s | 115,010 MB |
+| MI300X | 1 | 978.0886 s | 36.0142 s | 137,626 MB |
+
+For a measured lower-count AMD deployment, set both `--num-gpus` and
+`--ulysses-degree` to 4, 2, or 1. AITER packed attention matched segment-wise
+BF16 SDPA at cosine similarity `0.9999991655` on MI355X and `0.9999991059` on
+MI300X.
@@ -0,0 +1,283 @@
+---
+title: Qwen-Image-Edit-2511
+metatags:
+    description: "Deploy Qwen-Image-Edit-2511 with SGLang - 20B image editing model with text rendering, character consistency, and geometric reasoning."
+---
+
+import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';
+import { QwenImageEditDeployment } from '/src/snippets/diffusion/qwen-image-edit-deployment.jsx';
+
+<DiffusionModelTags tags={["image", "image editing", "text rendering", "character consistency"]} />
+
+## 1. Model Introduction
+
+[Qwen-Image-Edit-2511](https://huggingface.co/Qwen/Qwen-Image-Edit-2511) is an enhanced version over Qwen-Image-Edit-2509, featuring multiple improvements—including notably better consistency. Built upon the 20B Qwen-Image model, Qwen-Image-Edit-2511 successfully extends Qwen-Image's unique text rendering capabilities to image editing tasks, enabling precise text editing.
+
+Key Enhancements in Qwen-Image-Edit-2511:
+
+- **Mitigate Image Drift**: Reduces unwanted changes in non-edited regions of the image.
+- **Improved Character Consistency**: The model can perform imaginative edits based on an input portrait while preserving the identity and visual characteristics of the subject.
+- **Multi-Person Consistency**: Enhanced consistency in multi-person group photos, enabling high-fidelity fusion of two separate person images into a coherent group shot.
+- **Integrated LoRA Capabilities**: Selected popular community-created LoRAs are integrated directly into the base model, unlocking their effects without extra tuning (e.g., lighting enhancement, viewpoint generation).
+- **Enhanced Industrial Design Generation**: Special attention to practical engineering scenarios, including batch industrial product design and material replacement for industrial components.
+- **Strengthened Geometric Reasoning**: Stronger geometric reasoning capability for generating auxiliary construction lines for design or annotation purposes.
+
+For more details, please refer to the [official Qwen-Image-Edit-2511 HuggingFace page](https://huggingface.co/Qwen/Qwen-Image-Edit-2511), the [Blog](https://qwenlm.github.io/blog/qwen-image-edit-2511/), and the [Tech Report](https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-Image/Qwen_Image.pdf).
+
+## 2. SGLang-diffusion Installation
+
+SGLang-diffusion offers multiple installation methods. You can choose the most suitable installation method based on your hardware platform and requirements.
+
+Please refer to the [official SGLang-diffusion installation guide](https://docs.sglang.io/docs/sglang-diffusion/installation) for installation instructions.
+
+## 3. Model Deployment
+
+This section provides deployment configurations optimized for different hardware platforms and use cases.
+
+### 3.1 Basic Configuration
+
+Qwen-Image-Edit-2511 is a 20B parameter model optimized for image editing tasks. The recommended launch configurations vary by hardware.
+
+**Interactive Command Generator**: Use the configuration selector below to automatically generate the appropriate deployment command for your hardware platform.
+
+<QwenImageEditDeployment />
+
+### 3.2 Configuration Tips
+
+Currently supported optimizations are listed [here](/docs/sglang-diffusion/compatibility_matrix).
+
+- `--vae-path`: Path to a custom VAE model or HuggingFace model ID (e.g., fal/FLUX.2-Tiny-AutoEncoder). If not specified, the VAE will be loaded from the main model path.
+- `--num-gpus`: Number of GPUs to use
+- `--tp-size`: Tensor parallelism size (only for the encoder; should not be larger than 1 if text encoder offload is enabled, as layer-wise offload plus prefetch is faster)
+- `--sp-degree`: Sequence parallelism size (typically should match the number of GPUs)
+- `--ulysses-degree`: The degree of DeepSpeed-Ulysses-style SP in USP
+- `--ring-degree`: The degree of ring attention-style SP in USP
+
+## 4. API Usage
+
+For complete API documentation, please refer to the [official API usage guide](/docs/sglang-diffusion/api/openai_api).
+
+### 4.1 Edit an Image
+
+```python Example
+import base64
+from openai import OpenAI
+
+client = OpenAI(api_key="EMPTY", base_url="http://localhost:3000/v1")
+
+response = client.images.edit(
+    model="Qwen/Qwen-Image-Edit-2511",
+    image=open("input.png", "rb"),
+    prompt="Change the color of the taxi to black.",
+    n=1,
+    response_format="b64_json",
+)
+
+# Save the edited image
+image_bytes = base64.b64decode(response.data[0].b64_json)
+with open("output.png", "wb") as f:
+    f.write(image_bytes)
+```
+
+### 4.2 Advanced Usage
+
+#### 4.2.1 Cache-DiT Acceleration
+
+SGLang integrates [Cache-DiT](https://github.com/vipshop/cache-dit), a caching acceleration engine for Diffusion Transformers (DiT), to achieve up to 7.4x inference speedup with minimal quality loss. You can set `SGLANG_CACHE_DIT_ENABLED=True` to enable it. For more details, please refer to the SGLang Cache-DiT [documentation](/docs/sglang-diffusion/cache_dit).
+
+**Basic Usage**
+
+```bash Command
+SGLANG_CACHE_DIT_ENABLED=true sglang serve --model-path Qwen/Qwen-Image-Edit-2511
+```
+
+**Advanced Usage**
+
+- DBCache Parameters: DBCache controls block-level caching behavior:
+
+<table style={{width: "100%", borderCollapse: "collapse", tableLayout: "fixed"}}>
+  <colgroup>
+    <col style={{width: "25.0%"}} />
+    <col style={{width: "25.0%"}} />
+    <col style={{width: "25.0%"}} />
+    <col style={{width: "25.0%"}} />
+  </colgroup>
+  <thead>
+    <tr style={{borderBottom: "2px solid #d55816"}}>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Parameter</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>Env Variable</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Default</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>Description</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Fn</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_FN`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>1</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Number of first blocks to always compute</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Bn</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_BN`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>0</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Number of last blocks to always compute</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>W</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_WARMUP`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>4</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Warmup steps before caching starts</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>R</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_RDT`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>0.24</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Residual difference threshold</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>MC</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_MC`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>3</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Maximum continuous cached steps</td>
+    </tr>
+  </tbody>
+</table>
+- TaylorSeer Configuration: TaylorSeer improves caching accuracy using Taylor expansion:
+
+<table style={{width: "100%", borderCollapse: "collapse", tableLayout: "fixed"}}>
+  <colgroup>
+    <col style={{width: "25.0%"}} />
+    <col style={{width: "25.0%"}} />
+    <col style={{width: "25.0%"}} />
+    <col style={{width: "25.0%"}} />
+  </colgroup>
+  <thead>
+    <tr style={{borderBottom: "2px solid #d55816"}}>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Parameter</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>Env Variable</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Default</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>Description</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Enable</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_TAYLORSEER`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>false</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Enable TaylorSeer calibrator</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Order</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_TS_ORDER`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>1</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Taylor expansion order (1 or 2)</td>
+    </tr>
+  </tbody>
+</table>
+
+  Combined Configuration Example:
+
+```bash Command
+SGLANG_CACHE_DIT_ENABLED=true \
+SGLANG_CACHE_DIT_FN=2 \
+SGLANG_CACHE_DIT_BN=1 \
+SGLANG_CACHE_DIT_WARMUP=4 \
+SGLANG_CACHE_DIT_RDT=0.4 \
+SGLANG_CACHE_DIT_MC=4 \
+SGLANG_CACHE_DIT_TAYLORSEER=true \
+SGLANG_CACHE_DIT_TS_ORDER=2 \
+sglang serve --model-path Qwen/Qwen-Image-Edit-2511
+```
+
+#### 4.2.2 CPU Offload
+
+- `--dit-cpu-offload`: Use CPU offload for DiT inference. Enable if run out of memory.
+- `--text-encoder-cpu-offload`: Use CPU offload for text encoder inference.
+- `--image-encoder-cpu-offload`: Use CPU offload for image encoder inference.
+- `--vae-cpu-offload`: Use CPU offload for VAE.
+- `--pin-cpu-memory`: Pin memory for CPU offload. Only added as a temp workaround if it throws "CUDA error: invalid argument".
+
+## 5. Benchmark
+
+Test Environment:
+
+- Hardware: NVIDIA B200 GPU (1x)
+- Model: Qwen/Qwen-Image-Edit-2511
+- sglang diffusion version: 0.5.6.post2
+
+### 5.1 Speedup Benchmark
+
+#### 5.1.1 Edit a image
+
+**Server Command**:
+
+```shell Command
+sglang serve --model-path Qwen/Qwen-Image-Edit-2511 --port 30000
+```
+
+**Benchmark Command**:
+
+```shell Command
+python3 -m sglang.multimodal_gen.benchmarks.bench_serving \
+    --backend sglang-image --dataset vbench --task ti2i --num-prompts 1 --max-concurrency 1
+```
+
+**Result**:
+
+```text Output
+================= Serving Benchmark Result =================
+Backend:                                 sglang-image
+Model:                                   Qwen/Qwen-Image-Edit-2511
+Dataset:                                 vbench
+Task:                                    ti2i
+--------------------------------------------------
+Benchmark duration (s):                  35.31
+Request rate:                            inf
+Max request concurrency:                 1
+Successful requests:                     1/1
+--------------------------------------------------
+Request throughput (req/s):              0.03
+Latency Mean (s):                        35.3053
+Latency Median (s):                      35.3053
+Latency P99 (s):                         35.3053
+--------------------------------------------------
+Peak Memory Max (MB):                    47959.35
+Peak Memory Mean (MB):                   47959.35
+Peak Memory Median (MB):                 47959.35
+============================================================
+```
+
+#### 5.1.2 Edit a image with high concurrency
+
+**Benchmark Command**:
+
+```shell Command
+python3 -m sglang.multimodal_gen.benchmarks.bench_serving \
+    --backend sglang-image --dataset vbench --task ti2i --num-prompts 20 --max-concurrency 20
+```
+
+**Result**:
+
+```text Output
+================= Serving Benchmark Result =================
+Backend:                                 sglang-image
+Model:                                   Qwen/Qwen-Image-Edit-2511
+Dataset:                                 vbench
+Task:                                    ti2i
+--------------------------------------------------
+Benchmark duration (s):                  286.11
+Request rate:                            inf
+Max request concurrency:                 20
+Successful requests:                     20/20
+--------------------------------------------------
+Request throughput (req/s):              0.07
+Latency Mean (s):                        150.0428
+Latency Median (s):                      150.0600
+Latency P99 (s):                         283.3843
+--------------------------------------------------
+Peak Memory Max (MB):                    47971.82
+Peak Memory Mean (MB):                   47971.49
+Peak Memory Median (MB):                 47971.29
+============================================================
+```
@@ -0,0 +1,383 @@
+---
+title: Qwen-Image
+metatags:
+    description: "Deploy Qwen-Image with SGLang - community contribution guide for Qwen's image generation model."
+---
+
+import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';
+import { QwenImageDeployment } from '/src/snippets/diffusion/qwen-image-deployment.jsx';
+
+<DiffusionModelTags tags={["image", "text-to-image", "text rendering", "NVFP4"]} />
+
+## 1. Model Introduction
+
+[Qwen-Image](https://huggingface.co/Qwen/Qwen-Image) is a text-to-image diffusion model developed by the Qwen team.
+
+For more details, please refer to the [official Qwen-Image HuggingFace page](https://huggingface.co/Qwen/Qwen-Image), the [Blog](https://qwenlm.github.io/blog/qwen-image/), and the [Tech Report](https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-Image/Qwen_Image.pdf).
+
+## 2. SGLang-diffusion Installation
+
+SGLang-diffusion offers multiple installation methods. You can choose the most suitable installation method based on your hardware platform and requirements.
+
+Please refer to the [official SGLang-diffusion installation guide](../../../docs/sglang-diffusion/installation) for installation instructions.
+
+## 3. Model Deployment
+
+This section provides deployment configurations optimized for different hardware platforms and use cases.
+
+### 3.1 Basic Configuration
+
+Qwen-Image is a text-to-image model. The recommended launch configurations vary by hardware. SGLang supports serving Qwen-Image on NVIDIA B200, B300, H200, H100, AMD MI300X, MI325X, MI355X GPUs and Ascend A2, A3 NPUs.
+
+**Interactive Command Generator**: Use the configuration selector below to automatically generate the appropriate deployment command for your hardware platform.
+
+<QwenImageDeployment />
+
+For the validated ModelOpt NVFP4 checkpoint on Blackwell, load the published
+Qwen-Image-2512 NVFP4 repo directly:
+
+```bash Command
+sglang serve \
+  --model-path lmsys/qwen-image-2512-modelopt-nvfp4-sglang \
+  --ulysses-degree=1 \
+  --ring-degree=1
+```
+
+For high-resolution B200 generations, the FlashInfer CUTLASS FP4 GEMM backend
+can be faster than the default TensorRT-LLM FP4 GEMM backend:
+
+```bash Command
+SGLANG_DIFFUSION_FLASHINFER_FP4_GEMM_BACKEND=cutlass \
+sglang generate \
+  --model-path lmsys/qwen-image-2512-modelopt-nvfp4-sglang \
+  --width 2048 --height 2048 \
+  --prompt "A tiny astronaut reading a book under a glass greenhouse" \
+  --save-output
+```
+
+### 3.2 Configuration Tips
+
+Currently supported optimizations are listed [here](/docs/sglang-diffusion/compatibility_matrix).
+
+- `--vae-path`: Path to a custom VAE model or HuggingFace model ID (e.g., fal/FLUX.2-Tiny-AutoEncoder). If not specified, the VAE will be loaded from the main model path.
+- `--num-gpus`: Number of GPUs to use
+- `--tp-size`: Tensor parallelism size (only for the encoder; should not be larger than 1 if text encoder offload is enabled, as layer-wise offload plus prefetch is faster)
+- `--sp-degree`: Sequence parallelism size (typically should match the number of GPUs)
+- `--ulysses-degree`: The degree of DeepSpeed-Ulysses-style SP in USP
+- `--ring-degree`: The degree of ring attention-style SP in USP
+
+**AMD ROCm Notes**: Requires SGLang >= v0.5.8.
+
+## 4. API Usage
+
+For complete API documentation, please refer to the [official API usage guide](../../../docs/sglang-diffusion/api/openai_api).
+
+### 4.1 Generate an Image
+
+```python Example
+import base64
+from openai import OpenAI
+
+client = OpenAI(api_key="EMPTY", base_url="http://localhost:30000/v1")
+
+response = client.images.generate(
+    model="Qwen/Qwen-Image",
+    prompt="A logo With Bold Large text: SGL Diffusion",
+    n=1,
+    response_format="b64_json",
+)
+
+# Save the generated image
+image_bytes = base64.b64decode(response.data[0].b64_json)
+with open("output.png", "wb") as f:
+    f.write(image_bytes)
+```
+
+### 4.2 Advanced Usage
+
+#### 4.2.1 Cache-DiT Acceleration
+
+SGLang integrates [Cache-DiT](https://github.com/vipshop/cache-dit), a caching acceleration engine for Diffusion Transformers (DiT), to achieve up to 7.4x inference speedup with minimal quality loss. You can set `SGLANG_CACHE_DIT_ENABLED=True` to enable it. For more details, please refer to the SGLang Cache-DiT [documentation](../../../docs/sglang-diffusion/cache_dit).
+
+**Basic Usage**
+
+```bash Command
+SGLANG_CACHE_DIT_ENABLED=true sglang serve --model-path Qwen/Qwen-Image
+```
+
+**Advanced Usage**
+
+- DBCache Parameters: DBCache controls block-level caching behavior:
+
+<table style={{width: "100%", borderCollapse: "collapse", tableLayout: "fixed"}}>
+  <colgroup>
+    <col style={{width: "25.0%"}} />
+    <col style={{width: "25.0%"}} />
+    <col style={{width: "25.0%"}} />
+    <col style={{width: "25.0%"}} />
+  </colgroup>
+  <thead>
+    <tr style={{borderBottom: "2px solid #d55816"}}>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Parameter</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>Env Variable</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Default</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>Description</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Fn</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_FN`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>1</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Number of first blocks to always compute</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Bn</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_BN`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>0</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Number of last blocks to always compute</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>W</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_WARMUP`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>4</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Warmup steps before caching starts</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>R</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_RDT`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>0.24</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Residual difference threshold</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>MC</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_MC`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>3</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Maximum continuous cached steps</td>
+    </tr>
+  </tbody>
+</table>
+- TaylorSeer Configuration: TaylorSeer improves caching accuracy using Taylor expansion:
+
+<table style={{width: "100%", borderCollapse: "collapse", tableLayout: "fixed"}}>
+  <colgroup>
+    <col style={{width: "25.0%"}} />
+    <col style={{width: "25.0%"}} />
+    <col style={{width: "25.0%"}} />
+    <col style={{width: "25.0%"}} />
+  </colgroup>
+  <thead>
+    <tr style={{borderBottom: "2px solid #d55816"}}>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Parameter</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>Env Variable</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Default</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>Description</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Enable</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_TAYLORSEER`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>false</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Enable TaylorSeer calibrator</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Order</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_TS_ORDER`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>1</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Taylor expansion order (1 or 2)</td>
+    </tr>
+  </tbody>
+</table>
+
+  Combined Configuration Example:
+
+```bash Command
+SGLANG_CACHE_DIT_ENABLED=true \
+SGLANG_CACHE_DIT_FN=2 \
+SGLANG_CACHE_DIT_BN=1 \
+SGLANG_CACHE_DIT_WARMUP=4 \
+SGLANG_CACHE_DIT_RDT=0.4 \
+SGLANG_CACHE_DIT_MC=4 \
+SGLANG_CACHE_DIT_TAYLORSEER=true \
+SGLANG_CACHE_DIT_TS_ORDER=2 \
+sglang serve --model-path Qwen/Qwen-Image
+```
+
+#### 4.2.2 CPU Offload
+
+- `--dit-cpu-offload`: Use CPU offload for DiT inference. Enable if run out of memory.
+- `--text-encoder-cpu-offload`: Use CPU offload for text encoder inference.
+- `--vae-cpu-offload`: Use CPU offload for VAE.
+- `--pin-cpu-memory`: Pin memory for CPU offload. Only added as a temp workaround if it throws "CUDA error: invalid argument".
+
+## 5. Benchmark
+
+Test Environment:
+
+- Hardware: AMD Instinct MI300X GPU (1x)
+- Model: Qwen/Qwen-Image
+- Docker Image: lmsysorg/sglang:v0.5.8-rocm700-mi30x
+- sglang diffusion version: 0.5.8
+
+### 5.1 Speedup Benchmark
+
+#### 5.1.1 Generate an image
+
+<Tabs>
+  <Tab title="AMD MI300X">
+    **Server Command**:
+
+    ```shell Command
+    sglang serve --model-path Qwen/Qwen-Image \
+        --ulysses-degree=1 --ring-degree=1 --port 30000
+    ```
+
+    **Benchmark Command**:
+
+    ```shell Command
+    python3 -m sglang.multimodal_gen.benchmarks.bench_serving \
+        --backend sglang-image --dataset vbench --task text-to-image --num-prompts 1 --max-concurrency 1
+    ```
+
+    **Result**:
+
+    ```text Output
+    ================= Serving Benchmark Result =================
+    Task:                                    text-to-image
+    Model:                                   Qwen/Qwen-Image
+    Dataset:                                 vbench
+    --------------------------------------------------
+    Benchmark duration (s):                  29.04
+    Request rate:                            inf
+    Max request concurrency:                 1
+    Successful requests:                     1/1
+    --------------------------------------------------
+    Request throughput (req/s):              0.03
+    Latency Mean (s):                        29.0378
+    Latency Median (s):                      29.0378
+    Latency P99 (s):                         29.0378
+    --------------------------------------------------
+    Peak Memory Max (MB):                    48018.83
+    Peak Memory Mean (MB):                   48018.83
+    Peak Memory Median (MB):                 48018.83
+    ============================================================
+    ```
+  </Tab>
+  <Tab title="Ascend A3">
+    **Server Command**:
+
+    ```shell Command
+    #One A3 card has 2 npu chips
+    sglang serve --tp-size 2 --sp-degree 1 --model-path Qwen/Qwen-Image --num-gpus 2
+    ```
+
+    **Benchmark Command**:
+
+    ```shell Command
+    python -m sglang.multimodal_gen.benchmarks.bench_serving --dataset vbench --task text-to-image --num-prompts 1 --max-concurrency 1
+    ```
+
+    **Result**:
+
+    ```text Output
+    ================= Serving Benchmark Result =================
+    Task:                                         text-to-image
+    Model:                                        Qwen/Qwen-Image
+    Dataset:                                      vbench
+    --------------------------------------------------
+    Benchmark duration (s):                       36.26
+    Request rate:                                 inf
+    Max request concurrency:                      1
+    Successful requests:                          1/1
+    Completed outputs:                            1
+    Outputs per prompt:                           1
+    --------------------------------------------------
+    Request throughput (req/s):                   0.03
+    Output throughput (outputs/s):                0.03
+    Latency Mean (s):                             36.26
+    Latency Median (s):                           36.26
+    Latency P90 (s):                              36.26
+    Latency P95 (s):                              36.26
+    Latency P99 (s):                              36.26
+    --------------------------------------------------
+    Peak Memory Max (MB):                         36984.00
+    Peak Memory Mean (MB):                        36984.00
+    Peak Memory Median (MB):                      36984.00
+    ------------------------------------------------------------
+    ```
+  </Tab>
+</Tabs>
+
+#### 5.1.2 Generate images with high concurrency
+
+<Tabs>
+  <Tab title="AMD MI300X">
+    **Benchmark Command**:
+
+    ```shell Command
+    python3 -m sglang.multimodal_gen.benchmarks.bench_serving \
+        --backend sglang-image --dataset vbench --task text-to-image --num-prompts 20 --max-concurrency 20 --port 30000
+    ```
+
+    **Result**:
+
+    ```text Output
+    ================= Serving Benchmark Result =================
+    Task:                                    text-to-image
+    Model:                                   Qwen/Qwen-Image
+    Dataset:                                 vbench
+    --------------------------------------------------
+    Benchmark duration (s):                  300.79
+    Request rate:                            inf
+    Max request concurrency:                 20
+    Successful requests:                     14/20
+    --------------------------------------------------
+    Request throughput (req/s):              0.05
+    Latency Mean (s):                        154.5368
+    Latency Median (s):                      154.8363
+    Latency P99 (s):                         285.4603
+    --------------------------------------------------
+    Peak Memory Max (MB):                    48030.31
+    Peak Memory Mean (MB):                   48030.30
+    Peak Memory Median (MB):                 48030.29
+    ============================================================
+    ```
+  </Tab>
+  <Tab title="Ascend A3">
+    **Benchmark Command**:
+
+    ```shell Command
+    python -m sglang.multimodal_gen.benchmarks.bench_serving --dataset vbench --task text-to-image --num-prompts 20 --max-concurrency 20
+    ```
+
+    **Result**:
+
+    ```text Output
+    ================= Serving Benchmark Result =================
+    Task:                                         text-to-image
+    Model:                                        Qwen/Qwen-Image
+    Dataset:                                      vbench
+    --------------------------------------------------
+    Benchmark duration (s):                       300.81
+    Request rate:                                 inf
+    Max request concurrency:                      20
+    Successful requests:                          8/20
+    Completed outputs:                            8
+    Outputs per prompt:                           1
+    --------------------------------------------------
+    Request throughput (req/s):                   0.03
+    Output throughput (outputs/s):                0.03
+    Latency Mean (s):                             166.61
+    Latency Median (s):                           167.02
+    Latency P90 (s):                              270.80
+    Latency P95 (s):                              283.48
+    Latency P99 (s):                              293.64
+    --------------------------------------------------
+    Peak Memory Max (MB):                         36984.00
+    Peak Memory Mean (MB):                        36984.00
+    Peak Memory Median (MB):                      36984.00
+    ------------------------------------------------------------
+    ```
+  </Tab>
+</Tabs>
@@ -0,0 +1,93 @@
+---
+title: "Diffusion Cookbook"
+description: "Cookbook recipes for running diffusion models with SGLang"
+metatags:
+    description: "Explore SGLang diffusion cookbook structure, categories, and contribution guidance for image and video generation recipes."
+---
+
+# SGLang Diffusion Cookbook
+
+<div style={{display: 'flex', gap: '8px'}}>
+  <a href="https://opensource.org/licenses/Apache-2.0"><img src="https://img.shields.io/badge/License-Apache_2.0-blue.svg" alt="License" /></a>
+  <a href="https://github.com/sgl-project/sglang/pulls"><img src="https://img.shields.io/badge/PRs-welcome-brightgreen.svg" alt="PRs Welcome" /></a>
+</div>
+
+Create a comprehensive cookbook for diffusion models in SGLang, demonstrating SGLang's performance advantages for image and video generation workloads.
+
+## 🎯 What You'll Find Here
+
+This cookbook aggregates battle-tested SGLang recipes covering:
+
+- **Models**: Mainstream Image and Video generation  Models
+- **Use Cases**: Inference serving, deployment strategies
+- **Hardware**: GPU and CPU configurations, optimization for different accelerators
+- **Best Practices**: Configuration templates, performance tuning, troubleshooting guides
+
+Each recipe provides step-by-step instructions to help you quickly implement SGLang solutions for your specific requirements.
+
+## 🚀 Quick Start
+
+1. Browse the recipe index above to find your model
+2. Follow the step-by-step instructions in each guide
+3. Adapt configurations to your specific hardware and requirements
+4. Join our community to share feedback and improvements
+
+The sglang diffusion cookbook directory structure are shown below:
+
+```text Example
+docs/cookbook/diffusion/
+├── README.mdx             # Main cookbook (this file)
+├── Qwen-Image/            # Qwen-Image series docs
+│   ├── Qwen-Image.mdx
+│   └── Qwen-Image-Edit.mdx
+├── Wan/                   # Wan series docs
+│   ├── Wan2.1.mdx
+│   └── Wan2.2.mdx
+├── Z-Image/               # Z-Image series docs
+│   └── Z-Image-Turbo.mdx
+├── Ernie-Image/           # ERNIE-Image series docs
+│   └── Ernie-Image.mdx
+└── ...
+```
+
+## 🤝 Contributing
+
+We believe the best documentation comes from practitioners. Whether you've optimized SGLang for a specific model, solved a tricky deployment challenge, or discovered performance improvements, we encourage you to contribute your recipes!
+
+**💪How to Contribute**
+
+- Comment below if interested (mention which role)
+- Join discussion on implementation details
+- Fork repo and work on assigned section
+- Submit PR following SGLang cookbook standards
+- Iterate based on review feedback
+
+**To contribute:**
+
+```shell Command
+# Fork the repo and clone locally
+git clone https://github.com/YOUR_USERNAME/sglang.git
+cd sglang
+
+# Create a new branch
+git checkout -b add-my-recipe
+
+# Add your recipe under docs/cookbook/diffusion/
+# Submit a PR!
+```
+
+## 📖 Resources
+
+- [SGLang GitHub](https://github.com/sgl-project/sglang)
+- [SGLang Documentation](/)
+- [SGLang Diffusion Documentation](/docs/sglang-diffusion/index)
+- [SLACK Channel](https://sgl-fru7574.slack.com/archives/C07GLLLESNR)
+- [Community Slack/Discord](https://discord.gg/MpEEuAeb)
+
+## 📄 License
+
+This project is licensed under the Apache License 2.0 - see the [LICENSE](https://github.com/sgl-project/sglang/blob/main/LICENSE) file for details.
+
+---
+
+**Let's build this resource together!** 🚀 Star the repo and contribute your recipes to help the SGLang community grow.
@@ -0,0 +1,502 @@
+---
+title: SANA-WM
+metatags:
+    description: "Deploy SANA-WM with SGLang - a camera-controlled text+image-to-video world model with WASD/IJKL 6-DoF camera control, served three ways: dense bidirectional and chunk-causal batch streaming over /v1/videos (SanaWMTwoStagePipeline), and live over a realtime WebSocket API (SanaWMRealtimePipeline, /v1/realtime_video/generate)."
+---
+
+import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';
+
+<DiffusionModelTags tags={["video", "realtime", "world model", "camera control", "two-stage"]} />
+
+## 1. Model Introduction
+
+[SANA-WM](https://huggingface.co/Efficient-Large-Model/SANA-WM_bidirectional) is an efficient open-source **world model** from NVLabs, trained natively for one-minute video generation. It is a **2.6B-parameter text+image-to-video (TI2V) diffusion transformer** that synthesizes **720p, minute-scale videos with precise 6-DoF camera control**, paired with an **LTX-2 refiner** for high-fidelity decoding. It builds on the [SANA](https://github.com/NVlabs/Sana) family — efficient high-resolution synthesis with a linear diffusion transformer.
+
+SANA-WM ships in two checkpoints: a **bidirectional** checkpoint (dense, one-shot) and a **streaming** checkpoint (chunk-causal, autoregressive — generated chunk-by-chunk, reusing causal DiT state across chunks for bounded memory → long, even endless, clips). From a single first frame, a text prompt, and a camera trajectory, this cookbook covers **all three serving modes** SGLang exposes:
+
+- **(A) Dense bidirectional** (§4) — the `SANA-WM_bidirectional` checkpoint generated in one shot (no chunking) via **`SanaWMTwoStagePipeline`** over the standard **`/v1/videos`** HTTP API. Highest single-clip quality (full bidirectional attention + dense LTX-2 refiner); matches the NVlabs dense reference.
+- **(B) Batch streaming** (§5) — the `SANA-WM_streaming` checkpoint generated chunk-by-chunk in one request via the same **`SanaWMTwoStagePipeline`** + `--streaming` over **`/v1/videos`**. This is SGLang's offline chunk-causal streaming path: the whole clip is produced chunk-by-chunk internally, then returned.
+- **(C) Live realtime** (§6–7) — the streaming pipeline exposed as **`SanaWMRealtimePipeline`** over a **WebSocket API** at `/v1/realtime_video/generate`, so a browser/client streams camera-action events frame-by-frame and receives video chunks back in real time. Realtime uses the same streaming checkpoint, but the incremental session path is not bit-identical to offline batch streaming.
+
+All three modes share the camera action DSL (§8) and the configuration knobs (§9). Modes (B) and (C) share the streaming checkpoint and the chunk-causal pipeline.
+
+**Key features** (per the official model):
+
+- **Hybrid Linear Attention** — frame-wise Gated DeltaNet (GDN) recurrent blocks combined with softmax attention (every 4th layer, block indices {3,7,11,15,19}) for memory-efficient long-context modeling.
+- **Dual-Branch Camera Control** — independent main and camera branches (UCPE + PRoPE) for precise per-frame 6-DoF trajectory adherence.
+- **Two-Stage Pipeline** — an LTX-2 long-video refiner on top of Stage-1 latents for quality and temporal consistency.
+
+In the **streaming / realtime** configuration this becomes a low-latency, interactive pipeline:
+
+- **Stage-1 chunk-causal DiT** — the streaming path carries a **per-block KV cache** (recurrent GDN state + a softmax K/V window) across chunks; bounded memory means it scales to long / endless sequences. Stage-1 is intentionally coarse.
+- **LTX-2 streaming refiner** — refines each Stage-1 latent chunk block-by-block with a **sink + sliding-history KV cache** (required for sharp output).
+- **Causal LTX-2 VAE** — decodes latents chunk-by-chunk with a carried conv-cache for seam-free frames.
+- **Camera control** — drive the camera with a compact **WASD/IJKL** action DSL (move with WASD, look with IJKL; see §8) — supplied at request time on the `/v1/videos` paths, or pushed over the WebSocket at init / as live per-chunk events on the realtime path (see §7).
+
+**Architecture & components**
+
+| Component | Value |
+|---|---|
+| Stage-1 DiT | 2.6B; 20 layers, hidden 2240, 20 heads (head_dim 112); ~10 GB |
+| Attention | frame-wise Gated DeltaNet + softmax every 4th block (hybrid linear) |
+| Camera | dual-branch, UCPE + PRoPE (raymap + Plücker), 6-DoF |
+| VAE | LTX-2 causal, strides (T, H, W) = (8, 32, 32); ~2 GB |
+| Refiner | LTX-2 Stage-2 distilled; ~41 GB |
+| Output | up to 720p (704×1280) @ 16 fps, minute-scale |
+
+For more details, see the [SANA-WM paper (arXiv)](https://arxiv.org/abs/2605.15178), the [SANA project page](https://nvlabs.github.io/Sana/), the [NVlabs/Sana GitHub](https://github.com/NVlabs/Sana), and the [SANA-WM_bidirectional model card](https://huggingface.co/Efficient-Large-Model/SANA-WM_bidirectional) (Apache-2.0).
+
+## 2. Installation
+
+SGLang-diffusion offers multiple installation methods depending on your hardware platform. Please refer to the [SGLang Diffusion installation guide](../../../docs/sglang-diffusion/installation).
+
+SANA-WM adds the `SanaWMTransformer3DModel` + GDN kernels, the `SanaWMTwoStagePipeline` (dense bidirectional + chunk-causal streaming), and the `SanaWMRealtimePipeline` with the `/v1/realtime_video` WebSocket router. Use `sglang serve` to launch the diffusion server.
+
+## 3. Model Setup
+
+Both SANA-WM checkpoints are **public** (Apache-2.0, no gating, no token) and load **directly** — there is no manual assembly step. Pass the HuggingFace repo id to `--model-path` and SGLang downloads, materializes, validates, and loads it:
+
+| Mode | `--model-path` |
+|---|---|
+| Dense bidirectional (§4) | `Efficient-Large-Model/SANA-WM_bidirectional` |
+| Batch streaming (§5) / realtime (§6) | `Efficient-Large-Model/SANA-WM_streaming` |
+
+Both repo ids are registered in SGLang's **built-in model-overlay registry**, so on first load the overlay transparently materializes the official release into a runnable Diffusers directory — for the streaming checkpoint this converts the DMD self-forcing checkpoint (`sana_dit/model.pt`) into a Diffusers `transformer/` and wires the LTX-2 causal VAE, the LTX-2 refiner, and the Gemma encoders. No environment variable or `build_model_dir.sh` step is needed. (You may also pass a local, already-materialized Diffusers directory.)
+
+The materialized checkpoint is a Diffusers directory whose `model_index.json` declares the loadable components:
+
+| Component (`model_index.json`) | Class |
+|---|---|
+| `transformer` (Stage-1 DiT) | `diffusers.SanaWMTransformer3DModel` |
+| `vae` | `diffusers.AutoencoderKLCausalLTX2Video` |
+| `text_encoder` | `transformers.Gemma2Model` |
+| `tokenizer` | `transformers.GemmaTokenizer` |
+| `scheduler` | `diffusers.FlowMatchEulerDiscreteScheduler` |
+
+How loading works:
+
+- The server resolves the checkpoint via `maybe_download_model(model_path, force_diffusers_model=True)` and verifies it contains a `model_index.json` plus the required component subdirectories (`transformer/`, `vae/`).
+- If `text_encoder` / `tokenizer` are not provided as component paths, the pipeline falls back to the default Stage-1 text encoder **`Efficient-Large-Model/gemma-2-2b-it`** (`DEFAULT_SANA_WM_TEXT_ENCODER`).
+- **Pick the path with `--pipeline-class-name`.** The checkpoint's `model_index.json` `_class_name` selects the default pipeline (`SanaWMTwoStagePipeline`). Pin it explicitly to choose: `--pipeline-class-name SanaWMTwoStagePipeline` for the `/v1/videos` paths (§4–5) or `--pipeline-class-name SanaWMRealtimePipeline` for live realtime (§6). Pinning is also required if you point `--model-path` at a bare safetensors file instead of a Diffusers directory.
+- **The Stage-2 LTX-2 refiner** lives under `refiner/` in the checkpoint: `refiner/transformer` (`transformer_2`), `refiner/connectors` (`connectors`), and `refiner/text_encoder` (the Gemma-3 encoder for `text_encoder_2`, whose tokenizer also serves as `tokenizer_2`). The refiner is **optional**: it is skipped (Stage-1-only output) when the env flag `SGLANG_SANA_WM_SKIP_REFINER` (or a `skip_refiner` request extra) is set, or when no `refiner/` is present (`transformer_2` unloaded). On the batch path it runs chunk-wise with `--refiner-chunked` (the official streaming path, default on) or whole-clip without it; on the realtime path the pipeline builds a `SanaWMChunkedRefinerChainStage` only when a refiner is available, and otherwise streams Stage-1 frames.
+
+<Note>
+Throughout this cookbook, `<checkpoint>` stands for the appropriate SANA-WM repo id from the table above (or a local materialized Diffusers directory).
+</Note>
+
+## 4. Dense bidirectional (offline `/v1/videos`)
+
+The **bidirectional** checkpoint generates the whole clip in **one shot** (full bidirectional attention, not chunked) followed by a dense LTX-2 refiner — the highest single-clip quality, matching the NVlabs dense reference.
+
+Launch with the two-stage pipeline and **no** `--streaming` flag (dense is the default — `streaming` defaults to `False`):
+
+```bash Command
+sglang serve \
+  --model-path Efficient-Large-Model/SANA-WM_bidirectional \
+  --pipeline-class-name SanaWMTwoStagePipeline \
+  --host 127.0.0.1 --port 30000
+```
+
+Then POST to **`/v1/videos`** exactly as in §5, but pass the NVlabs dense sampling defaults for closest parity — the dense path is denser than the distilled streaming few-step schedule:
+
+```bash Command
+curl -s http://127.0.0.1:30000/v1/videos \
+  -H 'content-type: application/json' -d '{
+    "prompt": "a camera moving forward and turning left",
+    "input_reference": "/path/to/first_frame.png",
+    "num_frames": 321,
+    "seed": 42,
+    "fps": 16,
+    "num_inference_steps": 60,
+    "guidance_scale": 5.0,
+    "diffusers_kwargs": {
+      "action": "w-80,wl-80,l-80,wj-80",
+      "intrinsics": "/path/to/intrinsics.npy"
+    }
+  }'
+```
+
+- `num_inference_steps` / `guidance_scale` — the dense path uses CFG; NVlabs' reference defaults to **60 steps, guidance 5.0** (the `SanaWMSamplingParams` defaults are the lighter 20 / 4.5 — pass 60 / 5.0 explicitly for dense parity).
+- The dense refiner drops the leading sink frame, so a `num_frames=321` request yields 320 output frames.
+
+## 5. Batch streaming (offline `/v1/videos`)
+
+The **streaming** checkpoint generates a **full camera-controlled clip in one request** — no websocket. This is SGLang's offline streaming path: the whole clip is generated chunk-by-chunk internally, refined, decoded, and returned as one video.
+
+Launch with the two-stage pipeline + the streaming flags:
+
+```bash Command
+sglang serve \
+  --model-path Efficient-Large-Model/SANA-WM_streaming \
+  --pipeline-class-name SanaWMTwoStagePipeline \
+  --streaming --refiner-chunked \
+  --host 127.0.0.1 --port 30000
+```
+
+- `--streaming` — chunk-causal `forward_long` Stage-1 (vs the dense one-shot path of §4).
+- `--refiner-chunked` — chunk-wise streaming LTX-2 refiner (**on by default**). To use the whole-clip dense refiner instead (also valid, higher peak memory), pass `--refiner-chunked false` — simply omitting the flag keeps the default chunked refiner.
+- `--num-frame-per-block N` — latent frames per chunk (default `3`).
+
+Then POST to **`/v1/videos`** (JSON body shown below; multipart/form-data with an uploaded `input_reference` file also works). Camera control goes in `diffusers_kwargs` — the action-DSL string (§8) and the intrinsics:
+
+```bash Command
+curl -s http://127.0.0.1:30000/v1/videos \
+  -H 'content-type: application/json' -d '{
+    "prompt": "a camera moving forward and turning left",
+    "input_reference": "/path/to/first_frame.png",
+    "num_frames": 321,
+    "seed": 42,
+    "fps": 16,
+    "diffusers_kwargs": {
+      "action": "w-80,wl-80,l-80,wj-80",
+      "intrinsics": "/path/to/intrinsics.npy"
+    }
+  }'
+```
+
+| Field | Notes |
+|---|---|
+| `prompt` | text prompt |
+| `input_reference` | first-frame image — a server-side path, or (multipart) an uploaded file. For an `http(s)://` URL in a JSON body, use the separate `reference_url` field (the server downloads it and assigns it to `input_reference`) |
+| `num_frames` | total pixel frames (e.g. `321` → 41 latent frames, 13 chunks; output 704×1280) |
+| `seed` | RNG seed (default `42`) |
+| `fps` | output frame rate — **pass `16`** (SANA-WM's native rate). The generic `/v1/videos` default is `24`, which would encode the same frames at 24 fps and make the clip play ~33% shorter (16/24 of the duration) |
+| `diffusers_kwargs.action` | camera action-DSL string (§8) |
+| `diffusers_kwargs.intrinsics` | path to a camera-intrinsics `.npy` (per-frame `(T,3,3)`) or an inline 3×3 / `(T,3,3)` list |
+
+The response is a `VideoResponse`; fetch the rendered MP4 via the returned reference or `GET /v1/videos/{id}/content`. The streaming hyperparameters (`num_frame_per_block`, `denoising_step_list`, `sink_size`, `num_cached_blocks`, `streaming_cfg_scale`) are **pipeline-config** defaults on `SanaWMPipelineConfig`, not request fields — see §9.
+
+## 6. Launch the Realtime Server
+
+Launch with the realtime pipeline **pinned** — the checkpoint defaults to `SanaWMTwoStagePipeline`, so realtime must be selected explicitly (see §3). The `/v1/realtime_video` router is always mounted and becomes functional once the realtime config is active, because `SanaWMRealtimeConfig` has a registered realtime adapter (`SanaWMRealtimeAdapter`).
+
+```bash Command
+sglang serve \
+  --model-path Efficient-Large-Model/SANA-WM_streaming \
+  --pipeline-class-name SanaWMRealtimePipeline \
+  --host 127.0.0.1 --port 30000
+```
+
+Common launch variants:
+
+```bash Command
+# recommended multi-GPU realtime profile
+sglang serve \
+  --model-path Efficient-Large-Model/SANA-WM_streaming \
+  --pipeline-class-name SanaWMRealtimePipeline \
+  --num-gpus 8 --sp-degree 8 \
+  --host 127.0.0.1 --port 30000
+
+# single GPU
+sglang serve \
+  --model-path Efficient-Large-Model/SANA-WM_streaming \
+  --pipeline-class-name SanaWMRealtimePipeline \
+  --num-gpus 1 --host 127.0.0.1 --port 30000
+
+# offload DiT + text encoder to CPU (tight VRAM)
+sglang serve \
+  --model-path Efficient-Large-Model/SANA-WM_streaming \
+  --pipeline-class-name SanaWMRealtimePipeline \
+  --host 127.0.0.1 --port 30000 \
+  --dit-cpu-offload --text-encoder-cpu-offload
+```
+
+Notes on launch behavior:
+
+- **Default endpoint** is `127.0.0.1:30000` (`--host` / `--port` override).
+- **CPU offload flags are optional.** `--dit-cpu-offload`, `--text-encoder-cpu-offload`, and `--image-encoder-cpu-offload` are available; defaults are auto-adjusted from GPU memory (GPUs under 30 GB get more aggressive offloading).
+- **Multi-GPU realtime.** Prefer explicit sequence parallelism (`--sp-degree` equal to the number of GPUs for a single session). Do not enable CFG parallel for the realtime profile: the default realtime request uses `guidance_scale=1.0`, while CFG parallel requires active cond/uncond branches.
+- **FSDP.** Use `--use-fsdp-inference` only when you specifically need weight sharding for memory. For the low-latency realtime profile, prefer keeping components resident and using SP first.
+- **Warmup.** Server warmup is **automatically skipped** for the realtime pipeline — a synthetic warmup request has no WebSocket session, so the server detects the registered realtime adapter and skips it. No `--warmup` flag is needed.
+
+Once up, the realtime WebSocket endpoint lives at `ws://127.0.0.1:30000/v1/realtime_video/generate` (use the Python client in §7 to connect — plain `curl` does not speak the `ws://` upgrade).
+
+## 7. Realtime WebSocket API
+
+The realtime API is a single WebSocket at **`/v1/realtime_video/generate`**. All messages — client → server and server → client — are **msgpack** (`msgspec.msgpack.encode` / `decode`), not JSON.
+
+The lifecycle is:
+
+<Steps>
+<Step title="Connect & send INIT">
+The client opens the WebSocket and sends exactly one **init** message (`type: "init"`), carrying the prompt, the required `first_frame`, output/sampling options, and optional camera conditions in `condition_inputs`.
+</Step>
+<Step title="Stream live EVENTs (optional)">
+While generation runs, the client may push **event** messages (`type: "event"`) to steer the camera — either `kind: "camera_actions"` (frame-by-frame lists or state transitions) or `kind: "action"` (an action-DSL string).
+</Step>
+<Step title="Receive frame batches">
+The server streams **frame batches** back. Each chunk arrives as one or more `frame_batch` messages (header fields + payload bytes); `is_final_frame_batch: true` marks the end of a chunk. The server also emits `chunk_stats` timing messages.
+</Step>
+</Steps>
+
+### INIT message
+
+`RealtimeVideoGenerationsRequest` (`type` is the literal `"init"`). Key fields:
+
+| Field | Type | Notes |
+|---|---|---|
+| `type` | `"init"` | Required literal |
+| `prompt` | str | Text prompt |
+| `first_frame` | bytes \| str | **Required by the SANA-WM adapter** (`on_init` raises if absent), though the generic request schema defines it as optional. Raw image bytes, a server-side path, or an `http(s)://` URL (downloaded & cached) |
+| `condition_inputs` | dict | Camera/conditioning inputs (see below) |
+| `num_frames` | int | Total frames to generate. **Omit it for an open-ended, continuous session** — the adapter leaves `num_frames` unset and flags an open-ended run (`condition_inputs["sana_wm_open_ended"] = True`), generating uniform chunks indefinitely (until `max_chunks` or the client disconnects). Provide an integer for a fixed-length clip |
+| `seed` | int | RNG seed (default `42`) |
+| `size` | str | `"WIDTHxHEIGHT"`; realtime requests default to `"832x480"` for latency. Pass `"1280x704"` for the native landscape resolution |
+| `max_chunks` | int | Optional cap on total chunks generated |
+| `num_inference_steps` | int | Default `4` for SANA-WM (realtime adapter) |
+| `guidance_scale` | float | Default `1.0` |
+| `realtime_output_format` | `"raw"` \| `"webp"` \| `"jpeg"` | Frame encoding for output (see below) |
+| `realtime_causal_sink_size` | int | Optional override |
+| `realtime_causal_kv_cache_num_frames` | int | Optional override |
+
+`condition_inputs` accepts (all optional; pass **only one** of `action` / `camera_actions`):
+
+| Key | Type | Meaning |
+|---|---|---|
+| `camera_actions` | `list[list[str]]` or `{mode: "state", transitions: [...]}` | Frame-by-frame camera actions, or state-based transitions |
+| `action` | str | Action-DSL string, e.g. `"w-10,none-5,a-8"` (see §8) |
+| `intrinsics_path` | str | Server-side path to a camera-intrinsics **`.npy`** file (loaded via `np.load`; shapes `(4,)`, `(3,3)`, or `(F,3,3)`) |
+| `intrinsics` | list | Inline intrinsics with shape `(4,)`, `(3,3)`, `(F,4)`, or `(F,3,3)` |
+
+If you omit both `intrinsics_path` and `intrinsics`, SGLang uses a centered heuristic intrinsic matrix derived from the first-frame size. Pass explicit intrinsics when you need closer camera parity with a prepared trajectory.
+
+```json INIT (msgpack dict) — open-ended (omit num_frames)
+{
+  "type": "init",
+  "prompt": "beautiful landscape video",
+  "first_frame": "<bytes or url>",
+  "size": "832x480",
+  "seed": 42,
+  "max_chunks": 10,
+  "realtime_output_format": "raw",
+  "num_inference_steps": 4,
+  "guidance_scale": 1.0,
+  "condition_inputs": {
+    "camera_actions": [["w"], [], ["a", "s"]],
+    "intrinsics_path": "/path/to/intrinsics.npy"
+  }
+}
+```
+
+### Live EVENT messages
+
+`RealtimeEvent` (`type: "event"`). Use `kind` + `payload` (optional `event_id` correlates the response back to this event).
+
+```json EVENT - camera_actions (frame-by-frame list[list[str]])
+{
+  "type": "event",
+  "kind": "camera_actions",
+  "event_id": 1,
+  "payload": [["w"], ["w"], ["a"], []]
+}
+```
+
+```json EVENT - camera_actions (state-based transitions)
+{
+  "type": "event",
+  "kind": "camera_actions",
+  "event_id": 2,
+  "payload": {
+    "mode": "state",
+    "transitions": [
+      {"actions": ["w"], "client_ts_ms": 1000},
+      {"actions": ["a", "w"], "client_ts_ms": 1500}
+    ]
+  }
+}
+```
+
+```json EVENT - action (DSL string)
+{
+  "type": "event",
+  "kind": "action",
+  "event_id": 3,
+  "payload": "w-10,none-5,a-8,d-10"
+}
+```
+
+### Server frame output
+
+The server streams **frame batches**. Every batch arrives as a **single** msgpack message with `type: "frame_batch"` — the header fields below plus an inline `payload` bytes field (the wire `type` is always `"frame_batch"`; there is no separate header-then-bytes message).
+
+Header fields:
+
+| Field | Meaning |
+|---|---|
+| `type` | `"frame_batch"` (always) |
+| `request_id` | Generation id |
+| `chunk_index` | Chunk index |
+| `content_type` | `application/x-raw-rgb`, `application/x-raw-rgb-delta-gzip`, `image/webp`, or `image/jpeg` |
+| `num_frames` | Frames in this batch |
+| `total_size` | Payload size in bytes (`len(payload)` — the compressed size for delta-gzip) |
+| `width`, `height`, `channels` | Frame geometry (`channels: 3`) |
+| `bytes_per_frame` | Bytes per uncompressed frame (`width*height*3`) |
+| `format` | `rgb24` for raw |
+| `encoding` | `raw`, `delta-gzip`, `webp`, or `jpeg` |
+| `delta_reference` | `previous-frame` (present for delta-gzip) |
+| `event_id` | Echoes the steering event id; **omitted** from the header for INIT-only chunks |
+| `frame_batch_index`, `num_frame_batches` | Sequence multiple batches within a chunk |
+| `is_final_frame_batch` | `true` ends the chunk |
+
+```json Server output - frame_batch (msgpack dict)
+{
+  "type": "frame_batch",
+  "request_id": "uuid-string",
+  "chunk_index": 0,
+  "content_type": "application/x-raw-rgb-delta-gzip",
+  "num_frames": 3,
+  "total_size": 1048576,
+  "width": 1280,
+  "height": 704,
+  "channels": 3,
+  "bytes_per_frame": 2703360,
+  "format": "rgb24",
+  "encoding": "delta-gzip",
+  "delta_reference": "previous-frame",
+  "event_id": 1,
+  "frame_batch_index": 0,
+  "num_frame_batches": 1,
+  "is_final_frame_batch": true,
+  "payload": "<gzip-compressed bytes>"
+}
+```
+
+**Encodings.** `application/x-raw-rgb` is uncompressed RGB24 (3 × uint8, `bytes_per_frame = width*height*3`). `application/x-raw-rgb-delta-gzip` is the zlib-compressed **per-frame XOR delta** against the preceding frame (each frame in the batch is XOR'd against the previous one; sent by default). `realtime_output_format: "raw"` forces uncompressed RGB; `"webp"` / `"jpeg"` send preview-encoded frames.
+
+<Note>
+delta-gzip must be restored **frame-by-frame**: decompress the payload, then for each frame XOR it against the already-restored previous frame (the first frame of a batch references the last frame of the previous batch). See `restore_delta_gzip_raw_rgb_payload` in `runtime/utils/realtime_video.py`. The `"raw"` format below avoids this.
+</Note>
+
+### Minimal client example
+
+```python Python
+import msgspec
+import numpy as np
+import websockets  # pip install websockets
+
+WS_URL = "ws://127.0.0.1:30000/v1/realtime_video/generate"
+
+async def run():
+    async with websockets.connect(WS_URL, max_size=None) as ws:
+        # 1) INIT — omit num_frames for an open-ended session; "raw" = uncompressed RGB24
+        with open("first_frame.png", "rb") as f:
+            first_frame = f.read()
+        await ws.send(msgspec.msgpack.encode({
+            "type": "init",
+            "prompt": "a camera moving forward and turning right",
+            "first_frame": first_frame,
+            "size": "832x480",
+            "seed": 42,
+            "max_chunks": 10,
+            "realtime_output_format": "raw",
+            "num_inference_steps": 4,
+            "guidance_scale": 1.0,
+            "condition_inputs": {
+                "action": "w-100,wd-50,d-30",
+                "intrinsics_path": "/path/to/intrinsics.npy",  # optional; centered heuristic if omitted
+            },
+        }))
+
+        # 2) optional: steer mid-stream
+        await ws.send(msgspec.msgpack.encode({
+            "type": "event",
+            "kind": "camera_actions",
+            "event_id": 1,
+            "payload": [["w"], ["w"], ["a"], []],
+        }))
+
+        # 3) receive frame batches (raw RGB24)
+        async for message in ws:
+            msg = msgspec.msgpack.decode(message)
+            if msg.get("type") != "frame_batch":
+                continue  # skip chunk_stats etc.
+            n, h, w, c = msg["num_frames"], msg["height"], msg["width"], msg["channels"]
+            frames = np.frombuffer(msg["payload"], dtype=np.uint8).reshape(n, h, w, c)
+            # ... display/save frames ...
+            if msg.get("is_final_frame_batch") and msg.get("chunk_index", 0) >= 9:
+                break
+
+# asyncio.run(run())
+```
+
+## 8. Camera Action DSL
+
+Camera trajectories are described by a compact string of comma-separated `<keys>-<frames>` segments, e.g. `"w-100,wd-50,d-30,none-10"`. This is the format accepted by `condition_inputs.action` at init and by `kind: "action"` events.
+
+Parsing rules (`parse_action_string`):
+
+- Each segment is `<keys>-<frames>`; `<frames>` must be a positive integer.
+- `none` means no motion for that span: `none-10` = 10 static frames.
+- Keys are case-insensitive; combined keys apply simultaneously (`wd` = forward + right strafe). Allowed keys are exactly `wasdijkl`.
+
+| Key | Motion |
+|---|---|
+| `w` / `s` | move forward / backward |
+| `a` / `d` | strafe left / right |
+| `i` / `k` | look (pitch) up / down |
+| `j` / `l` | look (yaw) left / right |
+
+Pose generation (`action_string_to_c2w`):
+
+- **Translation** (`w`/`s`/`a`/`d`) moves at `translation_speed` (default `0.04` world-units/frame).
+- **Rotation** (`i`/`k` pitch, `j`/`l` yaw) turns at `rotation_speed_deg` (default `1.2`°/frame); pitch is clamped to ±85°.
+- **Strafe-yaw coupling** (coefficient `0.4`): a `d` (right) strafe also nudges yaw right and `a` (left) nudges yaw left, so `wd` traces a curving arc rather than a pure sidestep.
+- Produces `(F+1, 4, 4)` camera-to-world matrices; the realtime stage pads the trajectory to the requested frame count.
+
+Example: `"w-100,wd-50,d-30,none-10"` = 100 frames forward → 50 frames forward + sweep right → 30 frames right strafe → 10 frames static.
+
+## 9. Configuration Reference
+
+SANA-WM's defaults live in three places: **request-time** sampling params, the **pipeline config** (streaming/refiner knobs), and the **realtime adapter** (init-time overrides).
+
+### Request-time — `SanaWMSamplingParams` (`configs/sample/sana_wm.py`)
+
+| Field | Default | Purpose |
+|---|---|---|
+| `height` | `704` | Output height |
+| `width` | `1280` | Output width |
+| `num_frames` | `49` | Total pixel frames (must satisfy `(num_frames - 1) % 8 == 0`) |
+| `fps` | `16` | Output frame rate (overrides the base default of 24) |
+| `num_inference_steps` | `20` | Stage-1 step count |
+| `guidance_scale` | `4.5` | Dense-path CFG scale |
+| `negative_prompt` | `""` | Negative prompt |
+| `camera_to_world` | `None` | In-memory `(T,4,4)` c2w extrinsics (mutually exclusive with `action`) |
+| `intrinsics` | `None` | In-memory `(T,3,3)` pinhole intrinsics |
+| `action` | `None` | Action-DSL string (see §8) |
+| `translation_speed` | `0.04` | World-units/frame for W/S/A/D |
+| `rotation_speed_deg` | `1.2` | Degrees/frame for I/K/J/L |
+| `pitch_limit_deg` | `85.0` | Pitch clamp |
+
+`generator_device` is inherited from the base `SamplingParams` (default `None` = use the pipeline/model default). On the `/v1/videos` HTTP API the camera fields are passed inside `diffusers_kwargs` (`action` / `intrinsics`, as in §4–5).
+
+### Pipeline config — `SanaWMPipelineConfig` (`configs/pipeline_configs/sana_wm.py`)
+
+These are server-launch knobs (set via the `--streaming` / `--refiner-chunked` / `--num-frame-per-block` CLI flags or a pipeline-config override), **not** request fields:
+
+| Field | Default | Purpose |
+|---|---|---|
+| `streaming` | `False` | Chunk-causal `forward_long` (§5) vs dense one-shot (§4) |
+| `refiner_chunked` | `True` | Chunk-wise streaming refiner vs whole-clip dense refiner |
+| `num_frame_per_block` | `3` | Latent frames per Stage-1 / refiner chunk |
+| `num_cached_blocks` | `2` | Rolling KV-cache history window |
+| `denoising_step_list` | `(1000, 960, 889, 727, 0)` | 4-step streaming self-forcing timesteps (must end in 0) |
+| `streaming_cfg_scale` | `1.0` | CFG scale for the distilled streaming path (1.0 = off) |
+| `sink_size` | `1` | Sink (unrefined context) frames |
+| `refiner_block_size` | `3` | Refiner block size |
+| `refiner_kv_max_frames` | `11` | Refiner sliding KV window |
+
+### Realtime adapter init overrides — `SanaWMRealtimeAdapter`
+
+At WebSocket `init` the realtime adapter fills SANA-WM defaults that differ from the request/sampling defaults above:
+
+| Field | Realtime default | Note |
+|---|---|---|
+| `size` | `832x480` | Realtime request default; pass `1280x704` for native landscape output |
+| `num_frames` | *(unset)* | Omitting → open-ended continuous session (§7) |
+| `num_inference_steps` | `4` | Distilled few-step |
+| `guidance_scale` | `1.0` | CFG off |
+| `fps` | `16` | Native rate |
+
+<Note>
+`guidance_scale` applies to the dense path (§4) only; the distilled streaming path uses `streaming_cfg_scale` (default `1.0`, i.e. no CFG) so a `guidance_scale` override never accidentally enables CFG on the streaming stage. `denoising_step_list = (1000, 960, 889, 727, 0)` is the official 4-step streaming schedule (it must end in 0).
+</Note>
@@ -0,0 +1,394 @@
+---
+title: Wan2.1
+metatags:
+    description: "Deploy Wan2.1 video generation models with SGLang - community contribution guide for Wan Video's diffusion models."
+---
+
+import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';
+import { Wan21Deployment } from '/src/snippets/diffusion/wan21-deployment.jsx';
+
+<DiffusionModelTags tags={["video", "text-to-video", "image-to-video", "LoRA", "text rendering"]} />
+
+## 1. Model Introduction
+
+[Wan2.1 series](https://github.com/Wan-Video/Wan2.1) is an open and advanced suite of large-scale video generative models from Wan-AI.
+
+Key characteristics:
+
+- **State-of-the-art video quality**: Consistently outperforms many open-source and commercial video models on internal and public benchmarks, especially for motion richness and temporal consistency.
+- **Consumer GPU friendly**: The T2V-1.3B variant can generate 5-second 480P videos on consumer GPUs with modest VRAM requirements.
+- **Multi-capability suite**: Supports Text-to-Video (T2V), Image-to-Video (I2V), video editing, text-to-image, and video-to-audio generation.
+- **Robust text rendering**: First-generation Wan model capable of generating both Chinese and English text in videos with strong readability.
+- **Powerful Wan-VAE**: A 3D causal VAE that encodes/decodes long 1080P videos while preserving temporal information, enabling efficient high-resolution video generation.
+
+For more details, refer to the official Wan2.1 resources:
+
+- **GitHub**: [Wan-Video/Wan2.1](https://github.com/Wan-Video/Wan2.1)
+- **Hugging Face collection**: [Wan-AI Wan2.1](https://huggingface.co/Wan-AI/Wan2.1-T2V-14B)
+
+## 2. SGLang-diffusion Installation
+
+SGLang-diffusion offers multiple installation methods. You can choose the most suitable installation method based on your hardware platform and requirements.
+
+Please refer to the [official SGLang-diffusion installation guide](../../../docs/sglang-diffusion/installation) for installation instructions.
+
+## 3. Model Deployment
+
+This section provides deployment configurations optimized for different hardware platforms and use cases.
+
+### 3.1 Basic Configuration
+
+The Wan2.1 series offers models in multiple sizes and resolutions. SGLang supports Wan2.1 deployment on NVIDIA B200, B300, H200, H100, and AMD MI300X, MI325X, MI355X GPUs and Ascend A2, A3 NPUs. The recommended launch configurations vary by hardware, model size, and memory headroom.
+
+**Interactive Command Generator**: Use the configuration selector below to automatically generate an appropriate deployment command for your model variant and options.
+
+<Wan21Deployment />
+
+### 3.2 Configuration Tips
+
+Current supported optimization options are listed in the [SGLang diffusion support matrix](../../../docs/sglang-diffusion/attention_backends#platform-support-matrix).
+
+- `--vae-path`: Path to a custom VAE model or HuggingFace model ID. If not specified, the VAE will be loaded from the main model path.
+- `--num-gpus {NUM_GPUS}`: Number of GPUs to use.
+- `--tp-size {TP_SIZE}`: Tensor parallelism size (for the encoder/DiT; keep \(\leq 1\) if relying heavily on CPU offload).
+- `--sp-degree {SP_SIZE}`: Sequence parallelism degree.
+- `--ulysses-degree {ULYSSES_DEGREE}`: Degree of DeepSpeed-Ulysses-style SP in USP.
+- `--ring-degree {RING_DEGREE}`: Degree of ring attention-style SP in USP.
+- `--text-encoder-cpu-offload`, `--dit-cpu-offload`, `--vae-cpu-offload`: Use CPU offload to reduce peak GPU memory when needed.
+
+## 4. Model Invocation
+
+### 4.1 Basic Usage
+
+For more API usage and request examples, please refer to:
+[SGLang Diffusion OpenAI API](../../../docs/sglang-diffusion/api/openai_api)
+
+#### 4.1.1 Launch a server and then send requests
+
+```bash Command
+sglang serve --model-path Wan-AI/Wan2.1-T2V-14B-Diffusers --port 30000
+
+curl http://127.0.0.1:30000/v1/images/generations \
+  -o >(jq -r '.data[0].b64_json' | base64 --decode > example.png) \
+  -H "Content-Type: application/json" \
+  -H "Authorization: Bearer $OPENAI_API_KEY" \
+  -d '{
+    "model": "Wan-AI/Wan2.1-T2V-14B-Diffusers",
+    "prompt": "A cute baby sea otter",
+    "n": 1,
+    "size": "1024x1024",
+    "response_format": "b64_json"
+  }'
+```
+
+#### 4.1.2 Generate a video without launching a server
+
+```bash Command
+SERVER_ARGS=(
+  --model-path Wan-AI/Wan2.1-T2V-14B-Diffusers
+  --text-encoder-cpu-offload
+  --pin-cpu-memory
+  --num-gpus 4
+  --ulysses-degree=2
+  --enable-cfg-parallel
+)
+
+SAMPLING_ARGS=(
+  --prompt "A curious raccoon"
+  --save-output
+  --output-path outputs
+  --output-file-name "A curious raccoon.mp4"
+)
+
+sglang generate "${SERVER_ARGS[@]}" "${SAMPLING_ARGS[@]}"
+```
+
+### 4.2 Advanced Usage
+
+#### 4.2.1 Cache-DiT Acceleration
+
+SGLang integrates [Cache-DiT](https://github.com/vipshop/cache-dit), a caching acceleration engine for Diffusion Transformers (DiT), to achieve significant inference speedups with minimal quality loss. You can set `SGLANG_CACHE_DIT_ENABLED=True` to enable it. For more details, please refer to the SGLang Cache-DiT [documentation](../../../docs/sglang-diffusion/cache_dit).
+
+**Basic Usage**
+
+```bash Command
+SGLANG_CACHE_DIT_ENABLED=true sglang serve --model-path Wan-AI/Wan2.1-T2V-14B-Diffusers
+```
+
+**Advanced Usage**
+
+  Combined Configuration Example:
+  ```bash Command
+  SGLANG_CACHE_DIT_ENABLED=true \
+  SGLANG_CACHE_DIT_FN=2 \
+  SGLANG_CACHE_DIT_BN=1 \
+  SGLANG_CACHE_DIT_WARMUP=4 \
+  SGLANG_CACHE_DIT_RDT=0.4 \
+  SGLANG_CACHE_DIT_MC=4 \
+  SGLANG_CACHE_DIT_TAYLORSEER=true \
+  SGLANG_CACHE_DIT_TS_ORDER=2 \
+  sglang serve --model-path Wan-AI/Wan2.1-T2V-14B-Diffusers
+  ```
+
+#### 4.2.2 GPU Optimization
+
+- `--dit-cpu-offload`: Use CPU offload for DiT inference. Enable if you run out of memory with FSDP.
+- `--text-encoder-cpu-offload`: Use CPU offload for text encoder inference.
+- `--image-encoder-cpu-offload`: Use CPU offload for image encoder inference.
+- `--vae-cpu-offload`: Use CPU offload for VAE.
+- `--pin-cpu-memory`: Pin memory for CPU offload. Use as a workaround if you see "CUDA error: invalid argument".
+
+#### 4.2.3 Supported LoRA Registry
+
+SGLang supports applying Wan2.1 LoRA adapters on top of base models:
+
+<table style={{width: "100%", borderCollapse: "collapse", tableLayout: "fixed"}}>
+  <colgroup>
+    <col style={{width: "50%"}} />
+    <col style={{width: "50%"}} />
+  </colgroup>
+  <thead>
+    <tr style={{borderBottom: "2px solid #d55816"}}>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>origin model</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>supported LoRA</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>[Wan-AI/Wan2.1-T2V-14B](https://huggingface.co/Wan-AI/Wan2.1-T2V-14B)</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>[NIVEDAN/wan2.1-lora](https://huggingface.co/NIVEDAN/wan2.1-lora)</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>[Wan-AI/Wan2.1-I2V-14B-720P](https://huggingface.co/Wan-AI/Wan2.1-I2V-14B-720P)</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>[valiantcat/Wan2.1-Fight-LoRA](https://huggingface.co/valiantcat/Wan2.1-Fight-LoRA)</td>
+    </tr>
+  </tbody>
+</table>
+
+**Example**:
+
+```bash Command
+sglang serve --model-path Wan-AI/Wan2.1-T2V-14B-Diffusers --port 30000 \
+    --lora-path NIVEDAN/wan2.1-lora
+```
+
+## 5. Reference Benchmark
+
+The following benchmark is a point-in-time reference for one model, hardware platform, SGLang image, and parameter set. It is not a complete hardware support matrix.
+
+Test Environment:
+
+- Hardware: AMD MI300X GPU (1x)
+- Model: Wan-AI/Wan2.1-T2V-14B-Diffusers
+- SGLang Docker Image Version: 0.5.9
+
+### 5.1 How to Run Benchmarks with SGLang
+
+You can use the built-in SGLang diffusion benchmark script to evaluate Wan2.1 performance on your hardware.
+
+#### 5.1.1 Generate a single video
+
+<Tabs>
+  <Tab title="NVIDIA B200">
+    **Server Command**:
+
+    ```bash Command
+    sglang serve --model-path Wan-AI/Wan2.1-T2V-14B-Diffusers
+    ```
+
+    **Benchmark Command**:
+
+    ```bash Command
+    python3 -m sglang.multimodal_gen.benchmarks.bench_serving \
+        --backend sglang-video --dataset vbench --task text-to-video --num-prompts 1 --max-concurrency 1
+    ```
+
+    **Result**:
+
+    ```text Output
+    ================= Serving Benchmark Result =================
+    Task:                                    text-to-video
+    Model:                                   Wan-AI/Wan2.1-T2V-14B-Diffusers
+    Dataset:                                 vbench
+    --------------------------------------------------
+    Benchmark duration (s):                  1958.41
+    Request rate:                            inf
+    Max request concurrency:                 1
+    Successful requests:                     1/1
+    --------------------------------------------------
+    Request throughput (req/s):              0.00
+    Latency Mean (s):                        1958.4059
+    Latency Median (s):                      1958.4059
+    Latency P99 (s):                         1958.4059
+    --------------------------------------------------
+    Peak Memory Max (MB):                    59662.00
+    Peak Memory Mean (MB):                   59662.00
+    Peak Memory Median (MB):                 59662.00
+    ============================================================
+    ```
+  </Tab>
+
+  <Tab title="Ascend A3">
+    **Server Command**:
+
+    ```bash Command
+    #One A3 card has 2 npu chips. Benchmark was did with two A3 cards
+    sglang serve \
+      --model-path /models/Wan-AI/Wan2.1-T2V-14B-Diffusers/ \
+      --tp-size 2 \
+      --sp-degree 2 \
+      --num-gpus 4 \
+      --attention-backend laser_attn
+    ```
+
+    **Benchmark Command**:
+
+    ```bash Command
+    python -m sglang.multimodal_gen.benchmarks.bench_serving \
+      --dataset vbench \
+      --task text-to-video \
+      --num-prompts 1 \
+      --max-concurrency 1
+    ```
+
+    **Result**:
+
+    ```text Output
+    ================= Serving Benchmark Result =================
+    Task:                                         text-to-video
+    Model:                                        Wan-AI/Wan2.1-T2V-14B-Diffusers/
+    Dataset:                                      vbench
+    --------------------------------------------------
+    Benchmark duration (s):                       1282.90
+    Request rate:                                 inf
+    Max request concurrency:                      1
+    Successful requests:                          1/1
+    Completed outputs:                            1
+    Outputs per prompt:                           1
+    --------------------------------------------------
+    Request throughput (req/s):                   0.00
+    Output throughput (outputs/s):                0.00
+    Latency Mean (s):                             1282.90
+    Latency Median (s):                           1282.90
+    Latency P90 (s):                              1282.90
+    Latency P95 (s):                              1282.90
+    Latency P99 (s):                              1282.90
+    --------------------------------------------------
+    Peak Memory Max (MB):                         31938.00
+    Peak Memory Mean (MB):                        31938.00
+    Peak Memory Median (MB):                      31938.00
+    ============================================================
+    ```
+  </Tab>
+</Tabs>
+
+#### 5.1.2 Generate videos with Cache-DiT acceleration
+
+<Tabs>
+  <Tab title="NVIDIA B200">
+    **Server Command**:
+
+    ```bash Command
+    SGLANG_CACHE_DIT_ENABLED=true \
+    SGLANG_CACHE_DIT_FN=2 \
+    SGLANG_CACHE_DIT_BN=1 \
+    SGLANG_CACHE_DIT_WARMUP=4 \
+    SGLANG_CACHE_DIT_RDT=0.4 \
+    SGLANG_CACHE_DIT_MC=4 \
+    SGLANG_CACHE_DIT_TAYLORSEER=true \
+    SGLANG_CACHE_DIT_TS_ORDER=2 \
+    sglang serve --model-path Wan-AI/Wan2.1-T2V-14B-Diffusers
+    ```
+
+    **Benchmark Command**:
+
+    ```bash Command
+    python3 -m sglang.multimodal_gen.benchmarks.bench_serving \
+        --backend sglang-video --dataset vbench --task text-to-video --num-prompts 1 --max-concurrency 1
+    ```
+
+    **Result**:
+
+    ```text Output
+    ================= Serving Benchmark Result =================
+    Task:                                    text-to-video
+    Model:                                   Wan-AI/Wan2.1-T2V-14B-Diffusers
+    Dataset:                                 vbench
+    --------------------------------------------------
+    Benchmark duration (s):                  556.99
+    Request rate:                            inf
+    Max request concurrency:                 1
+    Successful requests:                     1/1
+    --------------------------------------------------
+    Request throughput (req/s):              0.00
+    Latency Mean (s):                        556.9885
+    Latency Median (s):                      556.9885
+    Latency P99 (s):                         556.9885
+    --------------------------------------------------
+    Peak Memory Max (MB):                    69306.00
+    Peak Memory Mean (MB):                   69306.00
+    Peak Memory Median (MB):                 69306.00
+    ============================================================
+    ```
+  </Tab>
+
+  <Tab title="Ascend A3">
+    **Server Command**:
+
+    ```bash Command
+    #One A3 card has 2 npu chips. Benchmark was did with two Atlas 3 cards
+    SGLANG_CACHE_DIT_FN=2 \
+    SGLANG_CACHE_DIT_BN=1 \
+    SGLANG_CACHE_DIT_WARMUP=4 \
+    SGLANG_CACHE_DIT_RDT=0.4 \
+    SGLANG_CACHE_DIT_MC=4 \
+    SGLANG_CACHE_DIT_TAYLORSEER=true \
+    SGLANG_CACHE_DIT_TS_ORDER=2 \
+    SGLANG_CACHE_DIT_ENABLED=true sglang serve \
+      --model-path /models/Wan-AI/Wan2.1-T2V-14B-Diffusers/ \
+      --tp-size 2 \
+      --sp-degree 2 \
+      --num-gpus 4 \
+      --attention-backend laser_attn
+    ```
+
+    **Benchmark Command**:
+
+    ```bash Command
+    python -m sglang.multimodal_gen.benchmarks.bench_serving \
+      --dataset vbench \
+      --task text-to-video \
+      --num-prompts 1 \
+      --max-concurrency 1
+    ```
+
+    **Result**:
+
+    ```text Output
+    ================= Serving Benchmark Result =================
+    Task:                                         text-to-video
+    Model:                                        Wan-AI/Wan2.1-T2V-14B-Diffusers/
+    Dataset:                                      vbench
+    --------------------------------------------------
+    Benchmark duration (s):                       413.88
+    Request rate:                                 inf
+    Max request concurrency:                      1
+    Successful requests:                          1/1
+    Completed outputs:                            1
+    Outputs per prompt:                           1
+    --------------------------------------------------
+    Request throughput (req/s):                   0.00
+    Output throughput (outputs/s):                0.00
+    Latency Mean (s):                             413.88
+    Latency Median (s):                           413.88
+    Latency P90 (s):                              413.88
+    Latency P95 (s):                              413.88
+    Latency P99 (s):                              413.88
+    --------------------------------------------------
+    Peak Memory Max (MB):                         32782.00
+    Peak Memory Mean (MB):                        32782.00
+    Peak Memory Median (MB):                      32782.00
+    ============================================================
+    ```
+  </Tab>
+</Tabs>
@@ -0,0 +1,464 @@
+---
+title: Wan2.2
+metatags:
+    description: "Deploy Wan2.2 video generation models with SGLang - MoE architecture, cinematic aesthetics, and efficient 720P@24fps generation."
+---
+
+import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';
+import { Wan22Deployment } from '/src/snippets/diffusion/wan22-deployment.jsx';
+
+<DiffusionModelTags tags={["video", "text-to-video", "image-to-video", "TI2V", "MoE"]} />
+
+## 1. Model Introduction
+
+[Wan2.2 series](https://github.com/Wan-Video/Wan2.2) are the most popular and open and advanced large-scale video generative models.
+
+This generation delivers comprehensive upgrades across the board:
+
+- **Effective MoE Architecture**: Introduces a Mixture-of-Experts (MoE) architecture into video diffusion models. By separating the denoising process cross timesteps with specialized powerful expert models, this enlarges the overall model capacity while maintaining the same computational cost.
+- **Cinematic-level Aesthetics**: Incorporates meticulously curated aesthetic data, complete with detailed labels for lighting, composition, contrast, color tone, and more. This allows for more precise and controllable cinematic style generation, facilitating the creation of videos with customizable aesthetic preferences.
+- **Complex Motion Generation**: Trained on a significantly larger data, with +65.6% more images and +83.2% more videos. This expansion notably enhances the model's generalization across multiple dimensions such as motions, semantics, and aesthetics, achieving TOP performance among all open-sourced and closed-sourced models.
+- **Efficient High-Definition Hybrid TI2V**: Open-sources a 5B model built with our advanced Wan2.2-VAE that achieves a compression ratio of 16×16×4. This model supports both text-to-video and image-to-video generation at 720P resolution with 24fps and can also run on consumer-grade graphics cards like 4090. It is one of the fastest 720P@24fps models currently available, capable of serving both the industrial and academic sectors simultaneously.
+
+For more details, please refer to the [official Wan2.2 GitHub Repository](https://github.com/Wan-Video/Wan2.2).
+
+## 2. SGLang-diffusion Installation
+
+SGLang-diffusion offers multiple installation methods. You can choose the most suitable installation method based on your hardware platform and requirements.
+
+Please refer to the [official SGLang-diffusion installation guide](https://docs.sglang.io/docs/sglang-diffusion/installation) for installation instructions.
+
+## 3. Model Deployment
+
+This section provides deployment configurations optimized for different hardware platforms and use cases.
+
+### 3.1 Basic Configuration
+
+The Wan2.2 series offers models in various sizes, architectures and input types, optimized for different hardware platforms. The recommended launch configurations vary by hardware and model size.
+
+**Interactive Command Generator**: Use the configuration selector below to automatically generate the appropriate deployment command for your hardware platform, model size. SGLang supports serving Wan2.2 on NVIDIA B200, H200, AMD MI300X, MI325X, MI355X GPUs and Ascend A2, A3 NPUs.
+
+<Wan22Deployment />
+
+### 3.2 Configuration Tips
+
+Currently supported optimizations are listed [here](/docs/sglang-diffusion/compatibility_matrix).
+
+- `--vae-path`: Path to a custom VAE model or HuggingFace model ID (e.g., fal/FLUX.2-Tiny-AutoEncoder). If not specified, the VAE will be loaded from the main model path.
+- `--num-gpus {NUM_GPUS}`: Number of GPUs to use
+- `--tp-size {TP_SIZE}`: Tensor parallelism size (only for the encoder; should not be larger than 1 if text encoder offload is enabled, as layer-wise offload plus prefetch is faster)
+- `--sp-degree {SP_SIZE}`: Sequence parallelism size (typically should match the number of GPUs)
+- `--ulysses-degree {ULYSSES_DEGREE}`: The degree of DeepSpeed-Ulysses-style SP in USP
+- `--ring-degree {RING_DEGREE}`: The degree of ring attention-style SP in USP
+
+## 4. Model Invocation
+
+### 4.1 Basic Usage
+
+For more API usage and request examples, please refer to:
+[SGLang Diffusion OpenAI API](/docs/sglang-diffusion/api/openai_api)
+
+#### 4.1.1 Launch a server and then send requests
+
+```shell Command
+sglang serve --model-path Wan-AI/Wan2.2-T2V-A14B-Diffusers --port 3000
+
+curl http://127.0.0.1:3000/v1/images/generations \
+  -o >(jq -r '.data[0].b64_json' | base64 --decode > example.png) \
+  -H "Content-Type: application/json" \
+  -H "Authorization: Bearer $OPENAI_API_KEY" \
+  -d '{
+    "model": "Wan-AI/Wan2.2-T2V-A14B-Diffusers",
+    "prompt": "A cute baby sea otter",
+    "n": 1,
+    "size": "1024x1024",
+    "response_format": "b64_json"
+  }'
+```
+
+#### 4.1.2 Generate a video without launching a server
+
+```shell Command
+SERVER_ARGS=(
+  --model-path Wan-AI/Wan2.2-T2V-A14B-Diffusers
+  --text-encoder-cpu-offload
+  --pin-cpu-memory
+  --num-gpus 4
+  --ulysses-degree=2
+  --enable-cfg-parallel
+)
+
+SAMPLING_ARGS=(
+  --prompt "A curious raccoon"
+  --save-output
+  --output-path outputs
+  --output-file-name "A curious raccoon.mp4"
+)
+
+sglang generate "${SERVER_ARGS[@]}" "${SAMPLING_ARGS[@]}"
+
+```
+
+### 4.2 Advanced Usage
+
+#### 4.2.1 Cache-DiT Acceleration
+
+SGLang integrates [Cache-DiT](https://github.com/vipshop/cache-dit), a caching acceleration engine for Diffusion Transformers (DiT), to achieve up to 7.4x inference speedup with minimal quality loss. You can set `SGLANG_CACHE_DIT_ENABLED=True` to enable it. For more details, please refer to the SGLang Cache-DiT [documentation](/docs/sglang-diffusion/cache_dit).
+
+**Basic Usage**
+
+```shell Command
+SGLANG_CACHE_DIT_ENABLED=true sglang serve --model-path Wan-AI/Wan2.2-T2V-A14B-Diffusers
+```
+
+**Advanced Usage**
+
+- DBCache Parameters: DBCache controls block-level caching behavior:
+
+<table style={{width: "100%", borderCollapse: "collapse", tableLayout: "fixed"}}>
+  <colgroup>
+    <col style={{width: "25.0%"}} />
+    <col style={{width: "25.0%"}} />
+    <col style={{width: "25.0%"}} />
+    <col style={{width: "25.0%"}} />
+  </colgroup>
+  <thead>
+    <tr style={{borderBottom: "2px solid #d55816"}}>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Parameter</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>Env Variable</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Default</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>Description</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Fn</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_FN`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>1</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Number of first blocks to always compute</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Bn</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_BN`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>0</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Number of last blocks to always compute</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>W</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_WARMUP`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>4</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Warmup steps before caching starts</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>R</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_RDT`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>0.24</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Residual difference threshold</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>MC</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_MC`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>3</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Maximum continuous cached steps</td>
+    </tr>
+  </tbody>
+</table>
+
+- TaylorSeer Configuration: TaylorSeer improves caching accuracy using Taylor expansion:
+
+<table style={{width: "100%", borderCollapse: "collapse", tableLayout: "fixed"}}>
+  <colgroup>
+    <col style={{width: "25.0%"}} />
+    <col style={{width: "25.0%"}} />
+    <col style={{width: "25.0%"}} />
+    <col style={{width: "25.0%"}} />
+  </colgroup>
+  <thead>
+    <tr style={{borderBottom: "2px solid #d55816"}}>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Parameter</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>Env Variable</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Default</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>Description</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Enable</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_TAYLORSEER`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>false</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Enable TaylorSeer calibrator</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Order</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_TS_ORDER`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>1</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Taylor expansion order (1 or 2)</td>
+    </tr>
+  </tbody>
+</table>
+
+  Combined Configuration Example:
+
+```shell Command
+SGLANG_CACHE_DIT_ENABLED=true \
+SGLANG_CACHE_DIT_FN=2 \
+SGLANG_CACHE_DIT_BN=1 \
+SGLANG_CACHE_DIT_WARMUP=4 \
+SGLANG_CACHE_DIT_RDT=0.4 \
+SGLANG_CACHE_DIT_MC=4 \
+SGLANG_CACHE_DIT_TAYLORSEER=true \
+SGLANG_CACHE_DIT_TS_ORDER=2 \
+sglang serve --model-path Wan-AI/Wan2.2-T2V-A14B-Diffusers
+```
+
+#### 4.2.2 GPU Optimization
+
+- `--dit-cpu-offload`: Use CPU offload for DiT inference. Enable if run out of memory with FSDP.
+- `--text-encoder-cpu-offload`: Use CPU offload for text encoder inference. Enable if run out of memory with FSDP.
+- `--image-encoder-cpu-offload`: Use CPU offload for image encoder inference. Enable if run out of memory with FSDP.
+- `--vae-cpu-offload`: Use CPU offload for VAE. Enable if run out of memory.
+- `--pin-cpu-memory`: Pin memory for CPU offload. Only added as a temp workaround if it throws "CUDA error: invalid argument".
+
+#### 4.2.3 Supported LoRA Registry
+<table style={{width: "100%", borderCollapse: "collapse", tableLayout: "fixed"}}>
+  <colgroup>
+    <col style={{width: "50%"}} />
+    <col style={{width: "50%"}} />
+  </colgroup>
+  <thead>
+    <tr style={{borderBottom: "2px solid #d55816"}}>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>origin model</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>supported LoRA</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>[Wan-AI/Wan2.2-I2V-A14B-Diffusers](https://huggingface.co/Wan-AI/Wan2.2-I2V-A14B-Diffusers)</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>[lightx2v/Wan2.2-Distill-Loras](https://huggingface.co/lightx2v/Wan2.2-Distill-Loras)</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>[Wan-AI/Wan2.2-T2V-A14B-Diffusers](https://huggingface.co/Wan-AI/Wan2.2-T2V-A14B-Diffusers)</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>[Cseti/wan2.2-14B-Arcane_Jinx-lora-v1](https://huggingface.co/Cseti/wan2.2-14B-Arcane_Jinx-lora-v1)</td>
+    </tr>
+  </tbody>
+</table>
+**Example**:
+```shell Command
+sglang serve --model-path Wan-AI/Wan2.2-T2V-A14B-Diffusers --port 3000 \
+    --lora-path Cseti/wan2.2-14B-Arcane_Jinx-lora-v1
+```
+
+## 5. Benchmark
+
+Test Environment:
+
+- Hardware: NVIDIA B200 GPU (1x)
+- Model: Wan-AI/Wan2.2-T2V-A14B-Diffusers
+- sglang diffusion version: 0.5.6.post2
+
+### 5.1 Speedup Benchmark
+
+### 5.1.1 Generate a video
+
+<Tabs>
+  <Tab title="NVIDIA B200">
+    **Server Command**:
+    ```shell Command
+    sglang serve --model-path Wan-AI/Wan2.2-T2V-A14B-Diffusers
+    ```
+
+    **Benchmark Command**:
+    ```shell Command
+    python3 -m sglang.multimodal_gen.benchmarks.bench_serving \
+        --backend sglang-video --dataset vbench --task t2v --num-prompts 1 --max-concurrency 1
+    ```
+
+    **Result**:
+    ```text Output
+    ================= Serving Benchmark Result =================
+    Backend:                                 sglang-video
+    Model:                                   Wan-AI/Wan2.2-T2V-A14B-Diffusers
+    Dataset:                                 vbench
+    Task:                                    t2v
+    --------------------------------------------------
+    Benchmark duration (s):                  630.43
+    Request rate:                            inf
+    Max request concurrency:                 1
+    Successful requests:                     1/1
+    --------------------------------------------------
+    Request throughput (req/s):              0.00
+    Latency Mean (s):                        630.4277
+    Latency Median (s):                      630.4277
+    Latency P99 (s):                         630.4277
+    --------------------------------------------------
+    Peak Memory Max (MB):                    62627.41
+    Peak Memory Mean (MB):                   62627.41
+    Peak Memory Median (MB):                 62627.41
+
+    ============================================================
+    ```
+  </Tab>
+
+  <Tab title="Ascend A3">
+    **Server Command**:
+    ```shell Command
+    #One A3 card has 2 npu chips. Using four A3 cards in benchmarking
+    sglang serve \
+      --model-path /models/Wan-AI/Wan2.2-T2V-A14B-Diffusers/ \
+      --tp-size 2 \
+      --sp-degree 4 \
+      --num-gpus 8 \
+      --attention-backend laser_attn
+    ```
+
+    **Benchmark Command**:
+    ```shell Command
+    python -m sglang.multimodal_gen.benchmarks.bench_serving \
+      --dataset vbench \
+      --task text-to-video \
+      --num-prompts 1 \
+      --max-concurrency 1
+    ```
+
+    **Result**:
+    ```text Output
+    ================= Serving Benchmark Result =================
+    Task:                                         text-to-video
+    Model:                                        Wan-AI/Wan2.2-T2V-A14B-Diffusers/
+    Dataset:                                      vbench
+    --------------------------------------------------
+    Benchmark duration (s):                       214.50
+    Request rate:                                 inf
+    Max request concurrency:                      1
+    Successful requests:                          1/1
+    Completed outputs:                            1
+    Outputs per prompt:                           1
+    --------------------------------------------------
+    Request throughput (req/s):                   0.00
+    Output throughput (outputs/s):                0.00
+    Latency Mean (s):                             214.50
+    Latency Median (s):                           214.50
+    Latency P90 (s):                              214.50
+    Latency P95 (s):                              214.50
+    Latency P99 (s):                              214.50
+    --------------------------------------------------
+    Peak Memory Max (MB):                         46692.00
+    Peak Memory Mean (MB):                        46692.00
+    Peak Memory Median (MB):                      46692.00
+    ------------------------------------------------------------
+    ```
+  </Tab>
+</Tabs>
+
+#### 5.1.2 Generate videos with high concurrency
+
+<Tabs>
+  <Tab title="NVIDIA B200">
+    **Server Command**:
+
+    ```shell Command
+    SGLANG_CACHE_DIT_ENABLED=true \
+    SGLANG_CACHE_DIT_FN=2 \
+    SGLANG_CACHE_DIT_BN=1 \
+    SGLANG_CACHE_DIT_WARMUP=4 \
+    SGLANG_CACHE_DIT_RDT=0.4 \
+    SGLANG_CACHE_DIT_MC=4 \
+    SGLANG_CACHE_DIT_TAYLORSEER=true \
+    SGLANG_CACHE_DIT_TS_ORDER=2 \
+    sglang serve --model-path Wan-AI/Wan2.2-T2V-A14B-Diffusers
+    ```
+
+    **Benchmark Command**:
+
+    ```shell Command
+    python3 -m sglang.multimodal_gen.benchmarks.bench_serving \
+        --backend sglang-video --dataset vbench --task t2v --num-prompts 20 --max-concurrency 20
+    ```
+
+    **Result**:
+
+    ```text Output
+    ================= Serving Benchmark Result =================
+    Backend:                                 sglang-video
+    Model:                                   Wan-AI/Wan2.2-T2V-A14B-Diffusers
+    Dataset:                                 vbench
+    Task:                                    t2v
+    --------------------------------------------------
+    Benchmark duration (s):                  5163.21
+    Request rate:                            inf
+    Max request concurrency:                 20
+    Successful requests:                     20/20
+    --------------------------------------------------
+    Request throughput (req/s):              0.00
+    Latency Mean (s):                        2739.7695
+    Latency Median (s):                      2742.0673
+    Latency P99 (s):                         5121.6331
+    --------------------------------------------------
+    Peak Memory Max (MB):                    72523.56
+    Peak Memory Mean (MB):                   70253.34
+    Peak Memory Median (MB):                 70824.46
+
+    ============================================================
+    ```
+  </Tab>
+
+  <Tab title="Ascend A3">
+    **Server Command**:
+
+    ```shell Command
+    #One A3 card has 2 npu chips. Using four A3 cards in benchmarking
+    SGLANG_CACHE_DIT_FN=2 \
+    SGLANG_CACHE_DIT_BN=1 \
+    SGLANG_CACHE_DIT_WARMUP=4 \
+    SGLANG_CACHE_DIT_RDT=0.4 \
+    SGLANG_CACHE_DIT_MC=4 \
+    SGLANG_CACHE_DIT_TAYLORSEER=true \
+    SGLANG_CACHE_DIT_TS_ORDER=2 \
+    SGLANG_CACHE_DIT_ENABLED=true sglang serve \
+      --model-path /models/Wan-AI/Wan2.2-T2V-A14B-Diffusers/ \
+      --tp-size 2 \
+      --sp-degree 4 \
+      --num-gpus 8 \
+      --attention-backend laser_attn
+    ```
+
+    **Benchmark Command**:
+
+    ```shell Command
+    python -m sglang.multimodal_gen.benchmarks.bench_serving \
+      --dataset vbench \
+      --task text-to-video \
+      --num-prompts 20 \
+      --max-concurrency 20
+    ```
+
+    **Result**:
+
+    ```text Output
+    ================= Serving Benchmark Result =================
+    Task:                                         text-to-video
+    Model:                                        Wan-AI/Wan2.2-T2V-A14B-Diffusers/
+    Dataset:                                      vbench
+    --------------------------------------------------
+    Benchmark duration (s):                       4384.65
+    Request rate:                                 inf
+    Max request concurrency:                      20
+    Successful requests:                          20/20
+    Completed outputs:                            20
+    Outputs per prompt:                           1
+    --------------------------------------------------
+    Request throughput (req/s):                   0.00
+    Output throughput (outputs/s):                0.00
+    Latency Mean (s):                             2304.17
+    Latency Median (s):                           2297.69
+    Latency P90 (s):                              3972.32
+    Latency P95 (s):                              4178.99
+    Latency P99 (s):                              4343.52
+    --------------------------------------------------
+    Peak Memory Max (MB):                         46692.00
+    Peak Memory Mean (MB):                        46691.90
+    Peak Memory Median (MB):                      46692.00
+    ------------------------------------------------------------
+    ```
+  </Tab>
+</Tabs>
@@ -0,0 +1,371 @@
+---
+title: Z-Image-Turbo
+metatags:
+    description: "Deploy Z-Image-Turbo with SGLang - community contribution guide for Z-Image's fast image generation model."
+---
+
+import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';
+import { ZImageTurboDeployment } from '/src/snippets/diffusion/zimage-turbo-deployment.jsx';
+
+<DiffusionModelTags tags={["image", "text-to-image", "turbo", "8-step"]} />
+
+## 1. Model Introduction
+
+[Z-Image](https://github.com/Tongyi-MAI/Z-Image) is a powerful and highly efficient image generation model family with 6B parameters, developed by Tongyi-MAI. It adopts a Scalable Single-Stream DiT (S3-DiT) architecture, where text, visual semantic tokens, and image VAE tokens are concatenated at the sequence level to serve as a unified input stream, maximizing parameter efficiency compared to dual-stream approaches.
+
+[Z-Image-Turbo](https://huggingface.co/Tongyi-MAI/Z-Image-Turbo) is a distilled version of Z-Image that matches or exceeds leading competitors with only 8 NFEs (Number of Function Evaluations). It is powered by two core techniques: **Decoupled-DMD** (few-step distillation) and **DMDR** (fusing DMD with Reinforcement Learning).
+
+**Key Features:**
+
+- **Sub-second Inference Latency**: Achieves sub-second inference on enterprise-grade H800 GPUs and fits comfortably within 16GB VRAM consumer devices
+- **Photorealistic Image Generation**: Excels in high-quality photorealistic image generation with rich aesthetics
+- **Bilingual Text Rendering**: Supports accurate bilingual text rendering in both English and Chinese
+- **Robust Instruction Adherence**: Strong prompt following and instruction adherence capabilities
+- **#1 Open-Source Model**: Ranked 8th overall and #1 among open-source models on the [Artificial Analysis Text-to-Image Leaderboard](https://artificialanalysis.ai/image/leaderboard/text-to-image)
+
+For more details, please refer to the [Z-Image-Turbo HuggingFace page](https://huggingface.co/Tongyi-MAI/Z-Image-Turbo), the [GitHub repository](https://github.com/Tongyi-MAI/Z-Image), and the [technical report (arXiv)](https://arxiv.org/abs/2511.22699).
+
+## 2. SGLang-diffusion Installation
+
+SGLang-diffusion offers multiple installation methods. You can choose the most suitable installation method based on your hardware platform and requirements.
+
+Please refer to the [official SGLang-diffusion installation guide](https://docs.sglang.io/docs/sglang-diffusion/installation) for installation instructions.
+
+## 3. Model Deployment
+
+This section provides deployment configurations optimized for different hardware platforms and use cases.
+
+### 3.1 Basic Configuration
+
+Z-Image-Turbo is optimized for high-quality image generation with only 8 inference steps. The recommended launch configurations vary by hardware.
+
+**Interactive Command Generator**: Use the configuration selector below to automatically generate the appropriate deployment command for your hardware platform.
+
+<ZImageTurboDeployment />
+
+### 3.2 Configuration Tips
+
+Currently supported optimizations are listed [here](/docs/sglang-diffusion/compatibility_matrix).
+
+- `--vae-path`: Path to a custom VAE model or HuggingFace model ID (e.g., fal/FLUX.2-Tiny-AutoEncoder). If not specified, the VAE will be loaded from the main model path.
+- `--num-gpus`: Number of GPUs to use
+- `--tp-size`: Tensor parallelism size (only for the encoder; should not be larger than 1 if text encoder offload is enabled, as layer-wise offload plus prefetch is faster)
+- `--sp-degree`: Sequence parallelism size (typically should match the number of GPUs)
+- `--ulysses-degree`: The degree of DeepSpeed-Ulysses-style SP in USP
+- `--ring-degree`: The degree of ring attention-style SP in USP
+
+**AMD ROCm Notes**: Requires SGLang >= v0.5.8.
+
+## 4. API Usage
+
+For complete API documentation, please refer to the [official API usage guide](/docs/sglang-diffusion/api/openai_api).
+
+### 4.1 Generate an Image
+
+```python Example
+import base64
+from openai import OpenAI
+
+client = OpenAI(api_key="EMPTY", base_url="http://localhost:30000/v1")
+
+response = client.images.generate(
+    model="Tongyi-MAI/Z-Image-Turbo",
+    prompt="A logo With Bold Large text: SGL Diffusion",
+    n=1,
+    response_format="b64_json",
+)
+
+# Save the generated image
+image_bytes = base64.b64decode(response.data[0].b64_json)
+with open("output.png", "wb") as f:
+    f.write(image_bytes)
+```
+
+### 4.2 Advanced Usage
+
+#### 4.2.1 Cache-DiT Acceleration
+
+SGLang integrates [Cache-DiT](https://github.com/vipshop/cache-dit), a caching acceleration engine for Diffusion Transformers (DiT), to achieve up to 7.4x inference speedup with minimal quality loss. You can set `SGLANG_CACHE_DIT_ENABLED=True` to enable it. For more details, please refer to the SGLang Cache-DiT [documentation](/docs/sglang-diffusion/cache_dit).
+
+**Basic Usage**
+
+```bash Command
+SGLANG_CACHE_DIT_ENABLED=true sglang serve --model-path Tongyi-MAI/Z-Image-Turbo
+```
+
+**Advanced Usage**
+
+- DBCache Parameters: DBCache controls block-level caching behavior:
+
+<table style={{width: "100%", borderCollapse: "collapse", tableLayout: "fixed"}}>
+  <colgroup>
+    <col style={{width: "25.0%"}} />
+    <col style={{width: "25.0%"}} />
+    <col style={{width: "25.0%"}} />
+    <col style={{width: "25.0%"}} />
+  </colgroup>
+  <thead>
+    <tr style={{borderBottom: "2px solid #d55816"}}>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Parameter</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>Env Variable</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Default</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>Description</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Fn</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_FN`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>1</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Number of first blocks to always compute</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Bn</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_BN`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>0</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Number of last blocks to always compute</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>W</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_WARMUP`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>4</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Warmup steps before caching starts</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>R</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_RDT`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>0.24</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Residual difference threshold</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>MC</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_MC`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>3</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Maximum continuous cached steps</td>
+    </tr>
+  </tbody>
+</table>
+- TaylorSeer Configuration: TaylorSeer improves caching accuracy using Taylor expansion:
+
+<table style={{width: "100%", borderCollapse: "collapse", tableLayout: "fixed"}}>
+  <colgroup>
+    <col style={{width: "25.0%"}} />
+    <col style={{width: "25.0%"}} />
+    <col style={{width: "25.0%"}} />
+    <col style={{width: "25.0%"}} />
+  </colgroup>
+  <thead>
+    <tr style={{borderBottom: "2px solid #d55816"}}>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Parameter</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>Env Variable</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Default</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>Description</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Enable</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_TAYLORSEER`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>false</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Enable TaylorSeer calibrator</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Order</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_TS_ORDER`</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>1</td>
+      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Taylor expansion order (1 or 2)</td>
+    </tr>
+  </tbody>
+</table>
+
+  Combined Configuration Example:
+
+```bash Command
+SGLANG_CACHE_DIT_ENABLED=true \
+SGLANG_CACHE_DIT_FN=2 \
+SGLANG_CACHE_DIT_BN=1 \
+SGLANG_CACHE_DIT_WARMUP=4 \
+SGLANG_CACHE_DIT_RDT=0.4 \
+SGLANG_CACHE_DIT_MC=4 \
+SGLANG_CACHE_DIT_TAYLORSEER=true \
+SGLANG_CACHE_DIT_TS_ORDER=2 \
+sglang serve --model-path Tongyi-MAI/Z-Image-Turbo
+```
+
+#### 4.2.2 CPU Offload
+
+- `--dit-cpu-offload`: Use CPU offload for DiT inference. Enable if run out of memory.
+- `--text-encoder-cpu-offload`: Use CPU offload for text encoder inference.
+- `--vae-cpu-offload`: Use CPU offload for VAE.
+- `--pin-cpu-memory`: Pin memory for CPU offload. Only added as a temp workaround if it throws "CUDA error: invalid argument".
+
+## 5. Benchmark
+
+Test Environment:
+
+- Hardware: AMD Instinct MI300X GPU (1x)
+- Model: Tongyi-MAI/Z-Image-Turbo
+- Docker Image: lmsysorg/sglang:v0.5.8-rocm700-mi30x
+- sglang diffusion version: 0.5.8
+
+### 5.1 Speedup Benchmark
+
+#### 5.1.1 Generate an image
+
+<Tabs>
+  <Tab title="AMD MI300X">
+    **Server Command**:
+
+    ```shell Command
+    sglang serve --model-path Tongyi-MAI/Z-Image-Turbo \
+        --ulysses-degree=1 --ring-degree=1 --port 30000
+    ```
+
+    **Benchmark Command**:
+
+    ```shell Command
+    python3 -m sglang.multimodal_gen.benchmarks.bench_serving \
+        --backend sglang-image --dataset vbench --task text-to-image --num-prompts 1 --max-concurrency 1
+    ```
+
+    **Result**:
+
+    ```text Output
+    ================= Serving Benchmark Result =================
+    Task:                                    text-to-image
+    Model:                                   Tongyi-MAI/Z-Image-Turbo
+    Dataset:                                 vbench
+    --------------------------------------------------
+    Benchmark duration (s):                  1.84
+    Request rate:                            inf
+    Max request concurrency:                 1
+    Successful requests:                     1/1
+    --------------------------------------------------
+    Request throughput (req/s):              0.54
+    Latency Mean (s):                        1.8435
+    Latency Median (s):                      1.8435
+    Latency P99 (s):                         1.8435
+    --------------------------------------------------
+    Peak Memory Max (MB):                    30689.20
+    Peak Memory Mean (MB):                   30689.20
+    Peak Memory Median (MB):                 30689.20
+    ============================================================
+    ```
+  </Tab>
+  <Tab title="Ascend A3">
+    **Server Command**:
+
+    ```shell Command
+    #One A3 card has 2 npu chips
+    sglang serve --model-path Tongyi-MAI/Z-Image-Turbo --tp-size 2 --sp-degree 1 --num-gpus 2
+    ```
+
+    **Benchmark Command**:
+
+    ```shell Command
+    python -m sglang.multimodal_gen.benchmarks.bench_serving --dataset vbench --task text-to-image --num-prompts 1 --max-concurrency 1
+    ```
+
+    **Result**:
+
+    ```text Output
+    ================= Serving Benchmark Result =================
+    Task:                                         text-to-image
+    Model:                                        Tongyi-MAI/Z-Image-Turbo
+    Dataset:                                      vbench
+    --------------------------------------------------
+    Benchmark duration (s):                       2.43
+    Request rate:                                 inf
+    Max request concurrency:                      1
+    Successful requests:                          1/1
+    Completed outputs:                            1
+    Outputs per prompt:                           1
+    --------------------------------------------------
+    Request throughput (req/s):                   0.41
+    Output throughput (outputs/s):                0.41
+    Latency Mean (s):                             2.43
+    Latency Median (s):                           2.43
+    Latency P90 (s):                              2.43
+    Latency P95 (s):                              2.43
+    Latency P99 (s):                              2.43
+    --------------------------------------------------
+    Peak Memory Max (MB):                         11052.00
+    Peak Memory Mean (MB):                        11052.00
+    Peak Memory Median (MB):                      11052.00
+    ------------------------------------------------------------
+    ```
+  </Tab>
+</Tabs>
+
+#### 5.1.2 Generate images with high concurrency
+
+<Tabs>
+  <Tab title="AMD MI300X">
+    **Benchmark Command**:
+
+    ```shell Command
+    python3 -m sglang.multimodal_gen.benchmarks.bench_serving \
+        --backend sglang-image --dataset vbench --task text-to-image --num-prompts 20 --max-concurrency 20
+    ```
+
+    **Result**:
+
+    ```text Output
+    ================= Serving Benchmark Result =================
+    Task:                                    text-to-image
+    Model:                                   Tongyi-MAI/Z-Image-Turbo
+    Dataset:                                 vbench
+    --------------------------------------------------
+    Benchmark duration (s):                  35.32
+    Request rate:                            inf
+    Max request concurrency:                 20
+    Successful requests:                     20/20
+    --------------------------------------------------
+    Request throughput (req/s):              0.57
+    Latency Mean (s):                        18.5672
+    Latency Median (s):                      18.5573
+    Latency P99 (s):                         34.9880
+    --------------------------------------------------
+    Peak Memory Max (MB):                    30689.26
+    Peak Memory Mean (MB):                   30689.21
+    Peak Memory Median (MB):                 30689.21
+    ============================================================
+    ```
+  </Tab>
+  <Tab title="Ascend A3">
+    **Benchmark Command**:
+
+    ```shell Command
+    python -m sglang.multimodal_gen.benchmarks.bench_serving --dataset vbench --task text-to-image --num-prompts 20 --max-concurrency 20
+    ```
+
+    **Result**:
+
+    ```text Output
+    ================= Serving Benchmark Result =================
+    Task:                                         text-to-image
+    Model:                                        /models/Tongyi-MAI/Z-Image-Turbo/Z-Image-Turbo
+    Dataset:                                      vbench
+    --------------------------------------------------
+    Benchmark duration (s):                       49.08
+    Request rate:                                 inf
+    Max request concurrency:                      20
+    Successful requests:                          20/20
+    Completed outputs:                            20
+    Outputs per prompt:                           1
+    --------------------------------------------------
+    Request throughput (req/s):                   0.41
+    Output throughput (outputs/s):                0.41
+    Latency Mean (s):                             25.78
+    Latency Median (s):                           25.77
+    Latency P90 (s):                              44.42
+    Latency P95 (s):                              46.75
+    Latency P99 (s):                              48.61
+    --------------------------------------------------
+    Peak Memory Max (MB):                         11054.00
+    Peak Memory Mean (MB):                        11054.00
+    Peak Memory Median (MB):                      11054.00
+    ------------------------------------------------------------
+    ```
+  </Tab>
+</Tabs>
@@ -0,0 +1,122 @@
+---
+title: Overview
+mode: wide
+description: Practical guides for deploying and using diffusion models with SGLang.
+metatags:
+    description: "Explore SGLang diffusion model cookbooks for image and video generation deployment, invocation, optimization, and benchmarking examples."
+---
+
+Choose a recipe by output modality. The sidebar stays organized by model family, while this overview separates image, video, and realtime/world workloads.
+
+## Image Models
+
+Image models generate one image request as a bounded denoising job, usually with bidirectional attention over the whole latent sequence.
+
+<CardGroup cols={3}>
+  <Card
+    title="FLUX"
+    mode="card"
+    href="/cookbook/diffusion/FLUX/FLUX"
+    img="/cards/logos/flux.png"
+  />
+  <Card
+    title="Ideogram 4"
+    mode="card"
+    href="/cookbook/diffusion/Ideogram/Ideogram4"
+    img="/cards/logos/ideogram.png"
+  />
+  <Card
+    title="Qwen-Image"
+    mode="card"
+    href="/cookbook/diffusion/Qwen-Image/Qwen-Image"
+    img="/cards/logos/qwen.png"
+  />
+  <Card
+    title="Z-Image"
+    mode="card"
+    href="/cookbook/diffusion/Z-Image/Z-Image-Turbo"
+    img="/cards/logos/zimage.png"
+  />
+  <Card
+    title="Krea"
+    mode="card"
+    href="/cookbook/diffusion/Krea/Krea-2"
+    img="/cards/logos/krea.png"
+  />
+  <Card
+    title="ERNIE-Image"
+    mode="card"
+    href="/cookbook/diffusion/Ernie-Image/Ernie-Image"
+    img="/cards/logos/ernie.png"
+  />
+</CardGroup>
+
+## Video Models
+
+Video models denoise a bounded latent video sequence for each request. Use these recipes for offline text-to-video, image-to-video, and video generation serving.
+
+<CardGroup cols={3}>
+  <Card
+    title="Cosmos"
+    mode="card"
+    href="/cookbook/diffusion/Cosmos/Cosmos3"
+    img="/cards/logos/nvidia.png"
+  />
+  <Card
+    title="Wan"
+    mode="card"
+    href="/cookbook/diffusion/Wan/Wan2.2"
+    img="/cards/logos/wan.png"
+  />
+  <Card
+    title="LongLive 2.0"
+    mode="card"
+    href="/cookbook/diffusion/LongLive/LongLive-2.0"
+    img="/cards/logos/nvidia.png"
+  />
+  <Card
+    title="LTX"
+    mode="card"
+    href="/cookbook/diffusion/LTX/LTX2 & LTX2.3"
+    img="/cards/logos/ltx.svg"
+  />
+  <Card
+    title="JoyAI-Echo"
+    mode="card"
+    href="/cookbook/diffusion/JoyEcho/JoyEcho"
+    img="/cards/logos/joyai-echo.svg"
+  />
+  <Card
+    title="MOVA"
+    mode="card"
+    href="/cookbook/diffusion/MOVA/MOVA"
+    img="/cards/logos/mova.png"
+  />
+  <Card
+    title="MiniMax-H3"
+    mode="card"
+    href="/cookbook/diffusion/MiniMax/MiniMax-H3"
+    img="/cards/logos/minimax.png"
+  />
+</CardGroup>
+
+## Realtime / World Models
+
+Realtime models keep a session alive and generate chunk by chunk with causal state, control signals, and cached video history.
+
+<CardGroup cols={3}>
+  <Card
+    title="LingBot World"
+    mode="card"
+    href="/cookbook/diffusion/LingBot-World/LingBot-World-2.0"
+    img="/cards/logos/inclusionai.png"
+  />
+  <Card
+    title="SANA-WM"
+    mode="card"
+    href="/cookbook/diffusion/SANA-WM/SANA-WM"
+    img="/cards/logos/sana.png"
+  />
+</CardGroup>
+
+Use the sidebar group for LingBot World family variants. The overview links the newer LingBot World 2.0 recipe directly.