minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

docs: refine diffusion cookbook overview (#29507)

Browse Source
This commit is contained in:
Mick
2026-06-28 00:10:21 +08:00
committed by GitHub
parent 1589603114
commit cfd911ad6e
5 changed files with 55 additions and 50 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs_new/cards/logos/ideogram.png
BIN
View File
Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 37 KiB

After

Width:  |  Height:  |  Size: 74 KiB

docs_new/cookbook/diffusion/LingBot-World/LingBot-World.mdx
+6 -26
View File
@@ -5,36 +5,16 @@ metatags:
tag: REALTIME
---
<div className="not-prose" style={{display: "flex", flexWrap: "wrap", gap: "6px", margin: "8px 0 30px"}}>
<span style={{display: "inline-block", padding: "4px 9px", borderRadius: "999px", background: "#ecfeff", color: "#155e75", fontSize: "12px", fontWeight: 750}}>realtime</span>
<span style={{display: "inline-block", padding: "4px 9px", borderRadius: "999px", background: "#ecfeff", color: "#155e75", fontSize: "12px", fontWeight: 750}}>world model</span>
<span style={{display: "inline-block", padding: "4px 9px", borderRadius: "999px", background: "#ecfeff", color: "#155e75", fontSize: "12px", fontWeight: 750}}>causal DiT</span>
</div>
## 1. Model Introduction
[LingBot World](https://huggingface.co/robbyant/lingbot-world-fast-diffusers) is a realtime camera-controlled video world model. In SGLang-diffusion, it belongs to the realtime causal path: the server keeps a live session, samples control signals per chunk, reuses causal DiT state, and decodes video frames incrementally.
<div style={{border: "1px solid #dbe3ef", borderRadius: "16px", overflow: "hidden", background: "linear-gradient(135deg, #ffffff 0%, #f8fafc 48%, #ecfeff 100%)", boxShadow: "0 14px 36px rgba(15, 23, 42, 0.08)", margin: "22px 0 28px 0"}}>
<div style={{display: "flex", alignItems: "center", justifyContent: "space-between", gap: "16px", padding: "18px 22px", borderBottom: "1px solid #e2e8f0"}}>
<div>
<div style={{fontSize: "20px", fontWeight: 750, color: "#0f172a", lineHeight: 1.2}}>LingBot World</div>
<div style={{fontSize: "13px", color: "#64748b", marginTop: "5px"}}>Realtime diffusion world model in SGLang-diffusion</div>
</div>
<div style={{display: "inline-flex", alignItems: "center", whiteSpace: "nowrap", border: "1px solid #bae6fd", background: "#e0f2fe", color: "#0369a1", borderRadius: "999px", padding: "8px 12px", fontSize: "13px", fontWeight: 800}}>
prompt + image + control &rarr; streaming video
</div>
</div>
<div style={{display: "grid", gridTemplateColumns: "170px 1fr"}}>
<div style={{padding: "12px 22px", borderBottom: "1px solid #edf2f7", color: "#64748b", fontWeight: 700, background: "rgba(248, 250, 252, 0.72)"}}>Category</div>
<div style={{padding: "12px 22px", borderBottom: "1px solid #edf2f7", color: "#1e293b"}}>
<span style={{display: "inline-block", marginRight: "6px", padding: "4px 9px", borderRadius: "999px", background: "#ecfeff", color: "#155e75", fontSize: "12px", fontWeight: 750}}>realtime</span>
<span style={{display: "inline-block", marginRight: "6px", padding: "4px 9px", borderRadius: "999px", background: "#ecfeff", color: "#155e75", fontSize: "12px", fontWeight: 750}}>world model</span>
<span style={{display: "inline-block", marginRight: "6px", padding: "4px 9px", borderRadius: "999px", background: "#ecfeff", color: "#155e75", fontSize: "12px", fontWeight: 750}}>causal DiT</span>
</div>
<div style={{padding: "12px 22px", borderBottom: "1px solid #edf2f7", color: "#64748b", fontWeight: 700, background: "rgba(248, 250, 252, 0.72)"}}>Inputs</div>
<div style={{padding: "12px 22px", borderBottom: "1px solid #edf2f7", color: "#1e293b"}}>Prompt, first frame, and per-chunk camera control signals</div>
<div style={{padding: "12px 22px", borderBottom: "1px solid #edf2f7", color: "#64748b", fontWeight: 700, background: "rgba(248, 250, 252, 0.72)"}}>Outputs</div>
<div style={{padding: "12px 22px", borderBottom: "1px solid #edf2f7", color: "#1e293b"}}>Streaming video frame chunks over <code>/v1/realtime_video/generate</code></div>
<div style={{padding: "12px 22px", color: "#64748b", fontWeight: 700, background: "rgba(248, 250, 252, 0.72)"}}>Core runtime</div>
<div style={{padding: "12px 22px", color: "#1e293b"}}>Condition queue, causal DiT KV cache, causal VAE decode cache, and realtime session state</div>
</div>
</div>
This is different from offline diffusion video models such as Wan or LTX. Offline models denoise a bounded latent sequence for one request. Realtime world models generate a continuing stream, so the runtime must manage session state, control events, causal attention cache, and VAE decode cache.
## 2. Deployment
docs_new/cookbook/diffusion/Wan/Wan2.1.mdx
+4 -2
View File
@@ -35,7 +35,7 @@ This section provides deployment configurations optimized for different hardware
### 3.1 Basic Configuration
The Wan2.1 series offers models in multiple sizes and resolutions, optimized for different hardware platforms. The recommended launch configurations vary by hardware and model size.
The Wan2.1 series offers models in multiple sizes and resolutions. SGLang supports Wan2.1 deployment on NVIDIA B200, B300, H200, H100, and AMD MI300X, MI325X, MI355X GPUs. The recommended launch configurations vary by hardware, model size, and memory headroom.
**Interactive Command Generator**: Use the configuration selector below to automatically generate an appropriate deployment command for your model variant and options.
@@ -169,7 +169,9 @@ sglang serve --model-path Wan-AI/Wan2.1-T2V-14B-Diffusers --port 30000 \
--lora-path NIVEDAN/wan2.1-lora
```
## 5. Benchmark
## 5. Reference Benchmark
The following benchmark is a point-in-time reference for one model, hardware platform, SGLang image, and parameter set. It is not a complete hardware support matrix.
Test Environment:
docs_new/cookbook/diffusion/intro.mdx
+35 -20
View File
@@ -6,17 +6,13 @@ metatags:
description: "Explore SGLang diffusion model cookbooks for image and video generation deployment, invocation, optimization, and benchmarking examples."
---
## Offline Diffusion Models
Choose a recipe by output modality. The sidebar stays organized by model family, while this overview separates image, video, and realtime/world workloads.
Offline models generate each image or video request as a bounded denoising job. Most of them use bidirectional attention over the whole latent sequence.
## Image Models
Image models generate one image request as a bounded denoising job, usually with bidirectional attention over the whole latent sequence.
<CardGroup cols={3}>
<Card
title="Cosmos"
mode="card"
href="/cookbook/diffusion/Cosmos/Cosmos3"
img="/cards/logos/nvidia.png"
/>
<Card
title="FLUX"
mode="card"
@@ -29,18 +25,6 @@ Offline models generate each image or video request as a bounded denoising job.
href="/cookbook/diffusion/Ideogram/Ideogram4"
img="/cards/logos/ideogram.png"
/>
<Card
title="Wan"
mode="card"
href="/cookbook/diffusion/Wan/Wan2.2"
img="/cards/logos/wan.png"
/>
<Card
title="LTX"
mode="card"
href="/cookbook/diffusion/LTX/LTX2 & LTX2.3"
img="/cards/logos/ltx.svg"
/>
<Card
title="Qwen-Image"
mode="card"
@@ -65,6 +49,37 @@ Offline models generate each image or video request as a bounded denoising job.
href="/cookbook/diffusion/Ernie-Image/Ernie-Image"
img="/cards/logos/ernie.png"
/>
</CardGroup>
## Video Models
Video models denoise a bounded latent video sequence for each request. Use these recipes for offline text-to-video, image-to-video, and video generation serving.
<CardGroup cols={3}>
<Card
title="Cosmos"
mode="card"
href="/cookbook/diffusion/Cosmos/Cosmos3"
img="/cards/logos/nvidia.png"
/>
<Card
title="Wan"
mode="card"
href="/cookbook/diffusion/Wan/Wan2.2"
img="/cards/logos/wan.png"
/>
<Card
title="LTX"
mode="card"
href="/cookbook/diffusion/LTX/LTX2 & LTX2.3"
img="/cards/logos/ltx.svg"
/>
<Card
title="JoyEcho"
mode="card"
href="/cookbook/diffusion/JoyEcho/JoyEcho"
img="/cards/logos/ltx.svg"
/>
<Card
title="MOVA"
mode="card"
docs_new/src/snippets/diffusion/wan21-deployment.jsx
+10 -2
View File
@@ -39,7 +39,15 @@ export const Wan21Deployment = () => {
hardware: {
name: 'hardware',
title: 'Hardware Platform',
items: [{ id: 'mi300x', label: 'MI300X/MI325X/MI355X', default: true }],
items: [
{ id: 'b200', label: 'B200', default: true },
{ id: 'b300', label: 'B300', default: false },
{ id: 'h200', label: 'H200', default: false },
{ id: 'h100', label: 'H100', default: false },
{ id: 'mi300x', label: 'MI300X', default: false },
{ id: 'mi325x', label: 'MI325X', default: false },
{ id: 'mi355x', label: 'MI355X', default: false },
],
},
task: {
name: 'task',
@@ -77,7 +85,7 @@ export const Wan21Deployment = () => {
const configKey = `${task}-${modelsize}`;
const supported = modelConfigs[configKey]?.supportedLoras || [];
return {
hardware: 'mi300x',
hardware: 'b200',
task,
modelsize,
bestPractice: 'off',