minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

[diffusion] doc: fix diffusion docs and cookbook drift (#29302)

Browse Source
This commit is contained in:
Mick
2026-06-25 21:13:00 +08:00
committed by GitHub
parent bc150173b2
commit 890b38c211
12 changed files with 51 additions and 46 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs_new/cookbook/diffusion/FLUX/FLUX.mdx
+3 -3
View File
@@ -50,7 +50,7 @@ FLUX models are optimized for high-quality image generation. The recommended lau
### 3.2 Configuration Tips
Current supported optimization all listed [here](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/docs/support_matrix.md).
Currently supported optimizations are listed [here](/docs/sglang-diffusion/compatibility_matrix).
- `--vae-path`: Path to a custom VAE model or HuggingFace model ID (e.g., fal/FLUX.2-Tiny-AutoEncoder). If not specified, the VAE will be loaded from the main model path.
- `--num-gpus`: Number of GPUs to use
@@ -61,7 +61,7 @@ Current supported optimization all listed [here](https://github.com/sgl-project/
## 4. API Usage
For complete API documentation, please refer to the [official API usage guide](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/docs/openai_api.md).
For complete API documentation, please refer to the [official API usage guide](/docs/sglang-diffusion/api/openai_api).
### 4.1 Generate an Image
@@ -89,7 +89,7 @@ with open("output.png", "wb") as f:
#### 4.2.1 Cache-DiT Acceleration
SGLang integrates [Cache-DiT](https://github.com/vipshop/cache-dit), a caching acceleration engine for Diffusion Transformers (DiT), to achieve up to 7.4x inference speedup with minimal quality loss. You can set `SGLANG_CACHE_DIT_ENABLED=True` to enable it. For more details, please refer to the SGLang Cache-DiT [documentation](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/docs/cache_dit.md).
SGLang integrates [Cache-DiT](https://github.com/vipshop/cache-dit), a caching acceleration engine for Diffusion Transformers (DiT), to achieve up to 7.4x inference speedup with minimal quality loss. You can set `SGLANG_CACHE_DIT_ENABLED=True` to enable it. For more details, please refer to the SGLang Cache-DiT [documentation](/docs/sglang-diffusion/cache_dit).
**Basic Usage**
docs_new/cookbook/diffusion/Krea/Krea-2.mdx
+2 -2
View File
@@ -46,14 +46,14 @@ The step count and guidance scale are **request-time** settings (see [API Usage]
### 3.2 Configuration Tips
Currently supported optimizations are listed [here](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/docs/support_matrix.md).
Currently supported optimizations are listed [here](/docs/sglang-diffusion/compatibility_matrix).
- `--num-gpus`: Number of GPUs to use.
- `--tp-size`: Tensor parallelism size (the recommended multi-GPU path for Krea-2). Its attention heads (48, with 12 KV heads) and text heads (20) are divisible by a tensor-parallel size of 1, 2, or 4.
## 4. API Usage
For complete API documentation, please refer to the [official API usage guide](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/docs/openai_api.md).
For complete API documentation, please refer to the [official API usage guide](/docs/sglang-diffusion/api/openai_api).
### 4.1 Generate an Image
docs_new/cookbook/diffusion/MOVA/MOVA.mdx
+3 -3
View File
@@ -41,7 +41,7 @@ import { MOVADeployment } from '/src/snippets/diffusion/mova-deployment.jsx'
### 3.2 Configuration Tips
Current supported optimization all listed [here](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/docs/support_matrix.md).
Currently supported optimizations are listed [here](/docs/sglang-diffusion/compatibility_matrix).
- `--num-gpus`: Number of GPUs to use
- `--tp`: Tensor parallelism size (should not be larger than 1 if text encoder offload is enabled, as layer-wise offload plus prefetch is faster)
@@ -52,7 +52,7 @@ Current supported optimization all listed [here](https://github.com/sgl-project/
## 4. API Usage
For complete API documentation, please refer to the [official API usage guide](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/docs/openai_api.md).
For complete API documentation, please refer to the [official API usage guide](/docs/sglang-diffusion/api/openai_api).
### 4.1 CLI Generation (sglang generate)
@@ -95,7 +95,7 @@ curl -X POST "http://0.0.0.0:30002/v1/videos" \
#### 4.3.1 Cache-DiT Acceleration
SGLang integrates [Cache-DiT](https://github.com/vipshop/cache-dit), a caching acceleration engine for Diffusion Transformers (DiT), to achieve up to 7.4x inference speedup with minimal quality loss. You can set `SGLANG_CACHE_DIT_ENABLED=True` to enable it. For more details, please refer to the SGLang Cache-DiT [documentation](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/docs/cache_dit.md).
SGLang integrates [Cache-DiT](https://github.com/vipshop/cache-dit), a caching acceleration engine for Diffusion Transformers (DiT), to achieve up to 7.4x inference speedup with minimal quality loss. You can set `SGLANG_CACHE_DIT_ENABLED=True` to enable it. For more details, please refer to the SGLang Cache-DiT [documentation](/docs/sglang-diffusion/cache_dit).
**Basic Usage**
docs_new/cookbook/diffusion/Qwen-Image/Qwen-Image-Edit.mdx
+3 -3
View File
@@ -41,7 +41,7 @@ Qwen-Image-Edit-2511 is a 20B parameter model optimized for image editing tasks.
### 3.2 Configuration Tips
Current supported optimization all listed [here](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/docs/support_matrix.md).
Currently supported optimizations are listed [here](/docs/sglang-diffusion/compatibility_matrix).
- `--vae-path`: Path to a custom VAE model or HuggingFace model ID (e.g., fal/FLUX.2-Tiny-AutoEncoder). If not specified, the VAE will be loaded from the main model path.
- `--num-gpus`: Number of GPUs to use
@@ -52,7 +52,7 @@ Current supported optimization all listed [here](https://github.com/sgl-project/
## 4. API Usage
For complete API documentation, please refer to the [official API usage guide](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/docs/openai_api.md).
For complete API documentation, please refer to the [official API usage guide](/docs/sglang-diffusion/api/openai_api).
### 4.1 Edit an Image
@@ -80,7 +80,7 @@ with open("output.png", "wb") as f:
#### 4.2.1 Cache-DiT Acceleration
SGLang integrates [Cache-DiT](https://github.com/vipshop/cache-dit), a caching acceleration engine for Diffusion Transformers (DiT), to achieve up to 7.4x inference speedup with minimal quality loss. You can set `SGLANG_CACHE_DIT_ENABLED=True` to enable it. For more details, please refer to the SGLang Cache-DiT [documentation](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/docs/cache_dit.md).
SGLang integrates [Cache-DiT](https://github.com/vipshop/cache-dit), a caching acceleration engine for Diffusion Transformers (DiT), to achieve up to 7.4x inference speedup with minimal quality loss. You can set `SGLANG_CACHE_DIT_ENABLED=True` to enable it. For more details, please refer to the SGLang Cache-DiT [documentation](/docs/sglang-diffusion/cache_dit).
**Basic Usage**
docs_new/cookbook/diffusion/Qwen-Image/Qwen-Image.mdx
+1 -1
View File
@@ -32,7 +32,7 @@ Qwen-Image is a text-to-image model. The recommended launch configurations vary
### 3.2 Configuration Tips
Current supported optimization all listed [here](../../../docs/sglang-diffusion/attention_backends#platform-support-matrix).
Currently supported optimizations are listed [here](/docs/sglang-diffusion/compatibility_matrix).
- `--vae-path`: Path to a custom VAE model or HuggingFace model ID (e.g., fal/FLUX.2-Tiny-AutoEncoder). If not specified, the VAE will be loaded from the main model path.
- `--num-gpus`: Number of GPUs to use
docs_new/cookbook/diffusion/README.mdx
+19 -19
View File
@@ -9,7 +9,7 @@ metatags:
<div style={{display: 'flex', gap: '8px'}}>
<a href="https://opensource.org/licenses/Apache-2.0"><img src="https://img.shields.io/badge/License-Apache_2.0-blue.svg" alt="License" /></a>
<a href="https://github.com/sgl-project/sgl-cookbook/pulls"><img src="https://img.shields.io/badge/PRs-welcome-brightgreen.svg" alt="PRs Welcome" /></a>
<a href="https://github.com/sgl-project/sglang/pulls"><img src="https://img.shields.io/badge/PRs-welcome-brightgreen.svg" alt="PRs Welcome" /></a>
</div>
Create a comprehensive cookbook for diffusion models in SGLang, demonstrating SGLang's performance advantages for image and video generation workloads.
@@ -35,18 +35,18 @@ Each recipe provides step-by-step instructions to help you quickly implement SGL
The sglang diffusion cookbook directory structure are shown below:
```text Example
sgl-cookbook/docs/diffusion/
├── README.md # Main cookbook (this file)
├── Qwen-Image/ # Qwen-Image series models docs
│ ├── Qwen-Image.md
│ └── Qwen-Image-Edit.md
├── Wan/ # Wan series models docs
│ ├── Wan2.1.md
│ └── Wan2.2.md
├── Z-Image/ # Z-Image series models docs
│ └── Z-Image-Turbo.md
├── Ernie-Image/ # ERNIE-Image series models docs
│ └── Ernie-Image.md
docs_new/cookbook/diffusion/
├── README.mdx # Main cookbook (this file)
├── Qwen-Image/ # Qwen-Image series docs
│ ├── Qwen-Image.mdx
│ └── Qwen-Image-Edit.mdx
├── Wan/ # Wan series docs
│ ├── Wan2.1.mdx
│ └── Wan2.2.mdx
├── Z-Image/ # Z-Image series docs
│ └── Z-Image-Turbo.mdx
├── Ernie-Image/ # ERNIE-Image series docs
│ └── Ernie-Image.mdx
└── ...
```
@@ -66,27 +66,27 @@ We believe the best documentation comes from practitioners. Whether you've optim
```shell Command
# Fork the repo and clone locally
git clone https://github.com/YOUR_USERNAME/sglang-cookbook.git
cd sglang-cookbook
git clone https://github.com/YOUR_USERNAME/sglang.git
cd sglang
# Create a new branch
git checkout -b add-my-recipe
# Add your recipe following the template in DeepSeek-V3.2
# Add your recipe under docs_new/cookbook/diffusion/
# Submit a PR!
```
## 📖 Resources
- [SGLang GitHub](https://github.com/sgl-project/sglang)
- [SGLang Documentation](https://sgl-project.github.io)
- [SGLANG Diffusion Documentation](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/README.md)
- [SGLang Documentation](/)
- [SGLang Diffusion Documentation](/docs/sglang-diffusion/index)
- [SLACK Channel](https://sgl-fru7574.slack.com/archives/C07GLLLESNR)
- [Community Slack/Discord](https://discord.gg/MpEEuAeb)
## 📄 License
This project is licensed under the Apache License 2.0 - see the [LICENSE](https://github.com/sgl-project/sgl-cookbook/blob/main/LICENSE) file for details.
This project is licensed under the Apache License 2.0 - see the [LICENSE](https://github.com/sgl-project/sglang/blob/main/LICENSE) file for details.
---
docs_new/cookbook/diffusion/Wan/Wan2.2.mdx
+3 -3
View File
@@ -39,7 +39,7 @@ The Wan2.2 series offers models in various sizes, architectures and input types,
### 3.2 Configuration Tips
Current supported optimization all listed [here](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/docs/support_matrix.md).
Currently supported optimizations are listed [here](/docs/sglang-diffusion/compatibility_matrix).
- `--vae-path`: Path to a custom VAE model or HuggingFace model ID (e.g., fal/FLUX.2-Tiny-AutoEncoder). If not specified, the VAE will be loaded from the main model path.
- `--num-gpus {NUM_GPUS}`: Number of GPUs to use
@@ -53,7 +53,7 @@ Current supported optimization all listed [here](https://github.com/sgl-project/
### 4.1 Basic Usage
For more API usage and request examples, please refer to:
[SGLang Diffusion OpenAI API](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/docs/openai_api.md)
[SGLang Diffusion OpenAI API](/docs/sglang-diffusion/api/openai_api)
#### 4.1.1 Launch a server and then send requests
@@ -100,7 +100,7 @@ sglang generate "${SERVER_ARGS[@]}" "${SAMPLING_ARGS[@]}"
#### 4.2.1 Cache-DiT Acceleration
SGLang integrates [Cache-DiT](https://github.com/vipshop/cache-dit), a caching acceleration engine for Diffusion Transformers (DiT), to achieve up to 7.4x inference speedup with minimal quality loss. You can set `SGLANG_CACHE_DIT_ENABLED=True` to enable it. For more details, please refer to the SGLang Cache-DiT [documentation](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/docs/cache/cache_dit.md).
SGLang integrates [Cache-DiT](https://github.com/vipshop/cache-dit), a caching acceleration engine for Diffusion Transformers (DiT), to achieve up to 7.4x inference speedup with minimal quality loss. You can set `SGLANG_CACHE_DIT_ENABLED=True` to enable it. For more details, please refer to the SGLang Cache-DiT [documentation](/docs/sglang-diffusion/cache_dit).
**Basic Usage**
docs_new/cookbook/diffusion/Z-Image/Z-Image-Turbo.mdx
+3 -3
View File
@@ -42,7 +42,7 @@ Z-Image-Turbo is optimized for high-quality image generation with only 8 inferen
### 3.2 Configuration Tips
Current supported optimization all listed [here](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/docs/support_matrix.md).
Currently supported optimizations are listed [here](/docs/sglang-diffusion/compatibility_matrix).
- `--vae-path`: Path to a custom VAE model or HuggingFace model ID (e.g., fal/FLUX.2-Tiny-AutoEncoder). If not specified, the VAE will be loaded from the main model path.
- `--num-gpus`: Number of GPUs to use
@@ -55,7 +55,7 @@ Current supported optimization all listed [here](https://github.com/sgl-project/
## 4. API Usage
For complete API documentation, please refer to the [official API usage guide](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/docs/openai_api.md).
For complete API documentation, please refer to the [official API usage guide](/docs/sglang-diffusion/api/openai_api).
### 4.1 Generate an Image
@@ -82,7 +82,7 @@ with open("output.png", "wb") as f:
#### 4.2.1 Cache-DiT Acceleration
SGLang integrates [Cache-DiT](https://github.com/vipshop/cache-dit), a caching acceleration engine for Diffusion Transformers (DiT), to achieve up to 7.4x inference speedup with minimal quality loss. You can set `SGLANG_CACHE_DIT_ENABLED=True` to enable it. For more details, please refer to the SGLang Cache-DiT [documentation](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/docs/cache_dit.md).
SGLang integrates [Cache-DiT](https://github.com/vipshop/cache-dit), a caching acceleration engine for Diffusion Transformers (DiT), to achieve up to 7.4x inference speedup with minimal quality loss. You can set `SGLANG_CACHE_DIT_ENABLED=True` to enable it. For more details, please refer to the SGLang Cache-DiT [documentation](/docs/sglang-diffusion/cache_dit).
**Basic Usage**
docs_new/cookbook/diffusion/intro.mdx
+1 -1
View File
@@ -79,7 +79,7 @@ Realtime models keep a session alive and generate chunk by chunk with causal sta
<CardGroup cols={3}>
<Card
title="LingBot World (Realtime)"
title="LingBot World"
mode="card"
href="/cookbook/diffusion/LingBot-World/LingBot-World"
img="/cards/logos/inclusionai.png"
docs_new/docs/sglang-diffusion/api/cli.mdx
+1
View File
@@ -83,6 +83,7 @@ Use `sglang generate --help` and `sglang serve --help` for the full argument lis
- `--sp-degree &#123;N&#125;`: sequence parallelism size
- `--ulysses-degree &#123;N&#125;` and `--ring-degree &#123;N&#125;`: USP parallelism controls
- `--enable-cfg-parallel &#123;true|false&#125;`: enable or explicitly disable CFG parallelism
- `--warmup-mode &#123;off|request|server&#125;`: control startup warmup for `sglang serve`; `off` skips warmup, `request` primes the request path, and `server` runs a full synthetic server warmup before serving traffic
- `--attention-backend &#123;BACKEND&#125;`: attention backend for native SGLang and diffusers pipelines
- `--component-attention-backends &#123;MAP&#125;`: per-component attention backend overrides, for example `text_encoder=torch_sdpa,transformer=fa`
- `--attention-backend-config &#123;CONFIG&#125;`: attention backend configuration
docs_new/docs/sglang-diffusion/caching-acceleration.mdx
+6 -2
View File
@@ -45,7 +45,11 @@ SGLang supports two complementary caching approaches:
[Cache-DiT](https://github.com/vipshop/cache-dit) provides block-level caching with
advanced strategies like DBCache and TaylorSeer. It can achieve up to **1.69x speedup**.
See [cache_dit.md](./cache_dit) for detailed configuration.
See [Cache-DiT](./cache_dit) for detailed configuration.
<Note>
Cache-DiT currently cannot be combined with `--use-fsdp-inference`. Keep FSDP disabled when enabling Cache-DiT, or use other residency/offload controls instead.
</Note>
### Quick Start
@@ -65,7 +69,7 @@ sglang generate --model-path Qwen/Qwen-Image \
TeaCache (Temporal similarity-based caching) accelerates diffusion inference by detecting when consecutive denoising steps are similar enough to skip computation entirely.
See [teacache.md](./teacache) for detailed documentation.
See [TeaCache](./teacache) for detailed documentation.
### Quick Overview
python/sglang/multimodal_gen/README.md
+6 -6
View File
@@ -21,15 +21,15 @@ SGLang Diffusion has the following features:
### AMD/ROCm Support
SGLang Diffusion supports AMD Instinct GPUs through ROCm. On AMD platforms, we use the Triton attention backend and leverage AITER kernels for optimized layernorm and other operations. See the [installation guide](https://github.com/sgl-project/sglang/tree/main/docs/diffusion/installation.md) for setup instructions.
SGLang Diffusion supports AMD Instinct GPUs through ROCm. On AMD platforms, we use the Triton attention backend and leverage AITER kernels for optimized layernorm and other operations. See the [installation guide](https://docs.sglang.io/docs/sglang-diffusion/installation) for setup instructions.
### Moore Threads/MUSA Support
SGLang Diffusion supports Moore Threads GPUs (MTGPU) through the MUSA software stack. On MUSA platforms, we use FlashAttention (FA3) when available; also supports Sage Attention when installed; otherwise falls back to the Torch SDPA backend. See the [installation guide](https://github.com/sgl-project/sglang/tree/main/docs/diffusion/installation.md) for setup instructions.
SGLang Diffusion supports Moore Threads GPUs (MTGPU) through the MUSA software stack. On MUSA platforms, we use FlashAttention (FA3) when available; also supports Sage Attention when installed; otherwise falls back to the Torch SDPA backend. See the [installation guide](https://docs.sglang.io/docs/sglang-diffusion/installation) for setup instructions.
### Apple MPS Support
SGLang Diffusion supports Apple Silicon (M-series) via the MPS backend. Since Triton is Linux-only, all Triton kernels are replaced with PyTorch-native fallbacks on MPS. Norm operations can be optionally accelerated with MLX fused Metal kernels (`SGLANG_USE_MLX=1`). See the [installation guide](https://github.com/sgl-project/sglang/tree/main/docs/diffusion/installation.md) for setup instructions.
SGLang Diffusion supports Apple Silicon (M-series) via the MPS backend. Since Triton is Linux-only, all Triton kernels are replaced with PyTorch-native fallbacks on MPS. Norm operations can be optionally accelerated with MLX fused Metal kernels (`SGLANG_USE_MLX=1`). See the [installation guide](https://docs.sglang.io/docs/sglang-diffusion/installation) for setup instructions.
## Getting Started
@@ -37,7 +37,7 @@ SGLang Diffusion supports Apple Silicon (M-series) via the MPS backend. Since Tr
uv pip install 'sglang[diffusion]' --prerelease=allow
```
For more installation methods (e.g. pypi, uv, docker, ROCm/AMD, MUSA/Moore Threads), check [install.md](https://github.com/sgl-project/sglang/tree/main/docs/diffusion/installation.md).
For more installation methods (e.g. pypi, uv, docker, ROCm/AMD, MUSA/Moore Threads), check the [installation guide](https://docs.sglang.io/docs/sglang-diffusion/installation).
## Inference
@@ -89,11 +89,11 @@ sglang generate \
--save-output
```
For more usage examples (e.g. OpenAI compatible API, server mode), check [cli.md](https://github.com/sgl-project/sglang/tree/main/docs/diffusion/api/cli.md).
For more usage examples (e.g. OpenAI compatible API, server mode), check the [CLI reference](https://docs.sglang.io/docs/sglang-diffusion/api/cli).
## Contributing
All contributions are welcome. The contribution guide is available [here](https://github.com/sgl-project/sglang/tree/main/docs/diffusion/contributing.md).
All contributions are welcome. The contribution guide is available [here](https://docs.sglang.io/docs/sglang-diffusion/contributing).
## Acknowledgement