diff --git a/docs_new/cookbook/diffusion/FLUX/FLUX.mdx b/docs_new/cookbook/diffusion/FLUX/FLUX.mdx index cd4783d0a..840905699 100644 --- a/docs_new/cookbook/diffusion/FLUX/FLUX.mdx +++ b/docs_new/cookbook/diffusion/FLUX/FLUX.mdx @@ -50,7 +50,7 @@ FLUX models are optimized for high-quality image generation. The recommended lau ### 3.2 Configuration Tips -Current supported optimization all listed [here](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/docs/support_matrix.md). +Currently supported optimizations are listed [here](/docs/sglang-diffusion/compatibility_matrix). - `--vae-path`: Path to a custom VAE model or HuggingFace model ID (e.g., fal/FLUX.2-Tiny-AutoEncoder). If not specified, the VAE will be loaded from the main model path. - `--num-gpus`: Number of GPUs to use @@ -61,7 +61,7 @@ Current supported optimization all listed [here](https://github.com/sgl-project/ ## 4. API Usage -For complete API documentation, please refer to the [official API usage guide](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/docs/openai_api.md). +For complete API documentation, please refer to the [official API usage guide](/docs/sglang-diffusion/api/openai_api). ### 4.1 Generate an Image @@ -89,7 +89,7 @@ with open("output.png", "wb") as f: #### 4.2.1 Cache-DiT Acceleration -SGLang integrates [Cache-DiT](https://github.com/vipshop/cache-dit), a caching acceleration engine for Diffusion Transformers (DiT), to achieve up to 7.4x inference speedup with minimal quality loss. You can set `SGLANG_CACHE_DIT_ENABLED=True` to enable it. For more details, please refer to the SGLang Cache-DiT [documentation](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/docs/cache_dit.md). +SGLang integrates [Cache-DiT](https://github.com/vipshop/cache-dit), a caching acceleration engine for Diffusion Transformers (DiT), to achieve up to 7.4x inference speedup with minimal quality loss. You can set `SGLANG_CACHE_DIT_ENABLED=True` to enable it. For more details, please refer to the SGLang Cache-DiT [documentation](/docs/sglang-diffusion/cache_dit). **Basic Usage** diff --git a/docs_new/cookbook/diffusion/Krea/Krea-2.mdx b/docs_new/cookbook/diffusion/Krea/Krea-2.mdx index 1e3299378..5a9a4a4af 100644 --- a/docs_new/cookbook/diffusion/Krea/Krea-2.mdx +++ b/docs_new/cookbook/diffusion/Krea/Krea-2.mdx @@ -46,14 +46,14 @@ The step count and guidance scale are **request-time** settings (see [API Usage] ### 3.2 Configuration Tips -Currently supported optimizations are listed [here](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/docs/support_matrix.md). +Currently supported optimizations are listed [here](/docs/sglang-diffusion/compatibility_matrix). - `--num-gpus`: Number of GPUs to use. - `--tp-size`: Tensor parallelism size (the recommended multi-GPU path for Krea-2). Its attention heads (48, with 12 KV heads) and text heads (20) are divisible by a tensor-parallel size of 1, 2, or 4. ## 4. API Usage -For complete API documentation, please refer to the [official API usage guide](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/docs/openai_api.md). +For complete API documentation, please refer to the [official API usage guide](/docs/sglang-diffusion/api/openai_api). ### 4.1 Generate an Image diff --git a/docs_new/cookbook/diffusion/MOVA/MOVA.mdx b/docs_new/cookbook/diffusion/MOVA/MOVA.mdx index c5b3df6f2..e594e2000 100644 --- a/docs_new/cookbook/diffusion/MOVA/MOVA.mdx +++ b/docs_new/cookbook/diffusion/MOVA/MOVA.mdx @@ -41,7 +41,7 @@ import { MOVADeployment } from '/src/snippets/diffusion/mova-deployment.jsx' ### 3.2 Configuration Tips -Current supported optimization all listed [here](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/docs/support_matrix.md). +Currently supported optimizations are listed [here](/docs/sglang-diffusion/compatibility_matrix). - `--num-gpus`: Number of GPUs to use - `--tp`: Tensor parallelism size (should not be larger than 1 if text encoder offload is enabled, as layer-wise offload plus prefetch is faster) @@ -52,7 +52,7 @@ Current supported optimization all listed [here](https://github.com/sgl-project/ ## 4. API Usage -For complete API documentation, please refer to the [official API usage guide](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/docs/openai_api.md). +For complete API documentation, please refer to the [official API usage guide](/docs/sglang-diffusion/api/openai_api). ### 4.1 CLI Generation (sglang generate) @@ -95,7 +95,7 @@ curl -X POST "http://0.0.0.0:30002/v1/videos" \ #### 4.3.1 Cache-DiT Acceleration -SGLang integrates [Cache-DiT](https://github.com/vipshop/cache-dit), a caching acceleration engine for Diffusion Transformers (DiT), to achieve up to 7.4x inference speedup with minimal quality loss. You can set `SGLANG_CACHE_DIT_ENABLED=True` to enable it. For more details, please refer to the SGLang Cache-DiT [documentation](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/docs/cache_dit.md). +SGLang integrates [Cache-DiT](https://github.com/vipshop/cache-dit), a caching acceleration engine for Diffusion Transformers (DiT), to achieve up to 7.4x inference speedup with minimal quality loss. You can set `SGLANG_CACHE_DIT_ENABLED=True` to enable it. For more details, please refer to the SGLang Cache-DiT [documentation](/docs/sglang-diffusion/cache_dit). **Basic Usage** diff --git a/docs_new/cookbook/diffusion/Qwen-Image/Qwen-Image-Edit.mdx b/docs_new/cookbook/diffusion/Qwen-Image/Qwen-Image-Edit.mdx index 430e4bd92..2626927df 100644 --- a/docs_new/cookbook/diffusion/Qwen-Image/Qwen-Image-Edit.mdx +++ b/docs_new/cookbook/diffusion/Qwen-Image/Qwen-Image-Edit.mdx @@ -41,7 +41,7 @@ Qwen-Image-Edit-2511 is a 20B parameter model optimized for image editing tasks. ### 3.2 Configuration Tips -Current supported optimization all listed [here](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/docs/support_matrix.md). +Currently supported optimizations are listed [here](/docs/sglang-diffusion/compatibility_matrix). - `--vae-path`: Path to a custom VAE model or HuggingFace model ID (e.g., fal/FLUX.2-Tiny-AutoEncoder). If not specified, the VAE will be loaded from the main model path. - `--num-gpus`: Number of GPUs to use @@ -52,7 +52,7 @@ Current supported optimization all listed [here](https://github.com/sgl-project/ ## 4. API Usage -For complete API documentation, please refer to the [official API usage guide](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/docs/openai_api.md). +For complete API documentation, please refer to the [official API usage guide](/docs/sglang-diffusion/api/openai_api). ### 4.1 Edit an Image @@ -80,7 +80,7 @@ with open("output.png", "wb") as f: #### 4.2.1 Cache-DiT Acceleration -SGLang integrates [Cache-DiT](https://github.com/vipshop/cache-dit), a caching acceleration engine for Diffusion Transformers (DiT), to achieve up to 7.4x inference speedup with minimal quality loss. You can set `SGLANG_CACHE_DIT_ENABLED=True` to enable it. For more details, please refer to the SGLang Cache-DiT [documentation](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/docs/cache_dit.md). +SGLang integrates [Cache-DiT](https://github.com/vipshop/cache-dit), a caching acceleration engine for Diffusion Transformers (DiT), to achieve up to 7.4x inference speedup with minimal quality loss. You can set `SGLANG_CACHE_DIT_ENABLED=True` to enable it. For more details, please refer to the SGLang Cache-DiT [documentation](/docs/sglang-diffusion/cache_dit). **Basic Usage** diff --git a/docs_new/cookbook/diffusion/Qwen-Image/Qwen-Image.mdx b/docs_new/cookbook/diffusion/Qwen-Image/Qwen-Image.mdx index dc1c0ecc5..2f92c67d7 100644 --- a/docs_new/cookbook/diffusion/Qwen-Image/Qwen-Image.mdx +++ b/docs_new/cookbook/diffusion/Qwen-Image/Qwen-Image.mdx @@ -32,7 +32,7 @@ Qwen-Image is a text-to-image model. The recommended launch configurations vary ### 3.2 Configuration Tips -Current supported optimization all listed [here](../../../docs/sglang-diffusion/attention_backends#platform-support-matrix). +Currently supported optimizations are listed [here](/docs/sglang-diffusion/compatibility_matrix). - `--vae-path`: Path to a custom VAE model or HuggingFace model ID (e.g., fal/FLUX.2-Tiny-AutoEncoder). If not specified, the VAE will be loaded from the main model path. - `--num-gpus`: Number of GPUs to use diff --git a/docs_new/cookbook/diffusion/README.mdx b/docs_new/cookbook/diffusion/README.mdx index 10570d314..3b020c5a8 100644 --- a/docs_new/cookbook/diffusion/README.mdx +++ b/docs_new/cookbook/diffusion/README.mdx @@ -9,7 +9,7 @@ metatags:
License - PRs Welcome + PRs Welcome
Create a comprehensive cookbook for diffusion models in SGLang, demonstrating SGLang's performance advantages for image and video generation workloads. @@ -35,18 +35,18 @@ Each recipe provides step-by-step instructions to help you quickly implement SGL The sglang diffusion cookbook directory structure are shown below: ```text Example -sgl-cookbook/docs/diffusion/ -├── README.md # Main cookbook (this file) -├── Qwen-Image/ # Qwen-Image series models docs -│ ├── Qwen-Image.md -│ └── Qwen-Image-Edit.md -├── Wan/ # Wan series models docs -│ ├── Wan2.1.md -│ └── Wan2.2.md -├── Z-Image/ # Z-Image series models docs -│ └── Z-Image-Turbo.md -├── Ernie-Image/ # ERNIE-Image series models docs -│ └── Ernie-Image.md +docs_new/cookbook/diffusion/ +├── README.mdx # Main cookbook (this file) +├── Qwen-Image/ # Qwen-Image series docs +│ ├── Qwen-Image.mdx +│ └── Qwen-Image-Edit.mdx +├── Wan/ # Wan series docs +│ ├── Wan2.1.mdx +│ └── Wan2.2.mdx +├── Z-Image/ # Z-Image series docs +│ └── Z-Image-Turbo.mdx +├── Ernie-Image/ # ERNIE-Image series docs +│ └── Ernie-Image.mdx └── ... ``` @@ -66,27 +66,27 @@ We believe the best documentation comes from practitioners. Whether you've optim ```shell Command # Fork the repo and clone locally -git clone https://github.com/YOUR_USERNAME/sglang-cookbook.git -cd sglang-cookbook +git clone https://github.com/YOUR_USERNAME/sglang.git +cd sglang # Create a new branch git checkout -b add-my-recipe -# Add your recipe following the template in DeepSeek-V3.2 +# Add your recipe under docs_new/cookbook/diffusion/ # Submit a PR! ``` ## 📖 Resources - [SGLang GitHub](https://github.com/sgl-project/sglang) -- [SGLang Documentation](https://sgl-project.github.io) -- [SGLANG Diffusion Documentation](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/README.md) +- [SGLang Documentation](/) +- [SGLang Diffusion Documentation](/docs/sglang-diffusion/index) - [SLACK Channel](https://sgl-fru7574.slack.com/archives/C07GLLLESNR) - [Community Slack/Discord](https://discord.gg/MpEEuAeb) ## 📄 License -This project is licensed under the Apache License 2.0 - see the [LICENSE](https://github.com/sgl-project/sgl-cookbook/blob/main/LICENSE) file for details. +This project is licensed under the Apache License 2.0 - see the [LICENSE](https://github.com/sgl-project/sglang/blob/main/LICENSE) file for details. --- diff --git a/docs_new/cookbook/diffusion/Wan/Wan2.2.mdx b/docs_new/cookbook/diffusion/Wan/Wan2.2.mdx index 7cdd2823a..b87ca4475 100644 --- a/docs_new/cookbook/diffusion/Wan/Wan2.2.mdx +++ b/docs_new/cookbook/diffusion/Wan/Wan2.2.mdx @@ -39,7 +39,7 @@ The Wan2.2 series offers models in various sizes, architectures and input types, ### 3.2 Configuration Tips -Current supported optimization all listed [here](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/docs/support_matrix.md). +Currently supported optimizations are listed [here](/docs/sglang-diffusion/compatibility_matrix). - `--vae-path`: Path to a custom VAE model or HuggingFace model ID (e.g., fal/FLUX.2-Tiny-AutoEncoder). If not specified, the VAE will be loaded from the main model path. - `--num-gpus {NUM_GPUS}`: Number of GPUs to use @@ -53,7 +53,7 @@ Current supported optimization all listed [here](https://github.com/sgl-project/ ### 4.1 Basic Usage For more API usage and request examples, please refer to: -[SGLang Diffusion OpenAI API](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/docs/openai_api.md) +[SGLang Diffusion OpenAI API](/docs/sglang-diffusion/api/openai_api) #### 4.1.1 Launch a server and then send requests @@ -100,7 +100,7 @@ sglang generate "${SERVER_ARGS[@]}" "${SAMPLING_ARGS[@]}" #### 4.2.1 Cache-DiT Acceleration -SGLang integrates [Cache-DiT](https://github.com/vipshop/cache-dit), a caching acceleration engine for Diffusion Transformers (DiT), to achieve up to 7.4x inference speedup with minimal quality loss. You can set `SGLANG_CACHE_DIT_ENABLED=True` to enable it. For more details, please refer to the SGLang Cache-DiT [documentation](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/docs/cache/cache_dit.md). +SGLang integrates [Cache-DiT](https://github.com/vipshop/cache-dit), a caching acceleration engine for Diffusion Transformers (DiT), to achieve up to 7.4x inference speedup with minimal quality loss. You can set `SGLANG_CACHE_DIT_ENABLED=True` to enable it. For more details, please refer to the SGLang Cache-DiT [documentation](/docs/sglang-diffusion/cache_dit). **Basic Usage** diff --git a/docs_new/cookbook/diffusion/Z-Image/Z-Image-Turbo.mdx b/docs_new/cookbook/diffusion/Z-Image/Z-Image-Turbo.mdx index b694ba25f..e336c9330 100644 --- a/docs_new/cookbook/diffusion/Z-Image/Z-Image-Turbo.mdx +++ b/docs_new/cookbook/diffusion/Z-Image/Z-Image-Turbo.mdx @@ -42,7 +42,7 @@ Z-Image-Turbo is optimized for high-quality image generation with only 8 inferen ### 3.2 Configuration Tips -Current supported optimization all listed [here](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/docs/support_matrix.md). +Currently supported optimizations are listed [here](/docs/sglang-diffusion/compatibility_matrix). - `--vae-path`: Path to a custom VAE model or HuggingFace model ID (e.g., fal/FLUX.2-Tiny-AutoEncoder). If not specified, the VAE will be loaded from the main model path. - `--num-gpus`: Number of GPUs to use @@ -55,7 +55,7 @@ Current supported optimization all listed [here](https://github.com/sgl-project/ ## 4. API Usage -For complete API documentation, please refer to the [official API usage guide](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/docs/openai_api.md). +For complete API documentation, please refer to the [official API usage guide](/docs/sglang-diffusion/api/openai_api). ### 4.1 Generate an Image @@ -82,7 +82,7 @@ with open("output.png", "wb") as f: #### 4.2.1 Cache-DiT Acceleration -SGLang integrates [Cache-DiT](https://github.com/vipshop/cache-dit), a caching acceleration engine for Diffusion Transformers (DiT), to achieve up to 7.4x inference speedup with minimal quality loss. You can set `SGLANG_CACHE_DIT_ENABLED=True` to enable it. For more details, please refer to the SGLang Cache-DiT [documentation](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/docs/cache_dit.md). +SGLang integrates [Cache-DiT](https://github.com/vipshop/cache-dit), a caching acceleration engine for Diffusion Transformers (DiT), to achieve up to 7.4x inference speedup with minimal quality loss. You can set `SGLANG_CACHE_DIT_ENABLED=True` to enable it. For more details, please refer to the SGLang Cache-DiT [documentation](/docs/sglang-diffusion/cache_dit). **Basic Usage** diff --git a/docs_new/cookbook/diffusion/intro.mdx b/docs_new/cookbook/diffusion/intro.mdx index 42c6d3fb2..35dbf76a8 100644 --- a/docs_new/cookbook/diffusion/intro.mdx +++ b/docs_new/cookbook/diffusion/intro.mdx @@ -79,7 +79,7 @@ Realtime models keep a session alive and generate chunk by chunk with causal sta +Cache-DiT currently cannot be combined with `--use-fsdp-inference`. Keep FSDP disabled when enabling Cache-DiT, or use other residency/offload controls instead. + ### Quick Start @@ -65,7 +69,7 @@ sglang generate --model-path Qwen/Qwen-Image \ TeaCache (Temporal similarity-based caching) accelerates diffusion inference by detecting when consecutive denoising steps are similar enough to skip computation entirely. -See [teacache.md](./teacache) for detailed documentation. +See [TeaCache](./teacache) for detailed documentation. ### Quick Overview diff --git a/python/sglang/multimodal_gen/README.md b/python/sglang/multimodal_gen/README.md index 3a87a8573..682739dca 100644 --- a/python/sglang/multimodal_gen/README.md +++ b/python/sglang/multimodal_gen/README.md @@ -21,15 +21,15 @@ SGLang Diffusion has the following features: ### AMD/ROCm Support -SGLang Diffusion supports AMD Instinct GPUs through ROCm. On AMD platforms, we use the Triton attention backend and leverage AITER kernels for optimized layernorm and other operations. See the [installation guide](https://github.com/sgl-project/sglang/tree/main/docs/diffusion/installation.md) for setup instructions. +SGLang Diffusion supports AMD Instinct GPUs through ROCm. On AMD platforms, we use the Triton attention backend and leverage AITER kernels for optimized layernorm and other operations. See the [installation guide](https://docs.sglang.io/docs/sglang-diffusion/installation) for setup instructions. ### Moore Threads/MUSA Support -SGLang Diffusion supports Moore Threads GPUs (MTGPU) through the MUSA software stack. On MUSA platforms, we use FlashAttention (FA3) when available; also supports Sage Attention when installed; otherwise falls back to the Torch SDPA backend. See the [installation guide](https://github.com/sgl-project/sglang/tree/main/docs/diffusion/installation.md) for setup instructions. +SGLang Diffusion supports Moore Threads GPUs (MTGPU) through the MUSA software stack. On MUSA platforms, we use FlashAttention (FA3) when available; also supports Sage Attention when installed; otherwise falls back to the Torch SDPA backend. See the [installation guide](https://docs.sglang.io/docs/sglang-diffusion/installation) for setup instructions. ### Apple MPS Support -SGLang Diffusion supports Apple Silicon (M-series) via the MPS backend. Since Triton is Linux-only, all Triton kernels are replaced with PyTorch-native fallbacks on MPS. Norm operations can be optionally accelerated with MLX fused Metal kernels (`SGLANG_USE_MLX=1`). See the [installation guide](https://github.com/sgl-project/sglang/tree/main/docs/diffusion/installation.md) for setup instructions. +SGLang Diffusion supports Apple Silicon (M-series) via the MPS backend. Since Triton is Linux-only, all Triton kernels are replaced with PyTorch-native fallbacks on MPS. Norm operations can be optionally accelerated with MLX fused Metal kernels (`SGLANG_USE_MLX=1`). See the [installation guide](https://docs.sglang.io/docs/sglang-diffusion/installation) for setup instructions. ## Getting Started @@ -37,7 +37,7 @@ SGLang Diffusion supports Apple Silicon (M-series) via the MPS backend. Since Tr uv pip install 'sglang[diffusion]' --prerelease=allow ``` -For more installation methods (e.g. pypi, uv, docker, ROCm/AMD, MUSA/Moore Threads), check [install.md](https://github.com/sgl-project/sglang/tree/main/docs/diffusion/installation.md). +For more installation methods (e.g. pypi, uv, docker, ROCm/AMD, MUSA/Moore Threads), check the [installation guide](https://docs.sglang.io/docs/sglang-diffusion/installation). ## Inference @@ -89,11 +89,11 @@ sglang generate \ --save-output ``` -For more usage examples (e.g. OpenAI compatible API, server mode), check [cli.md](https://github.com/sgl-project/sglang/tree/main/docs/diffusion/api/cli.md). +For more usage examples (e.g. OpenAI compatible API, server mode), check the [CLI reference](https://docs.sglang.io/docs/sglang-diffusion/api/cli). ## Contributing -All contributions are welcome. The contribution guide is available [here](https://github.com/sgl-project/sglang/tree/main/docs/diffusion/contributing.md). +All contributions are welcome. The contribution guide is available [here](https://docs.sglang.io/docs/sglang-diffusion/contributing). ## Acknowledgement