[diffusion] docs: consolidate diffusion documentation into docs (#18095)

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: JiaxinD <djx2048@gmail.com>

docs/advanced_features/server_arguments.md

@@ -373,6 +373,7 @@ Please consult the documentation below and [server_args.py](https://github.com/s
 | `--kt-max-deferred-experts-per-token` | [ktransformers parameter] Maximum number of experts deferred to CPU per token. All MoE layers except the final one use this value; the final layer always uses 0. | `None` | Type: int |
 
 ## Diffusion LLM
+
 | Argument | Description | Defaults | Options |
 | --- | --- | --- | --- |
 | `--dllm-algorithm` | The diffusion LLM algorithm, such as LowConfidence. | `None` | Type: str |

docs/basic_usage/diffusion.md

@@ -4,7 +4,7 @@ SGLang supports two categories of diffusion models for different use cases. This
 
 ## Image & Video Generation Models
 
-For generating images and videos from text prompts, SGLang supports [many](../supported_models/image_generation/diffusion_models.md#image-generation-models) models like:
+For generating images and videos from text prompts, SGLang supports [many](../diffusion/compatibility_matrix.md) models like:
 
 - **FLUX, Qwen-Image** - High-quality image generation
 - **Wan 2.2, HunyuanVideo** - Video generation
@@ -16,4 +16,4 @@ python3 -m sglang.launch_server \
   --host 0.0.0.0 --port 30000
 ```
 
-**Full model list:** [Diffusion Models](../supported_models/image_generation/diffusion_models.md)
+**Full model list:** [Diffusion Models](../diffusion/compatibility_matrix.md)

docs/diffusion/api/cli.md

@@ -5,7 +5,6 @@ The SGLang-diffusion CLI provides a quick way to access the inference pipeline f
 ## Prerequisites
 
 - A working SGLang diffusion installation and the `sglang` CLI available in `$PATH`.
-- Python 3.11+ if you plan to use the OpenAI Python SDK.
 
 
 ## Supported Arguments
@@ -35,7 +34,7 @@ The SGLang-diffusion CLI provides a quick way to access the inference pipeline f
 - `--seed {SEED}`: Random seed for reproducible generation
 
 
-#### Image/Video Configuration
+**Image/Video Configuration**
 
 - `--height {HEIGHT}`: Height of the generated output
 - `--width {WIDTH}`: Width of the generated output
@@ -43,7 +42,7 @@ The SGLang-diffusion CLI provides a quick way to access the inference pipeline f
 - `--fps {FPS}`: Frames per second for the saved output, if this is a video-generation task
 
 
-#### Output Options
+**Output Options**
 
 - `--output-path {PATH}`: Directory to save the generated video
 - `--save-output`: Whether to save the image/video to disk
@@ -168,7 +167,7 @@ When enabled, the server follows a **Generate -> Upload -> Delete** workflow:
 3. Upon successful upload, the local file is deleted.
 4. The API response returns the public URL of the uploaded object.
 
-#### Configuration
+**Configuration**
 
 Cloud storage is enabled via environment variables. Note that `boto3` must be installed separately (`pip install boto3`) to use this feature.
 
@@ -183,7 +182,7 @@ export SGLANG_S3_SECRET_ACCESS_KEY=your-secret-key
 export SGLANG_S3_ENDPOINT_URL=https://minio.example.com
 ```
 
-See [Environment Variables Documentation](environment_variables.md) for more details.
+See [Environment Variables Documentation](../environment_variables.md) for more details.
 
 ## Generate
 

docs/diffusion/api/openai_api.md

@@ -2,6 +2,10 @@
 
 The SGLang diffusion HTTP server implements an OpenAI-compatible API for image and video generation, as well as LoRA adapter management.
 
+## Prerequisites
+
+- Python 3.11+ if you plan to use the OpenAI Python SDK.
+
 ## Serve
 
 Launch the server using the `sglang serve` command.
@@ -25,7 +29,7 @@ sglang serve "${SERVER_ARGS[@]}"
 - **--model-path**: Path to the model or model ID.
 - **--port**: HTTP port to listen on (default: `30000`).
 
-#### Get Model Information
+**Get Model Information**
 
 **Endpoint:** `GET /models`
 
@@ -59,7 +63,7 @@ curl -sS -X GET "http://localhost:30010/models"
 
 The server implements an OpenAI-compatible Images API under the `/v1/images` namespace.
 
-#### Create an image
+**Create an image**
 
 **Endpoint:** `POST /v1/images/generations`
 
@@ -100,7 +104,7 @@ curl -sS -X POST "http://localhost:30010/v1/images/generations" \
 > **Note**
 > The `response_format=url` option is not supported for `POST /v1/images/generations` and will return a `400` error.
 
-#### Edit an image
+**Edit an image**
 
 **Endpoint:** `POST /v1/images/edits`
 
@@ -130,7 +134,7 @@ curl -sS -X POST "http://localhost:30010/v1/images/edits" \
   -F "response_format=url"
 ```
 
-#### Download image content
+**Download image content**
 
 When `response_format=url` is used with `POST /v1/images/edits`, the API returns a relative URL like `/v1/images/<IMAGE_ID>/content`.
 
@@ -148,7 +152,7 @@ curl -sS -L "http://localhost:30010/v1/images/<IMAGE_ID>/content" \
 
 The server implements a subset of the OpenAI Videos API under the `/v1/videos` namespace.
 
-#### Create a video
+**Create a video**
 
 **Endpoint:** `POST /v1/videos`
 
@@ -178,7 +182,7 @@ curl -sS -X POST "http://localhost:30010/v1/videos" \
       }'
 ```
 
-#### List videos
+**List videos**
 
 **Endpoint:** `GET /v1/videos`
 
@@ -197,7 +201,7 @@ curl -sS -X GET "http://localhost:30010/v1/videos" \
   -H "Authorization: Bearer sk-proj-1234567890"
 ```
 
-#### Download video content
+**Download video content**
 
 **Endpoint:** `GET /v1/videos/{video_id}/content`
 
@@ -239,7 +243,7 @@ The server supports dynamic loading, merging, and unmerging of LoRA adapters.
 - Switching: To switch LoRAs, you must first `unmerge` the current one, then `set` the new one
 - Caching: The server caches loaded LoRA weights in memory. Switching back to a previously loaded LoRA (same path) has little cost
 
-#### Set LoRA Adapter
+**Set LoRA Adapter**
 
 Loads one or more LoRA adapters and merges their weights into the model. Supports both single LoRA (backward compatible) and multiple LoRA adapters.
 
@@ -301,7 +305,7 @@ curl -X POST http://localhost:30010/v1/set_lora \
 > - Multiple LoRAs applied to the same target will be merged in order
 
 
-#### Merge LoRA Weights
+**Merge LoRA Weights**
 
 Manually merges the currently set LoRA weights into the base model.
 
@@ -323,7 +327,7 @@ curl -X POST http://localhost:30010/v1/merge_lora_weights \
 ```
 
 
-#### Unmerge LoRA Weights
+**Unmerge LoRA Weights**
 
 Unmerges the currently active LoRA weights from the base model, restoring it to its original state. This **must** be called before setting a different LoRA.
 
@@ -336,7 +340,7 @@ curl -X POST http://localhost:30010/v1/unmerge_lora_weights \
   -H "Content-Type: application/json"
 ```
 
-#### List LoRA Adapters
+**List LoRA Adapters**
 
 Returns loaded LoRA adapters and current application status per module.
 

docs/diffusion/ci_perf.md

@@ -1,5 +1,4 @@
-
+## Perf Baseline Generation Script
-## Perf baseline generation script
 
 `python/sglang/multimodal_gen/test/scripts/gen_perf_baselines.py` starts a local diffusion server, issues requests for selected test cases, aggregates stage/denoise-step/E2E timings from the perf log, and writes the results back to the `scenarios` section of `perf_baselines.json`.
 

docs/diffusion/compatibility_matrix.md

@@ -16,7 +16,7 @@ default parameters when initializing and generating videos.
 
 ### Video Generation Models
 
-| Model Name                   | Hugging Face Model ID                             | Resolutions         | TeaCache | Sliding Tile Attn | Sage Attn | Video Sparse Attention (VSA) | Sparse Linear Attention（SLA）| Sage Sparse Linear Attention（SageSLA）|
+| Model Name                   | Hugging Face Model ID                             | Resolutions         | TeaCache | Sliding Tile Attn | Sage Attn | Video Sparse Attention (VSA) | Sparse Linear Attention (SLA) | Sage Sparse Linear Attention (SageSLA) |
 |:-----------------------------|:--------------------------------------------------|:--------------------|:--------:|:-----------------:|:---------:|:----------------------------:|:----------------------------:|:-----------------------------------------------:|
 | FastWan2.1 T2V 1.3B          | `FastVideo/FastWan2.1-T2V-1.3B-Diffusers`         | 480p                |    ⭕     |         ⭕         |     ⭕     |              ✅               |              ❌               |              ❌               |
 | FastWan2.2 TI2V 5B Full Attn | `FastVideo/FastWan2.2-TI2V-5B-FullAttn-Diffusers` | 720p                |    ⭕     |         ⭕         |     ⭕     |              ✅               |              ❌               |              ❌               |
@@ -34,8 +34,8 @@ default parameters when initializing and generating videos.
 | TurboWan2.1 T2V 14B 720P     | `IPostYellow/TurboWan2.1-T2V-14B-720P-Diffusers`  | 720p                |    ✅     |         ❌         |     ❌     |              ❌               |              ✅               |              ✅               |
 | TurboWan2.2 I2V A14B         | `IPostYellow/TurboWan2.2-I2V-A14B-Diffusers`      | 720p                |    ✅     |         ❌         |     ❌     |              ❌               |              ✅               |              ✅               |
 
-**Note**: <br>
+**Note**:
-1.Wan2.2 TI2V 5B has some quality issues when performing I2V generation. We are working on fixing this issue.<br>
+1.Wan2.2 TI2V 5B has some quality issues when performing I2V generation. We are working on fixing this issue.
 2.SageSLA Based on SpargeAttn. Install it first with `pip install git+https://github.com/thu-ml/SpargeAttn.git --no-build-isolation`
 
 ### Image Generation Models
@@ -55,7 +55,7 @@ default parameters when initializing and generating videos.
 
 This section lists example LoRAs that have been explicitly tested and verified with each base model in the **SGLang Diffusion** pipeline.
 
-> Important: \
+> Important:
 > LoRAs that are not listed here are not necessarily incompatible.
 > In practice, most standard LoRAs are expected to work, especially those following common Diffusers or SD-style conventions.
 > The entries below simply reflect configurations that have been manually validated by the SGLang team.

docs/diffusion/contributing.md

@@ -2,7 +2,7 @@
 
 This guide outlines the requirements for contributing to the SGLang Diffusion module (`sglang.multimodal_gen`).
 
-## 1. Commit Message Convention
+## Commit Message Convention
 
 We follow a structured commit message format to maintain a clean history.
 
@@ -21,7 +21,7 @@ We follow a structured commit message format to maintain a clean history.
 - **Scope** (Optional): `cli`, `scheduler`, `model`, `pipeline`, `docs`, etc.
 - **Subject**: Imperative mood, short and clear (e.g., "add feature" not "added feature").
 
-## 2. Performance Reporting
+## Performance Reporting
 
 For PRs that impact **latency**, **throughput**, or **memory usage**, you **should** provide a performance comparison report.
 
@@ -45,7 +45,7 @@ For PRs that impact **latency**, **throughput**, or **memory usage**, you **shou
     ```
 4. **Paste**: paste the table into the PR description
 
-## 3. CI-Based Change Protection
+## CI-Based Change Protection
 
 Consider adding tests to the `pr-test` or `nightly-test` suites to safeguard your changes, especially for PRs that:
 

docs/diffusion/environment_variables.md

@@ -1,11 +1,11 @@
 ## Caching Acceleration
 
 These variables configure caching acceleration for Diffusion Transformer (DiT) models.
-SGLang supports multiple caching strategies - see [caching documentation](cache/caching.md) for an overview.
+SGLang supports multiple caching strategies - see [caching documentation](performance/cache/index.md) for an overview.
 
 ### Cache-DiT Configuration
 
-See [cache-dit documentation](cache/cache_dit.md) for detailed configuration.
+See [cache-dit documentation](performance/cache/cache_dit.md) for detailed configuration.
 
 | Environment Variable                | Default | Description                              |
 |-------------------------------------|---------|------------------------------------------|

docs/diffusion/index.md

@@ -0,0 +1,98 @@
+# SGLang Diffusion
+
+SGLang Diffusion is an inference framework for accelerated image and video generation using diffusion models. It provides an end-to-end unified pipeline with optimized kernels and an efficient scheduler loop.
+
+## Key Features
+
+- **Broad Model Support**: Wan series, FastWan series, Hunyuan, Qwen-Image, Qwen-Image-Edit, Flux, Z-Image, GLM-Image, and more
+- **Fast Inference**: Optimized kernels, efficient scheduler loop, and Cache-DiT acceleration
+- **Ease of Use**: OpenAI-compatible API, CLI, and Python SDK
+- **Multi-Platform**: NVIDIA GPUs (H100, H200, A100, B200, 4090) and AMD GPUs (MI300X, MI325X)
+
+---
+
+## Quick Start
+
+### Installation
+
+```bash
+uv pip install "sglang[diffusion]" --prerelease=allow
+```
+
+See [Installation Guide](installation.md) for more installation methods and ROCm-specific instructions.
+
+### Basic Usage
+
+Generate an image with the CLI:
+
+```bash
+sglang generate --model-path Qwen/Qwen-Image \
+    --prompt "A beautiful sunset over the mountains" \
+    --save-output
+```
+
+Or start a server with the OpenAI-compatible API:
+
+```bash
+sglang serve --model-path Qwen/Qwen-Image --port 30010
+```
+
+---
+
+## Documentation
+
+### Getting Started
+
+- **[Installation](installation.md)** - Install SGLang Diffusion via pip, uv, Docker, or from source
+- **[Compatibility Matrix](compatibility_matrix.md)** - Supported models and optimization compatibility
+
+### Usage
+
+- **[CLI Documentation](api/cli.md)** - Command-line interface for `sglang generate` and `sglang serve`
+- **[OpenAI API](api/openai_api.md)** - OpenAI-compatible API for image/video generation and LoRA management
+
+### Performance Optimization
+
+- **[Performance Overview](performance/index.md)** - Overview of all performance optimization strategies
+- **[Attention Backends](performance/attention_backends.md)** - Available attention backends (FlashAttention, SageAttention, etc.)
+- **[Caching Strategies](performance/cache/)** - Cache-DiT and TeaCache acceleration
+- **[Profiling](performance/profiling.md)** - Profiling techniques with PyTorch Profiler and Nsight Systems
+
+### Reference
+
+- **[Environment Variables](environment_variables.md)** - Configuration via environment variables
+- **[Support New Models](support_new_models.md)** - Guide for adding new diffusion models
+- **[Contributing](contributing.md)** - Contribution guidelines and commit message conventions
+- **[CI Performance](ci_perf.md)** - Performance baseline generation script
+
+---
+
+## CLI Quick Reference
+
+### Generate (one-off generation)
+
+```bash
+sglang generate --model-path <MODEL> --prompt "<PROMPT>" --save-output
+```
+
+### Serve (HTTP server)
+
+```bash
+sglang serve --model-path <MODEL> --port 30010
+```
+
+### Enable Cache-DiT acceleration
+
+```bash
+SGLANG_CACHE_DIT_ENABLED=true sglang generate --model-path <MODEL> --prompt "<PROMPT>"
+```
+
+---
+
+## References
+
+- [SGLang GitHub](https://github.com/sgl-project/sglang)
+- [Cache-DiT](https://github.com/vipshop/cache-dit)
+- [FastVideo](https://github.com/hao-ai-lab/FastVideo)
+- [xDiT](https://github.com/xdit-project/xDiT)
+- [Diffusers](https://github.com/huggingface/diffusers)

docs/diffusion/installation.md

@@ -0,0 +1,91 @@
+# Install SGLang-Diffusion
+
+You can install SGLang-Diffusion using one of the methods below.
+
+## Standard Installation (NVIDIA GPUs)
+
+### Method 1: With pip or uv
+
+It is recommended to use uv for a faster installation:
+
+```bash
+pip install --upgrade pip
+pip install uv
+uv pip install "sglang[diffusion]" --prerelease=allow
+```
+
+### Method 2: From source
+
+```bash
+# Use the latest release branch
+git clone https://github.com/sgl-project/sglang.git
+cd sglang
+
+# Install the Python packages
+pip install --upgrade pip
+pip install -e "python[diffusion]"
+
+# With uv
+uv pip install -e "python[diffusion]" --prerelease=allow
+```
+
+### Method 3: Using Docker
+
+The Docker images are available on Docker Hub at [lmsysorg/sglang](https://hub.docker.com/r/lmsysorg/sglang), built from the [Dockerfile](https://github.com/sgl-project/sglang/blob/main/docker/Dockerfile).
+Replace `<secret>` below with your HuggingFace Hub [token](https://huggingface.co/docs/hub/en/security-tokens).
+
+```bash
+docker run --gpus all \
+    --shm-size 32g \
+    -p 30000:30000 \
+    -v ~/.cache/huggingface:/root/.cache/huggingface \
+    --env "HF_TOKEN=<secret>" \
+    --ipc=host \
+    lmsysorg/sglang:dev \
+    zsh -c '\
+        echo "Installing diffusion dependencies..." && \
+        pip install -e "python[diffusion]" && \
+        echo "Starting SGLang-Diffusion..." && \
+        sglang generate \
+            --model-path black-forest-labs/FLUX.1-dev \
+            --prompt "A logo With Bold Large text: SGL Diffusion" \
+            --save-output \
+    '
+```
+
+## Platform-Specific: ROCm (AMD GPUs)
+
+For AMD Instinct GPUs (e.g., MI300X), you can use the ROCm-enabled Docker image:
+
+```bash
+docker run --device=/dev/kfd --device=/dev/dri --ipc=host \
+  -v ~/.cache/huggingface:/root/.cache/huggingface \
+  --env HF_TOKEN=<secret> \
+  lmsysorg/sglang:v0.5.5.post2-rocm700-mi30x \
+  sglang generate --model-path black-forest-labs/FLUX.1-dev --prompt "A logo With Bold Large text: SGL Diffusion" --save-output
+```
+
+For detailed ROCm system configuration and installation from source, see [AMD GPUs](../../platforms/amd_gpu.md).
+
+## Platform-Specific: MUSA (Moore Threads GPUs)
+
+For Moore Threads GPUs (MTGPU) with the MUSA software stack:
+
+```bash
+# Clone the repository
+git clone https://github.com/sgl-project/sglang.git
+cd sglang
+
+# Install the Python packages
+pip install --upgrade pip
+rm -f python/pyproject.toml && mv python/pyproject_other.toml python/pyproject.toml
+pip install -e "python[all_musa]"
+```
+
+Quick test:
+
+```bash
+sglang generate --model-path black-forest-labs/FLUX.1-dev \
+    --prompt "A logo With Bold Large text: SGL Diffusion" \
+    --save-output
+```

docs/diffusion/performance/attention_backends.md

@@ -47,7 +47,7 @@ Some backends require additional configuration. You can pass these parameters vi
 
 ### Supported Configuration Parameters
 
-#### Sliding Tile Attention (`sliding_tile_attn`)
+**Sliding Tile Attention (`sliding_tile_attn`)**
 
 | Parameter | Type | Description | Default |
 | :--- | :--- | :--- | :--- |
@@ -55,13 +55,13 @@ Some backends require additional configuration. You can pass these parameters vi
 | `sta_mode` | `str` | Mode of STA. | `STA_inference` |
 | `skip_time_steps` | `int` | Number of steps to use full attention before switching to sparse attention. | `15` |
 
-#### Video Sparse Attention (`video_sparse_attn`)
+**Video Sparse Attention (`video_sparse_attn`)**
 
 | Parameter | Type | Description | Default |
 | :--- | :--- | :--- | :--- |
 | `sparsity` | `float` | Validation sparsity (0.0 - 1.0). | `0.0` |
 
-#### V-MoBA (`vmoba_attn`)
+**V-MoBA (`vmoba_attn`)**
 
 | Parameter | Type | Description | Default |
 | :--- | :--- | :--- | :--- |

docs/diffusion/performance/cache/cache_dit.md

@@ -1,9 +1,5 @@
 # Cache-DiT Acceleration
 
-> **Note**: This is one of two caching strategies available in SGLang.
-> For an overview of all caching options, see [caching.md](caching.md).
-> For TeaCache documentation, see [teacache.md](teacache.md).
-
 SGLang integrates [Cache-DiT](https://github.com/vipshop/cache-dit), a caching acceleration engine for Diffusion Transformers (DiT), to achieve up to **1.69x inference speedup** with minimal quality loss.
 
 ## Overview
@@ -136,7 +132,7 @@ sglang generate --model-path black-forest-labs/FLUX.1-dev \
 SCM provides step-level caching control for additional speedup. It decides which denoising steps to compute fully and
 which to use cached results.
 
-#### SCM Presets
+**SCM Presets**
 
 SCM is configured with presets:
 
@@ -148,7 +144,7 @@ SCM is configured with presets:
 | `fast`   | ~35%          | ~3x      | Acceptable |
 | `ultra`  | ~25%          | ~4x      | Lower      |
 
-##### Usage
+**Usage**
 
 ```bash
 SGLANG_CACHE_DIT_ENABLED=true \
@@ -157,7 +153,7 @@ sglang generate --model-path Qwen/Qwen-Image \
     --prompt "A futuristic cityscape at sunset"
 ```
 
-#### Custom SCM Bins
+**Custom SCM Bins**
 
 For fine-grained control over which steps to compute vs cache:
 
@@ -169,7 +165,7 @@ sglang generate --model-path Qwen/Qwen-Image \
     --prompt "A futuristic cityscape at sunset"
 ```
 
-#### SCM Policy
+**SCM Policy**
 
 | Policy    | Env Variable                          | Description                                 |
 |-----------|---------------------------------------|---------------------------------------------|
@@ -178,22 +174,8 @@ sglang generate --model-path Qwen/Qwen-Image \
 
 ## Environment Variables
 
-All Cache-DiT parameters can be set via the following environment variables:
+All Cache-DiT parameters can be configured via environment variables.
-
+See [Environment Variables](../../environment_variables.md) for the complete list.
-| Environment Variable                | Default | Description                              |
-|-------------------------------------|---------|------------------------------------------|
-| `SGLANG_CACHE_DIT_ENABLED`          | false   | Enable Cache-DiT acceleration            |
-| `SGLANG_CACHE_DIT_FN`               | 1       | First N blocks to always compute         |
-| `SGLANG_CACHE_DIT_BN`               | 0       | Last N blocks to always compute          |
-| `SGLANG_CACHE_DIT_WARMUP`           | 4       | Warmup steps before caching              |
-| `SGLANG_CACHE_DIT_RDT`              | 0.24    | Residual difference threshold            |
-| `SGLANG_CACHE_DIT_MC`               | 3       | Max continuous cached steps              |
-| `SGLANG_CACHE_DIT_TAYLORSEER`       | false   | Enable TaylorSeer calibrator             |
-| `SGLANG_CACHE_DIT_TS_ORDER`         | 1       | TaylorSeer order (1 or 2)                |
-| `SGLANG_CACHE_DIT_SCM_PRESET`       | none    | SCM preset (none/slow/medium/fast/ultra) |
-| `SGLANG_CACHE_DIT_SCM_POLICY`       | dynamic | SCM caching policy                       |
-| `SGLANG_CACHE_DIT_SCM_COMPUTE_BINS` | not set | Custom SCM compute bins                  |
-| `SGLANG_CACHE_DIT_SCM_CACHE_BINS`   | not set | Custom SCM cache bins                    |
 
 ## Supported Models
 
@@ -240,4 +222,4 @@ acceleration still works.
 ## References
 
 - [Cache-Dit](https://github.com/vipshop/cache-dit)
-- [SGLang Diffusion](../README.md)
+- [SGLang Diffusion](../index.md)

docs/diffusion/performance/cache/teacache.md

@@ -1,7 +1,7 @@
 # TeaCache Acceleration
 
 > **Note**: This is one of two caching strategies available in SGLang.
-> For an overview of all caching options, see [caching.md](caching.md).
+> For an overview of all caching options, see [caching](../index.md).
 
 TeaCache (Temporal similarity-based caching) accelerates diffusion inference by detecting when consecutive denoising steps are similar enough to skip computation entirely.
 

docs/diffusion/performance/index.md

@@ -0,0 +1,72 @@
+# Performance Optimization
+
+SGLang-Diffusion provides multiple performance optimization strategies to accelerate inference. This section covers all available performance tuning options.
+
+## Overview
+
+| Optimization | Type | Description |
+|--------------|------|-------------|
+| **Cache-DiT** | Caching | Block-level caching with DBCache, TaylorSeer, and SCM |
+| **TeaCache** | Caching | Timestep-level caching using L1 similarity |
+| **Attention Backends** | Kernel | Optimized attention implementations (FlashAttention, SageAttention, etc.) |
+| **Profiling** | Diagnostics | PyTorch Profiler and Nsight Systems guidance |
+
+## Caching Strategies
+
+SGLang supports two complementary caching approaches:
+
+### Cache-DiT
+
+[Cache-DiT](https://github.com/vipshop/cache-dit) provides block-level caching with advanced strategies. It can achieve up to **1.69x speedup**.
+
+**Quick Start:**
+```bash
+SGLANG_CACHE_DIT_ENABLED=true \
+sglang generate --model-path Qwen/Qwen-Image \
+    --prompt "A beautiful sunset over the mountains"
+```
+
+**Key Features:**
+- **DBCache**: Dynamic block-level caching based on residual differences
+- **TaylorSeer**: Taylor expansion-based calibration for optimized caching
+- **SCM**: Step-level computation masking for additional speedup
+
+See [Cache-DiT Documentation](cache/cache_dit.md) for detailed configuration.
+
+### TeaCache
+
+TeaCache (Temporal similarity-based caching) accelerates diffusion inference by detecting when consecutive denoising steps are similar enough to skip computation entirely.
+
+**Quick Overview:**
+- Tracks L1 distance between modulated inputs across timesteps
+- When accumulated distance is below threshold, reuses cached residual
+- Supports CFG with separate positive/negative caches
+
+**Supported Models:** Wan (wan2.1, wan2.2), Hunyuan (HunyuanVideo), Z-Image
+
+See [TeaCache Documentation](cache/teacache.md) for detailed configuration.
+
+## Attention Backends
+
+Different attention backends offer varying performance characteristics depending on your hardware and model:
+
+- **FlashAttention**: Fastest on NVIDIA GPUs with fp16/bf16
+- **SageAttention**: Alternative optimized implementation
+- **xformers**: Memory-efficient attention
+- **SDPA**: PyTorch native scaled dot-product attention
+
+See [Attention Backends](attention_backends.md) for platform support and configuration options.
+
+## Profiling
+
+To diagnose performance bottlenecks, SGLang-Diffusion supports profiling tools:
+
+- **PyTorch Profiler**: Built-in Python profiling
+- **Nsight Systems**: GPU kernel-level analysis
+
+See [Profiling Guide](profiling.md) for detailed instructions.
+
+## References
+
+- [Cache-DiT Repository](https://github.com/vipshop/cache-dit)
+- [TeaCache Paper](https://arxiv.org/abs/2411.14324)

docs/diffusion/support_new_models.md

@@ -23,7 +23,7 @@ To add support for a new diffusion model, you will primarily need to define or c
 
 3.  **`ComposedPipeline` (not a config)**: This is the central class where you define the structure of your model's generation pipeline. You will create a new class that inherits from `ComposedPipelineBase` and, within it, instantiate and chain together the necessary `PipelineStage`s in the correct order. See `ComposedPipelineBase` and `PipelineStage` base definitions:
     - [`ComposedPipelineBase`](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/runtime/pipelines/composed_pipeline_base.py)
-    - [`PipelineStage`]( https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/runtime/pipelines/stages/base.py)
+    - [`PipelineStage`](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/runtime/pipelines/stages/base.py)
     - [Central registry (models/config mapping)](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/registry.py)
 
 4.  **Modules (components referenced by the pipeline)**: Each pipeline references a set of modules that are loaded from the model repository (e.g., Diffusers `model_index.json`) and assembled via the registry/loader. Common modules include:
@@ -37,7 +37,7 @@ To add support for a new diffusion model, you will primarily need to define or c
 
 ## Available Pipeline Stages
 
-You can build your custom `ComposedPipeline` by combining the following available stages as your will. Each stage is responsible for a specific part of the generation process.
+You can build your custom `ComposedPipeline` by combining the following available stages as needed. Each stage is responsible for a specific part of the generation process.
 
 | Stage Class                      | Description                                                                                             |
 | -------------------------------- | ------------------------------------------------------------------------------------------------------- |

docs/index.rst

@@ -74,11 +74,30 @@ Its core features include:
    :caption: Supported Models
 
    supported_models/text_generation/index
-   supported_models/image_generation/index
    supported_models/retrieval_ranking/index
    supported_models/specialized/index
    supported_models/extending/index
 
+.. toctree::
+   :maxdepth: 2
+   :caption: SGLang Diffusion
+
+   diffusion/index
+   diffusion/installation
+   diffusion/compatibility_matrix
+   diffusion/api/cli
+   diffusion/api/openai_api
+   diffusion/performance/index
+   diffusion/performance/attention_backends
+   diffusion/performance/profiling
+   diffusion/performance/cache/index
+   diffusion/performance/cache/cache_dit
+   diffusion/performance/cache/teacache
+   diffusion/support_new_models
+   diffusion/contributing
+   diffusion/ci_perf
+   diffusion/environment_variables
+
 .. toctree::
    :maxdepth: 1
    :caption: Hardware Platforms

docs/references/frontend/frontend_tutorial.ipynb

@@ -385,7 +385,7 @@
     "## Multi-modal Generation\n",
     "\n",
     "You may use SGLang frontend language to define multi-modal prompts.\n",
-    "See [here](https://docs.sglang.io/supported_models/text_generation/generative_models.html) for supported models."
+    "See [here](https://docs.sglang.io/supported_models/text_generation/multimodal_language_models.html) for supported models."
    ]
   },
   {

docs/supported_models/extending/index.rst

@@ -9,3 +9,4 @@ Adding new models and alternative backends.
    support_new_models.md
    transformers_fallback.md
    modelscope.md
+   mindspore_models.md

docs/supported_models/extending/mindspore_models.md

@@ -0,0 +1,151 @@
+# MindSpore Models
+
+## Introduction
+
+MindSpore is a high-performance AI framework optimized for Ascend NPUs. This doc guides users to run MindSpore models in SGLang.
+
+## Requirements
+
+MindSpore currently only supports Ascend NPU devices. Users need to first install CANN 8.5.
+The CANN software packages can be downloaded from the [Ascend Official Website](https://www.hiascend.com).
+
+## Supported Models
+
+Currently, the following models are supported:
+
+- **Qwen3**: Dense and MoE models
+- **DeepSeek V3/R1**
+- *More models coming soon...*
+
+## Installation
+
+> **Note**: Currently, MindSpore models are provided by an independent package `sgl-mindspore`. Support for MindSpore is built upon current SGLang support for Ascend NPU platform. Please first [install SGLang for Ascend NPU](../../platforms/ascend_npu.md) and then install `sgl-mindspore`:
+
+```shell
+git clone https://github.com/mindspore-lab/sgl-mindspore.git
+cd sgl-mindspore
+pip install -e .
+```
+
+
+## Run Model
+
+Current SGLang-MindSpore supports Qwen3 and DeepSeek V3/R1 models. This doc uses Qwen3-8B as an example.
+
+### Offline inference
+
+Use the following script for offline inference:
+
+```python
+import sglang as sgl
+
+# Initialize the engine with MindSpore backend
+llm = sgl.Engine(
+    model_path="/path/to/your/model",  # Local model path
+    device="npu",                      # Use NPU device
+    model_impl="mindspore",            # MindSpore implementation
+    attention_backend="ascend",        # Attention backend
+    tp_size=1,                         # Tensor parallelism size
+    dp_size=1                          # Data parallelism size
+)
+
+# Generate text
+prompts = [
+    "Hello, my name is",
+    "The capital of France is",
+    "The future of AI is"
+]
+
+sampling_params = {"temperature": 0, "top_p": 0.9}
+outputs = llm.generate(prompts, sampling_params)
+
+for prompt, output in zip(prompts, outputs):
+    print(f"Prompt: {prompt}")
+    print(f"Generated: {output['text']}")
+    print("---")
+```
+
+### Start server
+
+Launch a server with MindSpore backend:
+
+```bash
+# Basic server startup
+python3 -m sglang.launch_server \
+    --model-path /path/to/your/model \
+    --host 0.0.0.0 \
+    --device npu \
+    --model-impl mindspore \
+    --attention-backend ascend \
+    --tp-size 1 \
+    --dp-size 1
+```
+
+For distributed server with multiple nodes:
+
+```bash
+# Multi-node distributed server
+python3 -m sglang.launch_server \
+    --model-path /path/to/your/model \
+    --host 0.0.0.0 \
+    --device npu \
+    --model-impl mindspore \
+    --attention-backend ascend \
+    --dist-init-addr 127.0.0.1:29500 \
+    --nnodes 2 \
+    --node-rank 0 \
+    --tp-size 4 \
+    --dp-size 2
+```
+
+## Troubleshooting
+
+#### Debug Mode
+
+Enable sglang debug logging by log-level argument.
+
+```bash
+python3 -m sglang.launch_server \
+    --model-path /path/to/your/model \
+    --host 0.0.0.0 \
+    --device npu \
+    --model-impl mindspore \
+    --attention-backend ascend \
+    --log-level DEBUG
+```
+
+Enable mindspore info and debug logging by setting environments.
+
+```bash
+export GLOG_v=1  # INFO
+export GLOG_v=0  # DEBUG
+```
+
+#### Explicitly select devices
+
+Use the following environment variable to explicitly select the devices to use.
+
+```shell
+export ASCEND_RT_VISIBLE_DEVICES=4,5,6,7  # to set device
+```
+
+#### Some communication environment issues
+
+In case of some environment with special communication environment, users need set some environment variables.
+
+```shell
+export MS_ENABLE_LCCL=off # current not support LCCL communication mode in SGLang-MindSpore
+```
+
+#### Some dependencies of protobuf
+
+In case of some environment with special protobuf version, users need set some environment variables to avoid binary version mismatch.
+
+```shell
+export PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python  # to avoid protobuf binary version mismatch
+```
+
+## Support
+For MindSpore-specific issues:
+
+- Refer to the [MindSpore documentation](https://www.mindspore.cn/)

docs/supported_models/image_generation/index.rst

@@ -1,9 +0,0 @@
-Image Generation
-================
-
-Models for generating images and videos using diffusion.
-
-.. toctree::
-   :maxdepth: 1
-
-   diffusion_models.md

docs/supported_models/index.rst

@@ -8,7 +8,6 @@ Browse by category below to find models suited for your needs.
    :maxdepth: 2
 
    text_generation/index
-   image_generation/index
    retrieval_ranking/index
    specialized/index
    extending/index

docs/supported_models/specialized/reward_models.md

@@ -1,28 +1,28 @@
-# Reward Models
+# Reward Models
-
+
-These models output a scalar reward score or classification result, often used in reinforcement learning or content moderation tasks.
+These models output a scalar reward score or classification result, often used in reinforcement learning or content moderation tasks.
-
+
-```{important}
+```{important}
-They are executed with `--is-embedding` and some may require `--trust-remote-code`.
+They are executed with `--is-embedding` and some may require `--trust-remote-code`.
-```
+```
-
+
-## Example launch Command
+## Example launch Command
-
+
-```shell
+```shell
-python3 -m sglang.launch_server \
+python3 -m sglang.launch_server \
-  --model-path Qwen/Qwen2.5-Math-RM-72B \  # example HF/local path
+  --model-path Qwen/Qwen2.5-Math-RM-72B \  # example HF/local path
-  --is-embedding \
+  --is-embedding \
-  --host 0.0.0.0 \
+  --host 0.0.0.0 \
-  --tp-size=4 \                          # set for tensor parallelism
+  --tp-size=4 \                          # set for tensor parallelism
-  --port 30000 \
+  --port 30000 \
-```
+```
-
+
-## Supported models
+## Supported models
-
+
-| Model Family (Reward)                                                     | Example HuggingFace Identifier                              | Description                                                                     |
+| Model Family (Reward)                                                     | Example HuggingFace Identifier                              | Description                                                                     |
-|---------------------------------------------------------------------------|-----------------------------------------------------|---------------------------------------------------------------------------------|
+|---------------------------------------------------------------------------|-----------------------------------------------------|---------------------------------------------------------------------------------|
-| **Llama (3.1 Reward / `LlamaForSequenceClassification`)**                   | `Skywork/Skywork-Reward-Llama-3.1-8B-v0.2`            | Reward model (preference classifier) based on Llama 3.1 (8B) for scoring and ranking responses for RLHF.  |
+| **Llama (3.1 Reward / `LlamaForSequenceClassification`)**                   | `Skywork/Skywork-Reward-Llama-3.1-8B-v0.2`            | Reward model (preference classifier) based on Llama 3.1 (8B) for scoring and ranking responses for RLHF.  |
-| **Gemma 2 (27B Reward / `Gemma2ForSequenceClassification`)**                | `Skywork/Skywork-Reward-Gemma-2-27B-v0.2`             | Derived from Gemma‑2 (27B), this model provides human preference scoring for RLHF and multilingual tasks.  |
+| **Gemma 2 (27B Reward / `Gemma2ForSequenceClassification`)**                | `Skywork/Skywork-Reward-Gemma-2-27B-v0.2`             | Derived from Gemma‑2 (27B), this model provides human preference scoring for RLHF and multilingual tasks.  |
-| **InternLM 2 (Reward / `InternLM2ForRewardMode`)**                         | `internlm/internlm2-7b-reward`                       | InternLM 2 (7B)–based reward model used in alignment pipelines to guide outputs toward preferred behavior.  |
+| **InternLM 2 (Reward / `InternLM2ForRewardMode`)**                         | `internlm/internlm2-7b-reward`                       | InternLM 2 (7B)–based reward model used in alignment pipelines to guide outputs toward preferred behavior.  |
-| **Qwen2.5 (Reward - Math / `Qwen2ForRewardModel`)**                         | `Qwen/Qwen2.5-Math-RM-72B`                           | A 72B math-specialized RLHF reward model from the Qwen2.5 series, tuned for evaluating and refining responses.  |
+| **Qwen2.5 (Reward - Math / `Qwen2ForRewardModel`)**                         | `Qwen/Qwen2.5-Math-RM-72B`                           | A 72B math-specialized RLHF reward model from the Qwen2.5 series, tuned for evaluating and refining responses.  |
-| **Qwen2.5 (Reward - Sequence / `Qwen2ForSequenceClassification`)**          | `jason9693/Qwen2.5-1.5B-apeach`                      | A smaller Qwen2.5 variant used for sequence classification, offering an alternative RLHF scoring mechanism.  |
+| **Qwen2.5 (Reward - Sequence / `Qwen2ForSequenceClassification`)**          | `jason9693/Qwen2.5-1.5B-apeach`                      | A smaller Qwen2.5 variant used for sequence classification, offering an alternative RLHF scoring mechanism.  |

docs/supported_models/text_generation/diffusion_language_models.md

@@ -1,7 +1,5 @@
 # Diffusion Language Models
 
-> This page covers **text generation** using diffusion-based LLMs. For **image and video generation**, see [Diffusion Models](../image_generation/diffusion_models.md).
-
 Diffusion language models have shown promise for non-autoregressive text generation with parallel decoding capabilities. Unlike auto-regressive language models, different diffusion language models require different decoding strategies.
 
 ## Example Launch Command

python/sglang/multimodal_gen/README.md

@@ -16,11 +16,11 @@ SGLang Diffusion has the following features:
 
 ### AMD/ROCm Support
 
-SGLang Diffusion supports AMD Instinct GPUs through ROCm. On AMD platforms, we use the Triton attention backend and leverage AITER kernels for optimized layernorm and other operations. See the [ROCm installation guide](https://github.com/sgl-project/sglang/tree/main/python/sglang/multimodal_gen/docs/install_rocm.md) for setup instructions.
+SGLang Diffusion supports AMD Instinct GPUs through ROCm. On AMD platforms, we use the Triton attention backend and leverage AITER kernels for optimized layernorm and other operations. See the [installation guide](https://github.com/sgl-project/sglang/tree/main/docs/diffusion/installation.md) for setup instructions.
 
 ### Moore Threads/MUSA Support
 
-SGLang Diffusion supports Moore Threads GPUs (MTGPU) through the MUSA software stack. On MUSA platforms, we use the Torch SDPA backend for attention. See the [MUSA installation guide](https://github.com/sgl-project/sglang/tree/main/python/sglang/multimodal_gen/docs/install_musa.md) for setup instructions.
+SGLang Diffusion supports Moore Threads GPUs (MTGPU) through the MUSA software stack. On MUSA platforms, we use the Torch SDPA backend for attention. See the [installation guide](https://github.com/sgl-project/sglang/tree/main/docs/diffusion/installation.md) for setup instructions.
 
 ## Getting Started
 
@@ -28,9 +28,7 @@ SGLang Diffusion supports Moore Threads GPUs (MTGPU) through the MUSA software s
 uv pip install 'sglang[diffusion]' --prerelease=allow
 ```
 
-For more installation methods (e.g. pypi, uv, docker), check [install.md](https://github.com/sgl-project/sglang/tree/main/python/sglang/multimodal_gen/docs/install.md).
+For more installation methods (e.g. pypi, uv, docker, ROCm/AMD, MUSA/Moore Threads), check [install.md](https://github.com/sgl-project/sglang/tree/main/docs/diffusion/installation.md).
-* ROCm/AMD users should follow the [ROCm quickstart](https://github.com/sgl-project/sglang/tree/main/python/sglang/multimodal_gen/docs/install_rocm.md) that includes the additional kernel builds and attention backend settings we validated on MI300X.
-* MUSA/Moore Threads users should follow the [MUSA quickstart](https://github.com/sgl-project/sglang/tree/main/python/sglang/multimodal_gen/docs/install_musa.md) that includes the attention backend settings we validated on MTT S5000.
 
 ## Inference
 
@@ -82,11 +80,11 @@ sglang generate \
   --save-output
 ```
 
-For more usage examples (e.g. OpenAI compatible API, server mode), check [cli.md](https://github.com/sgl-project/sglang/tree/main/python/sglang/multimodal_gen/docs/cli.md).
+For more usage examples (e.g. OpenAI compatible API, server mode), check [cli.md](https://github.com/sgl-project/sglang/tree/main/docs/diffusion/cli.md).
 
 ## Contributing
 
-All contributions are welcome. The contribution guide is available [here](https://github.com/sgl-project/sglang/tree/main/python/sglang/multimodal_gen/docs/contributing.md).
+All contributions are welcome. The contribution guide is available [here](https://github.com/sgl-project/sglang/tree/main/docs/diffusion/contributing.md).
 
 ## Acknowledgement
 

python/sglang/multimodal_gen/docs/install.md

@@ -1,56 +0,0 @@
-# Install SGLang-diffusion
-
-You can install sglang-diffusion using one of the methods below.
-
-This page primarily applies to common NVIDIA GPU platforms.
-* For AMD Instinct/ROCm environments see the dedicated [ROCm quickstart](install_rocm.md), which lists the exact steps (including kernel builds) we used to validate sgl-diffusion on MI300X.
-* For Moore Threads GPU (MTGPU) with the MUSA software stack, see the [MUSA quickstart](install_musa.md), which lists the exact steps we used to validate sgl-diffusion on MTT S5000.
-
-## Method 1: With pip or uv
-
-It is recommended to use uv for a faster installation:
-
-```bash
-pip install --upgrade pip
-pip install uv
-uv pip install "sglang[diffusion]" --prerelease=allow
-```
-
-## Method 2: From source
-
-```bash
-# Use the latest release branch
-git clone https://github.com/sgl-project/sglang.git
-cd sglang
-
-# Install the Python packages
-pip install --upgrade pip
-pip install -e "python[diffusion]"
-
-# With uv
-uv pip install -e "python[diffusion]" --prerelease=allow
-```
-
-## Method 3: Using Docker
-
-The Docker images are available on Docker Hub at [lmsysorg/sglang](https://hub.docker.com/r/lmsysorg/sglang), built from the [Dockerfile](https://github.com/sgl-project/sglang/blob/main/docker/Dockerfile).
-Replace `<secret>` below with your HuggingFace Hub [token](https://huggingface.co/docs/hub/en/security-tokens).
-
-```bash
-docker run --gpus all \
-    --shm-size 32g \
-    -p 30000:30000 \
-    -v ~/.cache/huggingface:/root/.cache/huggingface \
-    --env "HF_TOKEN=<secret>" \
-    --ipc=host \
-    lmsysorg/sglang:dev \
-    zsh -c '\
-        echo "Installing diffusion dependencies..." && \
-        pip install -e "python[diffusion]" && \
-        echo "Starting SGLang-Diffusion..." && \
-        sglang generate \
-            --model-path black-forest-labs/FLUX.1-dev \
-            --prompt "A logo With Bold Large text: SGL Diffusion" \
-            --save-output \
-    '
-```

python/sglang/multimodal_gen/docs/install_musa.md

@@ -1,24 +0,0 @@
-# MUSA Quickstart for SGLang-Diffusion
-
-This page covers installation and usage of SGLang-Diffusion on Moore Threads GPU (MTGPU) with the MUSA software stack.
-
-## Install from Source
-
-```bash
-# Clone the repository
-git clone https://github.com/sgl-project/sglang.git
-cd sglang
-
-# Install the Python packages
-pip install --upgrade pip
-rm -f python/pyproject.toml && mv python/pyproject_other.toml python/pyproject.toml
-pip install -e "python[all_musa]"
-```
-
-## Quick Test
-
-```bash
-sglang generate --model-path black-forest-labs/FLUX.1-dev \
-    --prompt "A logo With Bold Large text: SGL Diffusion" \
-    --save-output
-```

python/sglang/multimodal_gen/docs/install_rocm.md

@@ -1,9 +0,0 @@
-# ROCm quickstart for sgl-diffusion
-
-```bash
-docker run --device=/dev/kfd --device=/dev/dri --ipc=host \
-  -v ~/.cache/huggingface:/root/.cache/huggingface \
-  --env HF_TOKEN=<secret> \
-  lmsysorg/sglang:v0.5.5.post2-rocm700-mi30x \
-  sglang generate --model-path black-forest-labs/FLUX.1-dev --prompt "A logo With Bold Large text: SGL Diffusion" --save-output
-```