minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 10 Packages Projects Releases Wiki Activity

[diffusion] doc: update doc (#23052)

Browse Source
This commit is contained in:
Mick
2026-04-17 16:23:46 +08:00
committed by GitHub
parent 0d031335ed
commit 0b2058853d
7 changed files with 136 additions and 27 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/diffusion/api/cli.md
+1
View File
@@ -83,6 +83,7 @@ Use `sglang generate --help` and `sglang serve --help` for the full argument lis
### Sampling and output
- `--prompt {PROMPT}` and `--negative-prompt {PROMPT}`
- `--image-path {PATH} [{PATH} ...]`: input image(s) for image-to-video or image-to-image generation
- `--num-inference-steps {STEPS}` and `--seed {SEED}`
- `--height {HEIGHT}`, `--width {WIDTH}`, `--num-frames {N}`, `--fps {FPS}`
- `--output-path {PATH}`, `--output-file-name {NAME}`, `--save-output`, `--return-frames`
docs/diffusion/api/openai_api.md
+28 -1
View File
@@ -154,7 +154,7 @@ curl -sS -L "http://localhost:30010/v1/images/<IMAGE_ID>/content" \
The server implements a subset of the OpenAI Videos API under the `/v1/videos` namespace.
**Create a video**
**Create a video (text-to-video)**
**Endpoint:** `POST /v1/videos`
@@ -184,6 +184,33 @@ curl -sS -X POST "http://localhost:30010/v1/videos" \
}'
```
**Create a video (image-to-video)**
For I2V or TI2V models (e.g., Wan2.1 I2V, LTX-2.3 two-stage), pass an input image via multipart form upload or a reference URL.
**Curl Example (multipart form upload):**
```bash
curl -sS -X POST "http://localhost:30010/v1/videos" \
-H "Authorization: Bearer sk-proj-1234567890" \
-F "prompt=A cat playing a piano" \
-F "input_reference=@input_image.png" \
-F "size=1280x720"
```
**Curl Example (reference URL):**
```bash
curl -sS -X POST "http://localhost:30010/v1/videos" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer sk-proj-1234567890" \
-d '{
"prompt": "A cat playing a piano",
"reference_url": "https://example.com/input_image.png",
"size": "1280x720"
}'
```
**List videos**
**Endpoint:** `GET /v1/videos`
docs/diffusion/compatibility_matrix.md
+34 -12
View File
@@ -33,6 +33,10 @@ default parameters when initializing and generating videos.
| TurboWan2.1 T2V 14B | `IPostYellow/TurboWan2.1-T2V-14B-Diffusers` | 480p | ✅ | ❌ | ❌ | ❌ | ✅ | ✅ | ⭕ |
| TurboWan2.1 T2V 14B 720P | `IPostYellow/TurboWan2.1-T2V-14B-720P-Diffusers` | 720p | ✅ | ❌ | ❌ | ❌ | ✅ | ✅ | ⭕ |
| TurboWan2.2 I2V A14B | `IPostYellow/TurboWan2.2-I2V-A14B-Diffusers` | 720p | ✅ | ❌ | ❌ | ❌ | ✅ | ✅ | ⭕ |
| Wan2.1 Fun 1.3B InP | `weizhou03/Wan2.1-Fun-1.3B-InP-Diffusers` | 480p | ✅ | ✅ | ✅ | ⭕ | ❌ | ❌ | ✅ |
| Helios Base | `BestWishYsh/Helios-Base` | 720p | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
| Helios Mid | `BestWishYsh/Helios-Mid` | 720p | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
| Helios Distilled | `BestWishYsh/Helios-Distilled` | 720p | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
| LTX-2 (one and two stages) | `Lightricks/LTX-2` | 768×512<br>1536×1024 | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
| LTX-2.3 (one and two stages) | `Lightricks/LTX-2.3` | 768×512<br>1536×1024 | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
@@ -40,23 +44,41 @@ default parameters when initializing and generating videos.
1. Wan2.2 TI2V 5B has some quality issues when performing I2V generation. We are working on fixing this issue.
2. SageSLA is based on SpargeAttn. Install it first with `pip install git+https://github.com/thu-ml/SpargeAttn.git --no-build-isolation`
3. LTX-2 two-stage generation uses `--pipeline-class-name LTX2TwoStagePipeline`. The spatial upsampler and distilled LoRA are auto-resolved from the model snapshot by default, and can still be overridden with `--spatial-upsampler-path` and `--distilled-lora-path`.
3. LTX-2 and LTX-2.3 two-stage generation uses `--pipeline-class-name LTX2TwoStagePipeline`. The spatial upsampler and distilled LoRA are auto-resolved from the model snapshot by default, and can still be overridden with `--spatial-upsampler-path` and `--distilled-lora-path`.
- For LTX models, the `Resolutions` column uses output video `width×height` semantics, matching `sglang generate --width ... --height ...`.
### Image Generation Models
| Model Name | HuggingFace Model ID |
|:---------------------|:------------------------------------|
| FLUX.1-dev | `black-forest-labs/FLUX.1-dev` |
| FLUX.2-dev | `black-forest-labs/FLUX.2-dev` |
| FLUX.2-Klein | `black-forest-labs/FLUX.2-klein-4B` |
| Z-Image-Turbo | `Tongyi-MAI/Z-Image-Turbo` |
| GLM-Image | `zai-org/GLM-Image` |
| Qwen Image | `Qwen/Qwen-Image` |
| Qwen Image 2512 | `Qwen/Qwen-Image-2512` |
| Qwen Image Edit | `Qwen/Qwen-Image-Edit` |
| Qwen Image Edit 2511 | `Qwen/Qwen-Image-Edit-2511` |
| Model Name | HuggingFace Model ID |
|:--------------------------|:---------------------------------------------------------|
| FLUX.1-dev | `black-forest-labs/FLUX.1-dev` |
| FLUX.2-dev | `black-forest-labs/FLUX.2-dev` |
| FLUX.2-dev-NVFP4 | `black-forest-labs/FLUX.2-dev-NVFP4` |
| FLUX.2-Klein-4B | `black-forest-labs/FLUX.2-klein-4B` |
| FLUX.2-Klein-9B | `black-forest-labs/FLUX.2-klein-9B` |
| Z-Image | `Tongyi-MAI/Z-Image` |
| Z-Image-Turbo | `Tongyi-MAI/Z-Image-Turbo` |
| GLM-Image | `zai-org/GLM-Image` |
| Qwen Image | `Qwen/Qwen-Image` |
| Qwen Image 2512 | `Qwen/Qwen-Image-2512` |
| Qwen Image Edit | `Qwen/Qwen-Image-Edit` |
| Qwen Image Edit 2509 | `Qwen/Qwen-Image-Edit-2509` |
| Qwen Image Edit 2511 | `Qwen/Qwen-Image-Edit-2511` |
| Qwen Image Layered | `Qwen/Qwen-Image-Layered` |
| SD3 Medium | `stabilityai/stable-diffusion-3-medium-diffusers` |
| SD3.5 Medium | `stabilityai/stable-diffusion-3.5-medium-diffusers` |
| SD3.5 Large | `stabilityai/stable-diffusion-3.5-large-diffusers` |
| Hunyuan3D-2 | `tencent/Hunyuan3D-2` |
| SANA 1.5 1.6B | `Efficient-Large-Model/SANA1.5_1.6B_1024px_diffusers` |
| SANA 1.5 4.8B | `Efficient-Large-Model/SANA1.5_4.8B_1024px_diffusers` |
| SANA 1600M 1024px | `Efficient-Large-Model/Sana_1600M_1024px_diffusers` |
| SANA 600M 1024px | `Efficient-Large-Model/Sana_600M_1024px_diffusers` |
| SANA 1600M 512px | `Efficient-Large-Model/Sana_1600M_512px_diffusers` |
| SANA 600M 512px | `Efficient-Large-Model/Sana_600M_512px_diffusers` |
| FireRed-Image-Edit 1.0 | `FireRedTeam/FireRed-Image-Edit-1.0` |
| FireRed-Image-Edit 1.1 | `FireRedTeam/FireRed-Image-Edit-1.1` |
| ERNIE-Image | `baidu/ERNIE-Image` |
| ERNIE-Image-Turbo | `baidu/ERNIE-Image-Turbo` |
## Supported Components
docs/diffusion/environment_variables.md
+46 -1
View File
@@ -1,11 +1,42 @@
# Environment Variables
## Apple MPS
## Runtime
| Environment Variable | Default | Description |
|----------------------|---------|-------------|
| `SGLANG_DIFFUSION_TARGET_DEVICE` | `cuda` | Target device for inference (`cuda`, `rocm`, `xpu`, `npu`, `musa`, `mps`, `cpu`) |
| `SGLANG_DIFFUSION_ATTENTION_BACKEND` | not set | Override attention backend via env var (e.g. `fa`, `torch_sdpa`, `sage_attn`) |
| `SGLANG_DIFFUSION_ATTENTION_CONFIG` | not set | Path to attention backend configuration file (JSON/YAML) |
| `SGLANG_DIFFUSION_STAGE_LOGGING` | false | Enable per-stage timing logs |
| `SGLANG_DIFFUSION_SERVER_DEV_MODE` | false | Enable dev-only HTTP endpoints for debugging |
| `SGLANG_DIFFUSION_TORCH_PROFILER_DIR` | not set | Directory for torch profiler traces (absolute path). Enables profiling when set |
| `SGLANG_DIFFUSION_CACHE_ROOT` | `~/.cache/sgl_diffusion` | Root directory for cache files |
| `SGLANG_DIFFUSION_CONFIG_ROOT` | `~/.config/sgl_diffusion` | Root directory for configuration files |
| `SGLANG_DIFFUSION_LOGGING_LEVEL` | `INFO` | Default logging level |
| `SGLANG_DIFFUSION_WORKER_MULTIPROC_METHOD` | `fork` | Multiprocess context for workers (`fork` or `spawn`) |
| `SGLANG_USE_RUNAI_MODEL_STREAMER` | true | Use Run:AI model streamer for model loading |
## Platform-Specific
### Apple MPS
| Environment Variable | Default | Description |
|----------------------|---------|--------------------------------------------------------------|
| `SGLANG_USE_MLX` | not set | Set to `1` to enable MLX fused Metal kernels for norm ops on MPS |
### ROCm (AMD GPUs)
| Environment Variable | Default | Description |
|----------------------|---------|-------------|
| `SGLANG_USE_ROCM_VAE` | false | Use AITer GroupNorm in VAE for improved performance on ROCm |
| `SGLANG_USE_ROCM_CUDNN_BENCHMARK` | false | Enable MIOpen auto-tuning for VAE conv layers on ROCm |
### Quantization
| Environment Variable | Default | Description |
|----------------------|---------|-------------|
| `SGLANG_DIFFUSION_FLASHINFER_FP4_GEMM_BACKEND` | not set | FlashInfer FP4 GEMM backend for generic NVFP4 fallback |
## Caching Acceleration
These variables configure caching acceleration for Diffusion Transformer (DiT) models.
@@ -30,6 +61,20 @@ See [cache-dit documentation](performance/cache/cache_dit.md) for detailed confi
| `SGLANG_CACHE_DIT_SCM_COMPUTE_BINS` | not set | Custom SCM compute bins |
| `SGLANG_CACHE_DIT_SCM_CACHE_BINS` | not set | Custom SCM cache bins |
### Cache-DiT Secondary Transformer
For dual-transformer models (e.g., Wan2.2 with high/low-noise experts), these variables configure caching for the secondary transformer. Each falls back to its primary counterpart if not set.
| Environment Variable | Default | Description |
|-------------------------------------|---------|------------------------------------------|
| `SGLANG_CACHE_DIT_SECONDARY_FN` | (from primary) | First N blocks to always compute |
| `SGLANG_CACHE_DIT_SECONDARY_BN` | (from primary) | Last N blocks to always compute |
| `SGLANG_CACHE_DIT_SECONDARY_WARMUP` | (from primary) | Warmup steps before caching |
| `SGLANG_CACHE_DIT_SECONDARY_RDT` | (from primary) | Residual difference threshold |
| `SGLANG_CACHE_DIT_SECONDARY_MC` | (from primary) | Max continuous cached steps |
| `SGLANG_CACHE_DIT_SECONDARY_TAYLORSEER` | (from primary) | Enable TaylorSeer calibrator |
| `SGLANG_CACHE_DIT_SECONDARY_TS_ORDER` | (from primary) | TaylorSeer order (1 or 2) |
## Cloud Storage
These variables configure S3-compatible cloud storage for automatically uploading generated images and videos.
docs/diffusion/index.md
+1 -1
View File
@@ -7,7 +7,7 @@ SGLang Diffusion is a high-performance inference framework for image and video g
- Broad model support across Wan, Hunyuan, Qwen-Image, FLUX, Z-Image, GLM-Image, and more
- Fast inference with `sgl-kernel`, JIT kernels, scheduler improvements, and caching acceleration
- Multiple interfaces: `sglang generate`, `sglang serve`, and an OpenAI-compatible API
- Multi-platform support for NVIDIA, AMD, Ascend, Apple Silicon, and Moore Threads
- Multi-platform support for NVIDIA, AMD, Intel XPU, Ascend, Apple Silicon, and Moore Threads
## Quick Start
docs/diffusion/installation.md
+8
View File
@@ -82,6 +82,14 @@ rm -f python/pyproject.toml && mv python/pyproject_other.toml python/pyproject.t
pip install -e "python[all_musa]"
```
## Platform-Specific: Intel XPU
For Intel Data Center GPU Max or Arc GPUs, follow the [XPU installation guide](../platforms/xpu.md) to set up the base environment, then install diffusion dependencies:
```bash
pip install -e "python[diffusion]"
```
## Platform-Specific: Ascend NPU
For Ascend NPU, please follow the [NPU installation guide](../platforms/ascend/ascend_npu.md).
docs/diffusion/performance/attention_backends.md
+18 -12
View File
@@ -13,6 +13,8 @@ When using the diffusers backend, `--attention-backend` is passed through to dif
- **CUDA**: prefers FlashAttention (FA3/FA4) when supported; otherwise falls back to PyTorch SDPA.
- **ROCm**: uses FlashAttention when available; otherwise falls back to PyTorch SDPA.
- **Intel XPU**: uses XPU Flash Attention backend (fp16/bf16, head sizes 64/96/128/192/256); otherwise falls back to PyTorch SDPA.
- **MUSA**: uses FlashAttention when available; otherwise falls back to PyTorch SDPA.
- **MPS**: always uses PyTorch SDPA.
- **NPU**: for ring attention uses FA otherwise uses PyTorch SDPA.
@@ -31,6 +33,8 @@ For SGLang-native pipelines, the CLI accepts the lowercase names of `AttentionBa
| `vmoba_attn` | `VMOBA_ATTN` | Requires `kernel.attn.vmoba_attn.vmoba`. Configure via `--attention-backend-config`. |
| `aiter` | `AITER` | Requires `aiter`. |
| `aiter_sage` | `AITER_SAGE` | Requires `aiter`. |
| `sla_attn` | `SLA_ATTN` | Sparse Linear Attention. Requires `SpargeAttn`. Install with `pip install git+https://github.com/thu-ml/SpargeAttn.git --no-build-isolation`. |
| `sage_sla_attn` | `SAGE_SLA_ATTN` | SageAttention + Sparse Linear Attention. Requires `SpargeAttn` (same install as SLA). |
| `sparse_video_gen_2_attn` | `SPARSE_VIDEO_GEN_2_ATTN` | Requires `svg`. See installation instructions at https://github.com/svg-project/Sparse-VideoGen. |
## Selection priority
@@ -85,18 +89,20 @@ Some backends require additional configuration. You can pass these parameters vi
## Platform support matrix
| Backend | CUDA | ROCm | MPS | NPU | Notes |
|---|---:|---:|---:|---:|---|
| `fa` | ✅ | ✅ | ❌ | ✅ | CUDA requires SM80+ and fp16/bf16. FlashAttention is only used when the required runtime is installed; otherwise it falls back to `torch_sdpa`. No extra installations are required for NPU |
| `torch_sdpa` | ✅ | ✅ | ✅ | ✅ | Most compatible option across platforms. |
| `sliding_tile_attn` | ✅ | ❌ | ❌ | ❌ | CUDA-only. Requires `st_attn`. Configure via `--attention-backend-config`. |
| `sage_attn` | ✅ | ❌ | ❌ | ❌ | CUDA-only (optional dependency). |
| `sage_attn_3` | ✅ | ❌ | ❌ | ❌ | CUDA-only (optional dependency). |
| `video_sparse_attn` | ✅ | ❌ | ❌ | ❌ | CUDA-only. Requires `vsa`. Configure `sparsity` via `--attention-backend-config`. |
| `vmoba_attn` | ✅ | ❌ | ❌ | ❌ | CUDA-only. Requires `kernel.attn.vmoba_attn.vmoba`. Configure via `--attention-backend-config`. |
| `aiter` | ❌ | ✅ | ❌ | ❌ | Requires `aiter`. |
| `aiter_sage` | ❌ | ✅ | ❌ | ❌ | Requires `aiter`. |
| `sparse_video_gen_2_attn` | ✅ | ❌ | ❌ | ❌ | CUDA-only. Requires `svg`. |
| Backend | CUDA | ROCm | XPU | MUSA | MPS | NPU | Notes |
|---|---:|---:|---:|---:|---:|---:|---|
| `fa` | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | CUDA requires SM80+ and fp16/bf16. XPU uses its own flash attention backend. FlashAttention is only used when the required runtime is installed; otherwise it falls back to `torch_sdpa`. No extra installations are required for NPU |
| `torch_sdpa` | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | Most compatible option across platforms. |
| `sliding_tile_attn` | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | CUDA-only. Requires `st_attn`. Configure via `--attention-backend-config`. |
| `sage_attn` | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | CUDA-only (optional dependency). |
| `sage_attn_3` | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | CUDA-only (optional dependency). |
| `video_sparse_attn` | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | CUDA-only. Requires `vsa`. Configure `sparsity` via `--attention-backend-config`. |
| `sla_attn` | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | CUDA-only. Requires `SpargeAttn`. |
| `sage_sla_attn` | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | CUDA-only. Requires `SpargeAttn`. |
| `vmoba_attn` | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | CUDA-only. Requires `kernel.attn.vmoba_attn.vmoba`. Configure via `--attention-backend-config`. |
| `aiter` | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | Requires `aiter`. |
| `aiter_sage` | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | Requires `aiter`. |
| `sparse_video_gen_2_attn` | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | CUDA-only. Requires `svg`. |
## Usage