[diffusion] doc: consolidate documentation (#21373)
This commit is contained in:
+1
-1
@@ -1,4 +1,4 @@
|
||||
# Cache-DiT Acceleration
|
||||
# Cache-DiT
|
||||
|
||||
SGLang integrates [Cache-DiT](https://github.com/vipshop/cache-dit), a caching acceleration engine for Diffusion Transformers (DiT), to achieve up to **1.69x inference speedup** with minimal quality loss.
|
||||
|
||||
|
||||
+9
-4
@@ -1,6 +1,6 @@
|
||||
# Caching Acceleration for Diffusion Models
|
||||
# Caching Acceleration
|
||||
|
||||
SGLang provides multiple caching acceleration strategies for Diffusion Transformer (DiT) models. These strategies can significantly reduce inference time by skipping redundant computation.
|
||||
SGLang provides two complementary caching strategies for Diffusion Transformer (DiT) models. Both reduce denoising cost by skipping redundant computation, but they operate at different levels.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -11,8 +11,6 @@ SGLang supports two complementary caching approaches:
|
||||
| **Cache-DiT** | Block-level | Skip individual transformer blocks dynamically | Advanced, higher speedup |
|
||||
| **TeaCache** | Timestep-level | Skip entire denoising steps based on L1 similarity | Simple, built-in |
|
||||
|
||||
|
||||
|
||||
## Cache-DiT
|
||||
|
||||
[Cache-DiT](https://github.com/vipshop/cache-dit) provides block-level caching with
|
||||
@@ -54,6 +52,13 @@ See [teacache.md](teacache.md) for detailed documentation.
|
||||
|
||||
For Flux and Qwen models, TeaCache is automatically disabled when CFG is enabled.
|
||||
|
||||
```{toctree}
|
||||
:maxdepth: 1
|
||||
|
||||
cache_dit
|
||||
teacache
|
||||
```
|
||||
|
||||
## References
|
||||
|
||||
- [Cache-DiT Repository](https://github.com/vipshop/cache-dit)
|
||||
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
# TeaCache Acceleration
|
||||
# TeaCache
|
||||
|
||||
> **Note**: This is one of two caching strategies available in SGLang.
|
||||
> For an overview of all caching options, see [caching](../index.md).
|
||||
|
||||
@@ -1,71 +1,35 @@
|
||||
# Performance Optimization
|
||||
# Performance
|
||||
|
||||
SGLang-Diffusion provides multiple performance optimization strategies to accelerate inference. This section covers all available performance tuning options.
|
||||
This section covers the main performance levers for SGLang Diffusion: attention backends, caching acceleration, and profiling.
|
||||
|
||||
## Overview
|
||||
|
||||
| Optimization | Type | Description |
|
||||
|--------------|------|-------------|
|
||||
| **Cache-DiT** | Caching | Block-level caching with DBCache, TaylorSeer, and SCM |
|
||||
| **TeaCache** | Caching | Timestep-level caching using L1 similarity |
|
||||
| **TeaCache** | Caching | Timestep-level caching based on temporal similarity |
|
||||
| **Attention Backends** | Kernel | Optimized attention implementations (FlashAttention, SageAttention, etc.) |
|
||||
| **Profiling** | Diagnostics | PyTorch Profiler and Nsight Systems guidance |
|
||||
|
||||
## Caching Strategies
|
||||
## Start Here
|
||||
|
||||
SGLang supports two complementary caching approaches:
|
||||
- Use [Attention Backends](attention_backends.md) to choose the best backend for your model and hardware.
|
||||
- Use [Caching Acceleration](cache/index.md) to reduce denoising cost with Cache-DiT or TeaCache.
|
||||
- Use [Profiling](profiling.md) when you need to diagnose a bottleneck rather than guess.
|
||||
|
||||
### Cache-DiT
|
||||
## Caching at a Glance
|
||||
|
||||
[Cache-DiT](https://github.com/vipshop/cache-dit) provides block-level caching with advanced strategies. It can achieve up to **1.69x speedup**.
|
||||
- [Cache-DiT](cache/cache_dit.md) is block-level caching for diffusers pipelines and higher speedup-oriented tuning.
|
||||
- [TeaCache](cache/teacache.md) is timestep-level caching built into SGLang model families.
|
||||
|
||||
**Quick Start:**
|
||||
```bash
|
||||
SGLANG_CACHE_DIT_ENABLED=true \
|
||||
sglang generate --model-path Qwen/Qwen-Image \
|
||||
--prompt "A beautiful sunset over the mountains"
|
||||
```{toctree}
|
||||
:maxdepth: 1
|
||||
|
||||
attention_backends
|
||||
cache/index
|
||||
profiling
|
||||
```
|
||||
|
||||
**Key Features:**
|
||||
- **DBCache**: Dynamic block-level caching based on residual differences
|
||||
- **TaylorSeer**: Taylor expansion-based calibration for optimized caching
|
||||
- **SCM**: Step-level computation masking for additional speedup
|
||||
|
||||
See [Cache-DiT Documentation](cache/cache_dit.md) for detailed configuration.
|
||||
|
||||
### TeaCache
|
||||
|
||||
TeaCache (Temporal similarity-based caching) accelerates diffusion inference by detecting when consecutive denoising steps are similar enough to skip computation entirely.
|
||||
|
||||
**Quick Overview:**
|
||||
- Tracks L1 distance between modulated inputs across timesteps
|
||||
- When accumulated distance is below threshold, reuses cached residual
|
||||
- Supports CFG with separate positive/negative caches
|
||||
|
||||
**Supported Models:** Wan (wan2.1, wan2.2), Hunyuan (HunyuanVideo), Z-Image
|
||||
|
||||
See [TeaCache Documentation](cache/teacache.md) for detailed configuration.
|
||||
|
||||
## Attention Backends
|
||||
|
||||
Different attention backends offer varying performance characteristics depending on your hardware and model:
|
||||
|
||||
- **FlashAttention**: Fastest on NVIDIA GPUs with fp16/bf16
|
||||
- **SageAttention**: Alternative optimized implementation
|
||||
- **xformers**: Memory-efficient attention
|
||||
- **SDPA**: PyTorch native scaled dot-product attention
|
||||
|
||||
See [Attention Backends](attention_backends.md) for platform support and configuration options.
|
||||
|
||||
## Profiling
|
||||
|
||||
To diagnose performance bottlenecks, SGLang-Diffusion supports profiling tools:
|
||||
|
||||
- **PyTorch Profiler**: Built-in Python profiling
|
||||
- **Nsight Systems**: GPU kernel-level analysis
|
||||
|
||||
See [Profiling Guide](profiling.md) for detailed instructions.
|
||||
|
||||
## References
|
||||
|
||||
- [Cache-DiT Repository](https://github.com/vipshop/cache-dit)
|
||||
|
||||
Reference in New Issue
Block a user