[diffusion] hardware: support diffusion models on MTGPU (doc, 6/N) (#17346)
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
This commit is contained in:
@@ -18,14 +18,19 @@ SGLang Diffusion has the following features:
|
|||||||
|
|
||||||
SGLang Diffusion supports AMD Instinct GPUs through ROCm. On AMD platforms, we use the Triton attention backend and leverage AITER kernels for optimized layernorm and other operations. See the [ROCm installation guide](https://github.com/sgl-project/sglang/tree/main/python/sglang/multimodal_gen/docs/install_rocm.md) for setup instructions.
|
SGLang Diffusion supports AMD Instinct GPUs through ROCm. On AMD platforms, we use the Triton attention backend and leverage AITER kernels for optimized layernorm and other operations. See the [ROCm installation guide](https://github.com/sgl-project/sglang/tree/main/python/sglang/multimodal_gen/docs/install_rocm.md) for setup instructions.
|
||||||
|
|
||||||
|
### Moore Threads/MUSA Support
|
||||||
|
|
||||||
|
SGLang Diffusion supports Moore Threads GPUs (MTGPU) through the MUSA software stack. On MUSA platforms, we use the Torch SDPA backend for attention. See the [MUSA installation guide](https://github.com/sgl-project/sglang/tree/main/python/sglang/multimodal_gen/docs/install_musa.md) for setup instructions.
|
||||||
|
|
||||||
## Getting Started
|
## Getting Started
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
uv pip install 'sglang[diffusion]' --prerelease=allow
|
uv pip install 'sglang[diffusion]' --prerelease=allow
|
||||||
```
|
```
|
||||||
|
|
||||||
For more installation methods (e.g. pypi, uv, docker), check [install.md](https://github.com/sgl-project/sglang/tree/main/python/sglang/multimodal_gen/docs/install.md). ROCm/AMD users should follow the [ROCm quickstart](https://github.com/sgl-project/sglang/tree/main/python/sglang/multimodal_gen/docs/install_rocm.md) that includes the additional kernel builds and attention backend settings we validated on MI300X.
|
For more installation methods (e.g. pypi, uv, docker), check [install.md](https://github.com/sgl-project/sglang/tree/main/python/sglang/multimodal_gen/docs/install.md).
|
||||||
|
* ROCm/AMD users should follow the [ROCm quickstart](https://github.com/sgl-project/sglang/tree/main/python/sglang/multimodal_gen/docs/install_rocm.md) that includes the additional kernel builds and attention backend settings we validated on MI300X.
|
||||||
|
* MUSA/Moore Threads users should follow the [MUSA quickstart](https://github.com/sgl-project/sglang/tree/main/python/sglang/multimodal_gen/docs/install_musa.md) that includes the attention backend settings we validated on MTT S5000.
|
||||||
|
|
||||||
## Inference
|
## Inference
|
||||||
|
|
||||||
|
|||||||
@@ -2,7 +2,9 @@
|
|||||||
|
|
||||||
You can install sglang-diffusion using one of the methods below.
|
You can install sglang-diffusion using one of the methods below.
|
||||||
|
|
||||||
This page primarily applies to common NVIDIA GPU platforms. For AMD Instinct/ROCm environments see the dedicated [ROCm quickstart](install_rocm.md), which lists the exact steps (including kernel builds) we used to validate sgl-diffusion on MI300X.
|
This page primarily applies to common NVIDIA GPU platforms.
|
||||||
|
* For AMD Instinct/ROCm environments see the dedicated [ROCm quickstart](install_rocm.md), which lists the exact steps (including kernel builds) we used to validate sgl-diffusion on MI300X.
|
||||||
|
* For Moore Threads GPU (MTGPU) with the MUSA software stack, see the [MUSA quickstart](install_musa.md), which lists the exact steps we used to validate sgl-diffusion on MTT S5000.
|
||||||
|
|
||||||
## Method 1: With pip or uv
|
## Method 1: With pip or uv
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,24 @@
|
|||||||
|
# MUSA Quickstart for SGLang-Diffusion
|
||||||
|
|
||||||
|
This page covers installation and usage of SGLang-Diffusion on Moore Threads GPU (MTGPU) with the MUSA software stack.
|
||||||
|
|
||||||
|
## Install from Source
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Clone the repository
|
||||||
|
git clone https://github.com/sgl-project/sglang.git
|
||||||
|
cd sglang
|
||||||
|
|
||||||
|
# Install the Python packages
|
||||||
|
pip install --upgrade pip
|
||||||
|
rm -f python/pyproject.toml && mv python/pyproject_other.toml python/pyproject.toml
|
||||||
|
pip install -e "python[all_musa]"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Quick Test
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sglang generate --model-path black-forest-labs/FLUX.1-dev \
|
||||||
|
--prompt "A logo With Bold Large text: SGL Diffusion" \
|
||||||
|
--save-output
|
||||||
|
```
|
||||||
Reference in New Issue
Block a user