[diffusion] docs: recommend docker for gpu deployments (#41966)

Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
This commit is contained in:
Mick
2026-10-01 09:52:31 +08:00
committed by GitHub
co-authored by Mick Qian
parent 36330b8c08
commit 2ab1b537f8
18 changed files with 96 additions and 144 deletions
@@ -13,8 +13,8 @@ import { config } from '/src/snippets/configs/__HF_ORG__/__MODEL_SLUG__.jsx';
## 1. Quick start
Install with `uv pip install "sglang[diffusion]" --prerelease=allow`, then choose
a verified recipe below.
Follow the [SGLang Diffusion installation guide](/docs/sglang-diffusion/installation),
then choose a verified recipe below. Docker is recommended for Linux GPU deployments.
<Deployment config={config} />
@@ -11,13 +11,7 @@ import { config } from '/src/snippets/configs/CircleStone/anima.jsx';
## 1. Quick start
Install the diffusion dependencies on Linux with NVIDIA CUDA, then install this
integration from its source checkout:
```bash Installation
uv pip install "sglang[diffusion]" --prerelease=allow
uv pip install -e "python[diffusion]"
```
On Linux with NVIDIA CUDA, use a nightly Docker image containing this integration, or install from source, following the [SGLang Diffusion installation guide](/docs/sglang-diffusion/installation). Run the commands below inside that environment.
Use the official `circlestone-labs/Anima-Base-v1.0-Diffusers` checkpoint. SGLang reads
its `modular_model_index.json` directly; no checkpoint conversion or newer Diffusers
+1 -5
View File
@@ -32,11 +32,7 @@ The default `flow_shift` is `3.0` for T2I, `10.0` for non-Edge video and all act
## 2. Installation
Install SGLang with the diffusion dependencies:
```bash Command
pip install -e "python[diffusion]"
```
Follow the [SGLang Diffusion installation guide](/docs/sglang-diffusion/installation). Docker is recommended for Linux GPU deployments; run the commands below inside the container or your installed Python environment.
Cosmos3 guardrails are enabled by default when the package is available:
@@ -21,13 +21,7 @@ This integration currently targets text-only image generation rather than image
## 2. Installation
Install SGLang with the diffusion dependencies:
```bash Command
pip install -e "python[diffusion]"
```
For full installation options, see the [SGLang Diffusion installation guide](/docs/sglang-diffusion/installation).
Follow the [SGLang Diffusion installation guide](/docs/sglang-diffusion/installation). Docker is recommended for Linux GPU deployments; run the commands below inside the container or your installed Python environment.
## 3. Serve the model
+1 -7
View File
@@ -31,13 +31,7 @@ Review the model license on the [JoyAI-Echo Hugging Face page](https://huggingfa
## 2. SGLang-diffusion Installation
Install SGLang with diffusion dependencies:
```bash
uv pip install "sglang[diffusion]" --prerelease=allow
```
For platform-specific setup, see the [SGLang Diffusion installation guide](/docs/sglang-diffusion/installation).
Follow the [SGLang Diffusion installation guide](/docs/sglang-diffusion/installation). Docker is recommended for Linux GPU deployments; run the commands below inside the container or your installed Python environment.
## 3. Model Deployment
@@ -24,13 +24,7 @@ Use `Lightricks/LTX-2` or `Lightricks/LTX-2.3` as `--model-path`. SGLang loads t
## 2. SGLang-diffusion Installation
Install SGLang with diffusion dependencies:
```bash
uv pip install "sglang[diffusion]" --prerelease=allow
```
For platform-specific setup, see the [SGLang Diffusion installation guide](/docs/sglang-diffusion/installation).
Follow the [SGLang Diffusion installation guide](/docs/sglang-diffusion/installation). Docker is recommended for Linux GPU deployments; run the commands below inside the container or your installed Python environment.
## 3. Model Deployment
+1 -5
View File
@@ -66,11 +66,7 @@ decoder choice does not change anything upstream of it.
## 2. SGLang-diffusion Installation
```bash
uv pip install "sglang[diffusion]" --prerelease=allow
```
For platform-specific setup, see the [SGLang Diffusion installation guide](/docs/sglang-diffusion/installation).
Follow the [SGLang Diffusion installation guide](/docs/sglang-diffusion/installation). Docker is recommended for Linux GPU deployments; run the commands below inside the container or your installed Python environment.
NATTEN is an optional extra, worth installing only if you plan to use the
[diffusion decoder](#4-6-diffusion-decoder) — see that section for why.
@@ -34,14 +34,7 @@ it is not an `extra_params` object.
## 2. Installation
Install SGLang with the diffusion dependencies:
```bash Command
uv pip install "sglang[diffusion]" --prerelease=allow
```
See the [SGLang Diffusion installation guide](/docs/sglang-diffusion/installation)
for platform-specific setup.
Follow the [SGLang Diffusion installation guide](/docs/sglang-diffusion/installation). Docker is recommended for Linux GPU deployments; run the commands below inside the container or your installed Python environment.
## 3. Serve LingBot Video MoE
@@ -20,14 +20,7 @@ each request can disable it when lower latency matters more than the rewritten p
## 2. Installation
Install SGLang with the diffusion dependencies:
```bash Command
pip install -e "python[diffusion]"
```
For other installation options, see the
[SGLang Diffusion installation guide](/docs/sglang-diffusion/installation).
Follow the [SGLang Diffusion installation guide](/docs/sglang-diffusion/installation). Docker is recommended for Linux GPU deployments; run the commands below inside the container or your installed Python environment.
## 3. Serve the model
@@ -13,8 +13,9 @@ import { config } from "/src/snippets/configs/MiniMaxAI/minimax-h3.jsx";
## 1. Quick start
Install with `uv pip install "sglang[diffusion]" --prerelease=allow`, then choose
a verified recipe below. Setup changes the deployment; Server and Request expose
Follow the [SGLang Diffusion installation guide](/docs/sglang-diffusion/installation),
then choose a verified recipe below. Docker is recommended where the picker offers it.
Setup changes the deployment; Server and Request expose
orthogonal startup and sampling choices.
<Deployment config={config} />
@@ -25,8 +26,8 @@ Change a Server option only for a deliberate trade-off; Request options do not
reload the model.
</Note>
The Docker form installs the platform-specific diffusion extra from the source
bundled in the image. For conditioned requests, set **Host media directory**
The Docker form uses the image's bundled diffusion dependencies without reinstalling
them at startup. For conditioned requests, set **Host media directory**
under **Variables**; the builder mounts it read-only at `/data/minimax-h3`.
AMD and Grace Blackwell currently offer the Python form; the other NVIDIA
recipes also offer Docker.
@@ -12,9 +12,9 @@ import { config } from '/src/snippets/configs/Qwen/qwen-image-2.1.jsx';
## 1. Quick start
Install the runtime dependencies with `uv pip install "sglang[diffusion]" --prerelease=allow`,
then install this integration from its source checkout with
`uv pip install -e "python[diffusion]"`. The picker uses `Qwen/Qwen-Image-2.1`;
Use a nightly Docker image containing this integration, or install from source,
following the [SGLang Diffusion installation guide](/docs/sglang-diffusion/installation).
Run the commands below inside that environment. The picker uses `Qwen/Qwen-Image-2.1`;
you can also set a local checkpoint directory under **Variables**. The recipes
target NVIDIA CUDA on Linux; the picker marks which single-GPU workloads have
been verified with the full checkpoint.
@@ -28,14 +28,7 @@ from the prompt with an optional `motion score: N.` suffix.
## 2. Installation
Install SGLang with the diffusion dependencies:
```bash Command
uv pip install "sglang[diffusion]" --prerelease=allow
```
See the [SGLang Diffusion installation guide](/docs/sglang-diffusion/installation)
for platform-specific setup.
Follow the [SGLang Diffusion installation guide](/docs/sglang-diffusion/installation). Docker is recommended for Linux GPU deployments; run the commands below inside the container or your installed Python environment.
## 3. Serve SANA-Video
@@ -12,12 +12,7 @@ import { config } from '/src/snippets/configs/inclusionAI/ming-image.jsx';
## 1. Quick start
Install the dependencies, then this integration from its source checkout:
```bash Installation
uv pip install "sglang[diffusion]" --prerelease=allow
uv pip install -e "python[diffusion]" --no-deps
```
Use a nightly Docker image containing this integration, or install from source, following the [SGLang Diffusion installation guide](/docs/sglang-diffusion/installation). Run the commands below inside that environment.
Choose **Design** for generation or editing and **Design-Layer** for decomposition.
The two checkpoints require separate servers. Start with one H200 on Linux/CUDA;
+6 -4
View File
@@ -13,9 +13,9 @@ SGLang Diffusion is a high-performance inference framework for image and video g
## Quick Start
```bash
uv pip install "sglang[diffusion]" --prerelease=allow
```
Start with the [recommended Docker setup](/docs/sglang-diffusion/installation) for Linux GPU deployments. The installation guide also covers pip/uv, source installation, and other platforms.
Run the following commands inside the container or your activated Python environment. Generate an image:
```bash
sglang generate --model-path Qwen/Qwen-Image \
@@ -23,8 +23,10 @@ sglang generate --model-path Qwen/Qwen-Image \
--save-output
```
Or start an HTTP server:
```bash
sglang serve --model-path Qwen/Qwen-Image --port 30010
sglang serve --model-path Qwen/Qwen-Image --host 0.0.0.0 --port 30000
```
## Start Here
+66 -50
View File
@@ -1,73 +1,89 @@
---
title: Install SGLang Diffusion
description: Install SGLang Diffusion on NVIDIA, AMD, MUSA, and Ascend platforms.
description: Run SGLang Diffusion with Docker, or install with pip, uv, or from source.
---
You can install SGLang-Diffusion using one of the methods below. The standard installation already includes SGLang's optimized kernel stack, including both `sgl-kernel` and JIT kernels used by diffusion workloads.
Docker is the recommended setup for Linux GPU deployments. The official NVIDIA images include SGLang, its diffusion dependencies, and the optimized kernel stack; no separate Python installation or `pip install` inside the container is needed for the standard diffusion runtime. Optional backends and model-specific dependencies are documented in the [cookbook](/cookbook/diffusion/intro).
## Standard Installation (NVIDIA GPUs)
## Standard installation (NVIDIA GPUs)
### Method 1: With pip or uv
<Tabs>
<Tab title="Docker (recommended)" id="method-3-using-docker">
It is recommended to use uv for a faster installation:
Use Linux with a supported NVIDIA GPU, a driver compatible with the image's CUDA version, [Docker Engine](https://docs.docker.com/engine/install/), and [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html). Configure Docker's GPU runtime before continuing. See the [SGLang installation guide](/docs/get-started/install) for current CUDA requirements.
```bash Command
pip install --upgrade pip
The example uses the latest stable release. For reproducible deployments, replace `latest` with a release tag or image digest from [Docker Hub](https://hub.docker.com/r/lmsysorg/sglang/tags). For models or features not yet released, choose a [nightly image](https://hub.docker.com/r/lmsysorg/sglang/tags?name=nightly) containing the required changes, or install from source.
```bash
docker pull lmsysorg/sglang:latest
```
Start a container with persistent model and output directories. The working directory is `/outputs`, so generated files remain in the host's `outputs` directory after the container exits. The server port is published only on the host's loopback interface.
```bash
mkdir -p "$HOME/.cache/huggingface" "$PWD/outputs"
docker run --rm -it --gpus all \
--ipc=host \
-p 127.0.0.1:30000:30000 \
-v "$HOME/.cache/huggingface:/root/.cache/huggingface" \
-v "$PWD/outputs:/outputs" \
-w /outputs \
lmsysorg/sglang:latest bash
```
For gated or private models, export your [Hugging Face token](https://huggingface.co/docs/hub/en/security-tokens) as `HF_TOKEN` on the host and add `--env HF_TOKEN` before the image name. Mount any local input media or checkpoints into the container and use their container paths in requests.
Run the commands in [Generate or serve](#generate-or-serve) inside this container. For development tools and source changes, see the [development environment guide](/docs/developer_guide/development_guide_using_docker#setup-docker-container).
</Tab>
<Tab title="pip / uv" id="method-1-with-pip-or-uv">
Use this option for an existing Python environment. Follow the [Python and CUDA prerequisites](/docs/get-started/install), then install the diffusion extra in a virtual environment:
```bash
pip install uv
uv venv --python 3.12
source .venv/bin/activate
uv pip install "sglang[diffusion]" --prerelease=allow
```
### Method 2: From source
</Tab>
<Tab title="From source" id="method-2-from-source">
```bash Command
# Use the latest release branch
Use source installation to develop SGLang or try changes that are not in a published image. Follow the [Python and CUDA prerequisites](/docs/get-started/install). The commands below install the current `main`; check out a release tag or commit before installation when you need a fixed revision.
```bash
git clone https://github.com/sgl-project/sglang.git
cd sglang
# Install the Python packages
pip install --upgrade pip
pip install -e "python[diffusion]"
# With uv
pip install uv
uv venv --python 3.12
source .venv/bin/activate
uv pip install -e "python[diffusion]" --prerelease=allow
```
### Method 3: Using Docker
</Tab>
</Tabs>
The Docker images are available on Docker Hub at [lmsysorg/sglang](https://hub.docker.com/r/lmsysorg/sglang), built from the [Dockerfile](https://github.com/sgl-project/sglang/blob/main/docker/Dockerfile).
Replace `<secret>` below with your HuggingFace Hub [token](https://huggingface.co/docs/hub/en/security-tokens).
## Generate or serve
```bash Command
docker run --gpus all \
--shm-size 32g \
-p 30000:30000 \
-v ~/.cache/huggingface:/root/.cache/huggingface \
--env "HF_TOKEN=<secret>" \
--ipc=host \
lmsysorg/sglang:dev \
zsh -c '\
echo "Installing diffusion dependencies..." && \
pip install -e "python[diffusion]" && \
echo "Starting SGLang-Diffusion..." && \
sglang generate \
--model-path black-forest-labs/FLUX.1-dev \
--prompt "A logo With Bold Large text: SGL Diffusion" \
--save-output \
'
Run these commands inside the container or your activated Python environment. For a one-off image:
```bash
sglang generate --model-path Qwen/Qwen-Image \
--prompt "A beautiful sunset over the mountains" \
--save-output
```
Alternatively, start an HTTP server. Binding to `0.0.0.0` makes the published container port reachable from the host:
```bash
sglang serve --model-path Qwen/Qwen-Image --host 0.0.0.0 --port 30000
```
Choose a model and GPU configuration from the [cookbook](/cookbook/diffusion/intro). See the [OpenAI-compatible API](/docs/sglang-diffusion/api/openai_api) to send requests.
## Platform-Specific: ROCm (AMD GPUs)
For AMD Instinct GPUs (e.g., MI300X), use a ROCm-enabled Docker image from [lmsysorg/sglang](https://hub.docker.com/r/lmsysorg/sglang). The tag below is an example for ROCm 7.0 on MI300X and may lag the latest release tag:
```bash Command
docker run --device=/dev/kfd --device=/dev/dri --ipc=host \
-v ~/.cache/huggingface:/root/.cache/huggingface \
--env HF_TOKEN=<secret> \
lmsysorg/sglang:v0.5.5.post2-rocm700-mi30x \
sglang generate --model-path black-forest-labs/FLUX.1-dev --prompt "A logo With Bold Large text: SGL Diffusion" --save-output
```
For detailed ROCm system configuration and installation from source, see [AMD GPUs](../hardware-platforms/amd_gpu).
Docker is also recommended for AMD GPUs. Use a ROCm image matched to your GPU and host driver, not the NVIDIA `latest` image. Follow [AMD GPUs](/docs/hardware-platforms/amd_gpu) for the current image tags, device passthrough flags, and source installation options.
## Platform-Specific: MUSA (Moore Threads GPUs)
@@ -87,11 +103,11 @@ pip install -e "python[all_musa]"
## Platform-Specific: Intel XPU
For Intel Data Center GPU Max or Arc GPUs, follow the installation with docker in [XPU installation guide](../hardware-platforms/xpu). The dockerfile already include diffusion dependencies:
For Intel Data Center GPU Max or Arc GPUs, follow the Docker instructions in the [XPU installation guide](/docs/hardware-platforms/xpu). The Dockerfile already includes diffusion dependencies.
## Platform-Specific: Ascend NPU
For Ascend NPU, please follow the [NPU installation guide](../hardware-platforms/ascend-npus/getting-started/installation).
For Ascend NPU, please follow the [NPU installation guide](/docs/hardware-platforms/ascend-npus/getting-started/installation).
Quick test:
@@ -103,7 +119,7 @@ sglang generate --model-path black-forest-labs/FLUX.1-dev \
## Platform-Specific: Apple MPS
For Apple MPS, follow the instructions below to install from source. If the source tree still requires the alternate platform `pyproject` fallback, keep a backup of the default file before switching:
For Apple MPS, install natively on macOS using the source instructions below; the Linux Docker image is not an MPS runtime. If the source tree still requires the alternate platform `pyproject` fallback, keep a backup of the default file before switching:
```bash Command
# Install ffmpeg
+2 -2
View File
@@ -473,8 +473,8 @@ for (const path of walkMdx(DIFFUSION_COOKBOOK)) {
if (!quickStartBody.includes("<Deployment config={config} />")) {
fail(where, "Quick start must render the command builder before model capabilities");
}
if (!quickStartBody.includes('uv pip install "sglang[diffusion]"')) {
fail(where, "Quick start must include the diffusion installation command");
if (!quickStartBody.includes("(/docs/sglang-diffusion/installation)")) {
fail(where, "Quick start must link to the diffusion installation guide");
}
}
@@ -1001,16 +1001,11 @@ return {
dockerMounts: ["{{MEDIA_DIR}}:/data/minimax-h3:ro"],
dockerRunCommand: (s) =>
["mi300x", "mi355x"].includes(s.hw)
? `bash -lc 'python -m pip install -e "/sgl-workspace/sglang/python[diffusion_hip]" && exec sglang serve "$@"' --`
: `bash -lc 'python -m pip install -e "/sgl-workspace/sglang/python[diffusion]" && exec sglang serve "$@"' --`,
// Publish Docker only after the platform's H3 image/command has been validated.
runModes: (s) =>
["mi300x", "mi355x", "gb200", "gb300"].includes(s.hw)
? ["python"]
: ["python", "docker"],
: ["docker", "python"],
dockerImages: {
b200: "lmsysorg/sglang:dev",
+1 -5
View File
@@ -35,11 +35,7 @@ SGLang Diffusion supports Apple Silicon (M-series) via the MPS backend. Since Tr
## Getting Started
```bash
uv pip install 'sglang[diffusion]' --prerelease=allow
```
For more installation methods (e.g. pypi, uv, docker, ROCm/AMD, MUSA/Moore Threads), check the [installation guide](https://docs.sglang.io/docs/sglang-diffusion/installation).
Docker is recommended for Linux GPU deployments. Follow the [installation guide](https://docs.sglang.io/docs/sglang-diffusion/installation) to start a container with diffusion dependencies included, or choose pip/uv, source installation, or another platform's setup. Run the examples below inside that environment.
## Inference