[diffusion] docs: recommend docker for gpu deployments (#41966)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
This commit is contained in:
@@ -13,8 +13,8 @@ import { config } from '/src/snippets/configs/__HF_ORG__/__MODEL_SLUG__.jsx';
|
||||
|
||||
## 1. Quick start
|
||||
|
||||
Install with `uv pip install "sglang[diffusion]" --prerelease=allow`, then choose
|
||||
a verified recipe below.
|
||||
Follow the [SGLang Diffusion installation guide](/docs/sglang-diffusion/installation),
|
||||
then choose a verified recipe below. Docker is recommended for Linux GPU deployments.
|
||||
|
||||
<Deployment config={config} />
|
||||
|
||||
|
||||
@@ -11,13 +11,7 @@ import { config } from '/src/snippets/configs/CircleStone/anima.jsx';
|
||||
|
||||
## 1. Quick start
|
||||
|
||||
Install the diffusion dependencies on Linux with NVIDIA CUDA, then install this
|
||||
integration from its source checkout:
|
||||
|
||||
```bash Installation
|
||||
uv pip install "sglang[diffusion]" --prerelease=allow
|
||||
uv pip install -e "python[diffusion]"
|
||||
```
|
||||
On Linux with NVIDIA CUDA, use a nightly Docker image containing this integration, or install from source, following the [SGLang Diffusion installation guide](/docs/sglang-diffusion/installation). Run the commands below inside that environment.
|
||||
|
||||
Use the official `circlestone-labs/Anima-Base-v1.0-Diffusers` checkpoint. SGLang reads
|
||||
its `modular_model_index.json` directly; no checkpoint conversion or newer Diffusers
|
||||
|
||||
@@ -32,11 +32,7 @@ The default `flow_shift` is `3.0` for T2I, `10.0` for non-Edge video and all act
|
||||
|
||||
## 2. Installation
|
||||
|
||||
Install SGLang with the diffusion dependencies:
|
||||
|
||||
```bash Command
|
||||
pip install -e "python[diffusion]"
|
||||
```
|
||||
Follow the [SGLang Diffusion installation guide](/docs/sglang-diffusion/installation). Docker is recommended for Linux GPU deployments; run the commands below inside the container or your installed Python environment.
|
||||
|
||||
Cosmos3 guardrails are enabled by default when the package is available:
|
||||
|
||||
|
||||
@@ -21,13 +21,7 @@ This integration currently targets text-only image generation rather than image
|
||||
|
||||
## 2. Installation
|
||||
|
||||
Install SGLang with the diffusion dependencies:
|
||||
|
||||
```bash Command
|
||||
pip install -e "python[diffusion]"
|
||||
```
|
||||
|
||||
For full installation options, see the [SGLang Diffusion installation guide](/docs/sglang-diffusion/installation).
|
||||
Follow the [SGLang Diffusion installation guide](/docs/sglang-diffusion/installation). Docker is recommended for Linux GPU deployments; run the commands below inside the container or your installed Python environment.
|
||||
|
||||
## 3. Serve the model
|
||||
|
||||
|
||||
@@ -31,13 +31,7 @@ Review the model license on the [JoyAI-Echo Hugging Face page](https://huggingfa
|
||||
|
||||
## 2. SGLang-diffusion Installation
|
||||
|
||||
Install SGLang with diffusion dependencies:
|
||||
|
||||
```bash
|
||||
uv pip install "sglang[diffusion]" --prerelease=allow
|
||||
```
|
||||
|
||||
For platform-specific setup, see the [SGLang Diffusion installation guide](/docs/sglang-diffusion/installation).
|
||||
Follow the [SGLang Diffusion installation guide](/docs/sglang-diffusion/installation). Docker is recommended for Linux GPU deployments; run the commands below inside the container or your installed Python environment.
|
||||
|
||||
## 3. Model Deployment
|
||||
|
||||
|
||||
@@ -24,13 +24,7 @@ Use `Lightricks/LTX-2` or `Lightricks/LTX-2.3` as `--model-path`. SGLang loads t
|
||||
|
||||
## 2. SGLang-diffusion Installation
|
||||
|
||||
Install SGLang with diffusion dependencies:
|
||||
|
||||
```bash
|
||||
uv pip install "sglang[diffusion]" --prerelease=allow
|
||||
```
|
||||
|
||||
For platform-specific setup, see the [SGLang Diffusion installation guide](/docs/sglang-diffusion/installation).
|
||||
Follow the [SGLang Diffusion installation guide](/docs/sglang-diffusion/installation). Docker is recommended for Linux GPU deployments; run the commands below inside the container or your installed Python environment.
|
||||
|
||||
## 3. Model Deployment
|
||||
|
||||
|
||||
@@ -66,11 +66,7 @@ decoder choice does not change anything upstream of it.
|
||||
|
||||
## 2. SGLang-diffusion Installation
|
||||
|
||||
```bash
|
||||
uv pip install "sglang[diffusion]" --prerelease=allow
|
||||
```
|
||||
|
||||
For platform-specific setup, see the [SGLang Diffusion installation guide](/docs/sglang-diffusion/installation).
|
||||
Follow the [SGLang Diffusion installation guide](/docs/sglang-diffusion/installation). Docker is recommended for Linux GPU deployments; run the commands below inside the container or your installed Python environment.
|
||||
|
||||
NATTEN is an optional extra, worth installing only if you plan to use the
|
||||
[diffusion decoder](#4-6-diffusion-decoder) — see that section for why.
|
||||
|
||||
@@ -34,14 +34,7 @@ it is not an `extra_params` object.
|
||||
|
||||
## 2. Installation
|
||||
|
||||
Install SGLang with the diffusion dependencies:
|
||||
|
||||
```bash Command
|
||||
uv pip install "sglang[diffusion]" --prerelease=allow
|
||||
```
|
||||
|
||||
See the [SGLang Diffusion installation guide](/docs/sglang-diffusion/installation)
|
||||
for platform-specific setup.
|
||||
Follow the [SGLang Diffusion installation guide](/docs/sglang-diffusion/installation). Docker is recommended for Linux GPU deployments; run the commands below inside the container or your installed Python environment.
|
||||
|
||||
## 3. Serve LingBot Video MoE
|
||||
|
||||
|
||||
@@ -20,14 +20,7 @@ each request can disable it when lower latency matters more than the rewritten p
|
||||
|
||||
## 2. Installation
|
||||
|
||||
Install SGLang with the diffusion dependencies:
|
||||
|
||||
```bash Command
|
||||
pip install -e "python[diffusion]"
|
||||
```
|
||||
|
||||
For other installation options, see the
|
||||
[SGLang Diffusion installation guide](/docs/sglang-diffusion/installation).
|
||||
Follow the [SGLang Diffusion installation guide](/docs/sglang-diffusion/installation). Docker is recommended for Linux GPU deployments; run the commands below inside the container or your installed Python environment.
|
||||
|
||||
## 3. Serve the model
|
||||
|
||||
|
||||
@@ -13,8 +13,9 @@ import { config } from "/src/snippets/configs/MiniMaxAI/minimax-h3.jsx";
|
||||
|
||||
## 1. Quick start
|
||||
|
||||
Install with `uv pip install "sglang[diffusion]" --prerelease=allow`, then choose
|
||||
a verified recipe below. Setup changes the deployment; Server and Request expose
|
||||
Follow the [SGLang Diffusion installation guide](/docs/sglang-diffusion/installation),
|
||||
then choose a verified recipe below. Docker is recommended where the picker offers it.
|
||||
Setup changes the deployment; Server and Request expose
|
||||
orthogonal startup and sampling choices.
|
||||
|
||||
<Deployment config={config} />
|
||||
@@ -25,8 +26,8 @@ Change a Server option only for a deliberate trade-off; Request options do not
|
||||
reload the model.
|
||||
</Note>
|
||||
|
||||
The Docker form installs the platform-specific diffusion extra from the source
|
||||
bundled in the image. For conditioned requests, set **Host media directory**
|
||||
The Docker form uses the image's bundled diffusion dependencies without reinstalling
|
||||
them at startup. For conditioned requests, set **Host media directory**
|
||||
under **Variables**; the builder mounts it read-only at `/data/minimax-h3`.
|
||||
AMD and Grace Blackwell currently offer the Python form; the other NVIDIA
|
||||
recipes also offer Docker.
|
||||
|
||||
@@ -12,9 +12,9 @@ import { config } from '/src/snippets/configs/Qwen/qwen-image-2.1.jsx';
|
||||
|
||||
## 1. Quick start
|
||||
|
||||
Install the runtime dependencies with `uv pip install "sglang[diffusion]" --prerelease=allow`,
|
||||
then install this integration from its source checkout with
|
||||
`uv pip install -e "python[diffusion]"`. The picker uses `Qwen/Qwen-Image-2.1`;
|
||||
Use a nightly Docker image containing this integration, or install from source,
|
||||
following the [SGLang Diffusion installation guide](/docs/sglang-diffusion/installation).
|
||||
Run the commands below inside that environment. The picker uses `Qwen/Qwen-Image-2.1`;
|
||||
you can also set a local checkpoint directory under **Variables**. The recipes
|
||||
target NVIDIA CUDA on Linux; the picker marks which single-GPU workloads have
|
||||
been verified with the full checkpoint.
|
||||
|
||||
@@ -28,14 +28,7 @@ from the prompt with an optional `motion score: N.` suffix.
|
||||
|
||||
## 2. Installation
|
||||
|
||||
Install SGLang with the diffusion dependencies:
|
||||
|
||||
```bash Command
|
||||
uv pip install "sglang[diffusion]" --prerelease=allow
|
||||
```
|
||||
|
||||
See the [SGLang Diffusion installation guide](/docs/sglang-diffusion/installation)
|
||||
for platform-specific setup.
|
||||
Follow the [SGLang Diffusion installation guide](/docs/sglang-diffusion/installation). Docker is recommended for Linux GPU deployments; run the commands below inside the container or your installed Python environment.
|
||||
|
||||
## 3. Serve SANA-Video
|
||||
|
||||
|
||||
@@ -12,12 +12,7 @@ import { config } from '/src/snippets/configs/inclusionAI/ming-image.jsx';
|
||||
|
||||
## 1. Quick start
|
||||
|
||||
Install the dependencies, then this integration from its source checkout:
|
||||
|
||||
```bash Installation
|
||||
uv pip install "sglang[diffusion]" --prerelease=allow
|
||||
uv pip install -e "python[diffusion]" --no-deps
|
||||
```
|
||||
Use a nightly Docker image containing this integration, or install from source, following the [SGLang Diffusion installation guide](/docs/sglang-diffusion/installation). Run the commands below inside that environment.
|
||||
|
||||
Choose **Design** for generation or editing and **Design-Layer** for decomposition.
|
||||
The two checkpoints require separate servers. Start with one H200 on Linux/CUDA;
|
||||
|
||||
@@ -13,9 +13,9 @@ SGLang Diffusion is a high-performance inference framework for image and video g
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
uv pip install "sglang[diffusion]" --prerelease=allow
|
||||
```
|
||||
Start with the [recommended Docker setup](/docs/sglang-diffusion/installation) for Linux GPU deployments. The installation guide also covers pip/uv, source installation, and other platforms.
|
||||
|
||||
Run the following commands inside the container or your activated Python environment. Generate an image:
|
||||
|
||||
```bash
|
||||
sglang generate --model-path Qwen/Qwen-Image \
|
||||
@@ -23,8 +23,10 @@ sglang generate --model-path Qwen/Qwen-Image \
|
||||
--save-output
|
||||
```
|
||||
|
||||
Or start an HTTP server:
|
||||
|
||||
```bash
|
||||
sglang serve --model-path Qwen/Qwen-Image --port 30010
|
||||
sglang serve --model-path Qwen/Qwen-Image --host 0.0.0.0 --port 30000
|
||||
```
|
||||
|
||||
## Start Here
|
||||
|
||||
@@ -1,73 +1,89 @@
|
||||
---
|
||||
title: Install SGLang Diffusion
|
||||
description: Install SGLang Diffusion on NVIDIA, AMD, MUSA, and Ascend platforms.
|
||||
description: Run SGLang Diffusion with Docker, or install with pip, uv, or from source.
|
||||
---
|
||||
You can install SGLang-Diffusion using one of the methods below. The standard installation already includes SGLang's optimized kernel stack, including both `sgl-kernel` and JIT kernels used by diffusion workloads.
|
||||
Docker is the recommended setup for Linux GPU deployments. The official NVIDIA images include SGLang, its diffusion dependencies, and the optimized kernel stack; no separate Python installation or `pip install` inside the container is needed for the standard diffusion runtime. Optional backends and model-specific dependencies are documented in the [cookbook](/cookbook/diffusion/intro).
|
||||
|
||||
## Standard Installation (NVIDIA GPUs)
|
||||
## Standard installation (NVIDIA GPUs)
|
||||
|
||||
### Method 1: With pip or uv
|
||||
<Tabs>
|
||||
<Tab title="Docker (recommended)" id="method-3-using-docker">
|
||||
|
||||
It is recommended to use uv for a faster installation:
|
||||
Use Linux with a supported NVIDIA GPU, a driver compatible with the image's CUDA version, [Docker Engine](https://docs.docker.com/engine/install/), and [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html). Configure Docker's GPU runtime before continuing. See the [SGLang installation guide](/docs/get-started/install) for current CUDA requirements.
|
||||
|
||||
```bash Command
|
||||
pip install --upgrade pip
|
||||
The example uses the latest stable release. For reproducible deployments, replace `latest` with a release tag or image digest from [Docker Hub](https://hub.docker.com/r/lmsysorg/sglang/tags). For models or features not yet released, choose a [nightly image](https://hub.docker.com/r/lmsysorg/sglang/tags?name=nightly) containing the required changes, or install from source.
|
||||
|
||||
```bash
|
||||
docker pull lmsysorg/sglang:latest
|
||||
```
|
||||
|
||||
Start a container with persistent model and output directories. The working directory is `/outputs`, so generated files remain in the host's `outputs` directory after the container exits. The server port is published only on the host's loopback interface.
|
||||
|
||||
```bash
|
||||
mkdir -p "$HOME/.cache/huggingface" "$PWD/outputs"
|
||||
docker run --rm -it --gpus all \
|
||||
--ipc=host \
|
||||
-p 127.0.0.1:30000:30000 \
|
||||
-v "$HOME/.cache/huggingface:/root/.cache/huggingface" \
|
||||
-v "$PWD/outputs:/outputs" \
|
||||
-w /outputs \
|
||||
lmsysorg/sglang:latest bash
|
||||
```
|
||||
|
||||
For gated or private models, export your [Hugging Face token](https://huggingface.co/docs/hub/en/security-tokens) as `HF_TOKEN` on the host and add `--env HF_TOKEN` before the image name. Mount any local input media or checkpoints into the container and use their container paths in requests.
|
||||
|
||||
Run the commands in [Generate or serve](#generate-or-serve) inside this container. For development tools and source changes, see the [development environment guide](/docs/developer_guide/development_guide_using_docker#setup-docker-container).
|
||||
|
||||
</Tab>
|
||||
<Tab title="pip / uv" id="method-1-with-pip-or-uv">
|
||||
|
||||
Use this option for an existing Python environment. Follow the [Python and CUDA prerequisites](/docs/get-started/install), then install the diffusion extra in a virtual environment:
|
||||
|
||||
```bash
|
||||
pip install uv
|
||||
uv venv --python 3.12
|
||||
source .venv/bin/activate
|
||||
uv pip install "sglang[diffusion]" --prerelease=allow
|
||||
```
|
||||
|
||||
### Method 2: From source
|
||||
</Tab>
|
||||
<Tab title="From source" id="method-2-from-source">
|
||||
|
||||
```bash Command
|
||||
# Use the latest release branch
|
||||
Use source installation to develop SGLang or try changes that are not in a published image. Follow the [Python and CUDA prerequisites](/docs/get-started/install). The commands below install the current `main`; check out a release tag or commit before installation when you need a fixed revision.
|
||||
|
||||
```bash
|
||||
git clone https://github.com/sgl-project/sglang.git
|
||||
cd sglang
|
||||
|
||||
# Install the Python packages
|
||||
pip install --upgrade pip
|
||||
pip install -e "python[diffusion]"
|
||||
|
||||
# With uv
|
||||
pip install uv
|
||||
uv venv --python 3.12
|
||||
source .venv/bin/activate
|
||||
uv pip install -e "python[diffusion]" --prerelease=allow
|
||||
```
|
||||
|
||||
### Method 3: Using Docker
|
||||
</Tab>
|
||||
</Tabs>
|
||||
|
||||
The Docker images are available on Docker Hub at [lmsysorg/sglang](https://hub.docker.com/r/lmsysorg/sglang), built from the [Dockerfile](https://github.com/sgl-project/sglang/blob/main/docker/Dockerfile).
|
||||
Replace `<secret>` below with your HuggingFace Hub [token](https://huggingface.co/docs/hub/en/security-tokens).
|
||||
## Generate or serve
|
||||
|
||||
```bash Command
|
||||
docker run --gpus all \
|
||||
--shm-size 32g \
|
||||
-p 30000:30000 \
|
||||
-v ~/.cache/huggingface:/root/.cache/huggingface \
|
||||
--env "HF_TOKEN=<secret>" \
|
||||
--ipc=host \
|
||||
lmsysorg/sglang:dev \
|
||||
zsh -c '\
|
||||
echo "Installing diffusion dependencies..." && \
|
||||
pip install -e "python[diffusion]" && \
|
||||
echo "Starting SGLang-Diffusion..." && \
|
||||
sglang generate \
|
||||
--model-path black-forest-labs/FLUX.1-dev \
|
||||
--prompt "A logo With Bold Large text: SGL Diffusion" \
|
||||
--save-output \
|
||||
'
|
||||
Run these commands inside the container or your activated Python environment. For a one-off image:
|
||||
|
||||
```bash
|
||||
sglang generate --model-path Qwen/Qwen-Image \
|
||||
--prompt "A beautiful sunset over the mountains" \
|
||||
--save-output
|
||||
```
|
||||
|
||||
Alternatively, start an HTTP server. Binding to `0.0.0.0` makes the published container port reachable from the host:
|
||||
|
||||
```bash
|
||||
sglang serve --model-path Qwen/Qwen-Image --host 0.0.0.0 --port 30000
|
||||
```
|
||||
|
||||
Choose a model and GPU configuration from the [cookbook](/cookbook/diffusion/intro). See the [OpenAI-compatible API](/docs/sglang-diffusion/api/openai_api) to send requests.
|
||||
|
||||
## Platform-Specific: ROCm (AMD GPUs)
|
||||
|
||||
For AMD Instinct GPUs (e.g., MI300X), use a ROCm-enabled Docker image from [lmsysorg/sglang](https://hub.docker.com/r/lmsysorg/sglang). The tag below is an example for ROCm 7.0 on MI300X and may lag the latest release tag:
|
||||
|
||||
```bash Command
|
||||
docker run --device=/dev/kfd --device=/dev/dri --ipc=host \
|
||||
-v ~/.cache/huggingface:/root/.cache/huggingface \
|
||||
--env HF_TOKEN=<secret> \
|
||||
lmsysorg/sglang:v0.5.5.post2-rocm700-mi30x \
|
||||
sglang generate --model-path black-forest-labs/FLUX.1-dev --prompt "A logo With Bold Large text: SGL Diffusion" --save-output
|
||||
```
|
||||
|
||||
For detailed ROCm system configuration and installation from source, see [AMD GPUs](../hardware-platforms/amd_gpu).
|
||||
Docker is also recommended for AMD GPUs. Use a ROCm image matched to your GPU and host driver, not the NVIDIA `latest` image. Follow [AMD GPUs](/docs/hardware-platforms/amd_gpu) for the current image tags, device passthrough flags, and source installation options.
|
||||
|
||||
## Platform-Specific: MUSA (Moore Threads GPUs)
|
||||
|
||||
@@ -87,11 +103,11 @@ pip install -e "python[all_musa]"
|
||||
|
||||
## Platform-Specific: Intel XPU
|
||||
|
||||
For Intel Data Center GPU Max or Arc GPUs, follow the installation with docker in [XPU installation guide](../hardware-platforms/xpu). The dockerfile already include diffusion dependencies:
|
||||
For Intel Data Center GPU Max or Arc GPUs, follow the Docker instructions in the [XPU installation guide](/docs/hardware-platforms/xpu). The Dockerfile already includes diffusion dependencies.
|
||||
|
||||
## Platform-Specific: Ascend NPU
|
||||
|
||||
For Ascend NPU, please follow the [NPU installation guide](../hardware-platforms/ascend-npus/getting-started/installation).
|
||||
For Ascend NPU, please follow the [NPU installation guide](/docs/hardware-platforms/ascend-npus/getting-started/installation).
|
||||
|
||||
Quick test:
|
||||
|
||||
@@ -103,7 +119,7 @@ sglang generate --model-path black-forest-labs/FLUX.1-dev \
|
||||
|
||||
## Platform-Specific: Apple MPS
|
||||
|
||||
For Apple MPS, follow the instructions below to install from source. If the source tree still requires the alternate platform `pyproject` fallback, keep a backup of the default file before switching:
|
||||
For Apple MPS, install natively on macOS using the source instructions below; the Linux Docker image is not an MPS runtime. If the source tree still requires the alternate platform `pyproject` fallback, keep a backup of the default file before switching:
|
||||
|
||||
```bash Command
|
||||
# Install ffmpeg
|
||||
|
||||
@@ -473,8 +473,8 @@ for (const path of walkMdx(DIFFUSION_COOKBOOK)) {
|
||||
if (!quickStartBody.includes("<Deployment config={config} />")) {
|
||||
fail(where, "Quick start must render the command builder before model capabilities");
|
||||
}
|
||||
if (!quickStartBody.includes('uv pip install "sglang[diffusion]"')) {
|
||||
fail(where, "Quick start must include the diffusion installation command");
|
||||
if (!quickStartBody.includes("(/docs/sglang-diffusion/installation)")) {
|
||||
fail(where, "Quick start must link to the diffusion installation guide");
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -1001,16 +1001,11 @@ return {
|
||||
|
||||
dockerMounts: ["{{MEDIA_DIR}}:/data/minimax-h3:ro"],
|
||||
|
||||
dockerRunCommand: (s) =>
|
||||
["mi300x", "mi355x"].includes(s.hw)
|
||||
? `bash -lc 'python -m pip install -e "/sgl-workspace/sglang/python[diffusion_hip]" && exec sglang serve "$@"' --`
|
||||
: `bash -lc 'python -m pip install -e "/sgl-workspace/sglang/python[diffusion]" && exec sglang serve "$@"' --`,
|
||||
|
||||
// Publish Docker only after the platform's H3 image/command has been validated.
|
||||
runModes: (s) =>
|
||||
["mi300x", "mi355x", "gb200", "gb300"].includes(s.hw)
|
||||
? ["python"]
|
||||
: ["python", "docker"],
|
||||
: ["docker", "python"],
|
||||
|
||||
dockerImages: {
|
||||
b200: "lmsysorg/sglang:dev",
|
||||
|
||||
@@ -35,11 +35,7 @@ SGLang Diffusion supports Apple Silicon (M-series) via the MPS backend. Since Tr
|
||||
|
||||
## Getting Started
|
||||
|
||||
```bash
|
||||
uv pip install 'sglang[diffusion]' --prerelease=allow
|
||||
```
|
||||
|
||||
For more installation methods (e.g. pypi, uv, docker, ROCm/AMD, MUSA/Moore Threads), check the [installation guide](https://docs.sglang.io/docs/sglang-diffusion/installation).
|
||||
Docker is recommended for Linux GPU deployments. Follow the [installation guide](https://docs.sglang.io/docs/sglang-diffusion/installation) to start a container with diffusion dependencies included, or choose pip/uv, source installation, or another platform's setup. Run the examples below inside that environment.
|
||||
|
||||
## Inference
|
||||
|
||||
|
||||
Reference in New Issue
Block a user