[diffusion] doc: minor update docs (#13177)
This commit is contained in:
@@ -2,9 +2,9 @@
|
|||||||
<img src=https://github.com/lm-sys/lm-sys.github.io/releases/download/test/sgl-diffusion-logo.png width="80%"/>
|
<img src=https://github.com/lm-sys/lm-sys.github.io/releases/download/test/sgl-diffusion-logo.png width="80%"/>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
**sgl-diffusion is an inference framework for accelerated image/video generation.**
|
**SGLang diffusion is an inference framework for accelerated image/video generation.**
|
||||||
|
|
||||||
SGLang-Diffusion features an end-to-end unified pipeline for accelerating diffusion models. It is designed to be modular and extensible, allowing users to easily add new models and optimizations.
|
SGLang diffusion features an end-to-end unified pipeline for accelerating diffusion models. It is designed to be modular and extensible, allowing users to easily add new models and optimizations.
|
||||||
|
|
||||||
## Key Features
|
## Key Features
|
||||||
|
|
||||||
|
|||||||
@@ -46,7 +46,7 @@ class ModelConfig:
|
|||||||
# Diffuser/Transformer parameters
|
# Diffuser/Transformer parameters
|
||||||
arch_config: ArchConfig = field(default_factory=ArchConfig)
|
arch_config: ArchConfig = field(default_factory=ArchConfig)
|
||||||
|
|
||||||
# sgl-diffusion-specific parameters here
|
# sglang-diffusion-specific parameters here
|
||||||
# i.e. STA, quantization, teacache
|
# i.e. STA, quantization, teacache
|
||||||
|
|
||||||
def __getattr__(self, name):
|
def __getattr__(self, name):
|
||||||
|
|||||||
@@ -43,7 +43,7 @@ class DiTArchConfig(ArchConfig):
|
|||||||
class DiTConfig(ModelConfig):
|
class DiTConfig(ModelConfig):
|
||||||
arch_config: DiTArchConfig = field(default_factory=DiTArchConfig)
|
arch_config: DiTArchConfig = field(default_factory=DiTArchConfig)
|
||||||
|
|
||||||
# sgl-diffusionDiT-specific parameters
|
# sglang-diffusion DiT-specific parameters
|
||||||
prefix: str = ""
|
prefix: str = ""
|
||||||
quant_config: QuantizationConfig | None = None
|
quant_config: QuantizationConfig | None = None
|
||||||
|
|
||||||
|
|||||||
@@ -26,7 +26,7 @@ class VAEArchConfig(ArchConfig):
|
|||||||
class VAEConfig(ModelConfig):
|
class VAEConfig(ModelConfig):
|
||||||
arch_config: VAEArchConfig = field(default_factory=VAEArchConfig)
|
arch_config: VAEArchConfig = field(default_factory=VAEArchConfig)
|
||||||
|
|
||||||
# sgl-diffusionVAE-specific parameters
|
# sglang-diffusion VAE-specific parameters
|
||||||
load_encoder: bool = True
|
load_encoder: bool = True
|
||||||
load_decoder: bool = True
|
load_decoder: bool = True
|
||||||
|
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
# Attention Kernel Used in sgl-diffusion
|
# Attention Kernel Used in SGLang diffusion
|
||||||
|
|
||||||
## VMoBA: Mixture-of-Block Attention for Video Diffusion Models (VMoBA)
|
## VMoBA: Mixture-of-Block Attention for Video Diffusion Models (VMoBA)
|
||||||
|
|
||||||
|
|||||||
@@ -1,11 +1,11 @@
|
|||||||
# sgl-diffusion CLI Inference
|
# SGLang diffusion CLI Inference
|
||||||
|
|
||||||
The sgl-diffusion CLI provides a quick way to access the sgl-diffusion inference pipeline for image and video generation.
|
The SGLang-diffusion CLI provides a quick way to access the inference pipeline for image and video generation.
|
||||||
|
|
||||||
## Prerequisites
|
## Prerequisites
|
||||||
|
|
||||||
- A working sgl-diffusion installation and the `sgl-diffusion` CLI available in `$PATH`.
|
- A working SGLang diffusion installation and the `sglang` CLI available in `$PATH`.
|
||||||
- Python 3.10+ if you plan to use the OpenAI Python SDK.
|
- Python 3.11+ if you plan to use the OpenAI Python SDK.
|
||||||
|
|
||||||
|
|
||||||
## Supported Arguments
|
## Supported Arguments
|
||||||
@@ -127,7 +127,7 @@ sglang generate --help
|
|||||||
|
|
||||||
## Serve
|
## Serve
|
||||||
|
|
||||||
Launch the sgl-diffusion HTTP server and interact with it using the OpenAI SDK and curl. The server implements an OpenAI-compatible subset for Videos under the `/v1/videos` namespace.
|
Launch the SGLang diffusion HTTP server and interact with it using the OpenAI SDK and curl. The server implements an OpenAI-compatible subset for Videos under the `/v1/videos` namespace.
|
||||||
|
|
||||||
### Start the server
|
### Start the server
|
||||||
|
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# Install sgl-diffusion
|
# Install SGLang-diffusion
|
||||||
|
|
||||||
You can install sgl-diffusion using one of the methods below.
|
You can install sglang-diffusion using one of the methods below.
|
||||||
|
|
||||||
This page primarily applies to common NVIDIA GPU platforms.
|
This page primarily applies to common NVIDIA GPU platforms.
|
||||||
|
|
||||||
|
|||||||
@@ -10,7 +10,7 @@ The symbols used have the following meanings:
|
|||||||
|
|
||||||
## Models x Optimization
|
## Models x Optimization
|
||||||
|
|
||||||
The `HuggingFace Model ID` can be passed directly to `from_pretrained()` methods, and sgl-diffusion will use the optimal
|
The `HuggingFace Model ID` can be passed directly to `from_pretrained()` methods, and sglang-diffusion will use the optimal
|
||||||
default parameters when initializing and generating videos.
|
default parameters when initializing and generating videos.
|
||||||
|
|
||||||
### Video Generation Models
|
### Video Generation Models
|
||||||
|
|||||||
@@ -1,10 +1,12 @@
|
|||||||
# How to Support New Diffusion Models
|
# How to Support New Diffusion Models
|
||||||
|
|
||||||
This document explains how to add support for new diffusion models in SGLang Diffusion.
|
This document explains how to add support for new diffusion models in SGLang diffusion.
|
||||||
|
|
||||||
## Architecture Overview
|
## Architecture Overview
|
||||||
|
|
||||||
SGLang Diffusion is engineered for both performance and flexibility, built upon a modular pipeline architecture. This design allows developers to easily construct complex, customized pipelines for various diffusion models by combining and reusing different components.
|
SGLang diffusion is engineered for both performance and flexibility, built upon a modular pipeline architecture. This
|
||||||
|
design allows developers to easily construct complex, customized pipelines for various diffusion models by combining and
|
||||||
|
reusing different components.
|
||||||
|
|
||||||
At its core, the architecture revolves around two key concepts, as highlighted in our [blog post](https://lmsys.org/blog/2025-11-07-sglang-diffusion/#architecture):
|
At its core, the architecture revolves around two key concepts, as highlighted in our [blog post](https://lmsys.org/blog/2025-11-07-sglang-diffusion/#architecture):
|
||||||
|
|
||||||
@@ -101,4 +103,5 @@ To illustrate the process, let's look at how `Qwen-Image-Edit` is implemented. T
|
|||||||
5. **Register Configs**:
|
5. **Register Configs**:
|
||||||
- Register the configs in the central registry ([`registry.py`](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/registry.py)) via `_register_configs` to enable automatic loading and instantiation for the model. Modules are automatically loaded and injected based on the config and repository structure.
|
- Register the configs in the central registry ([`registry.py`](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/registry.py)) via `_register_configs` to enable automatic loading and instantiation for the model. Modules are automatically loaded and injected based on the config and repository structure.
|
||||||
|
|
||||||
By following this modular pattern of defining configurations and composing pipelines, you can integrate new diffusion models into SGLang with clarity and ease.
|
By following this pattern of defining configurations and composing pipelines, you can integrate new diffusion models
|
||||||
|
into SGLang with ease.
|
||||||
|
|||||||
@@ -163,7 +163,7 @@ def maybe_convert_int(value: str | None) -> int | None:
|
|||||||
|
|
||||||
environment_variables: dict[str, Callable[[], Any]] = {
|
environment_variables: dict[str, Callable[[], Any]] = {
|
||||||
# ================== Installation Time Env Vars ==================
|
# ================== Installation Time Env Vars ==================
|
||||||
# Target device of sgl-diffusion, supporting [cuda (by default),
|
# Target device of sglang-diffusion, supporting [cuda (by default),
|
||||||
# rocm, neuron, cpu, openvino]
|
# rocm, neuron, cpu, openvino]
|
||||||
"SGLANG_DIFFUSION_TARGET_DEVICE": lambda: os.getenv(
|
"SGLANG_DIFFUSION_TARGET_DEVICE": lambda: os.getenv(
|
||||||
"SGLANG_DIFFUSION_TARGET_DEVICE", "cuda"
|
"SGLANG_DIFFUSION_TARGET_DEVICE", "cuda"
|
||||||
|
|||||||
@@ -74,7 +74,7 @@ class PyNcclCommunicator:
|
|||||||
self.available = True
|
self.available = True
|
||||||
self.disabled = False
|
self.disabled = False
|
||||||
|
|
||||||
logger.info("sgl-diffusion is using nccl==%s", self.nccl.ncclGetVersion())
|
logger.info("sglang-diffusion is using nccl==%s", self.nccl.ncclGetVersion())
|
||||||
|
|
||||||
if self.rank == 0:
|
if self.rank == 0:
|
||||||
# get the unique id from NCCL
|
# get the unique id from NCCL
|
||||||
|
|||||||
@@ -13,7 +13,7 @@
|
|||||||
# Copyright 2023 The vLLM team.
|
# Copyright 2023 The vLLM team.
|
||||||
# Copyright (c) 2022, NVIDIA CORPORATION. All rights reserved.
|
# Copyright (c) 2022, NVIDIA CORPORATION. All rights reserved.
|
||||||
|
|
||||||
"""sgl-diffusion distributed state.
|
"""sglang-diffusion distributed state.
|
||||||
|
|
||||||
It takes over the control of the distributed environment from PyTorch.
|
It takes over the control of the distributed environment from PyTorch.
|
||||||
The typical workflow is:
|
The typical workflow is:
|
||||||
|
|||||||
@@ -62,7 +62,7 @@ def generate_cmd(args: argparse.Namespace):
|
|||||||
|
|
||||||
|
|
||||||
class GenerateSubcommand(CLISubcommand):
|
class GenerateSubcommand(CLISubcommand):
|
||||||
"""The `generate` subcommand for the sgl-diffusion CLI"""
|
"""The `generate` subcommand for the sglang-diffusion CLI"""
|
||||||
|
|
||||||
def __init__(self) -> None:
|
def __init__(self) -> None:
|
||||||
self.name = "generate"
|
self.name = "generate"
|
||||||
|
|||||||
@@ -21,7 +21,7 @@ def cmd_init() -> list[CLISubcommand]:
|
|||||||
|
|
||||||
|
|
||||||
def main() -> None:
|
def main() -> None:
|
||||||
parser = FlexibleArgumentParser(description="sgl-diffusion CLI")
|
parser = FlexibleArgumentParser(description="sglang-diffusion CLI")
|
||||||
parser.add_argument("-v", "--version", action="version", version="0.1.0")
|
parser.add_argument("-v", "--version", action="version", version="0.1.0")
|
||||||
|
|
||||||
subparsers = parser.add_subparsers(required=False, dest="subparser")
|
subparsers = parser.add_subparsers(required=False, dest="subparser")
|
||||||
|
|||||||
@@ -35,7 +35,7 @@ def execute_serve_cmd(args: argparse.Namespace, unknown_args: list[str] | None =
|
|||||||
|
|
||||||
|
|
||||||
class ServeSubcommand(CLISubcommand):
|
class ServeSubcommand(CLISubcommand):
|
||||||
"""The `serve` subcommand for the sgl-diffusion CLI"""
|
"""The `serve` subcommand for the sglang-diffusion CLI"""
|
||||||
|
|
||||||
def __init__(self) -> None:
|
def __init__(self) -> None:
|
||||||
self.name = "serve"
|
self.name = "serve"
|
||||||
|
|||||||
@@ -2,7 +2,7 @@
|
|||||||
|
|
||||||
# SPDX-License-Identifier: Apache-2.0
|
# SPDX-License-Identifier: Apache-2.0
|
||||||
"""
|
"""
|
||||||
DiffGenerator module for sgl-diffusion.
|
DiffGenerator module for sglang-diffusion.
|
||||||
|
|
||||||
This module provides a consolidated interface for generating videos using
|
This module provides a consolidated interface for generating videos using
|
||||||
diffusion models.
|
diffusion models.
|
||||||
|
|||||||
@@ -2,7 +2,7 @@
|
|||||||
|
|
||||||
# SPDX-License-Identifier: Apache-2.0
|
# SPDX-License-Identifier: Apache-2.0
|
||||||
"""
|
"""
|
||||||
DiffGenerator module for sgl-diffusion.
|
DiffGenerator module for sglang-diffusion.
|
||||||
|
|
||||||
This module provides a consolidated interface for generating videos using
|
This module provides a consolidated interface for generating videos using
|
||||||
diffusion models.
|
diffusion models.
|
||||||
|
|||||||
@@ -66,7 +66,7 @@ def launch_server(server_args: ServerArgs, launch_http_server: bool = True):
|
|||||||
task_pipes_to_slaves_w,
|
task_pipes_to_slaves_w,
|
||||||
result_pipes_from_slaves_r,
|
result_pipes_from_slaves_r,
|
||||||
),
|
),
|
||||||
name=f"sgl-diffusionWorker-{i}",
|
name=f"sglang-diffusionWorker-{i}",
|
||||||
daemon=True,
|
daemon=True,
|
||||||
)
|
)
|
||||||
else: # Slave workers
|
else: # Slave workers
|
||||||
@@ -83,7 +83,7 @@ def launch_server(server_args: ServerArgs, launch_http_server: bool = True):
|
|||||||
task_pipes_to_slaves_r[i - 1],
|
task_pipes_to_slaves_r[i - 1],
|
||||||
result_pipes_from_slaves_w[i - 1],
|
result_pipes_from_slaves_w[i - 1],
|
||||||
),
|
),
|
||||||
name=f"sgl-diffusionWorker-{i}",
|
name=f"sglang-diffusionWorker-{i}",
|
||||||
daemon=True,
|
daemon=True,
|
||||||
)
|
)
|
||||||
scheduler_pipe_readers.append(reader)
|
scheduler_pipe_readers.append(reader)
|
||||||
|
|||||||
@@ -41,7 +41,7 @@ def backend_name_to_enum(backend_name: str) -> AttentionBackendEnum | None:
|
|||||||
|
|
||||||
def get_env_variable_attn_backend() -> AttentionBackendEnum | None:
|
def get_env_variable_attn_backend() -> AttentionBackendEnum | None:
|
||||||
"""
|
"""
|
||||||
Get the backend override specified by the sgl-diffusion attention
|
Get the backend override specified by the sglang-diffusion attention
|
||||||
backend environment variable, if one is specified.
|
backend environment variable, if one is specified.
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
@@ -169,7 +169,7 @@ def global_force_attn_backend_context_manager(
|
|||||||
attn_backend: AttentionBackendEnum,
|
attn_backend: AttentionBackendEnum,
|
||||||
) -> Generator[None, None, None]:
|
) -> Generator[None, None, None]:
|
||||||
"""
|
"""
|
||||||
Globally force a sgl-diffusion attention backend override within a
|
Globally force a sglang-diffusion attention backend override within a
|
||||||
context manager, reverting the global attention backend
|
context manager, reverting the global attention backend
|
||||||
override to its prior state upon exiting the context
|
override to its prior state upon exiting the context
|
||||||
manager.
|
manager.
|
||||||
|
|||||||
@@ -4,7 +4,7 @@
|
|||||||
|
|
||||||
# Adapted from torchtune
|
# Adapted from torchtune
|
||||||
# Copyright 2024 The TorchTune Authors.
|
# Copyright 2024 The TorchTune Authors.
|
||||||
# Copyright 2025 The sgl-diffusion Authors.
|
# Copyright 2025 The sglang-diffusion Authors.
|
||||||
|
|
||||||
import contextlib
|
import contextlib
|
||||||
from collections.abc import Callable, Generator
|
from collections.abc import Callable, Generator
|
||||||
|
|||||||
@@ -3,7 +3,7 @@
|
|||||||
# SPDX-License-Identifier: Apache-2.0
|
# SPDX-License-Identifier: Apache-2.0
|
||||||
# Adapted from diffusers
|
# Adapted from diffusers
|
||||||
|
|
||||||
# Copyright 2024 The Hunyuan Team, The HuggingFace Team and The sgl-diffusion Team. All rights reserved.
|
# Copyright 2024 The Hunyuan Team, The HuggingFace Team and The sglang-diffusion Team. All rights reserved.
|
||||||
#
|
#
|
||||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
# you may not use this file except in compliance with the License.
|
# you may not use this file except in compliance with the License.
|
||||||
|
|||||||
@@ -2,7 +2,7 @@
|
|||||||
|
|
||||||
# SPDX-License-Identifier: Apache-2.0
|
# SPDX-License-Identifier: Apache-2.0
|
||||||
# Inspired by SGLang: https://github.com/sgl-project/sglang/blob/main/python/sglang/srt/server_args.py
|
# Inspired by SGLang: https://github.com/sgl-project/sglang/blob/main/python/sglang/srt/server_args.py
|
||||||
"""The arguments of sgl-diffusion Inference."""
|
"""The arguments of sglang-diffusion Inference."""
|
||||||
import argparse
|
import argparse
|
||||||
import dataclasses
|
import dataclasses
|
||||||
import inspect
|
import inspect
|
||||||
@@ -393,7 +393,7 @@ class ServerArgs:
|
|||||||
type=str,
|
type=str,
|
||||||
choices=ExecutionMode.choices(),
|
choices=ExecutionMode.choices(),
|
||||||
default=ServerArgs.mode.value,
|
default=ServerArgs.mode.value,
|
||||||
help="The mode to run sgl-diffusion",
|
help="The mode to run SGLang-diffusion",
|
||||||
)
|
)
|
||||||
|
|
||||||
# Workload type
|
# Workload type
|
||||||
|
|||||||
@@ -450,8 +450,8 @@ def import_pynvml():
|
|||||||
status without initializing CUDA context in the current process.
|
status without initializing CUDA context in the current process.
|
||||||
Historically, there are two packages that provide pynvml:
|
Historically, there are two packages that provide pynvml:
|
||||||
- `nvidia-ml-py` (https://pypi.org/project/nvidia-ml-py/): The official
|
- `nvidia-ml-py` (https://pypi.org/project/nvidia-ml-py/): The official
|
||||||
wrapper. It is a dependency of sgl-diffusion, and is installed when users
|
wrapper. It is a dependency of sglang-diffusion, and is installed when users
|
||||||
install sgl-diffusion. It provides a Python module named `pynvml`.
|
install sglang-diffusion. It provides a Python module named `pynvml`.
|
||||||
- `pynvml` (https://pypi.org/project/pynvml/): An unofficial wrapper.
|
- `pynvml` (https://pypi.org/project/pynvml/): An unofficial wrapper.
|
||||||
Prior to version 12.0, it also provides a Python module `pynvml`,
|
Prior to version 12.0, it also provides a Python module `pynvml`,
|
||||||
and therefore conflicts with the official one which is a standalone Python file.
|
and therefore conflicts with the official one which is a standalone Python file.
|
||||||
|
|||||||
Reference in New Issue
Block a user