minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

[diffusion] Keep LongLive2 components resident on large GPUs (#35993)

Browse Source
This commit is contained in:
Xiaoyu Zhang
2026-08-24 12:06:52 +08:00
committed by GitHub
parent acba8921bf
commit 09592f5889
4 changed files with 47 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/cookbook/diffusion/LongLive/LongLive-2.0.mdx
+5
View File
@@ -25,6 +25,11 @@ Please refer to the [official SGLang-diffusion installation guide](/docs/sglang-
sglang serve --model-path Rabinovich/LongLive-2.0-5B-Diffusers
```
In `auto` mode, GPUs with at least 60 GiB available keep the DiT, text encoder,
and VAE resident. This uses about 44 GiB on H200 for the 832x480 preset and
avoids moving encoder and decoder layers from host memory on every request.
Smaller GPUs retain the layerwise-offload defaults.
If the GPU runs out of memory, move the text encoder, VAE, and DiT to CPU between stages:
```bash Command