[diffusion] model: support ltx-2.5 (#34471)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: Mick <mickjagger19@icloud.com>
This commit is contained in:
Yihao Wang
co-authored by
Claude Opus 5
Mick
Claude Opus 5
Mick
parent
e331baaaa8
commit
5c0ace30c0
@@ -0,0 +1,317 @@
|
||||
---
|
||||
title: LTX2.5
|
||||
description: Run LTX-2.5 video + audio generation with SGLang Diffusion.
|
||||
metatags:
|
||||
description: "Deploy and use the LTX-2.5 video and audio generation model with SGLang Diffusion, including one-stage, two-stage, image-to-video, auto-duration, and diffusion-decoder examples."
|
||||
---
|
||||
|
||||
import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';
|
||||
import { LTX25Deployment } from '/src/snippets/diffusion/ltx25-deployment.jsx';
|
||||
|
||||
<DiffusionModelTags tags={["video", "audio", "text-to-video", "image-to-video", "two-stage", "auto-duration", "diffusion decoder"]} />
|
||||
|
||||
## 1. Model Introduction
|
||||
|
||||
[LTX-2.5](https://huggingface.co/Lightricks/LTX-2.5) is an open world model from
|
||||
Lightricks, built for local execution and fine-tuning. Its established use is
|
||||
generating synchronized, high-fidelity video and audio from text, image and
|
||||
video inputs.
|
||||
|
||||
It is a 22B DiT paired with a Gemma-4-12B text encoder, separate video and audio
|
||||
VAEs, and a vocoder that outputs 48 kHz stereo. Video and audio are denoised
|
||||
jointly in one pass rather than dubbed afterwards, so they stay in sync.
|
||||
|
||||
Use **`Lightricks/LTX-2.5-Diffusers`** as `--model-path`.
|
||||
|
||||
<Warning>
|
||||
**License notice:** LTX-2.5 is released under the LTX-2.x Community License
|
||||
Agreement, not Apache 2.0. The license includes commercial-use restrictions for
|
||||
some entities. Review the [official Lightricks license](https://github.com/Lightricks/LTX-2/blob/main/LICENSE.md)
|
||||
before production or commercial use; SGLang support does not grant additional
|
||||
model usage rights.
|
||||
</Warning>
|
||||
|
||||
### 1.1 New in LTX-2.5
|
||||
|
||||
Two capabilities have no equivalent in LTX-2 / LTX-2.3:
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="Auto-duration" icon="clock" href="#4-3-auto-duration">
|
||||
A duration head predicts how long the shot the caption implies should run,
|
||||
and picks the frame count for you. Pass `--auto-duration` instead of
|
||||
`--num-frames`.
|
||||
</Card>
|
||||
<Card title="Diffusion decoder" icon="wand-magic-sparkles" href="#4-6-diffusion-decoder">
|
||||
A diffusion model replaces the convolutional VAE decoder for the
|
||||
latent-to-pixel step. Enable with `--use-diffusion-decoder`.
|
||||
</Card>
|
||||
</CardGroup>
|
||||
|
||||
Both are optional and off by default.
|
||||
|
||||
### 1.2 Components
|
||||
|
||||
| Path | Component | Used by |
|
||||
| --- | --- | --- |
|
||||
| `transformer/` | Distilled DiT (the default) | always |
|
||||
| `transformer_full/` | Full / SFT DiT | `--model-variant dev` |
|
||||
| `vae/` | Convolutional video VAE | encode always; decode by default |
|
||||
| `diffusion_decoder/` | Diffusion video decoder, decoder-only | `--use-diffusion-decoder` |
|
||||
| `latent_upsampler/` | Spatial x2 latent upsampler | `LTX2TwoStagePipeline` |
|
||||
| `duration_head/` | Predicts clip length from the caption | `--auto-duration` |
|
||||
| `audio_vae/`, `vocoder/`, `connectors/`, `text_encoder/`, `tokenizer/`, `scheduler/` | Shared | always |
|
||||
|
||||
Encoding always uses `vae/`, and both decoders consume the same latents, so the
|
||||
decoder choice does not change anything upstream of it.
|
||||
|
||||
## 2. SGLang-diffusion Installation
|
||||
|
||||
```bash
|
||||
uv pip install "sglang[diffusion]" --prerelease=allow
|
||||
```
|
||||
|
||||
For platform-specific setup, see the [SGLang Diffusion installation guide](/docs/sglang-diffusion/installation).
|
||||
|
||||
NATTEN is an optional extra, worth installing only if you plan to use the
|
||||
[diffusion decoder](#4-6-diffusion-decoder) — see that section for why.
|
||||
|
||||
## 3. Model Deployment
|
||||
|
||||
### 3.1 Basic Configuration
|
||||
|
||||
```bash
|
||||
sglang serve \
|
||||
--model-path Lightricks/LTX-2.5-Diffusers \
|
||||
--pipeline-class-name LTX2Pipeline
|
||||
```
|
||||
|
||||
On a single high-VRAM GPU no extra flags are needed.
|
||||
|
||||
**Interactive Command Generator**: pick a target and the features you want; the
|
||||
command updates below. Server-side choices (pipeline class, weights variant,
|
||||
parallelism) go on `sglang serve`, while per-request choices (auto-duration,
|
||||
diffusion decoder, resolution) are listed separately, since they belong on the
|
||||
`sglang generate` call or the request body.
|
||||
|
||||
<LTX25Deployment />
|
||||
|
||||
### 3.2 Configuration Tips
|
||||
|
||||
Choose the pipeline class based on the quality and latency target:
|
||||
|
||||
| Use case | Pipeline class | Notes |
|
||||
| --- | --- | --- |
|
||||
| One-stage generation | `LTX2Pipeline` | Fastest path. Supports T2V and TI2V, auto-duration and the diffusion decoder. |
|
||||
| Two-stage generation | `LTX2TwoStagePipeline` | Half-resolution base stage, x2 latent upsample, then a short refinement. Pass the **final** resolution. |
|
||||
|
||||
There is no HQ pipeline class for LTX-2.5, and no `--distilled-lora-path` for
|
||||
either weights variant: LTX-2.5 distils the weights themselves rather than
|
||||
merging a LoRA per stage, so `--ltx2-two-stage-device-mode` (which governs that
|
||||
swap) does not apply either.
|
||||
|
||||
Every feature on this page — text-to-video, image conditioning, auto-duration,
|
||||
the diffusion decoder, and either weights variant — works with both pipeline
|
||||
classes.
|
||||
|
||||
Selecting weights:
|
||||
|
||||
- `--model-variant dev` serves the full / SFT DiT from `transformer_full/`; the
|
||||
default is the distilled one. See [section 4.5](#4-5-the-dev-transformer).
|
||||
|
||||
### 3.3 Multi-GPU presets
|
||||
|
||||
| Target | Recommended server flags | Notes |
|
||||
| --- | --- | --- |
|
||||
| 1 high-VRAM GPU | *(no extra flags)* | 960×544 fits comfortably on an H200. |
|
||||
| 1 tight-VRAM GPU | `--quantization fp8` | Halves the DiT and cuts peak memory ~18 GB at unchanged speed. See [section 3.4](#3-4-fp8-quantization). |
|
||||
| 1 very tight GPU | `--dit-layerwise-offload` | Cuts peak memory by roughly 10 GB, at about 4x the wall clock. |
|
||||
| 2 GPUs, long sequences | `--num-gpus 2 --ulysses-degree 2` | Sequence parallel; the memory/long-sequence tool. |
|
||||
| 2 GPUs, large DiT | `--num-gpus 2 --tp-size 2` | Tensor parallel across attention heads. |
|
||||
| 2 GPUs, dev weights | `--num-gpus 2 --enable-cfg-parallel` | Splits the guided and unguided branches across GPUs. Measured 1.77x on denoising (15.1s to 8.5s, 960×544 / 57 frames / 30 steps). |
|
||||
|
||||
<Warning>
|
||||
**CFG parallelism does not apply on the default (distilled) path.** That DiT
|
||||
runs unguided, so there is no negative branch to split across GPUs and
|
||||
`--enable-cfg-parallel` buys nothing — the CFG-parallel presets on the
|
||||
LTX-2 / LTX-2.3 page do not carry over. It *is* worth using with
|
||||
`--model-variant dev`, which runs with guidance.
|
||||
</Warning>
|
||||
|
||||
### 3.4 fp8 quantization
|
||||
|
||||
`--quantization fp8` quantizes the DiT's linear layers as it loads them, so it
|
||||
needs no pre-quantized checkpoint:
|
||||
|
||||
```bash
|
||||
sglang serve \
|
||||
--model-path Lightricks/LTX-2.5-Diffusers \
|
||||
--pipeline-class-name LTX2Pipeline \
|
||||
--quantization fp8
|
||||
```
|
||||
|
||||
At 960×544 / 49 frames the transformer loads in 18.11 GB against 35.37 GB for
|
||||
bf16, and the run peaks at 53.5 GB against 71.1 GB. Denoising time is
|
||||
unchanged: the distilled 8-step path at this size is bound by memory traffic
|
||||
rather than matmul throughput, so fp8 buys headroom rather than speed.
|
||||
|
||||
Expect a different sample for a given seed. Quantization nudges the denoising
|
||||
trajectory and diffusion amplifies that, so the result differs from bf16
|
||||
without being worse.
|
||||
|
||||
## 4. Model Invocation
|
||||
|
||||
### 4.1 Text-to-video with audio
|
||||
|
||||
```bash
|
||||
sglang generate \
|
||||
--model-path Lightricks/LTX-2.5-Diffusers \
|
||||
--pipeline-class-name LTX2Pipeline \
|
||||
--prompt "A cinematic shot of a red fox walking through a snowy forest at dawn, the camera tracking alongside, snow crunching underfoot." \
|
||||
--save-output
|
||||
```
|
||||
|
||||
Defaults: 960×544, 121 frames, 24 fps. Video and audio are generated jointly and
|
||||
muxed into one MP4.
|
||||
|
||||
The default DiT is distilled and runs off a fixed 8-sigma schedule rather than a
|
||||
step count, so `--num-inference-steps` and `--guidance-scale` have no effect
|
||||
here. Use [`--model-variant dev`](#4-5-the-dev-transformer) when you want
|
||||
control over either.
|
||||
|
||||
### 4.2 Image-to-video
|
||||
|
||||
```bash
|
||||
sglang generate \
|
||||
--model-path Lightricks/LTX-2.5-Diffusers \
|
||||
--pipeline-class-name LTX2Pipeline \
|
||||
--image-path ./inputs/start.png \
|
||||
--prompt "The camera pushes forward as the subject turns toward the light." \
|
||||
--save-output
|
||||
```
|
||||
|
||||
The conditioning image is re-compressed to match the compression the model was
|
||||
trained against — CRF 18 for LTX-2.5, where LTX-2 / 2.3 use 33. SGLang picks the
|
||||
right one from the checkpoint, so nothing needs to be passed.
|
||||
|
||||
### 4.3 Auto-duration
|
||||
|
||||
<span style={{fontSize: "0.7em", verticalAlign: "middle", padding: "2px 8px", borderRadius: "9999px", background: "#16a34a", color: "#fff"}}>NEW</span>
|
||||
|
||||
LTX-2.5 ships a duration head — a small module that reads the encoded caption
|
||||
and regresses the natural length of the shot it describes. Use it when the
|
||||
prompt implies a duration ("a quick glance" vs "a slow pan across the valley")
|
||||
and you would rather not guess a frame count:
|
||||
|
||||
```bash
|
||||
sglang generate \
|
||||
--model-path Lightricks/LTX-2.5-Diffusers \
|
||||
--pipeline-class-name LTX2Pipeline \
|
||||
--prompt "A red fox walking through a snowy forest at dawn." \
|
||||
--auto-duration \
|
||||
--save-output
|
||||
```
|
||||
|
||||
The prediction is clamped to `--auto-duration-min-seconds` /
|
||||
`--auto-duration-max-seconds` (default 1–20 s) and snapped to the VAE's temporal
|
||||
grid, so the result is always a valid frame count. It overrides `--num-frames`.
|
||||
|
||||
### 4.4 Two-stage (higher quality)
|
||||
|
||||
Stage 1 runs at half the requested resolution, the latents are upsampled 2x, and
|
||||
a short sigma tail refines at full resolution. Pass the **final** size:
|
||||
|
||||
```bash
|
||||
sglang generate \
|
||||
--model-path Lightricks/LTX-2.5-Diffusers \
|
||||
--pipeline-class-name LTX2TwoStagePipeline \
|
||||
--prompt "A cinematic shot of a red fox walking through a snowy forest at dawn." \
|
||||
--height 1088 --width 1920 \
|
||||
--save-output
|
||||
```
|
||||
|
||||
Resolution must be divisible by 64. Unlike LTX-2.3, no `--distilled-lora-path`
|
||||
is needed: the LTX-2.5 transformer is already distilled.
|
||||
|
||||
### 4.5 The dev transformer
|
||||
|
||||
LTX-2.5 ships two DiTs. `model_index.json` points at the distilled one; the
|
||||
full / SFT weights live in `transformer_full/` and are deliberately left out of
|
||||
the index. Select them with `--model-variant dev`:
|
||||
|
||||
```bash
|
||||
sglang generate \
|
||||
--model-path Lightricks/LTX-2.5-Diffusers \
|
||||
--pipeline-class-name LTX2Pipeline \
|
||||
--model-variant dev \
|
||||
--prompt "A cinematic shot of a red fox walking through a snowy forest at dawn." \
|
||||
--num-inference-steps 30 --guidance-scale 3.0 \
|
||||
--save-output
|
||||
```
|
||||
|
||||
The dev variant is not distilled, so SGLang automatically drops the pinned
|
||||
distilled sigma schedule and re-enables the dynamic shifting that `scheduler/`
|
||||
turns off for the distilled DiT. Unlike the distilled path it *is* driven by a
|
||||
step count and *does* want CFG, so pass `--num-inference-steps` and
|
||||
`--guidance-scale` yourself.
|
||||
|
||||
Note that `from_pretrained` only fetches what `model_index.json` lists, so a
|
||||
partial snapshot download will not include `transformer_full/` (another 38 GB).
|
||||
|
||||
### 4.6 Diffusion decoder
|
||||
|
||||
<span style={{fontSize: "0.7em", verticalAlign: "middle", padding: "2px 8px", borderRadius: "9999px", background: "#16a34a", color: "#fff"}}>NEW</span>
|
||||
|
||||
LTX-2.5 adds a diffusion-based video decoder as an alternative to the
|
||||
convolutional VAE decoder. Rather than deconvolving the latent it denoises
|
||||
pixels conditioned on a context volume built from it, which recovers detail a
|
||||
convolutional decoder tends to smooth away:
|
||||
|
||||
```bash
|
||||
sglang generate \
|
||||
--model-path Lightricks/LTX-2.5-Diffusers \
|
||||
--pipeline-class-name LTX2Pipeline \
|
||||
--prompt "A red fox walking through a snowy forest at dawn." \
|
||||
--use-diffusion-decoder \
|
||||
--save-output
|
||||
```
|
||||
|
||||
It is a diffusion model in its own right and decodes more slowly than the VAE
|
||||
decoder, so it is off by default — matching upstream, where `LTX2Pipeline` also
|
||||
decodes with the VAE. The offline `generate` command loads the optional decoder
|
||||
automatically when `--use-diffusion-decoder` is present.
|
||||
|
||||
For an online server, opt into loading the decoder at startup, then select it per
|
||||
request with `use_diffusion_decoder: true`:
|
||||
|
||||
```bash
|
||||
sglang serve \
|
||||
--model-path Lightricks/LTX-2.5-Diffusers \
|
||||
--pipeline-class-name LTX2Pipeline \
|
||||
--load-diffusion-decoder
|
||||
```
|
||||
|
||||
This keeps the default server footprint unchanged while still allowing VAE and
|
||||
diffusion-decoder requests to share one server. When GPU memory is constrained,
|
||||
`--cpu-offload-components diffusion_decoder` keeps the optional decoder on CPU
|
||||
between uses.
|
||||
|
||||
<Tip>
|
||||
**Install NATTEN for this decoder.** Its stages run 3D neighborhood attention,
|
||||
and SGLang uses NATTEN's fused `na3d` kernel for it when the package is present.
|
||||
NATTEN is *not* a dependency of `sglang[diffusion]`: without it the decoder
|
||||
falls back to a compiled FlexAttention block mask. The two agree to bf16
|
||||
rounding, but the fallback is roughly **5x slower** on the decoder's largest
|
||||
attention grid, and has to build the mask on top of that.
|
||||
|
||||
NATTEN ships prebuilt wheels pinned to a specific torch and CUDA build, so
|
||||
install the one matching your environment rather than a bare version — check
|
||||
your combination at [natten.org](https://natten.org). For torch 2.11 / CUDA
|
||||
13.0, for example:
|
||||
|
||||
```bash
|
||||
uv pip install natten==0.21.6+torch2110cu130 -f https://whl.natten.org/
|
||||
```
|
||||
|
||||
Nothing else changes if you skip it: the decoder still produces the same video,
|
||||
just slower.
|
||||
</Tip>
|
||||
+3
-1
@@ -1469,8 +1469,10 @@
|
||||
},
|
||||
{
|
||||
"group": "LTX",
|
||||
"tag": "NEW",
|
||||
"pages": [
|
||||
"cookbook/diffusion/LTX/LTX2 & LTX2.3"
|
||||
"cookbook/diffusion/LTX/LTX2 & LTX2.3",
|
||||
"cookbook/diffusion/LTX/LTX2.5"
|
||||
]
|
||||
},
|
||||
{
|
||||
|
||||
@@ -145,6 +145,12 @@ Rows are grouped when a family shares the same runtime path or optimization supp
|
||||
<td>One-stage, two-stage, TI2V, HQ</td>
|
||||
<td><span className="sgd-muted">No dedicated optimization listed</span></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>LTX-2.5</td>
|
||||
<td><div className="sgd-id-list"><code>Lightricks/LTX-2.5-Diffusers</code></div></td>
|
||||
<td>One-stage, two-stage, TI2V, auto-duration, diffusion decode</td>
|
||||
<td><span className="sgd-muted">No dedicated optimization listed</span></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Cosmos3</td>
|
||||
<td><div className="sgd-id-list"><code>nvidia/Cosmos3-Nano</code><code>nvidia/Cosmos3-Super</code><code>nvidia/Cosmos3-Super-Text2Image</code><code>nvidia/Cosmos3-Super-Image2Video</code></div></td>
|
||||
@@ -605,6 +611,21 @@ Optimization columns are abbreviated to keep the matrix readable:
|
||||
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>❌</td>
|
||||
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>❌</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>LTX-2.5 (one/two-stage/TI2V)</td>
|
||||
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}><code>Lightricks/LTX-2.5-Diffusers</code></td>
|
||||
<td style={{padding: "9px 8px", backgroundColor: "rgba(255,255,255,0.02)"}}>960×544 (default)<br />1920×1088 (two-stage)</td>
|
||||
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>❌</td>
|
||||
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>❌</td>
|
||||
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>❌</td>
|
||||
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>❌</td>
|
||||
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>❌</td>
|
||||
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>❌</td>
|
||||
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>❌</td>
|
||||
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>❌</td>
|
||||
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>❌</td>
|
||||
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>❌</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Cosmos3-Nano (T2V / I2V / T2I)</td>
|
||||
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}><code>nvidia/Cosmos3-Nano</code></td>
|
||||
|
||||
@@ -0,0 +1,182 @@
|
||||
export const LTX25Deployment = () => {
|
||||
const options = {
|
||||
hardware: {
|
||||
name: 'hardware',
|
||||
title: 'Deployment Target',
|
||||
items: [
|
||||
{ id: 'h200', label: '1x H200', subtitle: 'no extra flags', default: true },
|
||||
{ id: 'tight', label: '1 GPU, tight VRAM', subtitle: 'layerwise offload', default: false },
|
||||
{ id: 'sp2', label: '2 GPUs', subtitle: 'sequence parallel', default: false },
|
||||
{ id: 'tp2', label: '2 GPUs', subtitle: 'tensor parallel', default: false },
|
||||
{ id: 'cfg2', label: '2 GPUs', subtitle: 'CFG parallel', default: false },
|
||||
],
|
||||
},
|
||||
precision: {
|
||||
name: 'precision',
|
||||
title: 'Precision',
|
||||
items: [
|
||||
{ id: 'bf16', label: 'bf16', subtitle: 'default', default: true },
|
||||
{ id: 'fp8', label: 'fp8', subtitle: 'online, -18 GB', default: false },
|
||||
],
|
||||
},
|
||||
weights: {
|
||||
name: 'weights',
|
||||
title: 'Weights',
|
||||
items: [
|
||||
{ id: 'distilled', label: 'Distilled', subtitle: '8 steps, unguided', default: true },
|
||||
{ id: 'dev', label: 'Dev / SFT', subtitle: 'steps + CFG', default: false },
|
||||
],
|
||||
},
|
||||
pipeline: {
|
||||
name: 'pipeline',
|
||||
title: 'Pipeline',
|
||||
items: [
|
||||
{ id: 'one-stage', label: 'One Stage', subtitle: '960x544', default: true },
|
||||
{ id: 'two-stage', label: 'Two Stage', subtitle: '1920x1088', default: false },
|
||||
],
|
||||
},
|
||||
decoder: {
|
||||
name: 'decoder',
|
||||
title: 'Decoder',
|
||||
items: [
|
||||
{ id: 'vae', label: 'VAE', subtitle: 'default, fast', default: true },
|
||||
{ id: 'diffusion', label: 'Diffusion', subtitle: 'slower, more detail', default: false },
|
||||
],
|
||||
},
|
||||
duration: {
|
||||
name: 'duration',
|
||||
title: 'Clip Length',
|
||||
items: [
|
||||
{ id: 'fixed', label: 'Fixed', subtitle: '--num-frames', default: true },
|
||||
{ id: 'auto', label: 'Auto', subtitle: 'duration head', default: false },
|
||||
],
|
||||
},
|
||||
};
|
||||
|
||||
const REPO_ID = 'Lightricks/LTX-2.5-Diffusers';
|
||||
const PIPELINE_CLASSES = {
|
||||
'one-stage': 'LTX2Pipeline',
|
||||
'two-stage': 'LTX2TwoStagePipeline',
|
||||
};
|
||||
|
||||
const [values, setValues] = useState({
|
||||
hardware: 'h200',
|
||||
precision: 'bf16',
|
||||
weights: 'distilled',
|
||||
pipeline: 'one-stage',
|
||||
decoder: 'vae',
|
||||
duration: 'fixed',
|
||||
});
|
||||
const [isDark, setIsDark] = useState(false);
|
||||
|
||||
useEffect(() => {
|
||||
const checkDarkMode = () => {
|
||||
const html = document.documentElement;
|
||||
const isDarkMode = html.classList.contains('dark') ||
|
||||
html.getAttribute('data-theme') === 'dark' ||
|
||||
html.style.colorScheme === 'dark';
|
||||
setIsDark(isDarkMode);
|
||||
};
|
||||
checkDarkMode();
|
||||
const observer = new MutationObserver(checkDarkMode);
|
||||
observer.observe(document.documentElement, { attributes: true, attributeFilter: ['class', 'data-theme', 'style'] });
|
||||
return () => observer.disconnect();
|
||||
}, []);
|
||||
|
||||
const handleRadioChange = (key, id) => {
|
||||
setValues((prev) => ({ ...prev, [key]: id }));
|
||||
};
|
||||
|
||||
const getParallelFlags = () => {
|
||||
const map = {
|
||||
tight: ` \\\n --dit-layerwise-offload`,
|
||||
sp2: ` \\\n --num-gpus 2 \\\n --ulysses-degree 2`,
|
||||
tp2: ` \\\n --num-gpus 2 \\\n --tp-size 2`,
|
||||
cfg2: ` \\\n --num-gpus 2 \\\n --enable-cfg-parallel`,
|
||||
};
|
||||
return map[values.hardware] || '';
|
||||
};
|
||||
|
||||
const generateCommand = () => {
|
||||
let command = `sglang serve \\\n --model-path ${REPO_ID}`;
|
||||
command += ` \\\n --pipeline-class-name ${PIPELINE_CLASSES[values.pipeline]}`;
|
||||
if (values.weights === 'dev') {
|
||||
command += ` \\\n --model-variant dev`;
|
||||
}
|
||||
if (values.precision === 'fp8') {
|
||||
command += ` \\\n --quantization fp8`;
|
||||
}
|
||||
command += getParallelFlags();
|
||||
command += ` \\\n --port 30000`;
|
||||
|
||||
// The distilled DiT runs unguided, so there is no negative branch to split.
|
||||
if (values.hardware === 'cfg2' && values.weights !== 'dev') {
|
||||
command += `\n\n# Note: CFG parallel does nothing on the distilled weights (they run\n# unguided). Pick "Dev / SFT" above, or use sequence/tensor parallel.`;
|
||||
}
|
||||
|
||||
// Per-request flags belong on the generate call, not the server.
|
||||
const requestFlags = [];
|
||||
if (values.pipeline === 'two-stage') {
|
||||
requestFlags.push('--height 1088 --width 1920');
|
||||
}
|
||||
if (values.weights === 'dev') {
|
||||
requestFlags.push('--num-inference-steps 30 --guidance-scale 3.0');
|
||||
}
|
||||
if (values.duration === 'auto') {
|
||||
requestFlags.push('--auto-duration');
|
||||
}
|
||||
if (values.decoder === 'diffusion') {
|
||||
requestFlags.push('--use-diffusion-decoder');
|
||||
}
|
||||
if (requestFlags.length > 0) {
|
||||
command += `\n\n# Per-request flags (pass these to \`sglang generate\`, or as request fields):\n# ${requestFlags.join(' ')}`;
|
||||
}
|
||||
return command;
|
||||
};
|
||||
|
||||
const containerStyle = { maxWidth: '900px', margin: '0 auto', display: 'flex', flexDirection: 'column', gap: '4px' };
|
||||
const cardStyle = { padding: '8px 12px', border: `1px solid ${isDark ? '#374151' : '#e5e7eb'}`, borderLeft: `3px solid ${isDark ? '#E85D4D' : '#D45D44'}`, borderRadius: '4px', display: 'flex', alignItems: 'center', gap: '12px', background: isDark ? '#1f2937' : '#fff' };
|
||||
const titleStyle = { fontSize: '13px', fontWeight: '600', minWidth: '140px', flexShrink: 0, color: isDark ? '#e5e7eb' : 'inherit' };
|
||||
const itemsStyle = { display: 'flex', rowGap: '2px', columnGap: '6px', flexWrap: 'wrap', alignItems: 'center', flex: 1 };
|
||||
const labelBaseStyle = { padding: '4px 10px', border: `1px solid ${isDark ? '#9ca3af' : '#d1d5db'}`, borderRadius: '3px', cursor: 'pointer', display: 'inline-flex', flexDirection: 'column', alignItems: 'center', justifyContent: 'center', fontWeight: '500', fontSize: '13px', transition: 'all 0.2s', userSelect: 'none', minWidth: '45px', textAlign: 'center', flex: 1, background: isDark ? '#374151' : '#fff', color: isDark ? '#e5e7eb' : 'inherit' };
|
||||
const checkedStyle = { background: '#D45D44', color: 'white', borderColor: '#D45D44' };
|
||||
const subtitleStyle = { display: 'block', fontSize: '9px', marginTop: '1px', lineHeight: '1.1', opacity: 0.7 };
|
||||
const commandDisplayStyle = { flex: 1, padding: '12px 16px', background: isDark ? '#111827' : '#f5f5f5', borderRadius: '6px', fontFamily: "'Menlo', 'Monaco', 'Courier New', monospace", fontSize: '12px', lineHeight: '1.5', color: isDark ? '#e5e7eb' : '#374151', whiteSpace: 'pre-wrap', overflowX: 'auto', margin: 0, border: `1px solid ${isDark ? '#374151' : '#e5e7eb'}` };
|
||||
|
||||
return (
|
||||
<div style={containerStyle} className="not-prose">
|
||||
{Object.entries(options).map(([key, option]) => (
|
||||
<div key={key} style={cardStyle}>
|
||||
<div style={titleStyle}>{option.title}</div>
|
||||
<div style={itemsStyle}>
|
||||
{option.items.map((item) => {
|
||||
const isChecked = values[option.name] === item.id;
|
||||
return (
|
||||
<label key={item.id} style={{ ...labelBaseStyle, ...(isChecked ? checkedStyle : {}) }}>
|
||||
<input
|
||||
type="radio"
|
||||
name={option.name}
|
||||
checked={isChecked}
|
||||
onChange={() => handleRadioChange(key, item.id)}
|
||||
style={{ display: 'none' }}
|
||||
/>
|
||||
{item.label}
|
||||
{item.subtitle && (
|
||||
<small style={{ ...subtitleStyle, color: isChecked ? 'rgba(255,255,255,0.85)' : 'inherit' }}>
|
||||
{item.subtitle}
|
||||
</small>
|
||||
)}
|
||||
</label>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
</div>
|
||||
))}
|
||||
|
||||
<div style={cardStyle}>
|
||||
<div style={titleStyle}>Run this Command:</div>
|
||||
<pre style={commandDisplayStyle}>{generateCommand()}</pre>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
};
|
||||
Reference in New Issue
Block a user