diff --git a/python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-benchmark-profile/SKILL.md b/python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-benchmark-profile/SKILL.md index d91f2c2a2..29b12dab0 100644 --- a/python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-benchmark-profile/SKILL.md +++ b/python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-benchmark-profile/SKILL.md @@ -27,7 +27,7 @@ Before running any benchmark, profiler, or kernel-validation command: ## Main Reference -- [benchmark-and-profile.md](benchmark-and-profile.md) — canonical denoise benchmark, perf dump, and `torch.profiler` workflow; uses the checked-in nightly-aligned presets, including `LTX-2` two-stage +- [benchmark-and-profile.md](benchmark-and-profile.md) — canonical denoise benchmark, perf dump, and `torch.profiler` workflow; uses the checked-in nightly-aligned presets, plus `LTX-2`, `LTX-2.3` one-stage, and `LTX-2.3` two-stage benchmark recipes - [existing-fast-paths.md](existing-fast-paths.md) — map bottlenecks to existing fused kernels, packed QKV paths, fused `QK norm + RoPE`, and distributed overlap patterns before proposing new code - [scripts/diffusion_skill_env.py](scripts/diffusion_skill_env.py) — preflight helper: repo root discovery via `sglang.__file__`, write-access probe, benchmark/profile output directories, idle GPU selection - [scripts/bench_diffusion_denoise.py](scripts/bench_diffusion_denoise.py) — end-to-end denoise benchmark preset runner via `sglang generate`; use `--list-models` to inspect preset order, then save perf dumps by label and compare them with `compare_perf.py` diff --git a/python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-benchmark-profile/benchmark-and-profile.md b/python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-benchmark-profile/benchmark-and-profile.md index be4aac215..4ea949fb7 100644 --- a/python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-benchmark-profile/benchmark-and-profile.md +++ b/python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-benchmark-profile/benchmark-and-profile.md @@ -100,6 +100,26 @@ PYTHONPATH=python python3 \ --output-dir "${BENCH_DIR}" ``` +Run the `LTX-2.3` one-stage skill preset: + +```bash +PYTHONPATH=python python3 \ + python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-benchmark-profile/scripts/bench_diffusion_denoise.py \ + --model ltx23-one-stage \ + --label baseline \ + --output-dir "${BENCH_DIR}" +``` + +Run the `LTX-2.3` two-stage skill preset: + +```bash +PYTHONPATH=python python3 \ + python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-benchmark-profile/scripts/bench_diffusion_denoise.py \ + --model ltx23-two-stage \ + --label baseline \ + --output-dir "${BENCH_DIR}" +``` + Run the full preset sweep: ```bash @@ -123,6 +143,8 @@ Nightly-aligned presets come first; skill-only presets stay available after them | `wan-ti2v` | `Wan-AI/Wan2.2-TI2V-5B-Diffusers` | Yes: `wan22_ti2v_5b_720p` | Uses the nightly cat image and motion prompt | | `ltx2` | `Lightricks/LTX-2` | Yes: `ltx2_twostage_t2v` | Uses `LTX2TwoStagePipeline`; nightly-aligned prompt, negative prompt, 1536x1024, 121 frames, fps 24, seed 1234 | | `wan-i2v` | `Wan-AI/Wan2.2-I2V-A14B-Diffusers` | Yes: `wan22_i2v_a14b_720p` | Aligned to nightly CFG-parallel 4-GPU launch | +| `ltx23-one-stage` | `Lightricks/LTX-2.3` | No | Skill-only extra preset for the native `LTX-2.3` one-stage baseline; 2 GPUs, 768x512, 121 frames, fps 24, 30 steps, guidance 3.0, seed 1234 | +| `ltx23-two-stage` | `Lightricks/LTX-2.3` | No | Skill-only extra preset for the native `LTX-2.3` two-stage path; uses `LTX2TwoStagePipeline`, 2 GPUs, 1536x1024, 121 frames, fps 24, 30 steps, guidance 3.0, seed 1234 | | `hunyuanvideo` | `hunyuanvideo-community/HunyuanVideo` | No | Skill-only extra preset | | `mova-720p` | `OpenMOSS-Team/MOVA-720p` | No | Skill-only extra preset | | `helios` | `BestWishYsh/Helios-Base` | No | Skill-only extra preset | @@ -151,6 +173,41 @@ After [PR #20707](https://github.com/sgl-project/sglang/pull/20707), `LTX2TwoStagePipeline` is a native path. The spatial upsampler and distilled LoRA are auto-resolved from the same model snapshot unless you override them. +### Manual command example: LTX-2.3 One-Stage + +```bash +sglang generate \ + --model-path=Lightricks/LTX-2.3 \ + --prompt="A beautiful sunset over the ocean" \ + --negative-prompt="shaky, glitchy, low quality, worst quality, deformed, distorted, disfigured, motion smear, motion artifacts, fused fingers, bad anatomy, weird hand, ugly, transition, static." \ + --width=768 --height=512 \ + --num-frames=121 --fps=24 \ + --num-inference-steps=30 --guidance-scale=3.0 \ + --seed=1234 --num-gpus=2 \ + --save-output --enable-torch-compile --warmup +``` + +Use this when you want the native `LTX2Pipeline` baseline for `LTX-2.3` at the +validated one-stage resolution. + +### Manual command example: LTX-2.3 Two-Stage + +```bash +sglang generate \ + --model-path=Lightricks/LTX-2.3 \ + --pipeline-class-name=LTX2TwoStagePipeline \ + --prompt="A beautiful sunset over the ocean" \ + --negative-prompt="shaky, glitchy, low quality, worst quality, deformed, distorted, disfigured, motion smear, motion artifacts, fused fingers, bad anatomy, weird hand, ugly, transition, static." \ + --width=1536 --height=1024 \ + --num-frames=121 --fps=24 \ + --num-inference-steps=30 --guidance-scale=3.0 \ + --seed=1234 --num-gpus=2 \ + --save-output --enable-torch-compile --warmup +``` + +This matches the new `ltx23-two-stage` skill preset and is a good benchmark target for +the recently merged `LTX-2.3` two-stage path. + ### Manual command example: Wan2.2-I2V-A14B 720P ```bash diff --git a/python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-benchmark-profile/scripts/bench_diffusion_denoise.py b/python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-benchmark-profile/scripts/bench_diffusion_denoise.py index 002c13429..224462896 100755 --- a/python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-benchmark-profile/scripts/bench_diffusion_denoise.py +++ b/python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-benchmark-profile/scripts/bench_diffusion_denoise.py @@ -12,7 +12,7 @@ Usage: # Tag the run for later compare_perf.py usage python3 python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-benchmark-profile/scripts/bench_diffusion_denoise.py --model flux --label tuned - # All 12 preset models + # All 14 preset models python3 python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-benchmark-profile/scripts/bench_diffusion_denoise.py --all # Show preset order, model path, and nightly mapping @@ -193,6 +193,39 @@ MODELS = { ], }, # 10. Skill-only extra preset + "ltx23-one-stage": { + "path": "Lightricks/LTX-2.3", + "prompt": "A beautiful sunset over the ocean", + "negative_prompt": "shaky, glitchy, low quality, worst quality, deformed, distorted, disfigured, motion smear, motion artifacts, fused fingers, bad anatomy, weird hand, ugly, transition, static.", + "seed": 1234, + "extra_args": [ + "--width=768", + "--height=512", + "--num-frames=121", + "--fps=24", + "--num-inference-steps=30", + "--guidance-scale=3.0", + "--num-gpus=2", + ], + }, + # 11. Skill-only extra preset + "ltx23-two-stage": { + "path": "Lightricks/LTX-2.3", + "prompt": "A beautiful sunset over the ocean", + "negative_prompt": "shaky, glitchy, low quality, worst quality, deformed, distorted, disfigured, motion smear, motion artifacts, fused fingers, bad anatomy, weird hand, ugly, transition, static.", + "seed": 1234, + "extra_args": [ + "--pipeline-class-name=LTX2TwoStagePipeline", + "--width=1536", + "--height=1024", + "--num-frames=121", + "--fps=24", + "--num-inference-steps=30", + "--guidance-scale=3.0", + "--num-gpus=2", + ], + }, + # 12. Skill-only extra preset "hunyuanvideo": { "path": "hunyuanvideo-community/HunyuanVideo", "prompt": "A cat and a dog baking a cake together in a kitchen. The cat is carefully measuring flour, while the dog is stirring the batter with a wooden spoon. The kitchen is cozy, with sunlight streaming through the window.", @@ -205,7 +238,7 @@ MODELS = { "--num-inference-steps=30", ], }, - # 11. Skill-only extra preset + # 13. Skill-only extra preset # Requires: /inputs/diffusion_benchmark/figs/mova_single_person.jpg "mova-720p": { "path": "OpenMOSS-Team/MOVA-720p", @@ -221,7 +254,7 @@ MODELS = { "--num-inference-steps=2", ], }, - # 12. Skill-only extra preset + # 14. Skill-only extra preset "helios": { "path": "BestWishYsh/Helios-Base", "prompt": "A curious raccoon", @@ -247,6 +280,8 @@ def required_gpus_for_model(model_key: str) -> int: return 4 if model_key == "mova-720p": return 4 + if model_key in {"ltx23-one-stage", "ltx23-two-stage"}: + return 2 return 1 @@ -260,11 +295,11 @@ def print_model_catalog(): print("=" * 95) print("MODEL PRESETS — Nightly-aligned first, skill-only extras after") print("=" * 95) - print(f"{'Preset':<14} {'Nightly':<28} {'Model Path':<46} {'GPUs':>4}") + print(f"{'Preset':<17} {'Nightly':<28} {'Model Path':<46} {'GPUs':>4}") print("-" * 95) for model_key, cfg in MODELS.items(): print( - f"{model_key:<14} {model_nightly_case_id(model_key):<28} {cfg['path']:<46} {required_gpus_for_model(model_key):>4}" + f"{model_key:<17} {model_nightly_case_id(model_key):<28} {cfg['path']:<46} {required_gpus_for_model(model_key):>4}" ) print("-" * 112) print( @@ -474,7 +509,7 @@ def main(): choices=list(MODELS.keys()), help="Model to benchmark (default: flux)", ) - parser.add_argument("--all", action="store_true", help="Benchmark all 12 models") + parser.add_argument("--all", action="store_true", help="Benchmark all 14 models") parser.add_argument( "--list-models", action="store_true", diff --git a/python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-performance/SKILL.md b/python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-performance/SKILL.md index e81449004..5c8d91e01 100644 --- a/python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-performance/SKILL.md +++ b/python/sglang/multimodal_gen/.claude/skills/sglang-diffusion-performance/SKILL.md @@ -88,6 +88,37 @@ sglang generate --model-path Lightricks/LTX-2 \ Note: this generate recipe is aligned with the nightly comparison case `ltx2_twostage_t2v`. After [PR #20707](https://github.com/sgl-project/sglang/pull/20707), `LTX2TwoStagePipeline` is a native path and auto-resolves the spatial upsampler plus distilled LoRA from the same model snapshot unless you override them. +### Native baseline, 2 GPUs: LTX-2.3 one-stage + +```bash +sglang generate --model-path Lightricks/LTX-2.3 \ + --prompt "A beautiful sunset over the ocean" \ + --negative-prompt "shaky, glitchy, low quality, worst quality, deformed, distorted, disfigured, motion smear, motion artifacts, fused fingers, bad anatomy, weird hand, ugly, transition, static." \ + --width 768 --height 512 \ + --num-frames 121 --fps 24 \ + --num-inference-steps 30 --guidance-scale 3.0 \ + --seed 1234 --num-gpus 2 \ + --enable-torch-compile --warmup --save-output +``` + +Note: use this as the native `LTX2Pipeline` baseline for `LTX-2.3`. It keeps the validated one-stage resolution and explicit `LTX-2.3` sampling defaults, and matches the `ltx23-one-stage` benchmark preset in `sglang-diffusion-benchmark-profile`. + +### Benchmark target, 2 GPUs: LTX-2.3 two-stage + +```bash +sglang generate --model-path Lightricks/LTX-2.3 \ + --pipeline-class-name LTX2TwoStagePipeline \ + --prompt "A beautiful sunset over the ocean" \ + --negative-prompt "shaky, glitchy, low quality, worst quality, deformed, distorted, disfigured, motion smear, motion artifacts, fused fingers, bad anatomy, weird hand, ugly, transition, static." \ + --width 1536 --height 1024 \ + --num-frames 121 --fps 24 \ + --num-inference-steps 30 --guidance-scale 3.0 \ + --seed 1234 --num-gpus 2 \ + --enable-torch-compile --warmup --save-output +``` + +Note: this is the recommended benchmark command for the new `LTX-2.3` two-stage path. It uses the native `LTX2TwoStagePipeline` and matches the `ltx23-two-stage` benchmark preset in `sglang-diffusion-benchmark-profile`. + ### Maximum speed, image model, single GPU, lossless ```bash