diff --git a/docs/cookbook/diffusion/Wan/Wan-Animate-2.mdx b/docs/cookbook/diffusion/Wan/Wan-Animate-2.mdx index b93b1358ab..51c3bd9fcf 100644 --- a/docs/cookbook/diffusion/Wan/Wan-Animate-2.mdx +++ b/docs/cookbook/diffusion/Wan/Wan-Animate-2.mdx @@ -1,9 +1,9 @@ --- title: Wan-Animate-2 tag: NEW -description: Run Wan-Animate-2 character animation (reference image + reference video) with SGLang Diffusion. +description: Run Wan-Animate-2 character animation (reference image + reference video) with SGLang Diffusion, with the base or the distilled checkpoint. metatags: - description: "Deploy and use the Wan2.2-Animate-2 14B character-animation model with SGLang Diffusion: reference-image + reference-video conditioning, single-expert native pipeline, config-file and /v1/videos invocation, and multi-GPU presets." + description: "Deploy and use the Wan2.2-Animate-2 14B character-animation model (base and distilled checkpoints) with SGLang Diffusion: reference-image + reference-video conditioning, single-expert native pipeline, config-file and /v1/videos invocation, and multi-GPU presets." --- import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx'; @@ -18,7 +18,9 @@ import { config } from '/src/snippets/configs/Wan/wan-animate-2.jsx'; ## 1. Quick start Follow the [installation guide](/docs/sglang-diffusion/installation) on Linux with -NVIDIA CUDA, then select a verified recipe. +NVIDIA CUDA, then select a verified recipe. **Checkpoint weights** under **Setup** +switches between the base and the distilled checkpoint; the server flags and the +Request command follow that choice. @@ -36,17 +38,38 @@ image-only conditioning of the other Wan2.2 checkpoints, it animates a **referen image** so that it follows the motion of a **reference video**, producing a new clip of that subject performing the reference video's motion. -Use **`Wan-AI/Wan2.2-Animate-2-14B-Diffusers`** as `--model-path`. The distilled -checkpoint (`Wan-AI/Wan2.2-Animate-2-14B-Distilled-Diffusers`, `log_scale=-1.3`, -10 steps, no CFG) is not supported in this release. +Two checkpoints are supported; select one under **Checkpoint weights** or pass it +as `--model-path`: + +| Variant | `--model-path` | Steps | `guidance_scale` | `log_scale` | CFG parallel | +| --- | --- | ---: | ---: | ---: | --- | +| Base | `Wan-AI/Wan2.2-Animate-2-14B-Diffusers` | `40` | `3.0` | `0` | yes | +| Distilled | `Wan-AI/Wan2.2-Animate-2-14B-Distilled-Diffusers` | `10` | `1.0` | `-1.3` | rejected at startup | + +The distilled checkpoint differs in its transformer weights, in the +attention-score bias `log_scale` (the official distillation value, applied by +default because the published `transformer/config.json` carries no such key) and +in the two defaults above. The distilled checkpoint is sampled with DPM-Solver++ +2M on the official sigma grid, like the base checkpoint: that is what the model +authors' command-line implementation runs for both checkpoints. The +`FlowMatchEulerDiscreteScheduler` named in the distilled repo's +`scheduler/scheduler_config.json` is read only by the Diffusers pipeline (which +also omits the `log_scale` bias) and is not used here; on the same seed it lands +1.8 dB farther from the authors' output. `--scheduler-class-override` is ignored +for this checkpoint and logged. A local directory routes by the pipeline class in +its `modular_model_index.json` (`WanAnimate2DistilledModularPipeline` / +`WanAnimate2ModularPipeline`) or by a `distill` token in the directory name (in a +path that also contains `wan2.2-animate-2` or `wan_animate_2`); the detectors are +checked in registration order, the distilled one first, so either signal selects +the distilled entry. `--model-id Wan2.2-Animate-2-14B-Distilled-Diffusers` pins +the variant explicitly. Unlike the A14B T2V/I2V checkpoints, Wan-Animate-2 is a **single-expert** model: -it has one `guidance_scale` (default `3.0`), no low-noise expert -(`guidance_scale_2`) and no boundary switching. Diffusers >= 0.40.0 serves -Wan-Animate-2 as a modular pipeline (`WanAnimate2ModularPipeline`); SGLang -implements the model natively and loads the same -`Wan-AI/Wan2.2-Animate-2-14B-Diffusers` checkpoint. There is no diffusers backend -fallback. +it has one `guidance_scale` (see the variant table), no low-noise expert +(`guidance_scale_2`) and no boundary switching. Diffusers >= 0.40.0 +serves Wan-Animate-2 as a modular pipeline (`WanAnimate2ModularPipeline`); SGLang +implements the model natively and loads the same checkpoints. There is no diffusers +backend fallback. Wan-Animate-2 is served through the native `WanAnimate2Pipeline`, selected @@ -77,14 +100,20 @@ video URLs use SGLang's shared media loader before decoding, without a disk cach | Parameter | Default | Notes | | --- | --- | --- | -| Resolution (`width`×`height`, the API's `size`) | `640`×`800` | A pixel-area budget, not the output dimensions: the frame keeps the reference image's aspect ratio and is the largest 16-aligned canvas with at most `width × height` pixels (see **Output resolution** below). `720`×`1280` is the larger supported budget. | +| Resolution (`width`×`height`, the API's `size`) | `640`×`800` | A pixel-area budget, not the output dimensions: the frame keeps the reference image's aspect ratio and is the largest 16-aligned canvas with at most `width × height` pixels (see **Output resolution** below). `720`×`1280` is the larger budget. | | `clip_len` | `37` | Per-clip denoising chunk length; must be `4k+1` (VAE temporal stride 4), e.g. `37`, `65`, `81`. Other values are rounded to `4k+1` with the same `k` and the rounding is logged (`40` becomes `41`; `38` and `39` become `37`); values below `5` are rejected. | | `fps` | `16` | The reference video is resampled to this rate and the output MP4 is written at it; duration is preserved. | -| `num_inference_steps` | `40` | Fewer steps (e.g. `20`) run faster. | -| `guidance_scale` | `3.0` | Single expert; there is no `guidance_scale_2`. | +| `num_inference_steps` | `40` (`10` for the distilled checkpoint) | Fewer steps (e.g. `20`) run faster on the base checkpoint. | +| `guidance_scale` | `3.0` (`1.0` for the distilled checkpoint) | Single expert; there is no `guidance_scale_2`. The distilled checkpoint samples at `1.0` only (one branch per step); a request with any other value is rejected at admission (HTTP 400 online, `ValueError` offline), see [section 6](#6-troubleshooting). | | `prompt_ref` | `人物动作的参考视频` ("reference video of the character's motion") | Text for the reference-video branch. The official demo conditions that branch on this fixed description of the reference video rather than on the scene prompt; leave it unchanged unless reproducing a different reference setup. | | `enable_audio` | `true` | Keep the reference video's audio track on the output (see **Audio** below). Per request, like `clip_len`. | +The distilled checkpoint inherits every other default. The official distilled demo +defaults to `clip_len` 81, `size` 720×1280 and `fps` 24; the distilled rows in +[section 5.3](#5-3-distilled-checkpoint-on-gb200) were measured at `clip_len` 37 +and `size` 640×800. `clip_len` 81 at the 640×800 budget runs on one 189 GB GPU at +about 230 s. + **Output length.** The output has as many frames as the (resampled) reference video, not `clip_len`. The video is generated clip by clip with `clip_len` frames per chunk; a larger `clip_len` improves temporal coherence at the cost of VRAM. @@ -96,8 +125,9 @@ it unset. **Output resolution.** `width`×`height` (the API's `size`) is a pixel-area budget, not the output dimensions. The reference image's aspect ratio is kept, the frame is the largest canvas with at most `width × height` pixels whose sides are multiples of -16, and the reference image and the driving-video frames are letterboxed onto it; -the output MP4 has the canvas size. At the default `640`×`800` (512,000 pixels) a +16, and the reference image is letterboxed onto it; the driving-video frames are +fitted the same way to their own canvas at the same budget. The output MP4 has +the canvas size. At the default `640`×`800` (512,000 pixels) a 9:16 reference gives `528`×`944`, a 1:1 reference gives `704`×`704`, and only a 4:5 reference gives exactly `640`×`800`. To obtain a specific output size, request a budget with the same aspect ratio as the reference image (for a 9:16 image, @@ -163,6 +193,12 @@ For two B300 GPUs, set **GPUs / node** to **2** and **Memory policy** to **CFG p **Auto** CFG parallelism enables the measured two-GPU recipe. The RTX 4090 entry selects its single-GPU memory recipe. +With **Checkpoint weights** set to the distilled checkpoint, CFG parallelism is +off (one branch per step), **Auto** topology resolves to Ulysses across the GPUs +and **Auto** text encoder to CPU offload, the policy of the measured GB200 +recipes (1, 2 and 4 GPUs, resident or DiT layerwise offload, Ulysses or tensor +parallel); see the [distilled benchmark](#5-3-distilled-checkpoint-on-gb200). + Routing to the native Wan-Animate-2 pipeline is automatic from the checkpoint's `model_index.json` (`_class_name: WanAnimate2Pipeline`); pass `--pipeline-class-name WanAnimate2Pipeline` only for a local mirror that lacks @@ -176,7 +212,7 @@ acceleration features and their runtime requirements. - `--num-gpus {NUM_GPUS}`: Number of GPUs to use. - `--tp-size {TP_SIZE}`: Tensor parallelism size. - `--ulysses-degree {ULYSSES_DEGREE}`: DeepSpeed-Ulysses-style sequence parallelism (USP). The **ring degree must be 1** for this model; `--ring-degree` greater than 1 is rejected with `NotImplementedError`. Combines with `--tp-size`: Ulysses splits the TP-local attention heads, so `--tp-size` must divide the 40 heads and `--ulysses-degree` must divide the per-shard count (see [section 3.3](#3-3-multi-gpu-and-memory-presets)). -- `--enable-cfg-parallel`: Split the guided and unguided branches across GPUs. +- `--enable-cfg-parallel`: Split the guided and unguided branches across GPUs. Base checkpoint only: the distilled checkpoint has one branch and rejects the flag at startup. - `--text-encoder-cpu-offload`: CPU-offload the text encoder to save memory; it is moved to the GPU for the one encode per request. ### 3.3 Multi-GPU and memory presets @@ -194,6 +230,22 @@ acceleration features and their runtime requirements. | FSDP sharding | `--use-fsdp-inference` is **not supported** | Rejected at startup, before any weights load; use `--tp-size` to shard the DiT or `--dit-layerwise-offload` to cut peak memory. | | 4+ GPUs, TP x Ulysses | `--num-gpus 4 --tp-size 2 --ulysses-degree 2` | Tensor parallel across attention heads, Ulysses across the TP-local heads: `--tp-size` must divide the 40 heads and `--ulysses-degree` the per-shard count (`40 / tp`). The memory-leaning 4-GPU layout: about 33 GiB peak per GPU (25 GiB torch reserved), the lowest 4-GPU peak after `--tp-size 4`, at about 13 percent slower than CFG-parallel x tensor parallel and 27 percent slower than the default CFG-parallel x Ulysses layout; use the default for speed and this layout for memory. CFG-parallel can be added on top with twice the GPUs (`--num-gpus 8 --enable-cfg-parallel --tp-size 2 --ulysses-degree 2`). | +The distilled checkpoint (`Wan-AI/Wan2.2-Animate-2-14B-Distilled-Diffusers`) runs +one branch per step, so the CFG-parallel rows above do not apply to it. The rows +below are the picker's distilled recipes, all measured with +`--text-encoder-cpu-offload` (its **Auto** text-encoder policy) on 189 GB GB200 +GPUs; the 80 to 96 GB advice is inferred from their peaks. The measurements, with +the request-phase and whole-run peaks defined, are in +[section 5.3](#5-3-distilled-checkpoint-on-gb200). + +| Target | Recommended server flags | Notes | +| --- | --- | --- | +| Distilled, 1 GPU, no offload, needs about 96 GiB over the whole run (69.5 GiB during the request) | *(no extra flags)* | DiT resident, the fastest single-GPU form. The whole-run peak sets the 96 GiB figure: it includes the server warmup's synthetic requests, one of which runs at the largest supported budget, 720×1280. | +| Distilled, 1 GPU, 80 to 96 GB | `--dit-layerwise-offload --layerwise-offload-components transformer` | Streams the DiT layer by layer at the same speed, with the output identical to the resident DiT, at about 40 GiB peak per GPU during the request (65 GiB over the whole run) versus 69.5 GiB resident. | +| Distilled, 2 GPUs | `--num-gpus 2 --ulysses-degree 2` | The fastest 2-GPU layout, 1.73x one GPU; the automatic 2-GPU assignment resolves to it. On 80 GB cards add `--dit-layerwise-offload --layerwise-offload-components transformer` (same speed, identical output); `--tp-size 2` is the resident lower-memory alternative. | +| Distilled, 4 GPUs | `--num-gpus 4 --ulysses-degree 4` | The fastest 4-GPU layout, 2.69x one GPU; the automatic 4-GPU assignment resolves to it. Lower-memory alternatives, fastest first: layerwise offload on top (identical output), `--tp-size 2 --ulysses-degree 2` and `--tp-size 4`. | +| Distilled, CFG parallel | `--enable-cfg-parallel` / `--cfg-parallel-size 2` are **rejected** | The server refuses to start with `Wan_Animate_2_14B_Distilled_Config does not support CFG parallelism`: the distilled model runs one branch per step, so there is nothing to split; use `--ulysses-degree` or `--tp-size`. | + ## 4. Model Invocation ### 4.1 Generate offline with a config file @@ -208,8 +260,6 @@ video_path: /path/to/reference_video.mp4 clip_len: 37 height: 800 width: 640 -num_inference_steps: 40 -guidance_scale: 3.0 fps: 16 seed: 42 ``` @@ -224,7 +274,11 @@ sglang generate \ Any value in the YAML can also be passed as a CLI flag (for example `--image-path`, `--video-path`, `--num-inference-steps`, `--guidance-scale`, -`--seed`). +`--seed`). The examples on this page set neither `num_inference_steps` nor +`guidance_scale`: each checkpoint takes the defaults of the variant table in +[section 2](#2-model-capabilities); set them only to override the defaults (the +distilled checkpoint accepts `guidance_scale` 1.0 only). The picker's Request +command sets both for the selected checkpoint. ### 4.2 Serve and request over HTTP @@ -240,8 +294,6 @@ job_id=$(curl -sS -X POST http://127.0.0.1:30010/v1/videos \ --form-string "video_path=/path/to/reference_video.mp4" \ --form-string "clip_len=37" \ --form-string "size=640x800" \ - --form-string "num_inference_steps=40" \ - --form-string "guidance_scale=3.0" \ --form-string "fps=16" \ --form-string "seed=42" \ --form-string "enable_audio=false" \ @@ -275,7 +327,6 @@ video = client.videos.create( extra_body={ "video_path": "/path/to/reference_video.mp4", "clip_len": 37, - "guidance_scale": 3.0, "enable_audio": False, # silent output; the reference video's audio is kept by default }, ) @@ -299,6 +350,9 @@ For DiT-only offload, choose **DiT layerwise offload** in the picker instead. On RTX 4090, prefix the command with `PYTORCH_ALLOC_CONF=expandable_segments:True` as shown in its hardware preset; memory mode alone OOMed in the tested workload. Lowering `clip_len` also reduces memory, but shortens each clip's temporal context. +The distilled checkpoint has the same DiT, so the same flags apply with the same +effect; its layerwise-offload rows are in +[section 5.3](#5-3-distilled-checkpoint-on-gb200). ## 5. Benchmark @@ -420,7 +474,63 @@ synchronizes the device so the per-stage timings are accurate. -### 5.3 Native regression coverage +### 5.3 Distilled checkpoint on GB200 + +`Wan-AI/Wan2.2-Animate-2-14B-Distilled-Diffusers` on the GB200 node of +[section 5.2](#5-2-gb200-reference-benchmark) with the same inputs at the distilled +defaults (10 steps, `guidance_scale` 1.0, seed 123, `clip_len` 37, `size` 640x800, +`fps` 30). The rows were measured through the server, not through +`sglang generate`, so the two GB200 tables are not like for like: the command +below plus the row's flags, then two identical `/v1/videos` requests per server +(the request of [section 4.2](#4-2-serve-and-request-over-http) with the benchmark +inputs). These rows are the picker's verified GB200 recipes. + +```bash Command +sglang serve \ + --model-path Wan-AI/Wan2.2-Animate-2-14B-Distilled-Diffusers \ + --num-gpus 1 --text-encoder-cpu-offload --warmup-mode server \ + --port 30010 +``` + +E2E is the server's `Pixel data generated` time for the steady state, the second +identical request on the same server. Peak is the `nvidia-smi` maximum over ranks +during the request. + +**Result** (peak memory per GPU from `nvidia-smi`): + +| Config | GPUs | Extra flags | E2E (s) | Peak per GPU (GiB) | +| --- | ---: | --- | ---: | ---: | +| 1 GPU, DiT resident | 1 | none | 150.2 | 69.5 | +| 1 GPU, layerwise DiT offload | 1 | `--dit-layerwise-offload --layerwise-offload-components transformer` | 150.0 | 39.7 | +| 2 GPUs, Ulysses (= framework default) | 2 | `--ulysses-degree 2` | 87.0 | 60.1 | +| 2 GPUs, Ulysses, layerwise DiT offload | 2 | `--ulysses-degree 2 --dit-layerwise-offload --layerwise-offload-components transformer` | 86.3 | 31.4 | +| 2 GPUs, tensor parallel | 2 | `--tp-size 2` | 97.1 | 40.0 | +| 4 GPUs, Ulysses (= framework default) | 4 | `--ulysses-degree 4` | 55.8 | 51.0 | +| 4 GPUs, Ulysses, layerwise DiT offload | 4 | `--ulysses-degree 4 --dit-layerwise-offload --layerwise-offload-components transformer` | 58.0 | 21.5 | +| 4 GPUs, tensor parallel x Ulysses | 4 | `--tp-size 2 --ulysses-degree 2` | 59.3 | 35.3 | +| 4 GPUs, tensor parallel | 4 | `--tp-size 4` | 64.1 | 30.4 | + +Whole-run peaks, which include the server warmup's synthetic requests (one at the +largest supported budget, 720x1280, to decide whether the DiT stays resident), are +higher: 95.5 / 64.8 / 72.3 / 41.9 / 47.3 / 57.4 / 27.3 / 38.9 / 37.8 GiB in row order. + +The first request after the server warmup adds a one-time flex-attention autotune +when its frame geometry differs from the warmup shapes (about 16 s on 1 GPU, 12 to +13 s on 2, 9 s on 4). With one branch per step there is no CFG split to +parallelize, so Ulysses is the distilled +checkpoint's fastest layout: 1.73x faster than one GPU on 2 GPUs and 2.69x on 4. +The framework default for `--num-gpus 2` and `--num-gpus 4` resolves to the same +Ulysses layout and produces the same output; the explicit flag pins it. Layerwise +DiT offload costs about 2 s and leaves the output identical. Tensor parallel x +Ulysses is the memory-leaning 4-GPU layout, 6 percent slower than Ulysses 4 at 31 +percent less memory and faster than `--tp-size 4`; tensor parallelism alone is 12 +to 15 percent slower than Ulysses with the lowest peak. With the default encoder +and VAE parallel settings no multi-GPU layout is bit-identical to the single GPU +(Ulysses 2 and tensor parallel 4 are the closest, above 36 dB PSNR). +`--enable-torch-compile` is not faster than eager on this model and is not +recommended. + +### 5.4 Native regression coverage A separate regression used two B300 GPUs, CFG-parallel with replicated encoders and serial VAE decode, a 576x1024 area budget, 40 steps, `clip_len=37`, CFG=3, @@ -451,6 +561,9 @@ accuracy must be validated separately with pinned versions and aligned settings. - **OOM**: lower `clip_len` (for example `37` instead of `65`/`81`) or add `--dit-layerwise-offload --layerwise-offload-components transformer`. - **`--ring-degree` greater than 1** is rejected at startup with `NotImplementedError`. Use `--ulysses-degree` for sequence parallelism and keep the ring degree at 1. - **`--ulysses-degree` that does not divide the TP-local attention heads** (40 / `--tp-size`; with TP 1, degrees 3, 6, 16, ... are rejected) fails at startup with `ValueError`; with TP 1 use 2, 4, 5, 8, 10, 20 or 40. The same check covers leftover GPUs auto-assigned to sequence parallelism (for example `--num-gpus 8 --tp-size 2` runs TP 2 x Ulysses 4). +- **`--enable-cfg-parallel` (or `--cfg-parallel-size 2`) with the distilled checkpoint**: the server refuses to start with `Wan_Animate_2_14B_Distilled_Config does not support CFG parallelism`. The distilled model runs one branch per step, so there is nothing to split; use `--ulysses-degree` or `--tp-size` for multiple GPUs. +- **`guidance_scale` other than `1.0` with the distilled checkpoint**: the request is rejected at admission (HTTP 400 on `/v1/videos`, `ValueError` offline) with `guidance_scale= is not supported by the distilled Wan-Animate-2 checkpoint: it samples at guidance_scale 1.0 only (the distilled recipe has no unconditional branch); omit the field or set it to 1.0.` The distilled checkpoint runs one branch per step; omit the field or set it to `1.0`. +- **`The expanded size of the tensor (N) must match the existing size (M)`** at the first denoising step (the request fails): the reference image and the reference video have different aspect ratios and at the requested `size` the video's 16-aligned grid has more tokens per frame than the image's. With this page's inputs it happens at the 720×1280 budget on both checkpoints; use the 640×800 budget. - **FSDP** (`--use-fsdp-inference`) is not supported for this model and is rejected at startup; use `--tp-size` or DiT layerwise offload for memory. - **`--attention-backend`** (or `--component-attention-backends transformer=...`) is rejected at startup with `ValueError`: the in-context self-attention runs on torch `flex_attention` only. The text encoder still accepts `--component-attention-backends text_encoder=`. - **`enable_teacache` / `enable_spectrum`** request fields are rejected with `ValueError`: every block runs at every step, so the cache heuristics do not apply. diff --git a/docs/scripts/tests/wan-animate-2-config.test.mjs b/docs/scripts/tests/wan-animate-2-config.test.mjs index 8e1c98e534..3230312fbf 100644 --- a/docs/scripts/tests/wan-animate-2-config.test.mjs +++ b/docs/scripts/tests/wan-animate-2-config.test.mjs @@ -71,8 +71,9 @@ test("all 233 legacy selections preserve flags, environment and verification", ( test("H3 builder uses scoped settings and preserves the default HTTP workload", () => { assert.equal(config.cells, undefined); + assert.deepEqual(config.overlayDims.filter((d) => d.scope === "base").map((d) => d.id), ["weights"]); assert.deepEqual(config.overlayDims.filter((d) => d.scope === "serve").map((d) => d.id), ["placement", "encoder", "cfg"]); - assert.deepEqual(config.overlayDims.filter((d) => d.scope === "request").map((d) => d.id), ["resolution", "clip_len", "steps", "audio"]); + assert.deepEqual(config.overlayDims.filter((d) => d.scope === "request").map((d) => d.id), ["resolution", "clip_len", "steps", "distilled_steps", "audio"]); assert.equal(config.overlayDims.find((d) => d.id === "steps").unit, "steps"); assert.equal(config.curl(selection()), [ "curl -sS -X POST http://{{CURL_HOST}}:{{CURL_PORT}}/v1/videos", @@ -91,7 +92,8 @@ test("H3 builder uses scoped settings and preserves the default HTTP workload", test("recommended recipes have accurate verification and consumer memory settings", () => { const recipes = config.commandBuilder.resource.verifiedRecipes; - assert.equal(recipes.filter((r) => !r.unverified).length, 7); + assert.equal(recipes.filter((r) => !r.unverified && r.weights !== "distilled").length, 7); + assert.equal(recipes.filter((r) => r.weights === "distilled").length, 9); for (const hw of config.supportedHardware) { assert.equal(recipes.filter((r) => r.hw === hw && r.default).length, 1); } @@ -153,3 +155,43 @@ test("invalid resource and topology combinations cannot be verified", () => { assert.deepEqual(result.builder.verification, { serve: "error", request: "error" }); } }); + +test("distilled checkpoint: GB200 recipes, Ulysses auto topology and no CFG parallelism", () => { + const recipes = config.commandBuilder.resource.verifiedRecipes.filter((r) => r.weights === "distilled"); + for (const recipe of recipes) { + assert.equal(recipe.hw, "gb200"); + const result = resolve({ ...recipe, topology_mode: "manual" }); + assert.deepEqual(result.builder.errors, [], recipe.id); + assert.equal(result.verified, true, recipe.id); + assert.deepEqual(result.builder.verification, { serve: "verified", request: "verified" }, recipe.id); + assert.ok(result.flags.includes("--model-path {{DISTILLED_MODEL_PATH}}"), recipe.id); + assert.ok(result.flags.includes("--text-encoder-cpu-offload"), recipe.id); + assert.ok(!result.flags.includes("--enable-cfg-parallel"), recipe.id); + assert.deepEqual(result.env, [], recipe.id); + } + // Auto topology is Ulysses across the GPUs; explicit CPU offload is the same command. + const four = resolve({ weights: "distilled", hw: "gb200", gpus_per_node: 4 }); + assert.deepEqual(four.builder.topology, { tp_size: 1, ulysses_degree: 4, ring_degree: 1 }); + assert.deepEqual(four.flags, ["--model-path {{DISTILLED_MODEL_PATH}}", "--port {{PORT}}", "--num-gpus 4", + "--ulysses-degree 4", "--text-encoder-cpu-offload"]); + assert.equal(four.verified, true); + assert.deepEqual(resolve({ weights: "distilled", hw: "gb200", encoder: "offload" }).flags, + resolve({ weights: "distilled", hw: "gb200" }).flags); + assert.equal(resolve({ weights: "distilled", hw: "gb200", encoder: "offload" }).verified, true); + assert.equal(resolve({ weights: "distilled", hw: "gb200", gpus_per_node: 2 }).builder.topology.ulysses_degree, 2); + assert.equal(resolve({ weights: "distilled", hw: "gb200" }).builder.resolvedSettings.cfg, "Disabled"); + // The request carries the distilled defaults; a step change leaves the verified serve recipe alone. + const request = config.curl(selection({ weights: "distilled" })); + assert.ok(request.includes("num_inference_steps=10")); + assert.ok(request.includes("guidance_scale=1.0")); + const steps = resolve({ weights: "distilled", hw: "gb200", distilled_steps: 20 }); + assert.deepEqual(steps.builder.verification, { serve: "verified", request: "unverified" }); + assert.ok(config.curl(selection({ weights: "distilled", distilled_steps: 20 })).includes("num_inference_steps=20")); + // Other hardware is unverified, CFG parallelism is rejected, and the base checkpoint keeps GB200 derived. + assert.equal(resolve({ weights: "distilled", hw: "h200" }).verified, false); + const cfgOn = resolve({ weights: "distilled", hw: "gb200", gpus_per_node: 2, cfg: "on" }); + assert.ok(cfgOn.builder.errors.length); + assert.deepEqual(cfgOn.builder.verification, { serve: "error", request: "error" }); + assert.equal(resolve({ hw: "gb200" }).verified, false); + assert.equal(resolve({ hw: "gb200" }).builder.verification.serve, "unverified"); +}); diff --git a/docs/src/snippets/configs/Wan/wan-animate-2.jsx b/docs/src/snippets/configs/Wan/wan-animate-2.jsx index a6f04cabf6..c93c867319 100644 --- a/docs/src/snippets/configs/Wan/wan-animate-2.jsx +++ b/docs/src/snippets/configs/Wan/wan-animate-2.jsx @@ -1,6 +1,10 @@ export const config = (() => { - const cfgDegree = (s) => s.cfg === "on" || (s.cfg === "auto" && Number(s.gpus_per_node) % 2 === 0) ? 2 : 1; + const distilled = (s) => s.weights === "distilled"; + // The distilled checkpoint runs one branch per step and rejects --enable-cfg-parallel. + const cfgDegree = (s) => !distilled(s) && (s.cfg === "on" || (s.cfg === "auto" && Number(s.gpus_per_node) % 2 === 0)) ? 2 : 1; const placementOf = (s) => s.hw === "rtx4090" && s.placement === "auto" ? "memory" : s.placement; + // The distilled recipes were measured with the text encoder on the CPU, so Auto resolves to that policy for them. + const encoderOf = (s) => distilled(s) && s.encoder === "auto" ? "offload" : s.encoder; const recipes = [ ["h200", 1, "auto"], ["h200", 1, "memory"], @@ -9,15 +13,37 @@ export const config = (() => { ["b300", 2, "auto"], ["b300", 2, "cfg-regression"], ["rtx4090", 1, "memory"], - ["gb200", 1, "auto", true], ["h100", 1, "offload", true], ].map(([hw, gpus, placement, unverified = false]) => ({ id: [hw, gpus, placement].join("-"), hw, nodes: 1, gpus_per_node: gpus, - placement, encoder: "auto", cfg: "auto", + weights: "default", placement, encoder: "auto", cfg: "auto", tp_size: 1, ulysses_degree: 1, ring_degree: 1, default: gpus === 1 && (placement !== "memory" || hw === "rtx4090"), unverified, + verifiedWhen: (s) => !distilled(s), })); + // Distilled checkpoint: GB200 HTTP runs, two identical /v1/videos requests per + // server, text encoder on the CPU. The 1-GPU resident entry is also GB200's + // derived shape for the base checkpoint. + recipes.push(...[ + [1, "auto", 1, 1], + [1, "offload", 1, 1], + [2, "auto", 1, 2], + [2, "offload", 1, 2], + [2, "auto", 2, 1], + [4, "auto", 1, 4], + [4, "offload", 1, 4], + [4, "auto", 2, 2], + [4, "auto", 4, 1], + ].map(([gpus, placement, tp, ulysses]) => ({ + id: ["gb200", gpus, placement, "tp" + tp, "sp" + ulysses].join("-"), + hw: "gb200", nodes: 1, gpus_per_node: gpus, + weights: "distilled", placement, encoder: "auto", cfg: "auto", + tp_size: tp, ulysses_degree: ulysses, ring_degree: 1, + default: gpus === 1 && placement === "auto", + unverified: false, + verifiedWhen: distilled, + }))); const config = { modelName: "Wan-Animate-2", @@ -28,6 +54,15 @@ export const config = (() => { groupHardware: false, matchDims: [], overlayDims: [ + { + id: "weights", title: "Checkpoint weights", scope: "base", default: "default", + description: "Both checkpoints run the native WanAnimate2Pipeline with the same inputs. Restart the server when changing weights.", + learnMore: "#2-model-capabilities", + options: [ + { id: "default", label: "Wan-Animate-2", description: "Base checkpoint: 40 steps at guidance 3.0 by default, CFG parallelism available." }, + { id: "distilled", label: "Wan-Animate-2 Distilled", description: "Distilled checkpoint: 10 steps at guidance 1.0 (the only accepted value), one branch per step; measured on GB200." }, + ], + }, { id: "placement", title: "Memory policy", scope: "serve", default: "auto", description: "CFG parity preserves the measured single-B300 output using offload, replicated encoders and serial VAE decode.", @@ -39,11 +74,12 @@ export const config = (() => { { id: "memory", label: "Memory" }, { id: "cfg-regression", label: "CFG parity", showWhen: (s) => s.hw !== "rtx4090", disabled: (s) => Number(s.gpus_per_node) !== 2 || cfgDegree(s) !== 2, - disableReason: "CFG parity requires 2 GPUs with CFG parallelism." }, + disableReason: "CFG parity requires 2 GPUs with CFG parallelism (base checkpoint)." }, ], }, { id: "encoder", title: "Text encoder", scope: "serve", default: "auto", + description: "For the distilled checkpoint, Auto resolves to CPU offload, the policy of the measured GB200 recipes.", showWhen: (s) => s.hw !== "rtx4090", options: [ { id: "auto", label: "Auto" }, @@ -52,12 +88,12 @@ export const config = (() => { }, { id: "cfg", title: "CFG parallelism", scope: "serve", default: "auto", - description: "Auto uses CFG parallelism for even GPU counts. TP and Ulysses are editable under Setup's advanced topology.", + description: "Auto uses CFG parallelism for even GPU counts with the base checkpoint; the distilled checkpoint always runs without it. TP and Ulysses are editable under Setup's advanced topology.", options: [ { id: "auto", label: "Auto" }, { id: "off", label: "Disabled" }, - { id: "on", label: "Enabled", disabled: (s) => Number(s.gpus_per_node) % 2 !== 0, - disableReason: "CFG parallelism requires an even GPU count." }, + { id: "on", label: "Enabled", disabled: (s) => distilled(s) || Number(s.gpus_per_node) % 2 !== 0, + disableReason: "CFG parallelism requires an even GPU count and the base checkpoint; the distilled checkpoint rejects it." }, ], }, { @@ -76,8 +112,15 @@ export const config = (() => { }, { id: "steps", title: "Denoising steps", scope: "request", kind: "number", + showWhen: (s) => !distilled(s), default: 40, min: 1, max: 100, unit: "steps", options: [], }, + { + id: "distilled_steps", title: "Denoising steps", scope: "request", kind: "number", + showWhen: distilled, + description: "10 is the distilled checkpoint's default; the request keeps guidance at 1.0, the only value the checkpoint accepts.", + default: 10, min: 1, max: 100, unit: "steps", options: [], + }, { id: "audio", title: "Reference audio", scope: "request", default: "false", options: [ @@ -96,6 +139,8 @@ export const config = (() => { verifiedRecipes: recipes, autoTopology: (s) => { const ranks = Number(s.gpus_per_node) / cfgDegree(s); + // Without a CFG split, Ulysses across the ranks is the distilled checkpoint's fastest measured layout. + if (distilled(s)) return { tp_size: 1, ulysses_degree: ranks, ring_degree: 1 }; const tp = ranks >= 4 && ranks % 2 === 0 ? 2 : 1; return { tp_size: tp, ulysses_degree: ranks / tp, ring_degree: 1 }; }, @@ -106,6 +151,8 @@ export const config = (() => { if (!Number.isInteger(gpus) || gpus < 1 || gpus > 8) errors.push("Choose 1-8 GPUs per node."); if (![t.tp_size, t.ulysses_degree, t.ring_degree].every((n) => Number.isInteger(n) && n >= 1)) errors.push("Parallel degrees must be positive integers."); + if (distilled(s) && s.cfg === "on") + errors.push("The distilled checkpoint runs one branch per step and does not support CFG parallelism."); if (gpus !== cfgDegree(s) * t.tp_size * t.ulysses_degree * t.ring_degree) errors.push("GPU count must equal CFG times TP times Ulysses times Ring."); if (40 % (t.tp_size * t.ulysses_degree)) @@ -125,14 +172,20 @@ export const config = (() => { : resource.autoTopology(s); const errors = resource.validateTopology(s, topology); const placement = placementOf(s); + const encoder = encoderOf(s); const recipe = recipes.find((r) => r.hw === s.hw && r.nodes === Number(s.nodes) && r.gpus_per_node === Number(s.gpus_per_node) && r.placement === placement && r.tp_size === topology.tp_size && r.ulysses_degree === topology.ulysses_degree && r.ring_degree === topology.ring_degree && cfgDegree(r) === cfgDegree(s)); - const served = !!recipe && !recipe.unverified && s.encoder === "auto" && errors.length === 0; + const encoderVerified = distilled(s) ? encoder === "offload" : s.encoder === "auto"; + const served = !!recipe && !recipe.unverified && recipe.verifiedWhen(s) && encoderVerified && errors.length === 0; + const steps = Number(distilled(s) ? s.distilled_steps : s.steps); const requested = served && s.resolution === "640x800" && Number(s.clip_len) === 37 - && Number(s.steps) === 40 && s.audio === "false"; - const flags = ["--model-path {{MODEL_NAME}}", "--port {{PORT}}", "--num-gpus " + s.gpus_per_node]; + && steps === (distilled(s) ? 10 : 40) && s.audio === "false"; + const flags = [ + distilled(s) ? "--model-path {{DISTILLED_MODEL_PATH}}" : "--model-path {{MODEL_NAME}}", + "--port {{PORT}}", "--num-gpus " + s.gpus_per_node, + ]; const env = s.hw === "rtx4090" ? ["PYTORCH_ALLOC_CONF=expandable_segments:True"] : []; if (cfgDegree(s) === 2) flags.push("--enable-cfg-parallel"); if (topology.tp_size > 1) flags.push("--tp-size " + topology.tp_size); @@ -144,13 +197,14 @@ export const config = (() => { env.push("SGLANG_DIFFUSION_VAE_CHANNELS_LAST_3D=1"); flags.push("--encoder-parallel replicate", "--vae-config.use-parallel-decode false"); } - if (s.encoder === "offload") flags.push("--text-encoder-cpu-offload"); + if (encoder === "offload") flags.push("--text-encoder-cpu-offload"); const warnings = []; if (!served && !errors.length) warnings.push("This exact deployment has not completed HTTP verification; see the benchmark for offline coverage."); if (served && !requested) warnings.push("These request settings are outside the verified HTTP workload."); if (s.hw === "rtx4090") warnings.push("24 GB recipe: memory mode plus an expandable allocator. The 24-frame benchmark uses nearly all VRAM; longer clips or larger pixel budgets may OOM. Offload also needs sufficient host RAM."); if (s.hw === "h100") warnings.push("Use DiT layerwise offload for the documented 80 GB memory preset. H100 recipes have not been measured in this cookbook."); + if (distilled(s) && s.hw !== "gb200") warnings.push("The distilled checkpoint was measured on GB200 only; see the distilled benchmark for its memory peaks."); const placementLabel = { auto: "Auto", speed: "Speed", memory: "Memory", offload: "DiT layerwise offload", "cfg-regression": "CFG parity" }[placement]; return { match: { hw: s.hw }, nnodes: Number(s.nodes), verified: served, flags, env, @@ -164,6 +218,7 @@ export const config = (() => { }, resolvedSettings: { placement: placementLabel, + encoder: encoder === "offload" ? "CPU offload" : "Auto", cfg: cfgDegree(s) === 2 ? "Enabled" : "Disabled", }, }, @@ -172,6 +227,7 @@ export const config = (() => { }, modelNames: { default: "Wan-AI/Wan2.2-Animate-2-14B-Diffusers" }, placeholders: { + DISTILLED_MODEL_PATH: { target: "command", label: "Distilled checkpoint repository or directory", default: "Wan-AI/Wan2.2-Animate-2-14B-Distilled-Diffusers" }, PORT: { target: "command", label: "Server port", default: "30010" }, CURL_HOST: { target: "curl", label: "Server host", default: "127.0.0.1" }, CURL_PORT: { target: "curl", label: "Request port", default: "30010" }, @@ -184,8 +240,8 @@ export const config = (() => { '--form-string "video_path={{INPUT_VIDEO}}"', '--form-string "clip_len=' + s.clip_len + '"', '--form-string "size=' + s.resolution + '"', - '--form-string "num_inference_steps=' + s.steps + '"', - '--form-string "guidance_scale=3.0"', + '--form-string "num_inference_steps=' + (distilled(s) ? s.distilled_steps : s.steps) + '"', + '--form-string "guidance_scale=' + (distilled(s) ? "1.0" : "3.0") + '"', '--form-string "fps=16"', '--form-string "seed=42"', '--form-string "enable_audio=' + s.audio + '"', diff --git a/docs/src/snippets/diffusion/model-catalog.jsx b/docs/src/snippets/diffusion/model-catalog.jsx index 4d5694bbf7..68d550cd5e 100644 --- a/docs/src/snippets/diffusion/model-catalog.jsx +++ b/docs/src/snippets/diffusion/model-catalog.jsx @@ -191,7 +191,10 @@ export const DiffusionModelCatalog = ({ category }) => { }, { name: "Wan-Animate-2", - modelIds: ["Wan-AI/Wan2.2-Animate-2-14B-Diffusers"], + modelIds: [ + "Wan-AI/Wan2.2-Animate-2-14B-Diffusers", + "Wan-AI/Wan2.2-Animate-2-14B-Distilled-Diffusers", + ], cookbook: "/cookbook/diffusion/Wan/Wan-Animate-2", }, { diff --git a/python/sglang/multimodal_gen/configs/models/dits/__init__.py b/python/sglang/multimodal_gen/configs/models/dits/__init__.py index 2b9d769252..31dd776b96 100644 --- a/python/sglang/multimodal_gen/configs/models/dits/__init__.py +++ b/python/sglang/multimodal_gen/configs/models/dits/__init__.py @@ -23,7 +23,10 @@ from sglang.multimodal_gen.configs.models.dits.sana_video import SanaVideoConfig from sglang.multimodal_gen.configs.models.dits.stablediffusion3 import ( StableDiffusion3TransformerConfig, ) -from sglang.multimodal_gen.configs.models.dits.wan_animate_2 import WanAnimate2Config +from sglang.multimodal_gen.configs.models.dits.wan_animate_2 import ( + WanAnimate2Config, + WanAnimate2DistilledConfig, +) from sglang.multimodal_gen.configs.models.dits.wanvideo import WanVideoConfig __all__ = [ @@ -38,6 +41,7 @@ __all__ = [ "LongLive2VideoConfig", "MiniMaxH3DiTConfig", "WanAnimate2Config", + "WanAnimate2DistilledConfig", "WanVideoConfig", "Hunyuan3DDiTConfig", "MOVAAudioConfig", diff --git a/python/sglang/multimodal_gen/configs/models/dits/wan_animate_2.py b/python/sglang/multimodal_gen/configs/models/dits/wan_animate_2.py index c3d5ddcd54..ea37aa94af 100644 --- a/python/sglang/multimodal_gen/configs/models/dits/wan_animate_2.py +++ b/python/sglang/multimodal_gen/configs/models/dits/wan_animate_2.py @@ -96,8 +96,9 @@ class WanAnimate2ArchConfig(WanVideoArchConfig): -1 ) # <0: resolved to the gen grid width in tokens at runtime (official convention) refer_stride: int = 1 - # Additive attention-score bias on the keys of generation frame 1 (official name); - # 0.0 disables it. Read from transformer/config.json when present; the base weights omit it. + # Additive attention-score bias on the keys of generation frame 1 (official name); 0.0 + # disables it. Neither published transformer/config.json carries the key, so the distilled + # default comes from WanAnimate2DistilledArchConfig; a checkpoint key wins via update_model_arch. log_scale: float = 0.0 # The image-embedding cross-attention (added_kv_proj_dim) is always built, as in the # official model; the checkpoint config states the same and nothing else reads this. @@ -128,3 +129,17 @@ class WanAnimate2ArchConfig(WanVideoArchConfig): class WanAnimate2Config(DiTConfig): arch_config: WanAnimate2ArchConfig = field(default_factory=WanAnimate2ArchConfig) prefix: str = "WanAnimate2" + + +@dataclass +class WanAnimate2DistilledArchConfig(WanAnimate2ArchConfig): + # Value of the official wan_animate_2_distillation.yaml; the published transformer/config.json + # omits the key, so a checkpoint value still wins through update_model_arch. + log_scale: float = -1.3 + + +@dataclass +class WanAnimate2DistilledConfig(WanAnimate2Config): + arch_config: WanAnimate2ArchConfig = field( + default_factory=WanAnimate2DistilledArchConfig + ) diff --git a/python/sglang/multimodal_gen/configs/pipeline_configs/wan.py b/python/sglang/multimodal_gen/configs/pipeline_configs/wan.py index 6ebf12c1cb..711a1bd180 100644 --- a/python/sglang/multimodal_gen/configs/pipeline_configs/wan.py +++ b/python/sglang/multimodal_gen/configs/pipeline_configs/wan.py @@ -2,14 +2,18 @@ # SPDX-License-Identifier: Apache-2.0 from collections.abc import Callable -from dataclasses import dataclass, field +from dataclasses import dataclass, field, replace from typing import Any import torch from sglang.multimodal_gen import envs from sglang.multimodal_gen.configs.models import DiTConfig, EncoderConfig, VAEConfig -from sglang.multimodal_gen.configs.models.dits import WanAnimate2Config, WanVideoConfig +from sglang.multimodal_gen.configs.models.dits import ( + WanAnimate2Config, + WanAnimate2DistilledConfig, + WanVideoConfig, +) from sglang.multimodal_gen.configs.models.encoders import ( BaseEncoderOutput, CLIPVisionConfig, @@ -428,6 +432,21 @@ class Wan_Animate_2_14B_Config(WanI2V720PConfig): super().validate_server_args(server_args) +@dataclass +class Wan_Animate_2_14B_Distilled_Config(Wan_Animate_2_14B_Config): + """Distilled Wan-Animate-2 14B: same components and stages, log_scale -1.3 in the DiT.""" + + dit_config: DiTConfig = field(default_factory=WanAnimate2DistilledConfig) + + def get_model_deployment_config(self) -> ModelDeploymentConfig: + # No unconditional branch to split: a CFG group would recompute one branch per rank, + # so an explicit CFG group is rejected. Auto-enable needs no flag: the server only + # auto-enables CFG parallel for a model whose default sampling params use guidance. + return replace( + super().get_model_deployment_config(), supports_cfg_parallel=False + ) + + # ============================================= # ============= Causal Self-Forcing ============= # ============================================= @@ -441,6 +460,19 @@ class SelfForcingWanT2V480PConfig(WanT2V480PConfig): warp_denoising_step: bool = True +def _is_wan_animate_2_path(p: str) -> bool: + # Registry probes are lower-cased. Require the "-2-"/"_2" token so a v1 + # Wan2.2-Animate-14B checkpoint (a different model / pipeline) does not route here. + return "wan_animate_2" in p or "wan2.2-animate-2" in p + + +def _is_wan_animate_2_distilled_path(p: str) -> bool: + # The distill token counts in the directory's own name only, so a base checkpoint + # kept under a parent such as distillation-runs/ keeps the base config. + short_name = p.rstrip("/").split("/")[-1] + return _is_wan_animate_2_path(p) and "distill" in short_name + + def register(): from sglang.multimodal_gen.configs.sample.wan import ( FastWanT2V480PConfig, @@ -449,6 +481,7 @@ def register(): Wan2_2_I2V_A14B_SamplingParam, Wan2_2_T2V_A14B_SamplingParam, Wan2_2_TI2V_5B_SamplingParam, + Wan_Animate_2_14B_Distilled_SamplingParam, Wan_Animate_2_14B_SamplingParam, WanI2V_14B_480P_SamplingParam, WanI2V_14B_720P_SamplingParam, @@ -551,20 +584,34 @@ def register(): "FastVideo/FastWan2.1-T2V-1.3B-Diffusers", ], ) + # Both repos ship model_index.json with the same _class_name (WanAnimate2Pipeline); + # modular_model_index.json names the variant, and the registry probes the detectors + # with both classes and the path (a distill token counts in the directory's own name + # only). Detectors are checked in registration order, and a directory that ships + # model_index.json matches the base detector through the shared class as well, so + # the distilled entry is registered first. register_configs( - sampling_param_cls=Wan_Animate_2_14B_SamplingParam, - pipeline_config_cls=Wan_Animate_2_14B_Config, - hf_model_paths=[ - "Wan-AI/Wan2.2-Animate-2-14B-Diffusers", - ], + sampling_param_cls=Wan_Animate_2_14B_Distilled_SamplingParam, + pipeline_config_cls=Wan_Animate_2_14B_Distilled_Config, + hf_model_paths=["Wan-AI/Wan2.2-Animate-2-14B-Distilled-Diffusers"], model_detectors=[ - # The registry calls this with the path and with model_index's lower-cased - # _class_name. Require the "-2-"/"_2" token so a v1 Wan2.2-Animate-14B - # checkpoint (a different model / pipeline) does not mis-route here. lambda p: ( - "wan_animate_2" in p.lower() - or "wan2.2-animate-2" in p.lower() - or p.lower() == "wananimate2pipeline" + p == "wananimate2distilledmodularpipeline" + or _is_wan_animate_2_distilled_path(p) + ) + ], + ) + register_configs( + sampling_param_cls=Wan_Animate_2_14B_SamplingParam, + pipeline_config_cls=Wan_Animate_2_14B_Config, + hf_model_paths=["Wan-AI/Wan2.2-Animate-2-14B-Diffusers"], + model_detectors=[ + lambda p: ( + p in ("wananimate2modularpipeline", "wananimate2pipeline") + or ( + _is_wan_animate_2_path(p) + and not _is_wan_animate_2_distilled_path(p) + ) ) ], ) diff --git a/python/sglang/multimodal_gen/configs/sample/wan.py b/python/sglang/multimodal_gen/configs/sample/wan.py index a66976a590..0903a9763b 100644 --- a/python/sglang/multimodal_gen/configs/sample/wan.py +++ b/python/sglang/multimodal_gen/configs/sample/wan.py @@ -395,6 +395,27 @@ class Wan_Animate_2_14B_SamplingParam(Wan2_2_Base_SamplingParams): ) +@dataclass +class Wan_Animate_2_14B_Distilled_SamplingParam(Wan_Animate_2_14B_SamplingParam): + """Sampling defaults for the distilled Wan-Animate-2 14B checkpoint (10 steps, no CFG).""" + + # Guidance is distilled into the weights; the stage runs a single forward at 1.0. + guidance_scale: float = 1.0 + num_inference_steps: int = 10 + + def _validate_with_pipeline_config(self, pipeline_config) -> None: + super()._validate_with_pipeline_config(pipeline_config) + # Runs once per request on the merged params. ValueError like the base hook's + # input checks: the /v1/videos handler maps it to HTTP 400. + if self.guidance_scale != 1.0: + raise ValueError( + f"guidance_scale={self.guidance_scale} is not supported by the " + "distilled Wan-Animate-2 checkpoint: it samples at guidance_scale 1.0 " + "only (the distilled recipe has no unconditional branch); omit the " + "field or set it to 1.0." + ) + + # Smallest clip: one conditioning frame plus one VAE temporal stride (4k+1, k >= 1). MIN_CLIP_LEN = 5 diff --git a/python/sglang/multimodal_gen/registry.py b/python/sglang/multimodal_gen/registry.py index 65c4d0a896..3693e24a17 100644 --- a/python/sglang/multimodal_gen/registry.py +++ b/python/sglang/multimodal_gen/registry.py @@ -49,6 +49,7 @@ from sglang.multimodal_gen.runtime.utils.external_model_package import ( load_external_model_package, ) from sglang.multimodal_gen.runtime.utils.hf_diffusers_utils import ( + load_dict, maybe_download_model_index, ) from sglang.multimodal_gen.runtime.utils.logging_utils import init_logger @@ -473,10 +474,22 @@ def _get_config_info( config = maybe_download_model_index(model_path) pipeline_name = config.get("_class_name", "") pipeline_name = pipeline_name.lower() + # A local checkpoint that ships modular_model_index.json next to model_index.json + # may name a finer-grained pipeline there (the Wan-Animate-2 variants); + # maybe_download_model_index reads the modular file only when model_index.json is + # absent, so probe the detectors with its class as well. + modular_pipeline_name = ( + load_dict(os.path.join(model_path, "modular_model_index.json")) + .get("_class_name", "") + .lower() + ) + probes = [model_path.lower(), pipeline_name] + if modular_pipeline_name and modular_pipeline_name != pipeline_name: + probes.append(modular_pipeline_name) matched_model_names = [] for model_id, detector in _MODEL_NAME_DETECTORS: - if detector(model_path.lower()) or detector(pipeline_name): + if any(detector(probe) for probe in probes): logger.debug( f"Matched model name '{model_id}' using a registered detector." ) diff --git a/python/sglang/multimodal_gen/runtime/pipelines/wan_animate_2_pipeline.py b/python/sglang/multimodal_gen/runtime/pipelines/wan_animate_2_pipeline.py index e2df14e238..2cfe9432e5 100644 --- a/python/sglang/multimodal_gen/runtime/pipelines/wan_animate_2_pipeline.py +++ b/python/sglang/multimodal_gen/runtime/pipelines/wan_animate_2_pipeline.py @@ -9,11 +9,20 @@ standard component loaders. Text uses TextEncodingStage; CLIP and VAE adapters preserve the reference preprocessing and latent scaling. """ +from sglang.multimodal_gen.configs.pipeline_configs.wan import ( + Wan_Animate_2_14B_Distilled_Config, +) from sglang.multimodal_gen.runtime.disaggregation.roles import RoleType from sglang.multimodal_gen.runtime.distributed import get_local_torch_device +from sglang.multimodal_gen.runtime.loader.component_loaders.scheduler_loader import ( + SchedulerLoader, +) from sglang.multimodal_gen.runtime.loader.component_loaders.transformer_loader import ( TransformerLoader, ) +from sglang.multimodal_gen.runtime.models.schedulers.scheduling_dpm_solver_multistep import ( + DPMSolverMultistepScheduler, +) from sglang.multimodal_gen.runtime.pipelines_core.composed_pipeline_base import ( ComposedPipelineBase, ) @@ -25,6 +34,9 @@ from sglang.multimodal_gen.runtime.pipelines_core.stages.model_specific_stages.w WanAnimate2VaeAdapter, ) from sglang.multimodal_gen.runtime.server_args import ServerArgs +from sglang.multimodal_gen.runtime.utils.logging_utils import init_logger + +logger = init_logger(__name__) class WanAnimate2TransformerLoader(TransformerLoader): @@ -34,20 +46,51 @@ class WanAnimate2TransformerLoader(TransformerLoader): strict_checkpoint_keys = True +class WanAnimate2SchedulerLoader(SchedulerLoader): + """Builds DPM-Solver++ 2M from the flow-matching settings for the distilled checkpoint + and defers to the shared loader (the checkpoint's own scheduler/) for the base one; + the cookbook page explains why the shipped scheduler config is not read.""" + + # Pipeline-local: an empty component_names keeps this subclass out of the shared + # loader registry (ComponentLoader.__init_subclass__ registers by component name). + component_names = [] + + def load_customized( + self, component_model_path: str, server_args: ServerArgs, *args + ): + pipeline_config = server_args.pipeline_config + if isinstance(pipeline_config, Wan_Animate_2_14B_Distilled_Config): + override = pipeline_config.scheduler_class_override + if override: + logger.warning( + "--scheduler-class-override %s is ignored for the distilled " + "Wan-Animate-2 checkpoint, which samples with DPM-Solver++ 2M on " + "the official sigma grid; drop the flag.", + override, + ) + return DPMSolverMultistepScheduler( + num_train_timesteps=1000, + prediction_type="flow_prediction", + use_flow_sigmas=True, + flow_shift=pipeline_config.flow_shift, + ) + return super().load_customized(component_model_path, server_args, *args) + + class WanAnimate2Pipeline(ComposedPipelineBase): """Wan-Animate-2 14B in-context reference image + reference video -> video pipeline.""" pipeline_name = "WanAnimate2Pipeline" is_video_pipeline = True - # All seven model_index.json components. The scheduler is built from the - # checkpoint's scheduler/scheduler_config.json (DPM-Solver++, flow prediction, - # flow sigmas) by the shared loader. Two of its settings are overridden at - # runtime: the stages call set_timesteps(sigmas=...) with the official Wan - # sigma grid at every clip (sigma_0 is exactly 1.0), shifted by - # pipeline_config.flow_shift, so the JSON's flow_shift and timestep_spacing - # are not consulted. The JSON's beta_schedule only shapes the beta-derived - # tables, which the flow-sigma path does not read. + # All seven model_index.json components; the scheduler comes from + # WanAnimate2SchedulerLoader (DPM-Solver++, flow prediction, flow sigmas for + # both checkpoints). Two of its settings are overridden at runtime: the + # stages call set_timesteps(sigmas=...) with the official Wan sigma grid at + # every clip (sigma_0 is exactly 1.0), shifted by pipeline_config.flow_shift, + # so the JSON's flow_shift and timestep_spacing are not consulted. The JSON's + # beta_schedule only shapes the beta-derived tables, which the flow-sigma + # path does not read. _required_config_modules: list[str] = [ "text_encoder", "tokenizer", @@ -57,7 +100,10 @@ class WanAnimate2Pipeline(ComposedPipelineBase): "image_processor", "scheduler", ] - component_loaders = {"transformer": WanAnimate2TransformerLoader} + component_loaders = { + "transformer": WanAnimate2TransformerLoader, + "scheduler": WanAnimate2SchedulerLoader, + } def create_pipeline_stages(self, server_args: ServerArgs) -> None: # Keep native modules visible to residency and layerwise-offload management; diff --git a/python/sglang/multimodal_gen/test/server/gpu_cases.py b/python/sglang/multimodal_gen/test/server/gpu_cases.py index 717a3ee1f6..7b5576371c 100644 --- a/python/sglang/multimodal_gen/test/server/gpu_cases.py +++ b/python/sglang/multimodal_gen/test/server/gpu_cases.py @@ -39,6 +39,7 @@ from sglang.multimodal_gen.test.server.testcase_configs import ( SANA_WM_TI2V_CI_sampling_params, T2I_sampling_params, T2V_sampling_params, + WAN_2_2_ANIMATE_2_14B_DISTILLED_sampling_params, WAN_2_2_ANIMATE_2_14B_sampling_params, _make_modelopt_ci_case, _with_default_num_gpus, @@ -64,6 +65,7 @@ from sglang.multimodal_gen.test.test_utils import ( DEFAULT_WAN_2_1_I2V_14B_720P_MODEL_NAME_FOR_TEST, DEFAULT_WAN_2_1_T2V_1_3B_MODEL_NAME_FOR_TEST, DEFAULT_WAN_2_1_T2V_14B_MODEL_NAME_FOR_TEST, + DEFAULT_WAN_2_2_ANIMATE_2_14B_DISTILLED_MODEL_NAME_FOR_TEST, DEFAULT_WAN_2_2_ANIMATE_2_14B_MODEL_NAME_FOR_TEST, DEFAULT_WAN_2_2_I2V_A14B_MODEL_NAME_FOR_TEST, DEFAULT_WAN_2_2_T2V_A14B_MODEL_NAME_FOR_TEST, @@ -478,6 +480,21 @@ ONE_GPU_CASES: list[DiffusionTestCase] = [ run_perf_check=True, run_component_accuracy_check=False, ), + # Distilled checkpoint on the same inputs and server layout: 10 steps at guidance + # 1.0, so the denoiser runs the single-forward path (no unconditional batch). + DiffusionTestCase( + "wan2_2_animate_2_14b_distilled_ref2v", + DiffusionServerArgs( + model_path=DEFAULT_WAN_2_2_ANIMATE_2_14B_DISTILLED_MODEL_NAME_FOR_TEST, + modality="video", + dit_layerwise_offload=True, + text_encoder_cpu_offload=True, + extras=["--layerwise-offload-components", "transformer"], + ), + WAN_2_2_ANIMATE_2_14B_DISTILLED_sampling_params, + run_perf_check=True, + run_component_accuracy_check=False, + ), # flaky # === Helios T2V === # DiffusionTestCase( diff --git a/python/sglang/multimodal_gen/test/server/perf_baselines/h100.json b/python/sglang/multimodal_gen/test/server/perf_baselines/h100.json index 872b0b70c8..27b78a9b66 100644 --- a/python/sglang/multimodal_gen/test/server/perf_baselines/h100.json +++ b/python/sglang/multimodal_gen/test/server/perf_baselines/h100.json @@ -3561,6 +3561,35 @@ "estimated_full_test_time_s": 489.9, "warmup_peak_vram_mb": 14598.0 }, + "wan2_2_animate_2_14b_distilled_ref2v": { + "stages_ms": { + "WanAnimate2BeforeDenoisingStage": 7846.04, + "WanAnimate2DenoisingStage": 69260.73, + "WanAnimate2OutputStage": 0.08 + }, + "denoise_step_ms": { + "0": 23165.59, + "1": 3081.14, + "2": 4366.18, + "3": 4368.35, + "4": 4373.6, + "5": 4377.11, + "6": 4376.38, + "7": 4380.89, + "8": 4382.1, + "9": 4374.26 + }, + "expected_e2e_ms": 76924.33, + "expected_load_ms": 48892.73, + "expected_avg_denoise_ms": 6117.67, + "expected_median_denoise_ms": 4375.32, + "load_peak_vram_mb": 1776.0, + "runtime_peak_vram_mb": 36322.0, + "load_peak_allocated_mb": 1737.0, + "runtime_peak_allocated_mb": 25748.0, + "estimated_full_test_time_s": 439.9, + "warmup_peak_vram_mb": 14598.0 + }, "wan2_2_animate_2_14b_ref2v_cfg_parallel_2gpu": { "stages_ms": { "WanAnimate2BeforeDenoisingStage": 5778.62, diff --git a/python/sglang/multimodal_gen/test/server/testcase_configs.py b/python/sglang/multimodal_gen/test/server/testcase_configs.py index cc6ecbf0c7..894de807ed 100644 --- a/python/sglang/multimodal_gen/test/server/testcase_configs.py +++ b/python/sglang/multimodal_gen/test/server/testcase_configs.py @@ -939,6 +939,17 @@ WAN_2_2_ANIMATE_2_14B_sampling_params = DiffusionSamplingParams( }, ) +# Distilled checkpoint: guidance is folded into the weights, so the request runs the +# model's 10-step, guidance-1.0 recipe over the same inputs, canvas and clip length. +WAN_2_2_ANIMATE_2_14B_DISTILLED_sampling_params = replace( + WAN_2_2_ANIMATE_2_14B_sampling_params, + extras={ + **WAN_2_2_ANIMATE_2_14B_sampling_params.extras, + "num_inference_steps": 10, + "guidance_scale": 1.0, + }, +) + HUNYUAN3D_SHAPE_sampling_params = DiffusionSamplingParams( prompt="", image_path="https://raw.githubusercontent.com/sgl-project/sgl-test-files/main/diffusion-ci/consistency_gt/1-gpu/hunyuan3d_2_0/hunyuan3d.png", diff --git a/python/sglang/multimodal_gen/test/test_utils.py b/python/sglang/multimodal_gen/test/test_utils.py index cb5b1270bd..bc9d8eee61 100644 --- a/python/sglang/multimodal_gen/test/test_utils.py +++ b/python/sglang/multimodal_gen/test/test_utils.py @@ -37,7 +37,7 @@ logger = init_logger(__name__) # NPU/ascend) is read from sgl-project/ci-data-diffusion, where the GT-gen workflows # publish. SGL_TEST_FILES_CI_DATA_REPO = "sgl-project/ci-data-diffusion" -SGL_TEST_FILES_CI_DATA_REVISION = "75992dbf1d6eaeacbc1c2cd6beb48f77dd7ae4f7" +SGL_TEST_FILES_CI_DATA_REVISION = "252df4958ae6cc9f6b146b2f70382f75459235ad" # The NPU pin is kept as a separate branch so ascend GT can be bumped independently # when it's regenerated on its own cadence. 2173af2f re-generated the MiniMax-H3 @@ -208,6 +208,9 @@ DEFAULT_WAN_2_2_I2V_A14B_MODEL_NAME_FOR_TEST = "Wan-AI/Wan2.2-I2V-A14B-Diffusers DEFAULT_WAN_2_2_ANIMATE_2_14B_MODEL_NAME_FOR_TEST = ( "Wan-AI/Wan2.2-Animate-2-14B-Diffusers" ) +DEFAULT_WAN_2_2_ANIMATE_2_14B_DISTILLED_MODEL_NAME_FOR_TEST = ( + "Wan-AI/Wan2.2-Animate-2-14B-Distilled-Diffusers" +) # MOVA video generation models DEFAULT_MOVA_360P_MODEL_NAME_FOR_TEST = "OpenMOSS-Team/MOVA-360p" diff --git a/python/sglang/multimodal_gen/test/unit/conftest.py b/python/sglang/multimodal_gen/test/unit/conftest.py index ed249b90af..8f68d0dd9d 100644 --- a/python/sglang/multimodal_gen/test/unit/conftest.py +++ b/python/sglang/multimodal_gen/test/unit/conftest.py @@ -1,6 +1,8 @@ # SPDX-License-Identifier: Apache-2.0 +from contextlib import contextmanager from types import SimpleNamespace +from unittest.mock import patch import pytest @@ -78,3 +80,58 @@ def default_global_server_args(): yield finally: set_global_server_args(previous) + + +@contextmanager +def mock_cuda_platform( + *, + memory_gb: int = 80, + available_memory_gb: int | dict[int, int] | None = None, +): + """current_platform as an idle CUDA device of memory_gb GB, so ServerArgs resolution + and parallelism auto-tuning run without a GPU.""" + + def get_available_gpu_memory(device_id=0, **_kwargs): + if isinstance(available_memory_gb, dict): + return available_memory_gb[device_id] + if available_memory_gb is not None: + return available_memory_gb + return memory_gb + + with ( + patch( + "sglang.multimodal_gen.runtime.platforms.current_platform.is_cpu", + return_value=False, + ), + patch( + "sglang.multimodal_gen.runtime.platforms.current_platform.is_mps", + return_value=False, + ), + patch( + "sglang.multimodal_gen.runtime.platforms.current_platform.is_cuda", + return_value=True, + ), + patch( + "sglang.multimodal_gen.runtime.platforms.current_platform.is_rocm", + return_value=False, + ), + patch( + "sglang.multimodal_gen.runtime.platforms.current_platform.get_device_total_memory", + return_value=memory_gb * 1024**3, + ), + patch( + "sglang.multimodal_gen.runtime.platforms.current_platform.get_available_gpu_memory", + side_effect=get_available_gpu_memory, + ), + patch( + "sglang.multimodal_gen.runtime.platforms.current_platform.enable_dit_layerwise_offload_by_default", + return_value=True, + ), + ): + yield + + +@pytest.fixture +def cuda_platform(): + with mock_cuda_platform(): + yield diff --git a/python/sglang/multimodal_gen/test/unit/test_registry.py b/python/sglang/multimodal_gen/test/unit/test_registry.py new file mode 100644 index 0000000000..b4e28a97a0 --- /dev/null +++ b/python/sglang/multimodal_gen/test/unit/test_registry.py @@ -0,0 +1,121 @@ +# SPDX-License-Identifier: Apache-2.0 +"""Model routing in the registry, without Hub access.""" + +import json + +import pytest + +from sglang.multimodal_gen import registry +from sglang.multimodal_gen.configs.pipeline_configs.wan import ( + Wan_Animate_2_14B_Config, + Wan_Animate_2_14B_Distilled_Config, +) + + +@pytest.fixture(autouse=True) +def clear_registry_cache(): + registry.get_model_info.cache_clear() + registry._get_config_info.cache_clear() + yield + registry.get_model_info.cache_clear() + registry._get_config_info.cache_clear() + + +def _checkpoint_dir( + root, + *, + name: str = "ckpt-7f3a", + index_class: str = "SomePipeline", + modular_class: str | None = None, +) -> str: + """Laid out like a published repo: model_index.json, the component folders the index + verifier looks for, and modular_model_index.json when a class is given.""" + ckpt = root / name + ckpt.mkdir(parents=True) + (ckpt / "transformer").mkdir() + (ckpt / "vae").mkdir() + index = {"_class_name": index_class, "_diffusers_version": "0.40.0.dev0"} + (ckpt / "model_index.json").write_text(json.dumps(index)) + if modular_class is not None: + (ckpt / "modular_model_index.json").write_text( + json.dumps({**index, "_class_name": modular_class}) + ) + return str(ckpt) + + +@pytest.mark.parametrize("modular_class", [None, "SomeModularPipeline"]) +def test_detectors_are_probed_with_the_modular_pipeline_class( + tmp_path, monkeypatch, modular_class +): + # A checkpoint may ship modular_model_index.json next to model_index.json and name + # a finer-grained pipeline there; without it the probes are the path and the + # model_index class only. + probes = [] + + def detector(probe: str) -> bool: + probes.append(probe) + return False + + monkeypatch.setattr(registry, "_MODEL_NAME_DETECTORS", [("probe", detector)]) + path = _checkpoint_dir(tmp_path, modular_class=modular_class) + assert registry._get_config_info(path) is None + expected = [path.lower(), "somepipeline"] + if modular_class is not None: + expected.append(modular_class.lower()) + assert probes == expected + + +# Wan-Animate-2: both repos ship model_index.json with the same _class_name; +# modular_model_index.json names the variant, and a distill token in the directory's +# own name selects the distilled entry. + +_WAN_ANIMATE_2 = "WanAnimate2Pipeline" + + +@pytest.mark.parametrize( + "modular_class, config_cls", + [ + ("WanAnimate2ModularPipeline", Wan_Animate_2_14B_Config), + ("WanAnimate2DistilledModularPipeline", Wan_Animate_2_14B_Distilled_Config), + # A mirror that skipped modular_model_index.json leaves only the shared class. + (None, Wan_Animate_2_14B_Config), + ], +) +def test_neutral_directory_routes_by_its_modular_pipeline_class( + tmp_path, modular_class, config_cls +): + path = _checkpoint_dir( + tmp_path, index_class=_WAN_ANIMATE_2, modular_class=modular_class + ) + assert registry._get_config_info(path).pipeline_config_cls is config_cls + + +def test_distill_token_in_the_directory_name_routes_to_the_distilled_config(tmp_path): + # The token routes without the modular index; the shared model_index class matches + # the base detector too, so the distilled entry is registered first. + path = _checkpoint_dir( + tmp_path, name="wan_animate_2_distilled", index_class=_WAN_ANIMATE_2 + ) + info = registry._get_config_info(path) + assert info.pipeline_config_cls is Wan_Animate_2_14B_Distilled_Config + + +def test_distill_token_in_a_parent_directory_leaves_the_checkpoint_in_charge(tmp_path): + # Only the directory's own name counts: a base checkpoint kept under a parent such + # as distillation-runs/ routes by its modular class. + path = _checkpoint_dir( + tmp_path / "distillation-runs", + name="wan_animate_2_base", + index_class=_WAN_ANIMATE_2, + modular_class="WanAnimate2ModularPipeline", + ) + assert ( + registry._get_config_info(path).pipeline_config_cls is Wan_Animate_2_14B_Config + ) + + +def test_model_id_short_name_selects_the_variant_for_a_neutral_directory(): + info = registry._get_config_info( + "/models/ckpt-7f3a", model_id="Wan2.2-Animate-2-14B-Distilled-Diffusers" + ) + assert info.pipeline_config_cls is Wan_Animate_2_14B_Distilled_Config diff --git a/python/sglang/multimodal_gen/test/unit/test_server_args.py b/python/sglang/multimodal_gen/test/unit/test_server_args.py index bd89b3b76a..94791096d0 100644 --- a/python/sglang/multimodal_gen/test/unit/test_server_args.py +++ b/python/sglang/multimodal_gen/test/unit/test_server_args.py @@ -4,7 +4,7 @@ import subprocess import sys import tempfile import unittest -from contextlib import ExitStack, contextmanager +from contextlib import ExitStack from unittest.mock import Mock, patch import zmq @@ -97,52 +97,7 @@ from sglang.multimodal_gen.runtime.server_args import ( ) from sglang.multimodal_gen.runtime.server_args import server_args as server_args_module from sglang.multimodal_gen.runtime.utils.argparse import FlexibleArgumentParser - - -@contextmanager -def _mock_cuda_platform( - *, - memory_gb: int = 80, - available_memory_gb: int | dict[int, int] | None = None, -): - def get_available_gpu_memory(device_id=0, **_kwargs): - if isinstance(available_memory_gb, dict): - return available_memory_gb[device_id] - if available_memory_gb is not None: - return available_memory_gb - return memory_gb - - with ( - patch( - "sglang.multimodal_gen.runtime.platforms.current_platform.is_cpu", - return_value=False, - ), - patch( - "sglang.multimodal_gen.runtime.platforms.current_platform.is_mps", - return_value=False, - ), - patch( - "sglang.multimodal_gen.runtime.platforms.current_platform.is_cuda", - return_value=True, - ), - patch( - "sglang.multimodal_gen.runtime.platforms.current_platform.is_rocm", - return_value=False, - ), - patch( - "sglang.multimodal_gen.runtime.platforms.current_platform.get_device_total_memory", - return_value=memory_gb * 1024**3, - ), - patch( - "sglang.multimodal_gen.runtime.platforms.current_platform.get_available_gpu_memory", - side_effect=get_available_gpu_memory, - ), - patch( - "sglang.multimodal_gen.runtime.platforms.current_platform.enable_dit_layerwise_offload_by_default", - return_value=True, - ), - ): - yield +from sglang.multimodal_gen.test.unit.conftest import mock_cuda_platform def _from_dict_without_model_resolution( @@ -151,7 +106,7 @@ def _from_dict_without_model_resolution( pipeline_config = pipeline_config or QwenImagePipelineConfig() with ( patch.object(PipelineConfig, "from_kwargs", return_value=pipeline_config), - _mock_cuda_platform(), + mock_cuda_platform(), ): return ServerArgs.from_dict(kwargs) @@ -191,7 +146,7 @@ class _CudaPlatformTestCase(unittest.TestCase): # Residency queries inspect the platform after argument construction too. stack = ExitStack() self.addCleanup(stack.close) - stack.enter_context(_mock_cuda_platform()) + stack.enter_context(mock_cuda_platform()) class TestSchedulerEndpoints(unittest.TestCase): @@ -284,7 +239,7 @@ class TestServerArgsPathExpansion(_CudaPlatformTestCase): patch.object( PipelineConfig, "from_kwargs", return_value=QwenImagePipelineConfig() ), - _mock_cuda_platform(), + mock_cuda_platform(), ): args, unknown_args = parser.parse_known_args(argv) server_args = ServerArgs.from_cli_args(args, unknown_args) @@ -570,7 +525,7 @@ class TestServerArgsPathExpansion(_CudaPlatformTestCase): patch.object( PipelineConfig, "from_kwargs", return_value=QwenImagePipelineConfig() ), - _mock_cuda_platform(), + mock_cuda_platform(), ): args, unknown_args = parser.parse_known_args(argv) server_args = ServerArgs.from_cli_args(args, unknown_args) @@ -591,7 +546,7 @@ class TestServerArgsPathExpansion(_CudaPlatformTestCase): patch.object( PipelineConfig, "from_kwargs", return_value=QwenImagePipelineConfig() ), - _mock_cuda_platform(), + mock_cuda_platform(), ): args, unknown_args = parser.parse_known_args(argv) server_args = ServerArgs.from_cli_args(args, unknown_args) @@ -3550,7 +3505,7 @@ class TestPipelineResolutionCliOverride(unittest.TestCase): with ( patch.object(sys, "argv", ["sglang"] + argv), - _mock_cuda_platform(), + mock_cuda_platform(), ): args, unknown_args = parser.parse_known_args(argv) server_args = ServerArgs.from_cli_args(args, unknown_args) @@ -3569,7 +3524,7 @@ class TestPipelineResolutionCliOverride(unittest.TestCase): with ( patch.object(sys, "argv", ["sglang"] + argv), - _mock_cuda_platform(), + mock_cuda_platform(), ): args, unknown_args = parser.parse_known_args(argv) server_args = ServerArgs.from_cli_args(args, unknown_args) diff --git a/python/sglang/multimodal_gen/test/unit/test_wan_animate_2_config.py b/python/sglang/multimodal_gen/test/unit/test_wan_animate_2_config.py index bd25910d66..f9db8a1334 100644 --- a/python/sglang/multimodal_gen/test/unit/test_wan_animate_2_config.py +++ b/python/sglang/multimodal_gen/test/unit/test_wan_animate_2_config.py @@ -23,12 +23,19 @@ from sglang.multimodal_gen.configs.models.dits.wan_animate_2 import ( WanAnimate2ArchConfig, WanAnimate2Config, ) -from sglang.multimodal_gen.configs.pipeline_configs.wan import Wan_Animate_2_14B_Config +from sglang.multimodal_gen.configs.pipeline_configs.base import PipelineConfig +from sglang.multimodal_gen.configs.pipeline_configs.wan import ( + Wan_Animate_2_14B_Config, + Wan_Animate_2_14B_Distilled_Config, +) from sglang.multimodal_gen.configs.sample.sampling_params import ( DataType, SamplingParams, ) -from sglang.multimodal_gen.configs.sample.wan import Wan_Animate_2_14B_SamplingParam +from sglang.multimodal_gen.configs.sample.wan import ( + Wan_Animate_2_14B_Distilled_SamplingParam, + Wan_Animate_2_14B_SamplingParam, +) from sglang.multimodal_gen.registry import _get_config_info from sglang.multimodal_gen.runtime.disaggregation.roles import RoleType from sglang.multimodal_gen.runtime.distributed.parallel_state import ( @@ -46,6 +53,7 @@ from sglang.multimodal_gen.runtime.layers.attention.backends.attention_backend i ) from sglang.multimodal_gen.runtime.layers.linear import UnquantizedLinearMethod from sglang.multimodal_gen.runtime.loader.component_loaders.scheduler_loader import ( + SchedulerLoader, _supported_init_kwargs, ) from sglang.multimodal_gen.runtime.loader.component_loaders.transformer_loader import ( @@ -67,6 +75,7 @@ from sglang.multimodal_gen.runtime.models.schedulers.scheduling_dpm_solver_multi ) from sglang.multimodal_gen.runtime.pipelines.wan_animate_2_pipeline import ( WanAnimate2Pipeline, + WanAnimate2SchedulerLoader, WanAnimate2TransformerLoader, ) from sglang.multimodal_gen.runtime.pipelines_core.schedule_batch import Req @@ -117,8 +126,12 @@ def test_pipeline_local_loader_preserves_shared_transformer_registry(): assert component_name_to_loader_cls[name] is TransformerLoader assert not TransformerLoader.strict_checkpoint_keys assert WanAnimate2TransformerLoader.strict_checkpoint_keys + for name in SchedulerLoader.component_names: + assert component_name_to_loader_cls[name] is SchedulerLoader + assert WanAnimate2SchedulerLoader.component_names == [] assert WanAnimate2Pipeline.component_loaders == { - "transformer": WanAnimate2TransformerLoader + "transformer": WanAnimate2TransformerLoader, + "scheduler": WanAnimate2SchedulerLoader, } @@ -241,6 +254,57 @@ def _parallel_server_args( ) +def _two_gpu_server_args(pipeline_config, **kwargs) -> ServerArgs: + """ServerArgs for the config's HF id with the config pinned and model_index.json + served from memory; under the cuda_platform fixture _adjust_parallelism runs + without a GPU or network.""" + hf_id = "Wan-AI/Wan2.2-Animate-2-14B-Diffusers" + if isinstance(pipeline_config, Wan_Animate_2_14B_Distilled_Config): + hf_id = "Wan-AI/Wan2.2-Animate-2-14B-Distilled-Diffusers" + with ( + patch.object(PipelineConfig, "from_kwargs", return_value=pipeline_config), + patch( + "sglang.multimodal_gen.registry.maybe_download_model_index", + return_value={"_class_name": "WanAnimate2Pipeline"}, + ), + patch( + "sglang.multimodal_gen.runtime.server_args.auto_tune.dit_parameter_count", + return_value=None, + ), + ): + return ServerArgs.from_dict({"model_path": hf_id, "num_gpus": 2, **kwargs}) + + +def test_two_gpu_default_layout_splits_cfg_only_for_the_base_checkpoint( + cuda_platform, +): + # With no parallelism flags the server auto-enables CFG parallel for a model whose + # defaults use guidance; the distilled checkpoint runs one branch, so its second GPU + # goes to sequence parallelism instead. + base = _two_gpu_server_args(Wan_Animate_2_14B_Config()) + distilled = _two_gpu_server_args(Wan_Animate_2_14B_Distilled_Config()) + assert base.enable_cfg_parallel and not distilled.enable_cfg_parallel + assert (base.cfg_parallel_degree, base.sp_degree) == (2, 1) + assert (distilled.cfg_parallel_degree, distilled.sp_degree) == (1, 2) + + +def test_distilled_rejects_explicit_cfg_parallel(cuda_platform): + # An explicit CFG group would silently run the single branch on every rank. + with pytest.raises(ValueError, match="does not support CFG parallelism"): + _two_gpu_server_args( + Wan_Animate_2_14B_Distilled_Config(), enable_cfg_parallel=True + ) + + +def test_distilled_deployment_config_keeps_the_base_residency_policy(): + # Only the CFG-parallel field differs; a change to the base residency policy must + # reach the distilled variant. + base = Wan_Animate_2_14B_Config().get_model_deployment_config() + distilled = Wan_Animate_2_14B_Distilled_Config().get_model_deployment_config() + assert distilled.supports_cfg_parallel is False + assert replace(distilled, supports_cfg_parallel=True) == base + + def test_validate_server_args_rejects_fsdp_before_any_weights_load(): with pytest.raises(NotImplementedError, match="use-fsdp-inference"): Wan_Animate_2_14B_Config().validate_server_args( @@ -487,6 +551,16 @@ def test_checkpoint_config_without_image_embedding_is_rejected(): config.update_model_arch({"use_img_emb": False}) +def test_distilled_log_scale_defaults_to_the_official_value_unless_the_checkpoint_sets_it(): + # -1.3 is the official wan_animate_2_distillation.yaml value; the published + # transformer/config.json has no log_scale key, so this default is what runs, and a + # checkpoint that does carry the key wins over it. + config = Wan_Animate_2_14B_Distilled_Config().dit_config + assert config.log_scale == -1.3 + config.update_model_arch({"log_scale": -0.7}) + assert config.log_scale == -0.7 + + # Wan_Animate_2_14B_SamplingParam @@ -580,20 +654,91 @@ def test_sampling_params_reject_the_shared_dit_cache_heuristics(field_name): Wan_Animate_2_14B_SamplingParam(**{field_name: True}) +def _distilled_request(**user_kwargs) -> SamplingParams: + """A /v1/videos-style request through the public entry point: registry defaults for + the distilled HF id merged with the user fields, adjusted and validated.""" + server_args = SimpleNamespace( + pipeline_class_name=None, + backend="sglang", + model_id=None, + pipeline_config=Wan_Animate_2_14B_Distilled_Config(), + output_path=None, + num_gpus=1, + comfyui_mode=False, + ) + with patch( + "sglang.multimodal_gen.registry.maybe_download_model_index", + return_value={"_class_name": "WanAnimate2Pipeline"}, + ): + return SamplingParams.from_user_sampling_params_args( + "Wan-AI/Wan2.2-Animate-2-14B-Distilled-Diffusers", + server_args, + prompt="a person", + image_path="reference.png", + video_path="driving.mp4", + **user_kwargs, + ) + + +@pytest.mark.parametrize( + "guidance_scale, rejected", + [(3.0, True), (0.5, True), (1.0, False), (None, False)], +) +def test_distilled_guidance_other_than_one_is_rejected(guidance_scale, rejected): + # The distilled recipe has no unconditional branch: the once-per-request validation + # hook raises ValueError (HTTP 400 on /v1/videos) for every guidance_scale but 1.0; + # an omitted field (None) takes the class default. + user_kwargs = {} if guidance_scale is None else {"guidance_scale": guidance_scale} + if rejected: + with pytest.raises( + ValueError, + match=rf"guidance_scale={guidance_scale} .*guidance_scale 1\.0 only", + ): + _distilled_request(**user_kwargs) + return + params = _distilled_request(**user_kwargs) + assert params.guidance_scale == 1.0 + + +def test_distilled_sampling_defaults_run_without_cfg(): + # The official distilled recipe: 10 steps, guidance 1.0; the rest is inherited. + params = Wan_Animate_2_14B_Distilled_SamplingParam() + assert params.guidance_scale == 1.0 + assert params.num_inference_steps == 10 + + # Registry resolution (no network) -def test_registered_hf_path_resolves_to_wan_animate_configs(): - info = _get_config_info("Wan-AI/Wan2.2-Animate-2-14B-Diffusers") +@pytest.mark.parametrize( + "hf_id, pipeline_cls, sampling_cls", + [ + ( + "Wan-AI/Wan2.2-Animate-2-14B-Diffusers", + Wan_Animate_2_14B_Config, + Wan_Animate_2_14B_SamplingParam, + ), + ( + "Wan-AI/Wan2.2-Animate-2-14B-Distilled-Diffusers", + Wan_Animate_2_14B_Distilled_Config, + Wan_Animate_2_14B_Distilled_SamplingParam, + ), + ], +) +def test_registered_hf_path_resolves_to_wan_animate_configs( + hf_id, pipeline_cls, sampling_cls +): + info = _get_config_info(hf_id) assert info is not None - assert info.pipeline_config_cls is Wan_Animate_2_14B_Config - assert info.sampling_param_cls is Wan_Animate_2_14B_SamplingParam + assert info.pipeline_config_cls is pipeline_cls + assert info.sampling_param_cls is sampling_cls @pytest.mark.parametrize( "path", ["some/local/dir/wan_animate_2", "my-wan2.2-animate-2-ckpt"] ) def test_detector_resolves_wan_animate_style_paths(path): + # No variant token in the name: the base classes, not the distilled ones. with patch( "sglang.multimodal_gen.registry.maybe_download_model_index", return_value={}, @@ -606,7 +751,8 @@ def test_detector_resolves_wan_animate_style_paths(path): def test_model_index_class_name_routes_a_checkpoint_at_any_path(): # A local checkpoint directory need not carry a model token in its name; its - # model_index.json _class_name is what identifies it. + # model_index.json _class_name is what identifies it. Both variants share that + # name; modular_model_index.json tells them apart (below). with patch( "sglang.multimodal_gen.registry.maybe_download_model_index", return_value={"_class_name": "WanAnimate2Pipeline"}, @@ -616,15 +762,22 @@ def test_model_index_class_name_routes_a_checkpoint_at_any_path(): assert info.pipeline_config_cls is Wan_Animate_2_14B_Config -def test_v1_animate_checkpoint_does_not_route_to_wan_animate_2(): +@pytest.mark.parametrize( + "path", ["Wan-AI/Wan2.2-Animate-14B", "ckpts/Wan2.2-Animate-14B-distill"] +) +def test_v1_animate_checkpoint_does_not_route_to_wan_animate_2(path): # Wan2.2-Animate-14B (v1) is a different model and pipeline; a detector loosened to - # "animate" would pass the positive cases above and mis-route it here. + # "animate" would pass the positive cases above and mis-route it here, and the + # distilled detector needs the "-2-"/"_2" token as well as "distill". with patch( "sglang.multimodal_gen.registry.maybe_download_model_index", return_value={"_class_name": "WanAnimatePipeline"}, ): - info = _get_config_info("Wan-AI/Wan2.2-Animate-14B") - assert info is None or info.pipeline_config_cls is not Wan_Animate_2_14B_Config + info = _get_config_info(path) + assert info is None or info.pipeline_config_cls not in ( + Wan_Animate_2_14B_Config, + Wan_Animate_2_14B_Distilled_Config, + ) # /v1/videos routing: the model-specific knobs are declared extras, the reference image and @@ -1305,8 +1458,10 @@ def test_extract_inputs_rejects_batched_single_value_fields(overrides, message): _WanAnimate2Inputs.from_request(batch) -# Scheduler: loaded from the checkpoint's scheduler/scheduler_config.json through the shared -# SchedulerLoader like the other components. The stages pass the official sigma grid to +# Scheduler: a loaded component like the others, through WanAnimate2SchedulerLoader. The +# base checkpoint's scheduler/scheduler_config.json is loaded as is; for the distilled +# config the loader builds DPMSolverMultistepScheduler from the flow-matching settings +# instead of from its Euler scheduler_config.json. The stages pass the official sigma grid to # set_timesteps explicitly, so the loaded configuration must denoise exactly like a # DPMSolverMultistepScheduler built from the flow-matching settings alone. @@ -1355,6 +1510,67 @@ _FLOW_MATCHING_SCHEDULER_KWARGS = { } +def _load_scheduler(pipeline_config, checkpoint_config: dict | None): + """Run WanAnimate2SchedulerLoader with the checkpoint's scheduler_config.json served + from memory (None: reading it is an error) and the flag-free server args it reads.""" + reader = {"return_value": dict(checkpoint_config)} if checkpoint_config else {} + server_args = SimpleNamespace(pipeline_config=pipeline_config) + with patch( + "sglang.multimodal_gen.runtime.loader.component_loaders.scheduler_loader" + ".get_diffusers_component_config", + **reader, + ) as read: + scheduler = WanAnimate2SchedulerLoader().load_customized( + "/ckpt/scheduler", server_args + ) + assert read.called is (checkpoint_config is not None) + return scheduler + + +def test_base_config_loads_the_checkpoint_scheduler_as_is(): + # The shared loader path: the class the checkpoint names, built from its own settings + # (beta_schedule=linear comes only from that file). + scheduler = _load_scheduler( + Wan_Animate_2_14B_Config(), + {"_class_name": "DPMSolverMultistepScheduler", **_CHECKPOINT_SCHEDULER_CONFIG}, + ) + assert isinstance(scheduler, DPMSolverMultistepScheduler) + assert scheduler._inner.config.beta_schedule == "linear" + assert scheduler._inner.config.flow_shift == 5.0 + + +def test_distilled_config_gets_dpm_solver_built_from_the_flow_matching_settings(): + # The wrapper comes from the flow-matching settings and the configured flow_shift; + # the shipped scheduler_config.json, which carries none of them, is never read. + config = Wan_Animate_2_14B_Distilled_Config() + config.update_config_from_dict({"flow_shift": 3.0}) + scheduler = _load_scheduler(config, None) + assert isinstance(scheduler, DPMSolverMultistepScheduler) + inner = scheduler._inner.config + assert inner.algorithm_type == "dpmsolver++" + assert inner.solver_order == 2 + for key, value in {**_FLOW_MATCHING_SCHEDULER_KWARGS, "flow_shift": 3.0}.items(): + assert getattr(inner, key) == value, key + + +def test_distilled_config_ignores_the_scheduler_class_override_with_one_warning(caplog): + # The flag would select the shipped Euler scheduler on the base path; the distilled + # path keeps DPM-Solver++ and says so instead of silently ignoring the flag. + config = Wan_Animate_2_14B_Distilled_Config() + config.scheduler_class_override = "FlowMatchEulerDiscreteScheduler" + with caplog.at_level(logging.WARNING): + scheduler = _load_scheduler(config, None) + assert isinstance(scheduler, DPMSolverMultistepScheduler) + inner = scheduler._inner.config + assert inner.algorithm_type == "dpmsolver++" + for key, value in _FLOW_MATCHING_SCHEDULER_KWARGS.items(): + assert getattr(inner, key) == value, key + warnings = [r.getMessage() for r in caplog.records if r.levelno == logging.WARNING] + assert len(warnings) == 1, warnings + assert "--scheduler-class-override FlowMatchEulerDiscreteScheduler" in warnings[0] + assert "ignored for the distilled Wan-Animate-2 checkpoint" in warnings[0] + + def test_loader_keeps_the_checkpoint_scheduler_settings(): kwargs = _supported_init_kwargs( DPMSolverMultistepScheduler, dict(_CHECKPOINT_SCHEDULER_CONFIG)