diff --git a/docs/cookbook/diffusion/MiniMax/MiniMax-H3.mdx b/docs/cookbook/diffusion/MiniMax/MiniMax-H3.mdx index 9060d5cc2c..1e1df5c250 100644 --- a/docs/cookbook/diffusion/MiniMax/MiniMax-H3.mdx +++ b/docs/cookbook/diffusion/MiniMax/MiniMax-H3.mdx @@ -495,6 +495,12 @@ HCCL_BUFFSIZE=256 sglang serve \ ## 4. Generate video and audio +`num_inference_steps: N` executes **N denoising updates** over N+1 sigma +points, including the terminal zero. Reference implementations that count +sigma points need N+1 to reproduce this schedule. Older SGLang versions ran +N-1 updates for the same request; use N-1 here to preserve their sampling +trajectory. Changing the step count changes the trajectory, not just latency. + MiniMax-H3 uses the asynchronous OpenAI-compatible video endpoint. Choose a generation mode below, submit a job, poll its status, and then download the completed MP4. @@ -714,8 +720,8 @@ The following pinned FL2VA adapters have distinct purposes: | Recipe | Repository and pinned file | Request setting | Prompt requirement | | --- | --- | --- | --- | -| Recommended speed/quality balance | [`larryvrh/MiniMax-H3-Turbo-Lora`](https://huggingface.co/larryvrh/MiniMax-H3-Turbo-Lora), `minimax_h3_turbo_v4_step600_ema.safetensors` | `num_inference_steps: 9` (8 denoiser evaluations), `lora_scale: 1.0` | None | -| Most aggressive speed preset (standard PEFT layout) | [`lightx2v/Minimax-h3-Turbo`](https://huggingface.co/lightx2v/Minimax-h3-Turbo), `minimax_h3_fl2v_turbo_4step_v0.1.safetensors` | `num_inference_steps: 5` (4 denoiser evaluations), `lora_scale: 1.0`, `lora_alpha: 8` | None | +| Recommended speed/quality balance | [`larryvrh/MiniMax-H3-Turbo-Lora`](https://huggingface.co/larryvrh/MiniMax-H3-Turbo-Lora), `minimax_h3_turbo_v4_step600_ema.safetensors` | `num_inference_steps: 8` (8 denoiser evaluations), `lora_scale: 1.0` | None | +| Most aggressive speed preset (standard PEFT layout) | [`lightx2v/Minimax-h3-Turbo`](https://huggingface.co/lightx2v/Minimax-h3-Turbo), `minimax_h3_fl2v_turbo_4step_v0.1.safetensors` | `num_inference_steps: 4` (4 denoiser evaluations), `lora_scale: 1.0`, `lora_alpha: 8` | None | | Realistic people style | [`fal/MiniMax-H3-Realism-People-LoRA`](https://huggingface.co/fal/MiniMax-H3-Realism-People-LoRA), `h3-realism-people-t2v-i2v-r2v.safetensors` | Keep the normal `num_inference_steps: 50` schedule; start with `lora_scale: 0.7` | Include `r34l1sm` in the prompt | | Spatial and physics | [`Jojocodex/minimax-h3-spatial-physics-lora`](https://huggingface.co/Jojocodex/minimax-h3-spatial-physics-lora), `wushu_spatial_physics_clean_3000_pruned.safetensors` | Keep the normal `num_inference_steps: 50` schedule; start with `lora_scale: 0.4` | None. Describe collisions, stacking, and gravity-consistent motion in the prompt | | Wushu action | [`Jojocodex/minimax-h3-wushu-action-lora`](https://huggingface.co/Jojocodex/minimax-h3-wushu-action-lora), `wushu_action_h3_lora_v5_3000_pruned.safetensors` | Keep the normal `num_inference_steps: 50` schedule; start with `lora_scale: 0.8` | Put `wushu_action` at the start of the prompt | @@ -723,10 +729,9 @@ The following pinned FL2VA adapters have distinct purposes: | Looping sketch anime | [`Inner-Reflections/MiniMax-H3-Looping-Sketch-Anime`](https://huggingface.co/Inner-Reflections/MiniMax-H3-Looping-Sketch-Anime), `minimax_h3_looping_sketch_anime_v1.safetensors` | Keep the normal `num_inference_steps: 50` schedule; start with `lora_scale: 1.0` | Include the author's hand-drawn sketch style phrase in the prompt | | 3D-to-real detail slider | [`siraxe/3d_to_real_detail_slider_H3`](https://huggingface.co/siraxe/3d_to_real_detail_slider_H3), `3d_to_real_H3_detail_slider_bf16.safetensors` | Keep the normal `num_inference_steps: 50` schedule; start with `lora_scale: 1.0` | None. Sweep the scale; this is a slider, not a fixed style | -The H3 request field controls the number of sigma grid points, including the -terminal zero; the denoising loop therefore runs one fewer model evaluation. -This is why an adapter described as 8-step uses `9`, and a 4-step adapter uses -`5`, in the request. +The H3 request field controls the number of denoiser evaluations. An 8-step +adapter uses `8`, and a 4-step adapter uses `4`; the sigma schedule includes +one additional terminal zero. Every recipe uses the same launch shape. Pinning the filename is required for repositories that publish multiple revisions, and is also recommended for a @@ -842,7 +847,7 @@ Keep `pdd_config.json` beside `pdd_fused_heads.safetensors`, outside the a separate output directory, and `--model-variant ref2va`. Use the requests in [Generate video and audio](#4-generate-video-and-audio) -with `num_inference_steps: 9`, `flow_shift: 12.0`, and `audio_flow_shift: 3.0`. +with `num_inference_steps: 8`, `flow_shift: 12.0`, and `audio_flow_shift: 3.0`. The nine sigma grid points give eight denoiser evaluations. These settings must match the fused heads; the server rejects a different step count or shift. Use `quality: "lossless"` in requests. @@ -872,7 +877,7 @@ sglang serve \ Requests use the same asynchronous video endpoint as the base model, with `task: "t2va"`, `conditions: []`, and a target such as `{"short_edge": 768, "aspect_ratio": "16:9", "duration_seconds": 5.0}`. The -request default is `num_inference_steps: 9`: nine sigma grid points, i.e. the +request default is `num_inference_steps: 8`: nine sigma grid points, i.e. the eight distilled DiT evaluations on the checkpoint's trained rungs `[999, 874, 749, 624, 500, 375, 250, 125]` (unshifted timesteps over 1000, shifted once per modality, as FastVideo runs them). Any other step count is @@ -958,7 +963,7 @@ Requests use the same video endpoint as the base model with `task: "t2va"` and `conditions: []`, or `task: "fl2va"` with the [keyframe conditions](#4-generate-video-and-audio), and a target such as `{"short_edge": 768, "aspect_ratio": "16:9", "duration_seconds": 14.375}` (the -345-frame paper workload). The default `num_inference_steps: 9` is the only +345-frame paper workload). The default `num_inference_steps: 8` is the only accepted value: nine sigma grid points, eight DiT forwards. Each keyframe adds about 2,000 dense rows, so an `fl2va` forward costs a little more than `t2va`. @@ -1353,7 +1358,7 @@ sglang serve \ entry per denoise step, each in `(0, 1]`. A ratio of `1.0` keeps a step on H3's native dense attention path; smaller values select the FlexAttention sparse path and drop more blocks. The example above matches a request with - `num_inference_steps: 8`, whose endpoint-inclusive sigma schedule has seven + `num_inference_steps: 7`, whose endpoint-inclusive sigma schedule has seven denoise updates. Cube labeling is coordinate-driven. FL2VA keyframes share the target video's @@ -1622,7 +1627,7 @@ python3 -m sglang.multimodal_gen.benchmarks.bench_serving \ A 4× GB300 host served [FastH3](#6-fasth3-8-step-distilled) with the VSA-H3 recipe above (1344×768 at 24 fps with audio, `task: "t2va"`, -`num_inference_steps: 9`, the default `quality: "extra-high"` and sparsity 0.8, +eight DiT forwards, the default `quality: "extra-high"` and sparsity 0.8, eager BF16, Ulysses4, `NCCL_NVLS_ENABLE=0`), measured with the MP4 encoded while the VAE decodes ([sgl-project/sglang#41819](https://github.com/sgl-project/sglang/pull/41819)). @@ -1650,7 +1655,7 @@ every MP4 carries 1344×768 H.264 at 24 fps with a 32 kHz stereo AAC track. An 8× B200 (SM100, 183 GB) host served [VDN-H3](#7-vdn-h3-hybrid-attention-8-step-distill) on the paper workload: 1344×768 at 24 fps with audio for 14.375 s (345 frames, 102 latent frames, about 104k packed rows), `task: "t2va"`, -`num_inference_steps: 9` (8 DiT forwards), seed 1000, `hybrid_window_attn_h3`, +nine sigma grid points (8 DiT forwards), seed 1000, `hybrid_window_attn_h3`, eager, and the warmup run at the served clip shape (`--warmup-num-frames 345 --warmup-resolutions 1344x768`), so every forward of the served request is steady state; without those flags the first forward diff --git a/python/sglang/multimodal_gen/configs/sample/minimax_h3.py b/python/sglang/multimodal_gen/configs/sample/minimax_h3.py index f679e836f0..da8d4a1b7f 100644 --- a/python/sglang/multimodal_gen/configs/sample/minimax_h3.py +++ b/python/sglang/multimodal_gen/configs/sample/minimax_h3.py @@ -25,8 +25,7 @@ def _optional_unit_float(value: Any, field_name: str) -> float | None: @dataclass class MiniMaxH3SamplingParams(SamplingParams): - # The video/audio sigma schedules include both interval endpoints. - min_num_inference_steps: ClassVar[int] = 2 + min_num_inference_steps: ClassVar[int] = 1 height: int = 512 width: int = 896 @@ -348,14 +347,15 @@ class MiniMaxH3SamplingParams(SamplingParams): class FastH3SamplingParams(MiniMaxH3SamplingParams): """FastH3 8-Step V2: nine sigma points, i.e. eight DiT forwards on the trained rungs.""" - num_inference_steps: int = 9 + num_inference_steps: int = 8 quality: str = "lossless" def _validate(self) -> None: super()._validate() - if self.num_inference_steps != 9: + if self.num_inference_steps != 8: raise ValueError( - "FastH3 is distilled for exactly nine sigma grid points (eight DiT " + "FastH3 is distilled for exactly eight inference steps (nine sigma " + "grid points, eight DiT " f"forwards); got num_inference_steps={self.num_inference_steps}. " "Use MiniMaxAI/MiniMax-H3 for other schedules." ) diff --git a/python/sglang/multimodal_gen/configs/sample/minimax_h3_vdn.py b/python/sglang/multimodal_gen/configs/sample/minimax_h3_vdn.py index b54d646f33..bd208374aa 100644 --- a/python/sglang/multimodal_gen/configs/sample/minimax_h3_vdn.py +++ b/python/sglang/multimodal_gen/configs/sample/minimax_h3_vdn.py @@ -9,17 +9,17 @@ from sglang.multimodal_gen.configs.sample.minimax_h3 import MiniMaxH3SamplingPar @dataclass class VDNH3SamplingParams(MiniMaxH3SamplingParams): """VDN-H3: nine sigma grid points, i.e. the eight distilled DiT forwards - (VDN counts NFEs; SGLang counts sigma grid points). The turbo adapter is - only valid at 8 NFE with video shift 12 / audio shift 3 (the defaults).""" + The turbo adapter is only valid at 8 NFE with video shift 12 / audio + shift 3 (the defaults).""" - num_inference_steps: int = 9 + num_inference_steps: int = 8 def _validate(self) -> None: super()._validate() - if self.num_inference_steps != 9: + if self.num_inference_steps != 8: raise ValueError( - "VDN-H3 is distilled for exactly nine sigma grid points (eight DiT " - f"forwards); got num_inference_steps={self.num_inference_steps}. " + "VDN-H3 is distilled for exactly eight inference steps (nine sigma " + f"grid points); got num_inference_steps={self.num_inference_steps}. " "Use MiniMaxAI/MiniMax-H3 for other schedules." ) if self.task is not None and self.task.strip().lower() not in ( diff --git a/python/sglang/multimodal_gen/envs.py b/python/sglang/multimodal_gen/envs.py index d78d980512..8cfa2a1cd8 100644 --- a/python/sglang/multimodal_gen/envs.py +++ b/python/sglang/multimodal_gen/envs.py @@ -334,7 +334,7 @@ environment_variables: dict[str, Callable[[], Any]] = { ), # Plan slots in the MiniMax-H3 --minimax-h3-adaln-online GPU slab # (9.25 MiB per slot-timestep; 64 x width 4 = 2.31 GiB). A request needs - # up to num_inference_steps - 1 slots; the default covers the 50-step + # up to num_inference_steps slots; the default covers the 50-step # serving schedule, so this is an escape hatch, not a deployment knob. "SGLANG_DIFFUSION_MINIMAX_H3_ADALN_GPU_PLANS": _lazy_int( "SGLANG_DIFFUSION_MINIMAX_H3_ADALN_GPU_PLANS", 64 diff --git a/python/sglang/multimodal_gen/runtime/models/dits/minimax_h3.py b/python/sglang/multimodal_gen/runtime/models/dits/minimax_h3.py index 9f1c2d1b84..bc41a2904b 100644 --- a/python/sglang/multimodal_gen/runtime/models/dits/minimax_h3.py +++ b/python/sglang/multimodal_gen/runtime/models/dits/minimax_h3.py @@ -1794,8 +1794,8 @@ class MiniMaxH3FinalLayer(nn.Module): if not 0 <= step < stack.shape[0]: raise ValueError( f"MiniMax-H3 PDD has {stack.shape[0]} fused heads but the loop is at " - f"step {step}; run with --num-inference-steps {stack.shape[0] + 1} " - "(H3 counts sigma grid points, so that is one more than the steps)." + f"step {step}; run with --num-inference-steps {stack.shape[0]} " + "(num_inference_steps counts denoise transitions)." ) weight = stack[step].to(device=h.device, dtype=h.dtype) bias = heads[f"{name}.bias"][step].to(device=h.device, dtype=h.dtype) diff --git a/python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/minimax_h3/release_metadata.py b/python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/minimax_h3/release_metadata.py index e33c19a8a5..af41661a01 100644 --- a/python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/minimax_h3/release_metadata.py +++ b/python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/minimax_h3/release_metadata.py @@ -172,20 +172,17 @@ class MiniMaxH3PartitionAdmissionStage(PipelineStage): self.metadata.canonical_task(task) min_steps = MiniMaxH3SamplingParams.min_num_inference_steps if batch.num_inference_steps < min_steps: - raise ValueError( - f"MiniMax H3 requires num_inference_steps >= {min_steps} because " - "its video/audio sigma schedules include both interval endpoints" - ) + raise ValueError(f"MiniMax H3 requires num_inference_steps >= {min_steps}") gpu_plans = envs.SGLANG_DIFFUSION_MINIMAX_H3_ADALN_GPU_PLANS if ( server_args.minimax_h3_adaln_online - and batch.num_inference_steps - 1 > gpu_plans + and batch.num_inference_steps > gpu_plans ): # Fail here, before the encode stages spend GPU time on a request # whose AdaLN rebuild is guaranteed to overflow the slab. raise ValueError( f"num_inference_steps={batch.num_inference_steps} needs up to " - f"{batch.num_inference_steps - 1} AdaLN plans but the online " + f"{batch.num_inference_steps} AdaLN plans but the online " f"slab holds {gpu_plans}; raise " "SGLANG_DIFFUSION_MINIMAX_H3_ADALN_GPU_PLANS" ) diff --git a/python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/minimax_h3/stages/denoising.py b/python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/minimax_h3/stages/denoising.py index f44355ec2b..7983186aaf 100644 --- a/python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/minimax_h3/stages/denoising.py +++ b/python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/minimax_h3/stages/denoising.py @@ -446,8 +446,7 @@ class MiniMaxH3DenoisingStage(DenoisingStage): def default_workload_iterations( self, batch: Req, num_inference_steps: int ) -> int | None: - # one denoise per sigma interval: steps - 1 - return max(1, num_inference_steps - 1) + return num_inference_steps def __init__(self, transformer, pipeline=None) -> None: super().__init__( diff --git a/python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/minimax_h3/stages/timestep_preparation.py b/python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/minimax_h3/stages/timestep_preparation.py index fc5f6fa3ec..fc027eea4b 100644 --- a/python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/minimax_h3/stages/timestep_preparation.py +++ b/python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/minimax_h3/stages/timestep_preparation.py @@ -47,7 +47,7 @@ class MiniMaxH3TimestepPreparationStage(PipelineStage): ) with safe_open(pdd_heads, "pt") as f: steps = f.get_slice("video_out.weight").get_shape()[0] - if self._pdd_config["num_inference_steps"] != steps + 1: + if self._pdd_config["num_inference_steps"] != steps: raise ValueError("MiniMax-H3 PDD config does not match the fused heads") def forward(self, batch: Req, server_args: ServerArgs) -> Req: @@ -104,7 +104,7 @@ class MiniMaxH3TimestepPreparationStage(PipelineStage): if batch.is_warmup: # Warmup may run fewer steps, but must use the same intervals # as serving rather than rescaling a shorter grid to [1, 0]. - sigmas[modality] = expected[: max(2, batch.num_inference_steps)] + sigmas[modality] = expected[: batch.num_inference_steps + 1] continue actual = sigmas[modality] if len(actual) != len(expected) or any( @@ -206,7 +206,7 @@ class MiniMaxH3TimestepPreparationStage(PipelineStage): num_steps = requested_num_steps if dmd_steps is not None and batch.is_warmup: # warmup truncates the trained rungs instead of rescaling a shorter grid - num_steps = len(dmd_steps) + 1 + num_steps = len(dmd_steps) sigmas: dict[str, list[float]] = {} for modality in ("video", "audio"): sigmas[modality] = minimax_h3_time_shift_sigmas( @@ -215,7 +215,7 @@ class MiniMaxH3TimestepPreparationStage(PipelineStage): dmd_steps=dmd_steps, ) if num_steps != requested_num_steps: - sigmas[modality] = sigmas[modality][: max(2, requested_num_steps)] + sigmas[modality] = sigmas[modality][: requested_num_steps + 1] batch.extra[MINIMAX_H3_SIGMAS_EXTRA_KEY] = sigmas def verify_input(self, batch: Req, server_args: ServerArgs) -> VerificationResult: diff --git a/python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/minimax_h3/time_request.py b/python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/minimax_h3/time_request.py index 3ba9f92014..91525c3655 100644 --- a/python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/minimax_h3/time_request.py +++ b/python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/minimax_h3/time_request.py @@ -43,7 +43,7 @@ def minimax_h3_time_shift_sigmas( import torch if dmd_steps is not None: - if int(num_steps) != len(dmd_steps) + 1: + if int(num_steps) != len(dmd_steps): raise ValueError( f"this checkpoint is distilled for {len(dmd_steps)} DiT forwards " f"({len(dmd_steps) + 1} sigma grid points); got " @@ -58,15 +58,12 @@ def minimax_h3_time_shift_sigmas( base = torch.linspace( 1.0, 0.0, - int(num_steps), + int(num_steps) + 1, device="cpu", dtype=torch.float32, ) shifted = float(shift_scale) * base / (1 + (float(shift_scale) - 1) * base) shifted, _ = torch.unique_consecutive(shifted, return_counts=True) - # A one-point request is still exactly one point. Normal serving uses - # multiple points, but preserving the requested cardinality keeps - # ``num_inference_steps`` the sole schedule-size control. - if num_steps > 1 and shifted[-1].item() > 0.0: + if shifted[-1].item() > 0.0: shifted = torch.cat([shifted, torch.tensor([0.0], dtype=shifted.dtype)]) return [float(value) for value in shifted.tolist()] diff --git a/python/sglang/multimodal_gen/test/server/gpu_cases.py b/python/sglang/multimodal_gen/test/server/gpu_cases.py index 4ea2a0e2f8..9a25008ef1 100644 --- a/python/sglang/multimodal_gen/test/server/gpu_cases.py +++ b/python/sglang/multimodal_gen/test/server/gpu_cases.py @@ -781,7 +781,7 @@ MINIMAX_H3_FOUR_GPU_H100_CASES = [ "aspect_ratio": "16:9", "duration_seconds": 5.0, }, - "num_inference_steps": 9, + "num_inference_steps": 8, "seed": 42, }, ), @@ -822,7 +822,7 @@ MINIMAX_H3_FOUR_GPU_H100_CASES = [ "aspect_ratio": "16:9", "duration_seconds": 5.0, }, - "num_inference_steps": 9, + "num_inference_steps": 8, "seed": 42, }, ), diff --git a/python/sglang/multimodal_gen/test/test_utils.py b/python/sglang/multimodal_gen/test/test_utils.py index 5a29794e5e..4accd433b3 100644 --- a/python/sglang/multimodal_gen/test/test_utils.py +++ b/python/sglang/multimodal_gen/test/test_utils.py @@ -37,7 +37,7 @@ logger = init_logger(__name__) # NPU/ascend) is read from sgl-project/ci-data-diffusion, where the GT-gen workflows # publish. SGL_TEST_FILES_CI_DATA_REPO = "sgl-project/ci-data-diffusion" -SGL_TEST_FILES_CI_DATA_REVISION = "5d8eb22d16624817c62e06ce4cd69d36fbe7d456" +SGL_TEST_FILES_CI_DATA_REVISION = "75992dbf1d6eaeacbc1c2cd6beb48f77dd7ae4f7" # The NPU pin is kept as a separate branch so ascend GT can be bumped independently # when it's regenerated on its own cadence. diff --git a/python/sglang/multimodal_gen/test/unit/test_minimax_h3_admission.py b/python/sglang/multimodal_gen/test/unit/test_minimax_h3_admission.py index 157c98c46b..d8e2dd31ad 100644 --- a/python/sglang/multimodal_gen/test/unit/test_minimax_h3_admission.py +++ b/python/sglang/multimodal_gen/test/unit/test_minimax_h3_admission.py @@ -451,6 +451,21 @@ def test_admission_rejects_steps_exceeding_online_adaln_gpu_plans(): stage.forward(batch, server_args) batch.num_inference_steps = 9 + with pytest.raises(ValueError, match="9 AdaLN plans.*slab holds 8"): + stage.forward(batch, server_args) + + batch.num_inference_steps = 8 + assert stage.forward(batch, server_args) is batch + + batch.num_inference_steps = 1 + assert stage.forward(batch, server_args) is batch + + batch.num_inference_steps = 0 + with pytest.raises(ValueError, match="num_inference_steps >= 1"): + stage.forward(batch, server_args) + + server_args.minimax_h3_adaln_online = False + batch.num_inference_steps = 50 assert stage.forward(batch, server_args) is batch diff --git a/python/sglang/multimodal_gen/test/unit/test_minimax_h3_denoise_loop.py b/python/sglang/multimodal_gen/test/unit/test_minimax_h3_denoise_loop.py index 4b364b582d..e21deabe5f 100644 --- a/python/sglang/multimodal_gen/test/unit/test_minimax_h3_denoise_loop.py +++ b/python/sglang/multimodal_gen/test/unit/test_minimax_h3_denoise_loop.py @@ -30,6 +30,9 @@ from sglang.multimodal_gen.runtime.pipelines_core.stages.model_specific_stages.m _build_cube_attn_metadata, _precompute_refined_prompt_embeds, ) +from sglang.multimodal_gen.runtime.pipelines_core.stages.model_specific_stages.minimax_h3.time_request import ( + minimax_h3_time_shift_sigmas, +) def _branch( @@ -64,6 +67,32 @@ def _branch( ) +def test_time_shift_sigmas_build_exact_denoise_step_count(): + for num_steps in (1, 2, 10, 50): + for shift_scale in (3.0, 12.0): + sigmas = minimax_h3_time_shift_sigmas( + num_steps=num_steps, + shift_scale=shift_scale, + ) + + assert len(sigmas) == num_steps + 1 + assert sigmas[0] == 1.0 + assert sigmas[-1] == 0.0 + assert all( + current > following for current, following in zip(sigmas, sigmas[1:]) + ) + # Independent closed form for N Euler intervals, including sigma=0. + expected = [ + shift_scale + * (num_steps - i) + / (num_steps + (shift_scale - 1) * (num_steps - i)) + for i in range(num_steps + 1) + ] + torch.testing.assert_close( + torch.tensor(sigmas), torch.tensor(expected), rtol=0, atol=2e-7 + ) + + def test_precomputed_timestep_plan_matches_full_unique_reference(): """Preplanning must preserve fp32 collisions and every packed row class.""" diff --git a/python/sglang/multimodal_gen/test/unit/test_minimax_h3_fasth3.py b/python/sglang/multimodal_gen/test/unit/test_minimax_h3_fasth3.py index 89ca14c2b0..a2c3105242 100644 --- a/python/sglang/multimodal_gen/test/unit/test_minimax_h3_fasth3.py +++ b/python/sglang/multimodal_gen/test/unit/test_minimax_h3_fasth3.py @@ -59,13 +59,13 @@ def test_registry_resolves_fasth3_configs() -> None: def test_fasth3_sampling_defaults_and_task_rejection() -> None: params = FastH3SamplingParams(prompt="p") - assert params.num_inference_steps == 9 + assert params.num_inference_steps == 8 assert params.guidance_scale == 1.0 assert params.quality == "lossless" assert FastH3SamplingParams(prompt="p", quality="extra-high").quality == "lossless" assert FastH3SamplingParams(prompt="p", quality="exact").quality == "exact" - with pytest.raises(ValueError, match="exactly nine sigma grid points"): + with pytest.raises(ValueError, match="exactly eight inference steps"): FastH3SamplingParams(prompt="p", num_inference_steps=50) with pytest.raises(ValueError, match="distilled for t2va only"): @@ -109,7 +109,7 @@ def test_fasth3_dmd_schedule_matches_trained_rungs() -> None: } for shift, expected in served.items(): sigmas = minimax_h3_time_shift_sigmas( - num_steps=9, shift_scale=shift, dmd_steps=DMD_STEPS + num_steps=8, shift_scale=shift, dmd_steps=DMD_STEPS ) assert sigmas == pytest.approx(expected, abs=1e-6) with pytest.raises(ValueError, match="8 DiT forwards"): diff --git a/python/sglang/multimodal_gen/test/unit/test_minimax_h3_pdd_offline.py b/python/sglang/multimodal_gen/test/unit/test_minimax_h3_pdd_offline.py index 51f3b211fa..527e139996 100644 --- a/python/sglang/multimodal_gen/test/unit/test_minimax_h3_pdd_offline.py +++ b/python/sglang/multimodal_gen/test/unit/test_minimax_h3_pdd_offline.py @@ -113,7 +113,8 @@ class TestMiniMaxH3PDDOffline(CustomTestCase): apply_schedule(batch) self.assertEqual( batch.extra[MINIMAX_H3_SIGMAS_EXTRA_KEY], - {name: values[:3] for name, values in sigmas.items()}, + # Three warmup updates need four sigma boundaries on the serving grid. + {name: values[:4] for name, values in sigmas.items()}, ) def test_conversion_keeps_heads_outside_transformer(self): diff --git a/python/sglang/multimodal_gen/test/unit/test_minimax_h3_vdn.py b/python/sglang/multimodal_gen/test/unit/test_minimax_h3_vdn.py index 6a330c5103..ae5d382594 100644 --- a/python/sglang/multimodal_gen/test/unit/test_minimax_h3_vdn.py +++ b/python/sglang/multimodal_gen/test/unit/test_minimax_h3_vdn.py @@ -73,9 +73,9 @@ def test_registry_resolves_vdn_h3_configs() -> None: def test_vdn_h3_sampling_defaults_and_rejections() -> None: params = VDNH3SamplingParams(prompt="p") - assert params.num_inference_steps == 9 # 8 NFE - with pytest.raises(ValueError, match="exactly nine sigma grid points"): - VDNH3SamplingParams(prompt="p", num_inference_steps=8) + assert params.num_inference_steps == 8 + with pytest.raises(ValueError, match="exactly eight inference steps"): + VDNH3SamplingParams(prompt="p", num_inference_steps=9) fl2va = VDNH3SamplingParams( prompt="p", task="fl2va", diff --git a/python/sglang/multimodal_gen/test/unit/test_publish_diffusion_gt.py b/python/sglang/multimodal_gen/test/unit/test_publish_diffusion_gt.py new file mode 100644 index 0000000000..acb1f4474a --- /dev/null +++ b/python/sglang/multimodal_gen/test/unit/test_publish_diffusion_gt.py @@ -0,0 +1,67 @@ +import importlib.util +import io +import json +import sys +import tempfile +import unittest +import wave +from pathlib import Path +from unittest.mock import patch + +import numpy as np + +from sglang.multimodal_gen.test.test_utils import encode_audio_gt_wav +from sglang.test.test_utils import CustomTestCase + +SCRIPT = ( + Path(__file__).resolve().parents[5] + / "scripts/ci/utils/diffusion/publish_diffusion_gt.py" +) +spec = importlib.util.spec_from_file_location("publish_diffusion_gt", SCRIPT) +publisher = importlib.util.module_from_spec(spec) +sys.modules[spec.name] = publisher +spec.loader.exec_module(publisher) + + +class TestPublishDiffusionAudioGT(CustomTestCase): + def test_generated_audio_is_collected_validated_and_deduplicated(self): + content = encode_audio_gt_wav(np.linspace(-0.5, 0.5, 16000)) + with tempfile.TemporaryDirectory() as directory: + Path(directory, "case_2gpu_audio.wav").write_bytes(content) + Path(directory, "case.mp4").write_bytes(b"not a GT") + files = publisher.collect_gt_files(directory, "gt/h100") + self.assertEqual(files, [("gt/h100/case_2gpu_audio.wav", content)]) + publisher.validate_gt_files(files, files, {}, None) + entries = [ + { + "type": "file", + "path": files[0][0], + "sha": publisher.git_blob_sha(content), + } + ] + with patch.object( + publisher, "make_github_request", return_value=json.dumps(entries) + ): + shas = publisher.get_remote_blob_shas("owner", "repo", "gt/h100", None) + self.assertEqual(publisher.filter_changed_files(files, shas), []) + + def test_invalid_audio_is_rejected(self): + valid = encode_audio_gt_wav(np.ones(16000) * 0.1) + output = io.BytesIO() + with wave.open(output, "wb") as wav: + wav.setnchannels(2) + wav.setsampwidth(2) + wav.setframerate(16000) + wav.writeframes(b"\0" * 32) + for content in [ + b"not a WAV", + valid[:-2], + encode_audio_gt_wav(np.zeros(0)), + output.getvalue(), + ]: + with self.subTest(length=len(content)), self.assertRaises(SystemExit): + publisher.validate_gt_files([("gt/audio.wav", content)], [], {}, None) + + +if __name__ == "__main__": + unittest.main() diff --git a/python/sglang/multimodal_gen/test/unit/test_warmup_step_floor.py b/python/sglang/multimodal_gen/test/unit/test_warmup_step_floor.py index bcc7e2570b..b21de0f984 100644 --- a/python/sglang/multimodal_gen/test/unit/test_warmup_step_floor.py +++ b/python/sglang/multimodal_gen/test/unit/test_warmup_step_floor.py @@ -27,10 +27,10 @@ def test_generic_warmup_keeps_the_requested_steps(): assert warmup.extra["warmup_target_num_inference_steps"] == 50 -def test_h3_warmup_is_raised_to_its_floor(): - assert MiniMaxH3SamplingParams.min_num_inference_steps == 2 +def test_h3_warmup_keeps_one_denoise_step(): + assert MiniMaxH3SamplingParams.min_num_inference_steps == 1 warmup = _req(MiniMaxH3SamplingParams(prompt="p"), 50).copy_as_warmup(1) - assert warmup.num_inference_steps == 2 + assert warmup.num_inference_steps == 1 assert warmup.extra["warmup_target_num_inference_steps"] == 50 diff --git a/python/sglang/multimodal_gen/tools/build_minimax_h3_adaln_cache.py b/python/sglang/multimodal_gen/tools/build_minimax_h3_adaln_cache.py index c4b58c9232..5c0fc93841 100644 --- a/python/sglang/multimodal_gen/tools/build_minimax_h3_adaln_cache.py +++ b/python/sglang/multimodal_gen/tools/build_minimax_h3_adaln_cache.py @@ -149,8 +149,8 @@ def _load_tensor( def main() -> None: args = _parse_args() - if args.num_inference_steps < 2 and args.timesteps is None: - raise ValueError("--num-inference-steps must be at least 2") + if args.num_inference_steps < 1 and args.timesteps is None: + raise ValueError("--num-inference-steps must be at least 1") mode_variant = _MODE_VARIANTS[args.mode] if args.model_variant != mode_variant: raise ValueError(f"--mode {args.mode} requires {mode_variant}") diff --git a/python/sglang/multimodal_gen/tools/fuse_minimax_h3_pdd_heads.py b/python/sglang/multimodal_gen/tools/fuse_minimax_h3_pdd_heads.py index 91ad5d96b8..0c8cf4f9d8 100755 --- a/python/sglang/multimodal_gen/tools/fuse_minimax_h3_pdd_heads.py +++ b/python/sglang/multimodal_gen/tools/fuse_minimax_h3_pdd_heads.py @@ -90,7 +90,7 @@ def main() -> int: } save_file(fused, str(d / "pdd_fused_heads.safetensors"), metadata={"format": "pt"}) cfg["fused_steps"] = n // block - cfg["num_inference_steps"] = n // block + 1 # H3 counts steps as sigma grid points + cfg["num_inference_steps"] = n // block cfg["video_shift"], cfg["audio_shift"] = args.video_shift, args.audio_shift (d / "pdd_config.json").write_text(json.dumps(cfg, indent=2) + "\n") diff --git a/scripts/ci/utils/diffusion/publish_diffusion_gt.py b/scripts/ci/utils/diffusion/publish_diffusion_gt.py index f270460c8f..643c37fcf4 100644 --- a/scripts/ci/utils/diffusion/publish_diffusion_gt.py +++ b/scripts/ci/utils/diffusion/publish_diffusion_gt.py @@ -1,5 +1,5 @@ """ -Publish diffusion CI ground-truth images to sgl-project/ci-data-diffusion +Publish diffusion CI ground-truth images and audio to sgl-project/ci-data-diffusion via the GitHub API (same pattern as publish_traces.py). """ @@ -10,6 +10,7 @@ import io import json import os import sys +import wave from dataclasses import dataclass from pathlib import Path from urllib.error import HTTPError @@ -53,6 +54,7 @@ BRANCH = "main" DEFAULT_TARGET_DIR = "diffusion-ci/consistency_gt/sglang_generated" IMAGE_EXTENSIONS = {".png", ".jpg", ".jpeg", ".webp"} +GT_EXTENSIONS = IMAGE_EXTENSIONS | {".wav"} QUALITY_MAX_SIDE = 256 LOW_DETAIL_STD_THRESHOLD = 0.075 LOW_DETAIL_ENTROPY_THRESHOLD = 0.55 @@ -81,12 +83,12 @@ class OldNewMetrics: mean_abs_diff: float -def collect_images(source_dir, target_dir): - """Collect image files from source_dir and return list of (repo_path, content) tuples.""" +def collect_gt_files(source_dir, target_dir): + """Collect image and audio GT files from source_dir and return list of (repo_path, content) tuples.""" files = [] for entry in sorted(os.listdir(source_dir)): ext = os.path.splitext(entry)[1].lower() - if ext not in IMAGE_EXTENSIONS: + if ext not in GT_EXTENSIONS: continue full_path = os.path.join(source_dir, entry) if not os.path.isfile(full_path): @@ -106,13 +108,13 @@ def git_blob_sha(content): def get_remote_blob_shas(repo_owner, repo_name, target_dir, token): return { path: item["sha"] - for path, item in get_remote_image_entries( + for path, item in get_remote_gt_entries( repo_owner, repo_name, target_dir, token ).items() } -def get_remote_image_entries(repo_owner, repo_name, target_dir, token): +def get_remote_gt_entries(repo_owner, repo_name, target_dir, token): url = ( f"https://api.github.com/repos/{repo_owner}/{repo_name}/contents/" f"{target_dir}?ref={BRANCH}" @@ -129,7 +131,7 @@ def get_remote_image_entries(repo_owner, repo_name, target_dir, token): for item in entries if item.get("type") == "file" and "sha" in item - and os.path.splitext(item["path"])[1].lower() in IMAGE_EXTENSIONS + and os.path.splitext(item["path"])[1].lower() in GT_EXTENSIONS } @@ -301,6 +303,22 @@ def _format_old_new_metrics(metrics): def validate_gt_files(files_to_upload, changed_files, remote_image_entries, token): failures = [] for path, content in files_to_upload: + if Path(path).suffix.lower() == ".wav": + try: + with wave.open(io.BytesIO(content), "rb") as wav: + if ( + wav.getnchannels() != 1 + or wav.getsampwidth() != 2 + or wav.getframerate() != 16000 + or wav.getnframes() == 0 + or len(wav.readframes(wav.getnframes())) != wav.getnframes() * 2 + ): + failures.append( + f"{path}: expected nonempty mono 16 kHz PCM16 audio" + ) + except (wave.Error, EOFError) as exc: + failures.append(f"{path}: invalid WAV ({exc})") + continue quality_metrics = compute_image_quality_metrics(content) quality_reasons = get_quality_failure_reasons(quality_metrics) if quality_reasons: @@ -310,6 +328,8 @@ def validate_gt_files(files_to_upload, changed_files, remote_image_entries, toke ) for path, content in changed_files: + if Path(path).suffix.lower() == ".wav": + continue remote_entry = remote_image_entries.get(path) if not remote_entry: continue @@ -339,8 +359,8 @@ def validate_gt_files(files_to_upload, changed_files, remote_image_entries, toke if not failures: print( - f"GT quality gate passed for {len(files_to_upload)} generated image(s) " - f"and {len(changed_files)} changed image(s)." + f"GT quality gate passed for {len(files_to_upload)} generated GT file(s) " + f"and {len(changed_files)} changed GT file(s)." ) return @@ -356,12 +376,12 @@ def check_quality(source_dir, target_dir): print("Error: GITHUB_TOKEN environment variable not set") sys.exit(1) - files_to_upload = collect_images(source_dir, target_dir) + files_to_upload = collect_gt_files(source_dir, target_dir) if not files_to_upload: - print(f"No image files found in {source_dir}") + print(f"No GT files found in {source_dir}") return - remote_image_entries = get_remote_image_entries( + remote_image_entries = get_remote_gt_entries( REPO_OWNER, REPO_NAME, target_dir, token ) remote_blob_shas = { @@ -377,13 +397,13 @@ def publish(source_dir, target_dir): print("Error: GITHUB_TOKEN environment variable not set") sys.exit(1) - files_to_upload = collect_images(source_dir, target_dir) + files_to_upload = collect_gt_files(source_dir, target_dir) if not files_to_upload: - print(f"No image files found in {source_dir}") + print(f"No GT files found in {source_dir}") return print( - f"Found {len(files_to_upload)} image(s) to upload to {REPO_OWNER}/{REPO_NAME}/{target_dir}" + f"Found {len(files_to_upload)} GT file(s) to upload to {REPO_OWNER}/{REPO_NAME}/{target_dir}" ) # Verify token @@ -401,7 +421,7 @@ def publish(source_dir, target_dir): try: branch_sha = get_branch_sha(REPO_OWNER, REPO_NAME, BRANCH, token) tree_sha = get_tree_sha(REPO_OWNER, REPO_NAME, branch_sha, token) - remote_image_entries = get_remote_image_entries( + remote_image_entries = get_remote_gt_entries( REPO_OWNER, REPO_NAME, target_dir, token ) remote_blob_shas = { @@ -412,7 +432,7 @@ def publish(source_dir, target_dir): files_to_upload, changed_files, remote_image_entries, token ) if not changed_files: - print("No image changes to publish.") + print("No GT changes to publish.") return try: @@ -436,13 +456,13 @@ def publish(source_dir, target_dir): print("No tree changes to publish.") return - commit_msg = f"diffusion-ci: update images in {target_dir} ({len(changed_files)} files) [automated]" + commit_msg = f"diffusion-ci: update GT in {target_dir} ({len(changed_files)} files) [automated]" commit_sha = create_commit( REPO_OWNER, REPO_NAME, new_tree_sha, branch_sha, commit_msg, token ) update_branch_ref(REPO_OWNER, REPO_NAME, BRANCH, commit_sha, token) print( - f"Successfully pushed {len(changed_files)} changed images (commit {commit_sha[:10]})" + f"Successfully pushed {len(changed_files)} changed GT files (commit {commit_sha[:10]})" ) return except Exception as e: diff --git a/test/registered/unit/multimodal_gen/test_minimax_h3_step_contract.py b/test/registered/unit/multimodal_gen/test_minimax_h3_step_contract.py new file mode 100644 index 0000000000..1b15b99d15 --- /dev/null +++ b/test/registered/unit/multimodal_gen/test_minimax_h3_step_contract.py @@ -0,0 +1,280 @@ +# SPDX-License-Identifier: Apache-2.0 + +import json +import os +import sys +from types import SimpleNamespace +from unittest.mock import patch + +import pytest +import torch +from safetensors.torch import save_file + +from sglang.multimodal_gen.configs.sample.minimax_h3 import ( + FastH3SamplingParams, + MiniMaxH3SamplingParams, +) +from sglang.multimodal_gen.configs.sample.minimax_h3_vdn import VDNH3SamplingParams +from sglang.multimodal_gen.runtime.pipelines_core.schedule_batch import Req +from sglang.multimodal_gen.runtime.pipelines_core.stages.model_specific_stages.minimax_h3.constants import ( + MINIMAX_H3_SIGMAS_EXTRA_KEY, +) +from sglang.multimodal_gen.runtime.pipelines_core.stages.model_specific_stages.minimax_h3.denoise_loop import ( + MiniMaxH3DenoiseBranch, + minimax_h3_denoise_loop, +) +from sglang.multimodal_gen.runtime.pipelines_core.stages.model_specific_stages.minimax_h3.packed_sequence import ( + minimax_h3_packed_sequence, +) +from sglang.multimodal_gen.runtime.pipelines_core.stages.model_specific_stages.minimax_h3.release_metadata import ( + MiniMaxH3PartitionAdmissionStage, + MiniMaxH3ReleaseMetadata, +) +from sglang.multimodal_gen.runtime.pipelines_core.stages.model_specific_stages.minimax_h3.resolved_plan import ( + MINIMAX_H3_CANONICAL_REQUEST_EXTRA_KEY, +) +from sglang.multimodal_gen.runtime.pipelines_core.stages.model_specific_stages.minimax_h3.stages.denoising import ( + MiniMaxH3DenoisingStage, +) +from sglang.multimodal_gen.runtime.pipelines_core.stages.model_specific_stages.minimax_h3.stages.timestep_preparation import ( + MiniMaxH3TimestepPreparationStage, +) +from sglang.multimodal_gen.runtime.server_args import server_args as server_args_module +from sglang.multimodal_gen.tools import ( + build_minimax_h3_adaln_cache, + fuse_minimax_h3_pdd_heads, +) +from sglang.test.ci.ci_register import register_cuda_ci + +register_cuda_ci(est_time=15, stage="base-b-kernel-unit", runner_config="1-gpu-large") + + +@pytest.fixture(autouse=True) +def global_server_args(monkeypatch): + monkeypatch.setattr( + server_args_module, + "_global_server_args", + SimpleNamespace(comfyui_mode=False), + ) + + +@pytest.fixture +def admission(): + metadata = MiniMaxH3ReleaseMetadata.from_model_index( + { + "_minimax_h3": { + "schema_version": 1, + "partition": "fl2va", + "tasks": ["t2va", "fl2va"], + "task_aliases": {}, + "sigma_shift_scales": {"video": 12.0, "audio": 3.0}, + } + } + ) + return MiniMaxH3PartitionAdmissionStage(metadata) + + +def _request(steps): + sampling = MiniMaxH3SamplingParams( + prompt="step contract", + task="t2va", + conditions=[], + target={"short_edge": 768, "aspect_ratio": "16:9", "duration_seconds": 5.0}, + num_inference_steps=steps, + ) + return Req(sampling_params=sampling, extra=sampling.build_request_extra()) + + +@pytest.mark.parametrize("steps", [1, 2, 10, 50]) +def test_admission_schedule_loop_and_workload_agree(admission, steps): + request = _request(steps) + server_args = SimpleNamespace(minimax_h3_adaln_online=False) + assert admission.forward(request, server_args) is request + assert MINIMAX_H3_CANONICAL_REQUEST_EXTRA_KEY in request.extra + preparation = MiniMaxH3TimestepPreparationStage() + preparation.forward(request, server_args) + sigmas = request.extra[MINIMAX_H3_SIGMAS_EXTRA_KEY] + assert len(request.timesteps) == steps + for schedule in sigmas.values(): + assert len(schedule) == steps + 1 + assert schedule[0] == 1.0 and schedule[-1] == 0.0 + stage = MiniMaxH3DenoisingStage.__new__(MiniMaxH3DenoisingStage) + assert stage.default_workload_iterations(request, steps) == steps + packed = minimax_h3_packed_sequence( + text_len=3, + latent_t=2, + latent_h=4, + latent_w=4, + audio_t=3, + include_keyframe_cond=False, + ) + branch = MiniMaxH3DenoiseBranch( + packed=packed, + text_embeddings=torch.zeros(3, 5120, device="cuda"), + token_tags=packed["token_tags"], + device=torch.device("cuda"), + ) + calls = [] + plans = [] + + def prepare_plans(timesteps): + plans.extend(timesteps) + return None + + def forward(_model, _kwargs, step): + calls.append(step) + return ( + torch.ones(int(branch.update_mask.sum()), 96, device="cuda"), + torch.ones(branch.audio_pos.numel(), 32, device="cuda"), + ) + + video, audio = minimax_h3_denoise_loop( + model=SimpleNamespace(prepare_adaln_plans=prepare_plans), + model_forward=forward, + positive=branch, + initial_video_rows=torch.zeros(branch.img_pos.numel(), 96, device="cuda"), + initial_audio_rows=torch.zeros(branch.audio_pos.numel(), 32, device="cuda"), + keyframe_cond_rows=None, + sigmas_video=sigmas["video"], + sigmas_audio=sigmas["audio"], + device=torch.device("cuda"), + ) + assert calls == list(range(steps)) and len(plans) == steps + # H3's velocity convention gives a +1 denoised target for a unit + # prediction from zero rows once the schedule reaches sigma=0. + torch.testing.assert_close( + video[branch.video_target_slice], + torch.ones_like(video[branch.video_target_slice]), + ) + torch.testing.assert_close( + audio[branch.audio_target_slice], + torch.ones_like(audio[branch.audio_target_slice]), + ) + + +def test_online_adaln_admission_capacity_boundary(admission): + server_args = SimpleNamespace(minimax_h3_adaln_online=True) + with patch.dict(os.environ, {"SGLANG_DIFFUSION_MINIMAX_H3_ADALN_GPU_PLANS": "8"}): + for steps in (1, 8): + request = _request(steps) + assert admission.forward(request, server_args) is request + with pytest.raises(ValueError, match="9 AdaLN plans.*slab holds 8"): + admission.forward(_request(9), server_args) + + +@pytest.mark.parametrize( + "params_class,steps", [(FastH3SamplingParams, 8), (VDNH3SamplingParams, 8)] +) +def test_distilled_models_keep_their_trained_grid(params_class, steps): + request = Req(sampling_params=params_class(prompt="step contract")) + assert request.num_inference_steps == steps + with pytest.raises(ValueError): + params_class(prompt="step contract", num_inference_steps=steps + 1) + + +def test_fasth3_trained_rungs_preserve_serving_and_warmup_counts(admission): + sampling = FastH3SamplingParams( + prompt="step contract", + task="t2va", + conditions=[], + target={"short_edge": 768, "aspect_ratio": "16:9", "duration_seconds": 5.0}, + flow_shift=10.0, + audio_flow_shift=3.0, + ) + request = Req(sampling_params=sampling, extra=sampling.build_request_extra()) + admission.forward(request, SimpleNamespace(minimax_h3_adaln_online=False)) + preparation = MiniMaxH3TimestepPreparationStage( + dmd_denoising_steps=(999, 874, 749, 624, 500, 375, 250, 125) + ) + warmup_requests = [request.copy_as_warmup(steps) for steps in (1, 2, 8)] + preparation.forward(request, SimpleNamespace()) + assert len(request.timesteps) == 8 + assert all( + len(schedule) == 9 and schedule[-1] == 0.0 + for schedule in request.extra[MINIMAX_H3_SIGMAS_EXTRA_KEY].values() + ) + for steps, warmup in zip((1, 2, 8), warmup_requests): + preparation.forward(warmup, SimpleNamespace()) + assert len(warmup.timesteps) == steps + for modality, schedule in warmup.extra[MINIMAX_H3_SIGMAS_EXTRA_KEY].items(): + assert ( + schedule + == request.extra[MINIMAX_H3_SIGMAS_EXTRA_KEY][modality][: steps + 1] + ) + + +@pytest.mark.parametrize("steps", [1, 2, 10, 50]) +@pytest.mark.parametrize( + "mode", ["t2va", "fl2va", "ref2va-image", "ref2va-audio", "ref2va-mixed"] +) +def test_offline_adaln_builder_prepares_one_plan_per_transition(steps, mode): + plans = build_minimax_h3_adaln_cache._cache_timestep_plans( + SimpleNamespace( + timesteps=None, + num_inference_steps=steps, + flow_shift=12.0, + audio_flow_shift=3.0, + imgvid_cond_noise_aug=0.999, + audio_cond_noise_aug=1.0, + mode=mode, + ) + ) + assert len(plans) == steps + assert all(plan.numel() > 0 for plan in plans) + + +def test_fused_pdd_config_and_warmup_match_head_count(tmp_path, admission): + heads = { + name: torch.ones(32, 2, 3) if name.endswith("weight") else torch.ones(32, 2) + for name in ( + "proj_out.weight", + "proj_out.bias", + "audio_proj_out.weight", + "audio_proj_out.bias", + ) + } + save_file(heads, tmp_path / "pdd_heads.safetensors") + config_path = tmp_path / "pdd_config.json" + config_path.write_text(json.dumps({"num_steps": 32, "block_size": 4})) + with patch.object(sys, "argv", ["fuse_minimax_h3_pdd_heads", str(tmp_path)]): + assert fuse_minimax_h3_pdd_heads.main() == 0 + config = json.loads(config_path.read_text()) + assert config["fused_steps"] == config["num_inference_steps"] == 8 + with patch.dict( + os.environ, + { + "SGLANG_DIFFUSION_MINIMAX_H3_PDD_HEADS": str( + tmp_path / "pdd_fused_heads.safetensors" + ) + }, + ): + preparation = MiniMaxH3TimestepPreparationStage() + serving = _request(8) + admission.forward(serving, SimpleNamespace(minimax_h3_adaln_online=False)) + preparation.forward(serving, SimpleNamespace()) + for steps in (1, 2, 8): + warmup = serving.copy_as_warmup(steps) + preparation.forward(warmup, SimpleNamespace()) + assert len(warmup.timesteps) == steps + for modality, schedule in warmup.extra[MINIMAX_H3_SIGMAS_EXTRA_KEY].items(): + assert ( + schedule + == serving.extra[MINIMAX_H3_SIGMAS_EXTRA_KEY][modality][: steps + 1] + ) + + config["num_inference_steps"] = 9 + config_path.write_text(json.dumps(config)) + with patch.dict( + os.environ, + { + "SGLANG_DIFFUSION_MINIMAX_H3_PDD_HEADS": str( + tmp_path / "pdd_fused_heads.safetensors" + ) + }, + ): + with pytest.raises(ValueError, match="config does not match the fused heads"): + MiniMaxH3TimestepPreparationStage() + + +if __name__ == "__main__": + sys.exit(pytest.main([__file__, "-v"]))