minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

[diffusion] MiniMax-H3: tiered AdaLN plan cache (pinned-host tier + per-plan LRU) (#37266)

Browse Source 
Co-authored-by: mickqian <mickqian@users.noreply.github.com>
This commit is contained in:
triple-mu
2026-09-03 22:09:21 +08:00
committed by GitHub
co-authored by mickqian
parent 4e37882a93
commit bf71035d39
15 changed files with 1621 additions and 335 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/cookbook/diffusion/MiniMax/MiniMax-H3.mdx
+28
View File
@@ -297,6 +297,34 @@ values. SGLang rejects a request outside that coverage instead of silently
changing conditioning. Cache mode supports the matching unquantized checkpoint
only.
### Advanced: online AdaLN rebuild with a host cache
`--minimax-h3-adaln-online true` needs no prebuilt artifact: the server drops
the 24.2 GiB of `adaln_proj` weights from the GPU and computes each request's
AdaLN outputs from the checkpoint on demand, bit-exact with the resident-weight
path. Because a rebuild pass streams the whole 24.2 GiB, the first request of
every new `(task shape, num_inference_steps, flow_shift, audio_flow_shift)`
combination pays several seconds; after that the plans are served from a
64-slot GPU slab (LRU per plan) backed by a pinned host cache
(`--minimax-h3-adaln-host-cache-gb`, LRU per schedule, default 8 GB per
rank), so mixed-schedule serving does not re-read the checkpoint. Plan sets
that exceed the host budget are simply recomputed on their next occurrence.
Expert escape hatches live in environment variables:
`SGLANG_DIFFUSION_MINIMAX_H3_ADALN_GPU_PLANS` resizes the GPU slab (needed
only beyond 65 inference steps) and `SGLANG_DIFFUSION_MINIMAX_H3_ADALN_FP32`
computes the one-time projections in fp32 (experimental; not bit-comparable
to resident weights, validate end-to-end before production). LoRA adapters
that modify `adaln_proj` are rejected in both cache modes rather than
silently ignored.
Both cache modes hold values derived from `adaln_proj`, so a runtime weight
update is only accepted when the cache can follow it: online mode takes a disk
update whose target directory carries native `adaln_proj` safetensors, and
rejects everything else (tensor updates, and directories without those
tensors) before a single weight is written. A sidecar is built offline and
cannot be regenerated in the server, so weight updates are rejected outright;
rebuild the sidecar against the new weights and restart.
### Serve MiniMax-H3 on Ascend NPUs
For Ascend NPU, follow the