[diffusion] feat: support loading serialized comfy w4a8 checkpoints (#36036)

This commit is contained in:
Mick
2026-08-24 10:32:21 +08:00
committed by GitHub
parent fd73d4b019
commit 2d84de5e69
8 changed files with 481 additions and 1 deletions
@@ -298,6 +298,14 @@ layers in their original dtype, and reuses SRT's MXFP8 dense kernels. The
selected SRT backend must support MXFP8 on the target GPU; FSDP is rejected for
this mixed per-layer layout.
W4A8 ConvRot DiT files use the same flagless flow. With
`comfy-kitchen>=0.2.27`, pass a file such as
`starsfriday/MiniMax-H3-w4a8/minimax_h3_fl2va_pruned_w4a8_mixed.safetensors`
to `--transformer-weights-path`; SGLang reads the serialized per-layer metadata
and packed INT4 tensors automatically. Do not add `--quantization`. TP remains
subject to each row-parallel shard preserving the checkpoint's ConvRot group
boundary, and FSDP is rejected.
### Advanced: precomputed AdaLN cache
The [model card](https://huggingface.co/MiniMaxAI/MiniMax-H3) notes that about
@@ -184,6 +184,14 @@ backend.
<td>SRT's platform MXFP8 backend</td>
<td>Serialized metadata is auto-detected. NVIDIA and ROCm reuse SRT's dense MXFP8 kernels; Ascend keeps its native online path. Mixed per-layer checkpoints do not support FSDP.</td>
</tr>
<tr>
<td><code>comfy-w4a8-convrot</code></td>
<td>Safetensors with serialized <code>asym_w4a8_int8</code> layer metadata and packed weights</td>
<td><code>--transformer-weights-path</code></td>
<td>MiniMax-H3 FL2VA / Ref2VA DiTs</td>
<td><code>comfy-kitchen&gt;=0.2.27</code></td>
<td>Auto-detected; omit <code>--quantization</code>. Requires SM80+ and validates packed weights, group/channel scales, and optional codebooks before model construction. TP must preserve ConvRot group boundaries; offload is supported and FSDP is not.</td>
</tr>
<tr>
<td><code>qvg-kv</code></td>
<td>Unquantized model with runtime causal KV-cache compression</td>