minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

[diffusion] feat: support loading serialized comfy w4a8 checkpoints (#36036)

Browse Source
This commit is contained in:
Mick
2026-08-24 10:32:21 +08:00
committed by GitHub
parent fd73d4b019
commit 2d84de5e69
8 changed files with 481 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/docs/sglang-diffusion/quantization.mdx
+8
View File
@@ -184,6 +184,14 @@ backend.
<td>SRT's platform MXFP8 backend</td>
<td>Serialized metadata is auto-detected. NVIDIA and ROCm reuse SRT's dense MXFP8 kernels; Ascend keeps its native online path. Mixed per-layer checkpoints do not support FSDP.</td>
</tr>
<tr>
<td><code>comfy-w4a8-convrot</code></td>
<td>Safetensors with serialized <code>asym_w4a8_int8</code> layer metadata and packed weights</td>
<td><code>--transformer-weights-path</code></td>
<td>MiniMax-H3 FL2VA / Ref2VA DiTs</td>
<td><code>comfy-kitchen&gt;=0.2.27</code></td>
<td>Auto-detected; omit <code>--quantization</code>. Requires SM80+ and validates packed weights, group/channel scales, and optional codebooks before model construction. TP must preserve ConvRot group boundaries; offload is supported and FSDP is not.</td>
</tr>
<tr>
<td><code>qvg-kv</code></td>
<td>Unquantized model with runtime causal KV-cache compression</td>