minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

[diffusion] feat: support loading serialized convrot w4a4 checkpoints (#36039)

Browse Source
This commit is contained in:
Mick
2026-08-24 19:13:35 +08:00
committed by GitHub
parent 716a6bf10c
commit bfeae4e79a
9 changed files with 407 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/docs/sglang-diffusion/quantization.mdx
+8
View File
@@ -200,6 +200,14 @@ backend.
<td><code>comfy-kitchen&gt;=0.2.27</code></td>
<td>Auto-detected; omit <code>--quantization</code>. Requires SM80+ and validates packed weights, group/channel scales, and optional codebooks before model construction. Mixed encoder files may keep their embedding tensorwise INT8. TP must preserve ConvRot group boundaries; offload is supported and FSDP is not.</td>
</tr>
<tr>
<td><code>comfy-w4a4-convrot</code></td>
<td>Safetensors with serialized <code>convrot_w4a4</code> layer metadata</td>
<td><code>--transformer-weights-path</code> or <code>--component-paths.text_encoder</code></td>
<td>Native DiTs and encoders with matching parameter mappings; MiniMax-H3 FL2VA / Ref2VA DiTs and Qwen3-VL encoder layouts are recognized</td>
<td><code>comfy-kitchen</code></td>
<td>Auto-detected; omit <code>--quantization</code>. Packed INT4 weights use the checkpoint's W4A4 ConvRot kernel and <code>linear_dtype</code>. CUDA requires SM75+; TP must preserve the 64-element quantization and ConvRot group boundaries. Offload is supported and FSDP is not.</td>
</tr>
<tr>
<td><code>quanto-int8</code></td>
<td>One native encoder safetensors file with an embedded Quanto quantization map</td>