[diffusion] feat: support serialized comfy convrot int8 dits (#35994)

2026-08-23 09:29:59 +08:00
parent 7f30d66045
commit a36c0746b9
11 changed files with 253 additions and 71 deletions
@@ -281,11 +281,13 @@ contract and low resident weight memory, but that part is slower than a fully
 quantized FP8 GEMM. TP, Ulysses/Ring sequence parallelism, and
 component/layerwise offload are supported; FSDP inference is rejected.

-The `pruned_int8_convrot` files are detected but remain unsupported. They
-require online regular-Hadamard ConvRot, dynamic INT8 activation quantization,
-and a matching W8A8 GEMM. SGLang fails before loading them instead of silently
-treating their stored INT8 values as ordinary weights. A native ConvRot kernel
-path should be added and benchmarked separately before these files are accepted.
+The `pruned_int8_convrot` files use the same override path and are detected
+automatically. Install `comfy-kitchen`, replace the FP8 filename above with
+`minimax_h3_fl2va_pruned_int8_convrot.safetensors`, and still omit
+`--quantization`. SGLang loads their serialized INT8 weights and row scales into
+the fused ConvRot kernel without requantizing them. TP1/2/4 and sequence or
+layerwise offload are supported; TP8 violates the checkpoint's 256-element
+ConvRot group boundary, and FSDP is rejected.

 ### Advanced: precomputed AdaLN cache

@@ -907,7 +909,7 @@ sglang generate \
 `--attention-backend sol_attn` with
 `--attention-backend-config dense_backend=sage_attn,dense_steps=10` and
 `--component-attention-backends text_encoder=torch_sdpa,transformer=sol_attn`.
-See [Quantization](/docs/sglang-diffusion/quantization#kitchen-int8-online-quantization)
+See [Quantization](/docs/sglang-diffusion/quantization#kitchen-int8)
 and [Attention Backends](/docs/sglang-diffusion/attention_backends#sage-then-sol-hybrid).

 <Warning>