[diffusion] feat: automatically infer comfy fp8 activation scaling (#36060)

This commit is contained in:
Mick
2026-08-23 18:43:33 +08:00
committed by GitHub
parent 44db041700
commit dd15fb57b5
5 changed files with 53 additions and 20 deletions
@@ -273,8 +273,10 @@ sglang serve \
--port 30010
```
SGLang uses its native static-activation FP8 linear path for attention and
`fc1`. The checkpoint marks `fc2` for full-precision matrix multiplication, so
SGLang uses its native static-activation FP8 linear path when the checkpoint
stores input scales, and automatically uses dynamic activation scaling for
Comfy FP8 exports that omit them. The checkpoint above marks `fc2` for
full-precision matrix multiplication, so
SGLang retains its FP8 storage but materializes and scales one compute-dtype
`fc2` matrix for each call. This preserves the checkpoint's mixed execution
contract and low resident weight memory, but that part is slower than a fully
+3 -1
View File
@@ -51,7 +51,9 @@ repo contains multiple candidate checkpoints, pass
`--transformer-weights-path` explicitly.
MiniMax-H3 is a verified example for Comfy safetensors with per-layer metadata,
including `pruned_fp8_scaled` and serialized ConvRot INT8. Pass one selected
including `pruned_fp8_scaled` and serialized ConvRot INT8. Other Comfy FP8
exports are also auto-detected: the presence of an input scale selects static
activation scaling, while its absence selects dynamic scaling. Pass one selected
FL2VA or Ref2VA DiT file by local path, `owner/repo/path/file.safetensors`, or
direct Hugging Face file URL; do not combine it with `--quantization`. Its GGUF
usage is documented in