[diffusion] feat: automatically infer comfy fp8 activation scaling (#36060)
This commit is contained in:
@@ -273,8 +273,10 @@ sglang serve \
|
||||
--port 30010
|
||||
```
|
||||
|
||||
SGLang uses its native static-activation FP8 linear path for attention and
|
||||
`fc1`. The checkpoint marks `fc2` for full-precision matrix multiplication, so
|
||||
SGLang uses its native static-activation FP8 linear path when the checkpoint
|
||||
stores input scales, and automatically uses dynamic activation scaling for
|
||||
Comfy FP8 exports that omit them. The checkpoint above marks `fc2` for
|
||||
full-precision matrix multiplication, so
|
||||
SGLang retains its FP8 storage but materializes and scales one compute-dtype
|
||||
`fc2` matrix for each call. This preserves the checkpoint's mixed execution
|
||||
contract and low resident weight memory, but that part is slower than a fully
|
||||
|
||||
@@ -51,7 +51,9 @@ repo contains multiple candidate checkpoints, pass
|
||||
`--transformer-weights-path` explicitly.
|
||||
|
||||
MiniMax-H3 is a verified example for Comfy safetensors with per-layer metadata,
|
||||
including `pruned_fp8_scaled` and serialized ConvRot INT8. Pass one selected
|
||||
including `pruned_fp8_scaled` and serialized ConvRot INT8. Other Comfy FP8
|
||||
exports are also auto-detected: the presence of an input scale selects static
|
||||
activation scaling, while its absence selects dynamic scaling. Pass one selected
|
||||
FL2VA or Ref2VA DiT file by local path, `owner/repo/path/file.safetensors`, or
|
||||
direct Hugging Face file URL; do not combine it with `--quantization`. Its GGUF
|
||||
usage is documented in
|
||||
|
||||
Reference in New Issue
Block a user