[Qwen3.5][MTP] Preserve online NVFP4 draft quantization for mixed checkpoints (#35545)
This commit is contained in:
@@ -1516,14 +1516,14 @@ class ModelConfig:
|
||||
"quant_method", "" if not self.quantization else self.quantization
|
||||
).lower()
|
||||
|
||||
# ModelOpt FP4 checkpoints quantize only the target model; an
|
||||
# embedded MTP draft may stay unquantized, so an explicit
|
||||
# ModelOpt FP4 and mixed checkpoints can quantize only the target
|
||||
# model; an embedded MTP draft may stay unquantized, so an explicit
|
||||
# nvfp4_online opt-in for the draft wins over checkpoint detection.
|
||||
# The online loader rejects already-packed weights at load time.
|
||||
preserve_online_draft_quantization = (
|
||||
self.is_draft_model
|
||||
and self.quantization == "nvfp4_online"
|
||||
and quant_method == "modelopt_fp4"
|
||||
and quant_method in ("modelopt_fp4", "modelopt_mixed")
|
||||
)
|
||||
# An explicit online-requantization request (e.g. quark_mxfp4 on top
|
||||
# of an NVFP4/mixed checkpoint) must not be overridden back to the
|
||||
|
||||
Reference in New Issue
Block a user