[Qwen3.5][MTP] Preserve online NVFP4 draft quantization for mixed checkpoints (#35545)
This commit is contained in:
@@ -1516,14 +1516,14 @@ class ModelConfig:
|
|||||||
"quant_method", "" if not self.quantization else self.quantization
|
"quant_method", "" if not self.quantization else self.quantization
|
||||||
).lower()
|
).lower()
|
||||||
|
|
||||||
# ModelOpt FP4 checkpoints quantize only the target model; an
|
# ModelOpt FP4 and mixed checkpoints can quantize only the target
|
||||||
# embedded MTP draft may stay unquantized, so an explicit
|
# model; an embedded MTP draft may stay unquantized, so an explicit
|
||||||
# nvfp4_online opt-in for the draft wins over checkpoint detection.
|
# nvfp4_online opt-in for the draft wins over checkpoint detection.
|
||||||
# The online loader rejects already-packed weights at load time.
|
# The online loader rejects already-packed weights at load time.
|
||||||
preserve_online_draft_quantization = (
|
preserve_online_draft_quantization = (
|
||||||
self.is_draft_model
|
self.is_draft_model
|
||||||
and self.quantization == "nvfp4_online"
|
and self.quantization == "nvfp4_online"
|
||||||
and quant_method == "modelopt_fp4"
|
and quant_method in ("modelopt_fp4", "modelopt_mixed")
|
||||||
)
|
)
|
||||||
# An explicit online-requantization request (e.g. quark_mxfp4 on top
|
# An explicit online-requantization request (e.g. quark_mxfp4 on top
|
||||||
# of an NVFP4/mixed checkpoint) must not be overridden back to the
|
# of an NVFP4/mixed checkpoint) must not be overridden back to the
|
||||||
|
|||||||
Reference in New Issue
Block a user