[AMD] Drop the redundant scale zero-fill before AITER per-tensor FP8 quant (#40557)
Co-authored-by: siliangchen-amd <{"message":"Although you appear to have the correct authorization credentials, the `amd-eng-emu` enterprise has an IP allow list enabled, and your IP address is not permitted to access this resource.","documentation_url":"https://docs.github.com/rest/users/users#get-the-authenticated-user","status":"403"}siliangchen-amd@users.noreply.github.com>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
This commit is contained in:
co-authored by
siliangchen-amd
Xiaoyu Zhang
parent
3f00fb7e2e
commit
debbb5cde9
@@ -2071,12 +2071,17 @@ if _is_hip:
|
||||
else:
|
||||
_native_dynamic_per_token_quant_fp8(output, input, scale)
|
||||
else:
|
||||
scale = torch.zeros(1, device=input.device, dtype=torch.float32)
|
||||
# Only vLLM needs `scale` pre-zeroed: segmented_max_reduction
|
||||
# accumulates into it with atomicMax. AITER zeroes it itself
|
||||
# and the native path overwrites it.
|
||||
if _use_aiter:
|
||||
scale = torch.empty(1, device=input.device, dtype=torch.float32)
|
||||
dynamic_per_tensor_quant(output, input, scale)
|
||||
elif _has_vllm:
|
||||
scale = torch.zeros(1, device=input.device, dtype=torch.float32)
|
||||
torch.ops._C.dynamic_scaled_fp8_quant(output, input, scale)
|
||||
else:
|
||||
scale = torch.empty(1, device=input.device, dtype=torch.float32)
|
||||
_native_dynamic_per_tensor_quant_fp8(output, input, scale)
|
||||
else:
|
||||
# Static scaling
|
||||
|
||||
Reference in New Issue
Block a user