✨ [llm][npu][quant] Add W8A8 MXFP8 quantization support for Qwen3 Dense on Ascend NPU (#22352)

Co-authored-by: ronnie_zheng <zl19940307@163.com>
This commit is contained in:
Junlin Wu
2026-06-16 09:45:18 +03:00
committed by GitHub
co-authored by ronnie_zheng
parent 72d962be88
commit 2a8ea70059
8 changed files with 376 additions and 11 deletions
@@ -30,7 +30,7 @@ The following table summarizes quantization method support across NVIDIA and AMD
<th>Method</th>
<th>NVIDIA GPUs</th>
<th>AMD GPUs (MI300X/MI325X/MI350X)</th>
<th>Ascend NPUs (A2/A3)</th>
<th>Ascend NPUs (A2/A3/A5)</th>
<th>Notes</th>
</tr>
</thead>
@@ -49,6 +49,13 @@ The following table summarizes quantization method support across NVIDIA and AMD
<td>WIP</td>
<td>Requires CDNA3/CDNA4 with MXFP support; uses Aiter</td>
</tr>
<tr>
<td><code>mxfp8</code></td>
<td>No</td>
<td>No</td>
<td>Yes (A5 for Diffusion and LLM Dense Linear)</td>
<td>Ascend NPU only; online MXFP8 quantization for Diffusion models (e.g., Wan2.2) and LLM Dense Linear on A5 series; uses CANN <code>npu_dynamic_mx_quant</code> / <code>npu_quant_matmul</code> kernels</td>
</tr>
<tr>
<td><code>blockwise_int8</code></td>
<td>Yes</td>
@@ -182,13 +189,7 @@ The following table summarizes quantization method support across NVIDIA and AMD
<td>Yes</td>
<td>Ascend quantization; Uses CANN kernels</td>
</tr>
<tr>
<td><code>mxfp8</code> (diffusion)</td>
<td>No</td>
<td>No</td>
<td>Yes (A2/A3)</td>
<td>Ascend NPU only; online MXFP8 quantization for diffusion models (e.g., Wan2.2); requires CANN ≥ 8.0.RC3</td>
</tr>
</tbody>
</table>