✨ [llm][npu][quant] Add W4A8 MXFP quantization support for Qwen3 Dense on Ascend NPU (#23650)

Co-authored-by: ronnie_zheng <zl19940307@163.com>
This commit is contained in:
Junlin Wu
2026-07-06 19:23:26 +03:00
committed by GitHub
co-authored by ronnie_zheng
parent 1b481deade
commit 3abdbab9bb
10 changed files with 613 additions and 5 deletions
@@ -56,6 +56,13 @@ The following table summarizes quantization method support across NVIDIA and AMD
<td>Yes (A5 for Diffusion and LLM Dense Linear)</td>
<td>Ascend NPU only; online MXFP8 quantization for Diffusion models (e.g., Wan2.2) and LLM Dense Linear on A5 series; uses CANN <code>npu_dynamic_mx_quant</code> / <code>npu_quant_matmul</code> kernels</td>
</tr>
<tr>
<td><code>mxfp_w4a8</code></td>
<td>No</td>
<td>No</td>
<td>Yes (A5)</td>
<td>Ascend NPU only; online W4A8 for Qwen3 dense LLM (MXFP4 weights + MXFP8 activations) on A5 series; offline <code>W4A8_MXFP</code> checkpoints are auto-detected via <code>modelslim</code></td>
</tr>
<tr>
<td><code>blockwise_int8</code></td>
<td>Yes</td>