✨ [llm][npu][quant] Add W8A8 MXFP8 quantization for Qwen3 MoE on Ascend NPU (#30768)

Co-authored-by: Артем Савкин <58187114+OrangeRedeng@users.noreply.github.com>
Co-authored-by: ronnie_zheng <zl19940307@163.com>
This commit is contained in:
Junlin Wu
2026-07-29 10:39:36 +03:00
committed by GitHub
co-authored by Артем Савкин ronnie_zheng
parent da5528db30
commit f05c92fb6d
18 changed files with 626 additions and 41 deletions
@@ -53,8 +53,8 @@ The following table summarizes quantization method support across NVIDIA and AMD
<td><code>mxfp8</code></td>
<td>No</td>
<td>No</td>
<td>Yes (A5 for Diffusion and LLM Dense Linear)</td>
<td>Ascend NPU only; online MXFP8 quantization for Diffusion models (e.g., Wan2.2) and LLM Dense Linear on A5 series; uses CANN <code>npu_dynamic_mx_quant</code> / <code>npu_quant_matmul</code> kernels</td>
<td>Yes (A5 for Diffusion, LLM Dense Linear and LLM MoE)</td>
<td>Ascend NPU only; online + offline MXFP8 for Diffusion models (e.g., Wan2.2), LLM Dense Linear, and LLM MoE (FusedMoE, e.g. Qwen3-30B-A3B) on A5 series; uses CANN <code>npu_dynamic_mx_quant</code> / <code>npu_quant_matmul</code> (dense) and <code>npu_grouped_matmul_swiglu_quant_v2</code> / <code>npu_grouped_matmul</code> (MoE) kernels</td>
</tr>
<tr>
<td><code>mxfp_w4a8</code></td>