minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

✨ [llm][npu][quant] Add W4A4 MXFP4 quantization support for Qwen3 Dense on Ascend NPU (#23795)

Browse Source
This commit is contained in:
Junlin Wu
2026-07-17 09:06:30 +03:00
committed by GitHub
parent 1ac1ffea0c
commit bbd2a3fe4a
8 changed files with 553 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs_new/docs/advanced_features/quantization.mdx
+2 -2
View File
@@ -46,8 +46,8 @@ The following table summarizes quantization method support across NVIDIA and AMD
<td><code>mxfp4</code></td>
<td>Yes</td>
<td>Yes</td>
<td>WIP</td>
<td>Requires CDNA3/CDNA4 with MXFP support; uses Aiter</td>
<td>Yes (A5)</td>
<td>On GPU: requires CDNA3/CDNA4 with MXFP support (uses Aiter). On Ascend NPU (A5): W4A4 MXFP4 for Qwen3 dense LLM (MXFP4 weights + activations) — online uses dual-level MXFP4, offline <code>W4A4_MXFP4</code> checkpoints (single-level) are auto-detected via <code>modelslim</code></td>
</tr>
<tr>
<td><code>mxfp8</code></td>