✨ [llm][npu][quant] Add W8A8 MXFP8 quantization support for Qwen3 Dense on Ascend NPU (#22352)

Co-authored-by: ronnie_zheng <zl19940307@163.com>

docs_new/docs/advanced_features/quantization.mdx

@@ -30,7 +30,7 @@ The following table summarizes quantization method support across NVIDIA and AMD
       <th>Method</th>
       <th>NVIDIA GPUs</th>
       <th>AMD GPUs (MI300X/MI325X/MI350X)</th>
-      <th>Ascend NPUs (A2/A3)</th>
+      <th>Ascend NPUs (A2/A3/A5)</th>
       <th>Notes</th>
     </tr>
   </thead>
@@ -49,6 +49,13 @@ The following table summarizes quantization method support across NVIDIA and AMD
       <td>WIP</td>
       <td>Requires CDNA3/CDNA4 with MXFP support; uses Aiter</td>
     </tr>
+    <tr>
+      <td><code>mxfp8</code></td>
+      <td>No</td>
+      <td>No</td>
+      <td>Yes (A5 for Diffusion and LLM Dense Linear)</td>
+      <td>Ascend NPU only; online MXFP8 quantization for Diffusion models (e.g., Wan2.2) and LLM Dense Linear on A5 series; uses CANN <code>npu_dynamic_mx_quant</code> / <code>npu_quant_matmul</code> kernels</td>
+    </tr>
     <tr>
       <td><code>blockwise_int8</code></td>
       <td>Yes</td>
@@ -182,13 +189,7 @@ The following table summarizes quantization method support across NVIDIA and AMD
       <td>Yes</td>
       <td>Ascend quantization; Uses CANN kernels</td>
     </tr>
-    <tr>
-      <td><code>mxfp8</code> (diffusion)</td>
-      <td>No</td>
-      <td>No</td>
-      <td>Yes (A2/A3)</td>
-      <td>Ascend NPU only; online MXFP8 quantization for diffusion models (e.g., Wan2.2); requires CANN ≥ 8.0.RC3</td>
-    </tr>
+
   </tbody>
 </table>
 

docs_new/docs/hardware-platforms/ascend-npus/ascend_npu_quantization.mdx

@@ -45,7 +45,7 @@ SGLang support **mix-bits** quantization (independently defines and loads each l
       <td><strong style={{color: 'green'}}>√</strong></td>
     </tr>
     <tr>
-      <td><a href="https://github.com/sgl-project/sglang/pull/20922">MXFP8</a></td>
+      <td>MXFP8 (<a href="https://github.com/sgl-project/sglang/pull/20922">Diffusion</a>, <a href="https://github.com/sgl-project/sglang/pull/22352">LLM dense</a>)</td>
       <td>Linear</td>
       <td><strong style={{color: 'red'}}>x</strong></td>
       <td><strong style={{color: 'red'}}>x</strong></td>
@@ -316,6 +316,36 @@ python3 -m sglang.launch_server \
 > - MoE layers use `npu_grouped_matmul` and `npu_moe_init_routing` / `npu_moe_finalize_routing` for high-performance expert computation.
 > - TP (tensor parallelism) sharding is supported for both dense and MoE GGUF models.
 
+**MXFP8 for LLM dense models (e.g. Qwen3 / Qwen3.5):**
+
+LLM dense W8A8 MXFP8 Linear support on Ascend was added in [PR #22352](https://github.com/sgl-project/sglang/pull/22352). Requires Ascend A5 series or newer (`npu_dynamic_mx_quant` is not available on A2 / A3).
+
+- Online MXFP8 quantization (BF16/FP16 weights → MXFP8 at load time):
+
+```bash Command
+python3 -m sglang.launch_server \
+    --model-path Qwen/Qwen3-8B \
+    --quantization mxfp8 \
+    --device npu --attention-backend ascend \
+    --host 0.0.0.0 --port 30000 \
+    --mem-fraction-static 0.8 --tp-size 1
+```
+
+- Offline MXFP8 quantization (msmodelslim pre-quantized weights, `W8A8_MXFP8` scheme; no `--quantization` flag needed — auto-detected from `quant_model_description.json`):
+
+```bash Command
+python3 -m sglang.launch_server \
+    --model-path /path/to/Qwen3-8B-W8A8-MXFP8 \
+    --device npu --attention-backend ascend \
+    --host 0.0.0.0 --port 30000 \
+    --mem-fraction-static 0.8 --tp-size 1
+```
+
+> **Implementation Notes:**
+> - Online path: `Fp8Config.get_quant_method()` dispatches to `NPUMXFP8LinearMethod`. Weights are quantized once at load via `npu_dynamic_mx_quant(weight, dst_type=torch_npu.float8_e4m3fn)` and pre-transposed to `[in, out]`; activations are per-token quantized at inference and matmul runs via `npu_quant_matmul(..., group_sizes=[1, 1, 32])` (block_size = 32).
+> - Offline path: `ModelSlimMXFP8Scheme` loads `float8_e4m3fn` weights + `float8_e8m0fnu` block scales pre-exported by msmodelslim. Transpose is kept as a non-contiguous view (`.data` assignment) — calling `.contiguous()` would physically reorder the pre-quantized layout and break the block-scale mapping.
+> - MoE MXFP8 (FusedMoE/TP) for LLMs is tracked separately and not part of this PR.
+
 ## Diffusion Model Quantization on Ascend NPU
 
 SGLang-Diffusion supports MXFP8 online and offline quantization for diffusion models (such as Wan2.2) on Ascend NPUs. MXFP8 requires A5; the ModelSlim W8A8/W4A4 schemes work on A2/A3.