docs: sync legacy docs/-only updates into docs_new (Mintlify) (#27308)

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
zijiexia
2026-06-04 19:45:13 -07:00
committed by GitHub
co-authored by Claude Opus 4.8
parent 46c58b5c70
commit c6c1f1a29a
18 changed files with 650 additions and 29 deletions
@@ -175,6 +175,13 @@ The following table summarizes quantization method support across NVIDIA and AMD
<td>Yes</td>
<td>Ascend quantization; Uses CANN kernels</td>
</tr>
<tr>
<td><code>mxfp8</code> (diffusion)</td>
<td>No</td>
<td>No</td>
<td>Yes (A2/A3)</td>
<td>Ascend NPU only; online MXFP8 quantization for diffusion models (e.g., Wan2.2); requires CANN ≥ 8.0.RC3</td>
</tr>
</tbody>
</table>
@@ -816,6 +823,36 @@ sglang serve --model-path Qwen/Qwen3-30B-A3B \
--quantization quark_mxfp4
```
## Diffusion Model Quantization on Ascend NPU
SGLang-Diffusion supports MXFP8 quantization for diffusion models (such as Wan2.2) on Ascend A5 NPUs, in both online and offline (ModelSlim) modes. This is separate from the LLM serving path and uses the `sglang serve` / `sglang generate` CLI.
**Requirements:** Ascend A5, CANN ≥ 8.0.RC3
### Online MXFP8
Pass `--quantization mxfp8` to dynamically quantize FP16/BF16 transformer weights to MXFP8 at load time:
```bash
sglang serve \
--model-path Wan-AI/Wan2.2-T2V-A14B-Diffusers \
--quantization mxfp8 \
--num-gpus 4
```
### Offline MXFP8 (ModelSlim)
Pre-quantize with [msModelSlim](https://gitcode.com/Ascend/msmodelslim) and load the checkpoint directly — the quantization scheme is auto-detected from `quant_model_description.json`:
```bash
sglang generate \
--model-path /path/to/wan2_2_mxfp8_diffusers \
--prompt "a beautiful sunset" \
--save-output
```
For the full quantization + format conversion workflow and a complete list of supported schemes, see [Diffusion Quantization on Ascend NPU](../hardware-platforms/ascend-npus/ascend_npu_quantization#diffusion-model-quantization-on-ascend-npu) and [SGLang-Diffusion Quantization](../sglang-diffusion/quantization#modelslim).
## Reference
- [GPTQModel](https://github.com/ModelCloud/GPTQModel)