Revert "[AMD][Quantization] Online MXFP4 quantization 2/N - FP8 to MXFP4 requantization on AMD GPUs" (#28213)

This commit is contained in:
Lianmin Zheng
2026-06-14 13:34:06 -07:00
committed by GitHub
parent 3cb29f6747
commit f18d38d040
15 changed files with 185 additions and 1097 deletions
@@ -846,7 +846,7 @@ Other layers (e.g. projections in the attention layers) have their weights quant
SGLang running on AMD GPUs with hardware FP4 support (CDNA4 architecture, e.g. MI355x) supports the quantization method `--quantization quark_mxfp4`, that will quantize BF16 model weights to MXFP4 at load time, use dynamic MXFP4 quantization for activations and MXFP4 GEMMs instead of BF16 GEMMs.
Example (BF16 to MXFP4 requantization):
Example:
```bash
sglang serve --model-path Qwen/Qwen3-30B-A3B \
@@ -854,22 +854,6 @@ sglang serve --model-path Qwen/Qwen3-30B-A3B \
--quantization quark_mxfp4
```
The option `--quantization quark_mxfp4` also supports converting FP8 dense and MOE models to MXFP4, following this logic:
1. Load an FP8 weight tensor,
2. Dequantize it to BF16,
3. Requantize it to MXFP4
progressively during weight loading.
Example (FP8 to MXFP4 requantization):
```bash
sglang serve --model-path Qwen/Qwen3-30B-A3B-Instruct-2507-FP8 \
--tensor-parallel-size 1 \
--quantization quark_mxfp4
```
## Diffusion Model Quantization on Ascend NPU
SGLang-Diffusion supports MXFP8 quantization for diffusion models (such as Wan2.2) on Ascend A5 NPUs, in both online and offline (ModelSlim) modes. This is separate from the LLM serving path and uses the `sglang serve` / `sglang generate` CLI.