[XPU] Support GPT-OSS MXFP4 checkpoints on Intel XPU (#35751)

Co-authored-by: Meng, Hengyu <hengyu.meng@intel.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Juan Muneton
2026-09-04 14:20:28 +08:00
committed by GitHub
co-authored by Meng, Hengyu Claude Opus 5
parent d122ca99b2
commit 72078cd7f5
5 changed files with 115 additions and 2 deletions
+1 -1
View File
@@ -47,7 +47,7 @@ The following table summarizes quantization method support across NVIDIA and AMD
<td>Yes</td>
<td>Yes</td>
<td>Yes (A5)</td>
<td>On GPU: requires CDNA3/CDNA4 with MXFP support (uses Aiter). On Ascend NPU (A5): W4A4 MXFP4 for Qwen3 dense and MoE LLMs (MXFP4 weights + activations) — dense models support online dual-level MXFP4; offline <code>W4A4_MXFP4</code> dense and MoE checkpoints (single-level) are auto-detected via <code>modelslim</code></td>
<td>On GPU: requires CDNA3/CDNA4 with MXFP support (uses Aiter). On Ascend NPU (A5): W4A4 MXFP4 for Qwen3 dense and MoE LLMs (MXFP4 weights + activations) — dense models support online dual-level MXFP4; offline <code>W4A4_MXFP4</code> dense and MoE checkpoints (single-level) are auto-detected via <code>modelslim</code>. On Intel GPUs (XPU): W4A16 MoE experts on Xe2/BMG via <code>sgl-kernel-xpu</code>, enabled automatically with <code>--device xpu</code> (see <a href="../hardware-platforms/xpu">Intel GPUs (XPU)</a>)</td>
</tr>
<tr>
<td><code>mxfp8</code></td>
+37
View File
@@ -41,6 +41,9 @@ A list of LLMs have been optimized on Intel GPU, and more are on the way:
**Note:** The model identifiers listed in the table above
have been verified on [Intel® Arc™ B580 Graphics](https://www.intel.com/content/www/us/en/products/sku/241598/intel-arc-b580-graphics/specifications.html).
Quantized MoE models are covered separately in
[MXFP4 MoE Quantization](#mxfp4-moe-quantization) below.
## Installation
### Install From Source
@@ -116,6 +119,40 @@ sglang serve \
--page-size \ # intel_xpu attention backend supports [32, 64, 128]
```
## MXFP4 MoE Quantization
Native MXFP4 MoE checkpoints (OCP microscaling FP4: packed `e2m1` weights plus
per-32-element `ue8m0` block scales) run on Intel GPUs through the
`sgl-kernel-xpu` W4A16 grouped GEMM. The expert weights stay in the checkpoint's
packed layout end to end — there is no dequantization to BF16 — so GPT-OSS-20B
loads in roughly 13 GB rather than the ~42 GB a BF16 upcast would need, which is
what lets it fit on a single 24 GB card.
The `mxfp4` method is registered automatically on `--device xpu`, which already
requires `sgl-kernel-xpu` — no extra flag is needed:
```bash
sglang serve \
--model-path openai/gpt-oss-20b \
--device xpu \
--attention-backend intel_xpu
```
**Tested models:**
| Model | Notes |
|:---|:---|
| [openai/gpt-oss-20b](https://huggingface.co/openai/gpt-oss-20b) | 32 experts, top-k 4, hidden 2880, intermediate 2880; clamped-swiglu activation (`gemm1_alpha` 1.702, limit 7.0) with expert biases |
**Requirements and limitations:**
| Item | Status |
|---|---|
| GPU architecture | Xe2 / BMG only (Intel® Arc™ B-Series and Arc™ Pro B-Series) |
| Quantized layers | Fused MoE experts only; attention, router, embeddings and LM head stay BF16 (per the checkpoint's `modules_to_not_convert`) |
| Activations | BF16 (W4A16). Clamped swiglu (GPT-OSS) and plain SiLU are supported |
| Expert parallelism | Not yet validated on this path |
## Benchmarking with Requests
You can benchmark the performance via the `bench_serving` script.