diff --git a/docs_new/docs/advanced_features/quantization.mdx b/docs_new/docs/advanced_features/quantization.mdx
index e13ca084e..bd59a0497 100644
--- a/docs_new/docs/advanced_features/quantization.mdx
+++ b/docs_new/docs/advanced_features/quantization.mdx
@@ -130,8 +130,8 @@ The following table summarizes quantization method support across NVIDIA and AMD
gguf |
Yes |
No |
- WIP |
- CUDA-only kernels in sgl-kernel |
+ Yes |
+ CUDA kernels in sgl-kernel; Ascend uses CPU pre-dequantization at load time |
modelopt / modelopt_fp8 |
diff --git a/docs_new/docs/hardware-platforms/ascend-npus/ascend_npu_quantization.mdx b/docs_new/docs/hardware-platforms/ascend-npus/ascend_npu_quantization.mdx
index 5d3c0b77b..c952a3606 100644
--- a/docs_new/docs/hardware-platforms/ascend-npus/ascend_npu_quantization.mdx
+++ b/docs_new/docs/hardware-platforms/ascend-npus/ascend_npu_quantization.mdx
@@ -23,66 +23,66 @@ SGLang support **mix-bits** quantization (independently defines and loads each l
| W4A4 dynamic |
Linear |
- <span style="color: green;">√</span> |
- <span style="color: green;">√</span> |
- <span style="color: yellow;">TBD</span> |
- <span style="color: green;">√</span> |
+ √ |
+ √ |
+ TBD |
+ √ |
| W8A8 static |
Linear |
- <span style="color: green;">√</span> |
- <span style="color: green;">√</span> |
- <span style="color: yellow;">TBD</span> |
- <span style="color: green;">√</span> |
+ √ |
+ √ |
+ TBD |
+ √ |
| W8A8 dynamic |
Linear |
- <span style="color: green;">√</span> |
- <span style="color: green;">√</span> |
- <span style="color: yellow;">TBD</span> |
- <span style="color: green;">√</span> |
+ √ |
+ √ |
+ TBD |
+ √ |
| MXFP8 |
Linear |
- <span style="color: red;">x</span> |
- <span style="color: red;">x</span> |
- <span style="color: blue;">WIP</span> |
- <span style="color: blue;">WIP</span> |
+ x |
+ x |
+ WIP |
+ WIP |
| W4A4 dynamic |
MoE |
- <span style="color: green;">√</span> |
- <span style="color: green;">√</span> |
- <span style="color: yellow;">TBD</span> |
- <span style="color: red;">x</span> |
+ √ |
+ √ |
+ TBD |
+ x |
| W4A8 dynamic |
MoE |
- <span style="color: green;">√</span> |
- <span style="color: green;">√</span> |
- <span style="color: yellow;">TBD</span> |
- <span style="color: red;">x</span> |
+ √ |
+ √ |
+ TBD |
+ x |
| W8A8 dynamic |
MoE |
- <span style="color: green;">√</span> |
- <span style="color: green;">√</span> |
- <span style="color: yellow;">TBD</span> |
- <span style="color: red;">x</span> |
+ √ |
+ √ |
+ TBD |
+ x |
| MXFP8 |
MoE |
- <span style="color: red;">x</span> |
- <span style="color: red;">x</span> |
- <span style="color: blue;">WIP</span> |
- <span style="color: red;">x</span> |
+ x |
+ x |
+ WIP |
+ x |
@@ -102,23 +102,23 @@ SGLang support **mix-bits** quantization (independently defines and loads each l
| W4A16 |
Linear |
- <span style="color: green;">√</span> |
- <span style="color: green;">√</span> |
- <span style="color: yellow;">TBD</span> |
+ √ |
+ √ |
+ TBD |
| W8A16 |
Linear |
- <span style="color: green;">√</span> |
- <span style="color: green;">√</span> |
- <span style="color: yellow;">TBD</span> |
+ √ |
+ √ |
+ TBD |
| W4A16 |
MoE |
- <span style="color: green;">√</span> |
- <span style="color: green;">√</span> |
- <span style="color: yellow;">TBD</span> |
+ √ |
+ √ |
+ TBD |
@@ -138,30 +138,30 @@ GPTQ on Ascend support
| W4A16 |
Linear |
- <span style="color: green;">√</span> |
- <span style="color: green;">√</span> |
- <span style="color: yellow;">TBD</span> |
+ √ |
+ √ |
+ TBD |
| W8A16 |
Linear |
- <span style="color: green;">√</span> |
- <span style="color: green;">√</span> |
- <span style="color: yellow;">TBD</span> |
+ √ |
+ √ |
+ TBD |
| W4A16 MOE |
MoE |
- <span style="color: green;">√</span> |
- <span style="color: green;">√</span> |
- <span style="color: yellow;">TBD</span> |
+ √ |
+ √ |
+ TBD |
| W8A16 MOE |
MoE |
- <span style="color: green;">√</span> |
- <span style="color: green;">√</span> |
- <span style="color: yellow;">TBD</span> |
+ √ |
+ √ |
+ TBD |
@@ -181,30 +181,30 @@ GPTQ on Ascend support
| W4A16 |
Linear |
- <span style="color: green;">√</span> |
- <span style="color: green;">√</span> |
- <span style="color: yellow;">TBD</span> |
+ √ |
+ √ |
+ TBD |
| W8A16 |
Linear |
- <span style="color: green;">√</span> |
- <span style="color: green;">√</span> |
- <span style="color: yellow;">TBD</span> |
+ √ |
+ √ |
+ TBD |
| W4A16 |
MoE |
- <span style="color: green;">√</span> |
- <span style="color: green;">√</span> |
- <span style="color: yellow;">TBD</span> |
+ √ |
+ √ |
+ TBD |
| W8A16 |
MoE |
- <span style="color: green;">√</span> |
- <span style="color: green;">√</span> |
- <span style="color: yellow;">TBD</span> |
+ √ |
+ √ |
+ TBD |
@@ -224,34 +224,86 @@ Compressed-tensors (LLM Compressor) on Ascend support:
| W8A8 dynamic |
Linear |
- <span style="color: green;">√</span> |
- <span style="color: green;">√</span> |
- <span style="color: yellow;">TBD</span> |
+ √ |
+ √ |
+ TBD |
| W4A8 dynamic with/without activation clip |
MoE |
- <span style="color: green;">√</span> |
- <span style="color: green;">√</span> |
- <span style="color: yellow;">TBD</span> |
+ √ |
+ √ |
+ TBD |
| W4A16 MOE |
MoE |
- <span style="color: green;">√</span> |
- <span style="color: green;">√</span> |
- <span style="color: yellow;">TBD</span> |
+ √ |
+ √ |
+ TBD |
| W8A8 dynamic |
MoE |
- <span style="color: green;">√</span> |
- <span style="color: green;">√</span> |
- <span style="color: yellow;">TBD</span> |
+ √ |
+ √ |
+ TBD |
[GGUF on Ascend support](https://github.com/sgl-project/sglang/pull/17883)
+
+
+
+ | Quantization type |
+ Layer type |
+ A2 Supported |
+ A3 Supported |
+ A5 Supported |
+
+
+
+
+ | All GGUF types (standard, K-quant) |
+ Linear |
+ √ |
+ √ |
+ TBD |
+
+
+ | All GGUF types (standard, K-quant) |
+ MoE |
+ √ |
+ √ |
+ TBD |
+
+
+
-in progress
+**Usage Examples:**
+
+- Dense model (e.g. Qwen3-14B-Q4_K_M.gguf):
+
+```bash Command
+python3 -m sglang.launch_server \
+ --model-path Qwen3-14B-Q4_K_M.gguf \
+ --device npu --attention-backend ascend \
+ --host 0.0.0.0 --port 30000 \
+ --mem-fraction-static 0.7 --tp-size 2
+```
+
+- MoE model (e.g. Qwen3-30B-A3B-Q4_K_M.gguf):
+
+```bash Command
+python3 -m sglang.launch_server \
+ --model-path Qwen3-30B-A3B-Q4_K_M.gguf \
+ --device npu --attention-backend ascend \
+ --host 0.0.0.0 --port 30000 \
+ --mem-fraction-static 0.8 --tp-size 2
+```
+
+> **Implementation Notes:**
+> - GGUF weights are pre-dequantized to FP16/BF16 during model loading on CPU, then transferred to NPU for inference. This trades higher memory usage for faster runtime performance (no per-forward-pass dequantization overhead).
+> - MoE layers use `npu_grouped_matmul` and `npu_moe_init_routing` / `npu_moe_finalize_routing` for high-performance expert computation.
+> - TP (tensor parallelism) sharding is supported for both dense and MoE GGUF models.
diff --git a/docs_new/docs/hardware-platforms/ascend-npus/ascend_npu_support_features.mdx b/docs_new/docs/hardware-platforms/ascend-npus/ascend_npu_support_features.mdx
index dfcbc451b..884d8f1dc 100644
--- a/docs_new/docs/hardware-platforms/ascend-npus/ascend_npu_support_features.mdx
+++ b/docs_new/docs/hardware-platforms/ascend-npus/ascend_npu_support_features.mdx
@@ -61,7 +61,7 @@ click [Server Arguments](../../advanced_features/server_arguments).
| `--load-format` |
`auto` |
- auto, safetensors |
+ auto, safetensors, gguf |
A2, A3 |