minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

Refactor FP4 quantization and remove deprecated JIT kernels (#30448)

Browse Source 
Co-authored-by: root <root@sgl-b300-inference.datacrunch.io>
This commit is contained in:
Brayden Zhong
2026-07-14 09:22:07 +08:00
committed by GitHub
co-authored by root
parent 0c01971eeb
commit 9756f768a6
38 changed files with 120 additions and 5705 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs_new/docs/advanced_features/quantization.mdx
+1 -6
View File
@@ -284,11 +284,6 @@ Backend selection applies to **blockwise FP8**, **MXFP8** (dense linear), and **
<td>SM80+</td>
<td>Auto-selects: <code>flashinfer_cutedsl</code> on SM100; <code>marlin</code> on SM80-SM90; <code>flashinfer_cutlass</code> otherwise (including SM120)</td>
</tr>
<tr>
<td><code>cutlass</code></td>
<td>SM100/120</td>
<td>SGLang CUTLASS kernel</td>
</tr>
<tr>
<td><code>flashinfer_cutlass</code></td>
<td>SM100/120</td>
@@ -317,7 +312,7 @@ Backend selection applies to **blockwise FP8**, **MXFP8** (dense linear), and **
</tbody>
</table>
On Blackwell, when FlashInfer is unavailable for NVFP4, the SGLang CUTLASS kernel is used as an automatic fallback. On SM80-SM90, `auto` selects Marlin for NVFP4.
On SM80-SM90, `auto` selects Marlin for NVFP4. NVFP4 GEMM requires FlashInfer to be installed.
## Offline Quantization