minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

docs: update checkpoint to Qwen3.5 NVFP4 V2 for InfX (#32945)

Browse Source
This commit is contained in:
Faradawn Yang
2026-08-07 15:26:07 -07:00
committed by GitHub
parent b2f9603f93
commit 115cd7bde1
2 changed files with 4 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/cookbook/autoregressive/Qwen/Qwen3.5.mdx
+2 -2
View File
@@ -42,7 +42,7 @@ Qwen3.5 features a Gated Delta Networks combined with sparse Mixture-of-Experts
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Qwen3.5-397B-A17B</td> <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Qwen3.5-397B-A17B</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>[Qwen/Qwen3.5-397B-A17B](https://huggingface.co/Qwen/Qwen3.5-397B-A17B)</td> <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>[Qwen/Qwen3.5-397B-A17B](https://huggingface.co/Qwen/Qwen3.5-397B-A17B)</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>[Qwen/Qwen3.5-397B-A17B-FP8](https://huggingface.co/Qwen/Qwen3.5-397B-A17B-FP8)</td> <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>[Qwen/Qwen3.5-397B-A17B-FP8](https://huggingface.co/Qwen/Qwen3.5-397B-A17B-FP8)</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>NVIDIA NVFP4: [nvidia/Qwen3.5-397B-A17B-NVFP4](https://huggingface.co/nvidia/Qwen3.5-397B-A17B-NVFP4)<br/>AMD MXFP4: [amd/Qwen3.5-397B-A17B-MXFP4](https://huggingface.co/amd/Qwen3.5-397B-A17B-MXFP4)</td> <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>NVIDIA NVFP4: [nvidia/Qwen3.5-397B-A17B-NVFP4-V2](https://huggingface.co/nvidia/Qwen3.5-397B-A17B-NVFP4-V2)<br/>AMD MXFP4: [amd/Qwen3.5-397B-A17B-MXFP4](https://huggingface.co/amd/Qwen3.5-397B-A17B-MXFP4)</td>
</tr> </tr>
<tr> <tr>
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Qwen3.5-122B-A10B</td> <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Qwen3.5-122B-A10B</td>
@@ -156,7 +156,7 @@ This section provides deployment configurations optimized for different hardware
- **MI300X (192GB)** runs with tp=4. - **MI300X (192GB)** runs with tp=4.
- **MI325X (256GB)** runs with tp=2. - **MI325X (256GB)** runs with tp=2.
- **MI355X (288GB)** runs with tp=2. - **MI355X (288GB)** runs with tp=2.
- **FP4**: The FP4 quantized model requires ~250GB for weights, cutting memory by almost 4x. NVFP4 ([nvidia/Qwen3.5-397B-A17B-NVFP4](https://huggingface.co/nvidia/Qwen3.5-397B-A17B-NVFP4)) requires B200/B300 (Blackwell architecture); AMD provides an MXFP4 checkpoint ([amd/Qwen3.5-397B-A17B-MXFP4](https://huggingface.co/amd/Qwen3.5-397B-A17B-MXFP4)) for MI355X. - **FP4**: The FP4 quantized model requires ~250GB for weights, cutting memory by almost 4x. NVFP4 ([nvidia/Qwen3.5-397B-A17B-NVFP4-V2](https://huggingface.co/nvidia/Qwen3.5-397B-A17B-NVFP4-V2)) requires B200/B300 (Blackwell architecture); AMD provides an MXFP4 checkpoint ([amd/Qwen3.5-397B-A17B-MXFP4](https://huggingface.co/amd/Qwen3.5-397B-A17B-MXFP4)) for MI355X.
- **B200 (183GB)** runs with tp=4. (NVFP4) - **B200 (183GB)** runs with tp=4. (NVFP4)
- **B300 (275GB)** runs with tp=2. (NVFP4) - **B300 (275GB)** runs with tp=2. (NVFP4)
- **MI355X (288GB)** runs with tp=2 (use tp=4 for low concurrency). (MXFP4) - **MI355X (288GB)** runs with tp=2 (use tp=4 for low concurrency). (MXFP4)
docs/src/snippets/autoregressive/qwen35-deployment.jsx
+2 -2
View File
@@ -318,10 +318,10 @@ export const Qwen35Deployment = () => {
let modelName; let modelName;
if (quantization === 'fp4') { if (quantization === 'fp4') {
// AMD MI355X uses the MXFP4 checkpoint; Blackwell uses NVFP4. // AMD MI355X uses the MXFP4 checkpoint; Blackwell uses NVFP4-V2.
modelName = hardware === 'mi355x' modelName = hardware === 'mi355x'
? 'amd/Qwen3.5-397B-A17B-MXFP4' ? 'amd/Qwen3.5-397B-A17B-MXFP4'
: 'nvidia/Qwen3.5-397B-A17B-NVFP4'; : 'nvidia/Qwen3.5-397B-A17B-NVFP4-V2';
} else { } else {
const suffix = MODEL_SUFFIX[model]; const suffix = MODEL_SUFFIX[model];
const quantSuffix = quantization === 'fp8' ? '-FP8' : ''; const quantSuffix = quantization === 'fp8' ? '-FP8' : '';