minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

docs(cookbook): revert MiniMax-M3 to dev image (model not yet in a release) (#31819)

Browse Source 
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Douglas Yang
2026-07-20 12:16:47 -07:00
committed by GitHub
co-authored by Claude Opus 4.8
parent 54aaedd76d
commit e149cdb337
2 changed files with 14 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs_new/cookbook/autoregressive/MiniMax/MiniMax-M3.mdx
+4 -3
View File
@@ -35,10 +35,11 @@ Then run the **Python** output of the command panel below in that environment. T
<Tab title="Docker">
```bash Command
docker pull lmsysorg/sglang:latest
# Pull the M3 image the command panel selects for your platform, e.g.:
docker pull lmsysorg/sglang:dev-cu13-minimax-m3
```
On NVIDIA the command panel below uses `lmsysorg/sglang:latest` (CUDA 13, multi-arch — H200 + all Blackwell). On AMD Instinct it uses the matching ROCm image (MI300X/MI325X → `aigmkt/minimax-m3-sglang-rocm700-mi30x`, MI350X/MI355X → `aigmkt/minimax-m3-sglang-rocm720-mi35x`). For how to launch the image, see [Install → Method 3: Using Docker](../../../docs/get-started/install#method-3-using-docker), substituting the inner `sglang serve ...` with what the command generator produces.
The command panel below fills in the right tag per platform: `dev-cu13-minimax-m3` (CUDA 13 — B300, GB200, GB300), `dev-cu12-minimax-m3` (CUDA 12 — Hopper H200), or `dev-minimax-m3` (default). On AMD Instinct it uses the matching ROCm image (MI300X/MI325X → `aigmkt/minimax-m3-sglang-rocm700-mi30x`, MI350X/MI355X → `aigmkt/minimax-m3-sglang-rocm720-mi35x`). For how to launch the image, see [Install → Method 3: Using Docker](../../../docs/get-started/install#method-3-using-docker), substituting the inner `sglang serve ...` with what the command generator produces.
<Note>
These M3 dev images now **bundle MiniMax's MSA sparse-attention kernel** (`fmha_sm100`), so Blackwell users get the recommended fast path automatically — no manual install needed (see **§2.1**). On a custom image without it, the same recipe still serves on the built-in Triton sparse path.
@@ -86,7 +87,7 @@ Key characteristics as served by SGLang:
### 2.1 MSA sparse-attention fast path (recommended for Blackwell users)
[MiniMax MSA](https://github.com/MiniMax-AI/MSA) (`fmha_sm100`, MIT-licensed) is the recommended Blackwell kernel for M3's main sparse-attention step — faster and more memory-efficient than the built-in Triton fallback. **It ships pre-installed in `lmsysorg/sglang:latest`**, so the Blackwell recipe above engages it automatically with no extra setup — `import fmha_sm100` works out of the box and the kernels JIT-compile on first use. It is otherwise purely additive: on a custom image, install it (below) and the recipe engages it automatically; without it the same recipe still serves on the built-in Triton path. The swap is numerically equivalent (cosine ≥ 0.99999 vs Triton), decode stays CUDA-graph-capturable, prefill TTFT drops ~9–12% at 8K–64K context, and the MSA path survives memory configurations where the Triton path OOMs.
[MiniMax MSA](https://github.com/MiniMax-AI/MSA) (`fmha_sm100`, MIT-licensed) is the recommended Blackwell kernel for M3's main sparse-attention step — faster and more memory-efficient than the built-in Triton fallback. **It ships pre-installed in the M3 dev image** (`lmsysorg/sglang:dev-minimax-m3`, also published under the `dev-cu13-minimax-m3` tag), so the Blackwell recipe above engages it automatically with no extra setup — `import fmha_sm100` works out of the box and the kernels JIT-compile on first use. It is otherwise purely additive: on a custom image, install it (below) and the recipe engages it automatically; without it the same recipe still serves on the built-in Triton path. The swap is numerically equivalent (cosine ≥ 0.99999 vs Triton), decode stays CUDA-graph-capturable, prefill TTFT drops ~9–12% at 8K–64K context, and the MSA path survives memory configurations where the Triton path OOMs.
**Requirements** (from the [MSA README](https://github.com/MiniMax-AI/MSA#requirements)):
docs_new/src/snippets/configs/MiniMaxAI/minimax-m3.jsx
+10 -8
View File
@@ -92,13 +92,15 @@ sgl-eval run mmmu_pro \\
],
dockerImages: {
// lmsysorg/sglang:latest (cu13, multi-arch amd64+arm64) covers H200 + all
// Blackwell (incl. sm_103 B300/GB300 and Grace arm64).
b200: "lmsysorg/sglang:latest",
b300: "lmsysorg/sglang:latest",
gb200: "lmsysorg/sglang:latest",
gb300: "lmsysorg/sglang:latest",
h200: "lmsysorg/sglang:latest",
// M3-specific dev images (multi-arch amd64+arm64). cu13 carries the sm_103
// (B300/GB300) + Grace arm64 builds; cu12 is the Hopper/CUDA-12 build;
// dev-minimax-m3 is the rolling default. M3 model support is not yet in a
// tagged release, so :latest cannot serve it.
b200: "lmsysorg/sglang:dev-minimax-m3",
b300: "lmsysorg/sglang:dev-cu13-minimax-m3",
gb200: "lmsysorg/sglang:dev-cu13-minimax-m3",
gb300: "lmsysorg/sglang:dev-cu13-minimax-m3",
h200: "lmsysorg/sglang:dev-cu12-minimax-m3",
// AMD ROCm images — published M3 builds, by arch (gfx942 -> mi30x, gfx950 -> mi35x).
mi300x: "aigmkt/minimax-m3-sglang-rocm700-mi30x",
mi325x: "aigmkt/minimax-m3-sglang-rocm700-mi30x",
@@ -198,7 +200,7 @@ sgl-eval run mmmu_pro \\
// B300 / GB200 / GB300, tp8 on B200. fa4 + page 128 + deep_gemm are the M3
// SM100 auto-defaults on current main, so this is also the bare-launch
// behavior; they engage MiniMax's MSA sparse-attention kernel (fmha_sm100,
// pre-installed in lmsysorg/sglang:latest; see Configuration Tips), Triton
// pre-installed in the dev-minimax-m3 images; see Configuration Tips), Triton
// fallback otherwise.
// AMD: tp8. MI350X/MI355X (gfx950) serve MXFP8 natively (backends auto). MI300X/
// MI325X (gfx942) need --attention-backend aiter + --moe-runner-backend triton,