diff --git a/docs_new/cookbook/autoregressive/MiniMax/MiniMax-M3.mdx b/docs_new/cookbook/autoregressive/MiniMax/MiniMax-M3.mdx index 19723cad6..bc081ee38 100644 --- a/docs_new/cookbook/autoregressive/MiniMax/MiniMax-M3.mdx +++ b/docs_new/cookbook/autoregressive/MiniMax/MiniMax-M3.mdx @@ -35,10 +35,11 @@ Then run the **Python** output of the command panel below in that environment. T ```bash Command -docker pull lmsysorg/sglang:latest +# Pull the M3 image the command panel selects for your platform, e.g.: +docker pull lmsysorg/sglang:dev-cu13-minimax-m3 ``` -On NVIDIA the command panel below uses `lmsysorg/sglang:latest` (CUDA 13, multi-arch — H200 + all Blackwell). On AMD Instinct it uses the matching ROCm image (MI300X/MI325X → `aigmkt/minimax-m3-sglang-rocm700-mi30x`, MI350X/MI355X → `aigmkt/minimax-m3-sglang-rocm720-mi35x`). For how to launch the image, see [Install → Method 3: Using Docker](../../../docs/get-started/install#method-3-using-docker), substituting the inner `sglang serve ...` with what the command generator produces. +The command panel below fills in the right tag per platform: `dev-cu13-minimax-m3` (CUDA 13 — B300, GB200, GB300), `dev-cu12-minimax-m3` (CUDA 12 — Hopper H200), or `dev-minimax-m3` (default). On AMD Instinct it uses the matching ROCm image (MI300X/MI325X → `aigmkt/minimax-m3-sglang-rocm700-mi30x`, MI350X/MI355X → `aigmkt/minimax-m3-sglang-rocm720-mi35x`). For how to launch the image, see [Install → Method 3: Using Docker](../../../docs/get-started/install#method-3-using-docker), substituting the inner `sglang serve ...` with what the command generator produces. These M3 dev images now **bundle MiniMax's MSA sparse-attention kernel** (`fmha_sm100`), so Blackwell users get the recommended fast path automatically — no manual install needed (see **§2.1**). On a custom image without it, the same recipe still serves on the built-in Triton sparse path. @@ -86,7 +87,7 @@ Key characteristics as served by SGLang: ### 2.1 MSA sparse-attention fast path (recommended for Blackwell users) -[MiniMax MSA](https://github.com/MiniMax-AI/MSA) (`fmha_sm100`, MIT-licensed) is the recommended Blackwell kernel for M3's main sparse-attention step — faster and more memory-efficient than the built-in Triton fallback. **It ships pre-installed in `lmsysorg/sglang:latest`**, so the Blackwell recipe above engages it automatically with no extra setup — `import fmha_sm100` works out of the box and the kernels JIT-compile on first use. It is otherwise purely additive: on a custom image, install it (below) and the recipe engages it automatically; without it the same recipe still serves on the built-in Triton path. The swap is numerically equivalent (cosine ≥ 0.99999 vs Triton), decode stays CUDA-graph-capturable, prefill TTFT drops ~9–12% at 8K–64K context, and the MSA path survives memory configurations where the Triton path OOMs. +[MiniMax MSA](https://github.com/MiniMax-AI/MSA) (`fmha_sm100`, MIT-licensed) is the recommended Blackwell kernel for M3's main sparse-attention step — faster and more memory-efficient than the built-in Triton fallback. **It ships pre-installed in the M3 dev image** (`lmsysorg/sglang:dev-minimax-m3`, also published under the `dev-cu13-minimax-m3` tag), so the Blackwell recipe above engages it automatically with no extra setup — `import fmha_sm100` works out of the box and the kernels JIT-compile on first use. It is otherwise purely additive: on a custom image, install it (below) and the recipe engages it automatically; without it the same recipe still serves on the built-in Triton path. The swap is numerically equivalent (cosine ≥ 0.99999 vs Triton), decode stays CUDA-graph-capturable, prefill TTFT drops ~9–12% at 8K–64K context, and the MSA path survives memory configurations where the Triton path OOMs. **Requirements** (from the [MSA README](https://github.com/MiniMax-AI/MSA#requirements)): diff --git a/docs_new/src/snippets/configs/MiniMaxAI/minimax-m3.jsx b/docs_new/src/snippets/configs/MiniMaxAI/minimax-m3.jsx index 2a0c47b62..397c18199 100644 --- a/docs_new/src/snippets/configs/MiniMaxAI/minimax-m3.jsx +++ b/docs_new/src/snippets/configs/MiniMaxAI/minimax-m3.jsx @@ -92,13 +92,15 @@ sgl-eval run mmmu_pro \\ ], dockerImages: { - // lmsysorg/sglang:latest (cu13, multi-arch amd64+arm64) covers H200 + all - // Blackwell (incl. sm_103 B300/GB300 and Grace arm64). - b200: "lmsysorg/sglang:latest", - b300: "lmsysorg/sglang:latest", - gb200: "lmsysorg/sglang:latest", - gb300: "lmsysorg/sglang:latest", - h200: "lmsysorg/sglang:latest", + // M3-specific dev images (multi-arch amd64+arm64). cu13 carries the sm_103 + // (B300/GB300) + Grace arm64 builds; cu12 is the Hopper/CUDA-12 build; + // dev-minimax-m3 is the rolling default. M3 model support is not yet in a + // tagged release, so :latest cannot serve it. + b200: "lmsysorg/sglang:dev-minimax-m3", + b300: "lmsysorg/sglang:dev-cu13-minimax-m3", + gb200: "lmsysorg/sglang:dev-cu13-minimax-m3", + gb300: "lmsysorg/sglang:dev-cu13-minimax-m3", + h200: "lmsysorg/sglang:dev-cu12-minimax-m3", // AMD ROCm images — published M3 builds, by arch (gfx942 -> mi30x, gfx950 -> mi35x). mi300x: "aigmkt/minimax-m3-sglang-rocm700-mi30x", mi325x: "aigmkt/minimax-m3-sglang-rocm700-mi30x", @@ -198,7 +200,7 @@ sgl-eval run mmmu_pro \\ // B300 / GB200 / GB300, tp8 on B200. fa4 + page 128 + deep_gemm are the M3 // SM100 auto-defaults on current main, so this is also the bare-launch // behavior; they engage MiniMax's MSA sparse-attention kernel (fmha_sm100, - // pre-installed in lmsysorg/sglang:latest; see Configuration Tips), Triton + // pre-installed in the dev-minimax-m3 images; see Configuration Tips), Triton // fallback otherwise. // AMD: tp8. MI350X/MI355X (gfx950) serve MXFP8 natively (backends auto). MI300X/ // MI325X (gfx942) need --attention-backend aiter + --moe-runner-backend triton,