minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

docs(cookbook): replace pinned nightly/dev images with :latest (#31610)

Browse Source 
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Douglas Yang
2026-07-17 23:33:19 +00:00
committed by GitHub
co-authored by Claude Opus 4.8
parent c95026aed3
commit a01a8e1ed9
15 changed files with 47 additions and 71 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs_new/cookbook/autoregressive/Google/Gemma4.mdx
+3 -9
View File
@@ -77,21 +77,15 @@ pip install 'git+https://github.com/sgl-project/sglang.git#subdirectory=python'
pip install 'git+https://github.com/huggingface/transformers.git@1423d22f7a3b62e8c70ad67b58ec25cd9b675897'
```
### Docker (prebuilt dev image)
### Docker
Prebuilt development images bundle SGLang together with the matching transformers commit preinstalled, so no manual install is needed. All tags are multi-arch (`amd64` + `arm64`):
| Tag | CUDA | Hardware |
| --- | --- | --- |
| `lmsysorg/sglang:dev-gemma-4-12B` | 13.0 | Default — amd64 (H200 / B200) + arm64 (GB200 / GB300) |
| `lmsysorg/sglang:dev-cu13-gemma-4-12B` | 13.0 | Alias of the default tag |
| `lmsysorg/sglang:dev-cu12-gemma-4-12B` | 12.9 | CUDA 12.x hosts |
`lmsysorg/sglang:latest` (CUDA 13.0, multi-arch `amd64` + `arm64`) runs on both Hopper (H200) and Blackwell (B200 / GB200 / GB300):
```bash Command
docker run --gpus all --ipc=host --shm-size 32g \
-v ~/.cache/huggingface:/root/.cache/huggingface \
-p 30000:30000 \
lmsysorg/sglang:dev-gemma-4-12B \
lmsysorg/sglang:latest \
sglang serve --model-path google/gemma-4-12B-it \
--reasoning-parser gemma4 --tool-call-parser gemma4 \
--host 0.0.0.0 --port 30000
docs_new/cookbook/autoregressive/InclusionAI/Ling-2.5-1T.mdx
+3 -6
View File
@@ -24,14 +24,11 @@ metatags:
## 2. SGLang Installation
Ling-2.5-1T requires a specific SGLang Docker image:
Ling-2.5-1T runs on the standard SGLang Docker image:
```bash Command
# For H200/B200
docker pull lmsysorg/sglang:nightly-dev-20260213-a0ebaa64
# For GB200/GB300
docker pull lmsysorg/sglang:nightly-dev-cu13-20260213-a0ebaa64
# NVIDIA (H200 / B200 / GB200 / GB300)
docker pull lmsysorg/sglang:latest
```
For other installation methods, please refer to the [official SGLang installation guide](../../../docs/get-started/install).
docs_new/cookbook/autoregressive/InclusionAI/Ring-2.5-1T.mdx
+3 -6
View File
@@ -25,14 +25,11 @@ metatags:
## 2. SGLang Installation
Ring-2.5-1T requires a specific SGLang Docker image:
Ring-2.5-1T runs on the standard SGLang Docker image:
```bash Command
# For H200/B200
docker pull lmsysorg/sglang:nightly-dev-20260213-a0ebaa64
# For GB200/GB300
docker pull lmsysorg/sglang:nightly-dev-cu13-20260213-a0ebaa64
# NVIDIA (H200 / B200 / GB200 / GB300)
docker pull lmsysorg/sglang:latest
# For MI300X/325X
docker pull lmsysorg/sglang:v0.5.9-rocm700-mi30x
docs_new/cookbook/autoregressive/MiniMax/MiniMax-M3.mdx
+3 -4
View File
@@ -35,11 +35,10 @@ Then run the **Python** output of the command panel below in that environment. T
<Tab title="Docker">
```bash Command
# Pull the M3 image the command panel selects for your platform, e.g.:
docker pull lmsysorg/sglang:dev-cu13-minimax-m3
docker pull lmsysorg/sglang:latest
```
The command panel below fills in the right tag per platform: `dev-cu13-minimax-m3` (CUDA 13 — B300, GB200, GB300), `dev-cu12-minimax-m3` (CUDA 12 — Hopper H200), or `dev-minimax-m3` (default). On AMD Instinct it uses the matching ROCm image (MI300X/MI325X → `aigmkt/minimax-m3-sglang-rocm700-mi30x`, MI350X/MI355X → `aigmkt/minimax-m3-sglang-rocm720-mi35x`). For how to launch the image, see [Install → Method 3: Using Docker](../../../docs/get-started/install#method-3-using-docker), substituting the inner `sglang serve ...` with what the command generator produces.
On NVIDIA the command panel below uses `lmsysorg/sglang:latest` (CUDA 13, multi-arch — H200 + all Blackwell). On AMD Instinct it uses the matching ROCm image (MI300X/MI325X → `aigmkt/minimax-m3-sglang-rocm700-mi30x`, MI350X/MI355X → `aigmkt/minimax-m3-sglang-rocm720-mi35x`). For how to launch the image, see [Install → Method 3: Using Docker](../../../docs/get-started/install#method-3-using-docker), substituting the inner `sglang serve ...` with what the command generator produces.
<Note>
These M3 dev images now **bundle MiniMax's MSA sparse-attention kernel** (`fmha_sm100`), so Blackwell users get the recommended fast path automatically — no manual install needed (see **§2.1**). On a custom image without it, the same recipe still serves on the built-in Triton sparse path.
@@ -87,7 +86,7 @@ Key characteristics as served by SGLang:
### 2.1 MSA sparse-attention fast path (recommended for Blackwell users)
[MiniMax MSA](https://github.com/MiniMax-AI/MSA) (`fmha_sm100`, MIT-licensed) is the recommended Blackwell kernel for M3's main sparse-attention step — faster and more memory-efficient than the built-in Triton fallback. **It ships pre-installed in the M3 dev image** (`lmsysorg/sglang:dev-minimax-m3`, also published under the `dev-cu13-minimax-m3` tag), so the Blackwell recipe above engages it automatically with no extra setup — `import fmha_sm100` works out of the box and the kernels JIT-compile on first use. It is otherwise purely additive: on a custom image, install it (below) and the recipe engages it automatically; without it the same recipe still serves on the built-in Triton path. The swap is numerically equivalent (cosine ≥ 0.99999 vs Triton), decode stays CUDA-graph-capturable, prefill TTFT drops ~9–12% at 8K–64K context, and the MSA path survives memory configurations where the Triton path OOMs.
[MiniMax MSA](https://github.com/MiniMax-AI/MSA) (`fmha_sm100`, MIT-licensed) is the recommended Blackwell kernel for M3's main sparse-attention step — faster and more memory-efficient than the built-in Triton fallback. **It ships pre-installed in `lmsysorg/sglang:latest`**, so the Blackwell recipe above engages it automatically with no extra setup — `import fmha_sm100` works out of the box and the kernels JIT-compile on first use. It is otherwise purely additive: on a custom image, install it (below) and the recipe engages it automatically; without it the same recipe still serves on the built-in Triton path. The swap is numerically equivalent (cosine ≥ 0.99999 vs Triton), decode stays CUDA-graph-capturable, prefill TTFT drops ~9–12% at 8K–64K context, and the MSA path survives memory configurations where the Triton path OOMs.
**Requirements** (from the [MSA README](https://github.com/MiniMax-AI/MSA#requirements)):
docs_new/cookbook/autoregressive/NVIDIA/Nemotron3-Nano-Omni.mdx
+1 -1
View File
@@ -49,7 +49,7 @@ pip install sglang
uv pip install 'git+https://github.com/sgl-project/sglang.git#subdirectory=python'
# Or use Docker
docker pull lmsysorg/sglang:dev-cu13-nemotronh-nano-omni-reasoning-v3
docker pull lmsysorg/sglang:latest
```
For the full Docker setup and other installation methods, refer to the [official SGLang installation guide](../../../docs/get-started/install).
docs_new/cookbook/autoregressive/Poolside/Laguna-M.1.mdx
+3 -3
View File
@@ -27,15 +27,15 @@ cd sglang
uv pip install -e python
```
Then run the **Python** output of the command panel below in that environment. The **Docker** tab is simpler — its image (`dev-cu13-618-nightly`) bundles the CUDA-13 runtime and the M.1 code. Once M.1 support lands in a tagged release, `uv pip install sglang` will pull it directly.
Then run the **Python** output of the command panel below in that environment. The **Docker** tab is simpler — `lmsysorg/sglang:latest` bundles the CUDA-13 runtime and the M.1 code.
</Tab>
<Tab title="Docker">
```bash Command
# Pinned nightly with the Laguna-M.1 build (PR #28400 + #28604; CUDA 13 — covers H200 + all Blackwell):
docker pull lmsysorg/sglang:dev-cu13-618-nightly
# CUDA 13 — covers H200 + all Blackwell:
docker pull lmsysorg/sglang:latest
```
For how to launch the image, see [Install → Method 3: Using Docker](../../../docs/get-started/install#method-3-using-docker). Substitute the inner `sglang serve ...` with what the command generator below produces.
docs_new/cookbook/autoregressive/Poolside/Laguna-XS-2.1.mdx
+1 -1
View File
@@ -34,7 +34,7 @@ Then run the **Python** output of the command panel below in that environment.
<Tab title="Docker">
```bash Command
docker pull lmsysorg/sglang:dev-cu13-laguna-xs-2-1
docker pull lmsysorg/sglang:latest
```
For how to launch the image, see [Install → Method 3: Using Docker](../../../docs/get-started/install#method-3-using-docker). Substitute the inner `sglang serve ...` with what the command generator below produces.
docs_new/cookbook/autoregressive/Poolside/Laguna-XS.2.mdx
+2 -3
View File
@@ -62,9 +62,8 @@ python3 -m pip install --extra-index-url https://docs.sglang.ai/whl/cu130 \
python3 -m pip install --extra-index-url https://docs.sglang.ai/whl/cu129 \
"sglang[all]==0.5.12.dev20260509+g096ad02b0"
# Or use Docker (multi-arch amd64/arm64)
docker pull lmsysorg/sglang:dev-cu13-laguna-xs2 # CUDA 13 (H200 / B200)
docker pull lmsysorg/sglang:dev-cu12-laguna-xs2 # CUDA 12 (H200)
# Or use Docker (multi-arch amd64/arm64; CUDA 13, H200 / B200)
docker pull lmsysorg/sglang:latest
```
For the full Docker setup and other installation methods, please refer to the [official SGLang installation guide](../../../docs/get-started/install).
docs_new/cookbook/autoregressive/Qwen/Qwen3.6.mdx
+1 -4
View File
@@ -78,11 +78,8 @@ uv pip install sglang
# Or install from source
uv pip install 'git+https://github.com/sgl-project/sglang.git#subdirectory=python'
# Or use Docker (NVIDIA GPUs)
# Or use Docker (NVIDIA GPUs; also serves the NVFP4 variant nvidia/Qwen3.6-27B-NVFP4)
docker pull lmsysorg/sglang:latest
# For the NVFP4 variant (nvidia/Qwen3.6-27B-NVFP4), use the dedicated dev image
docker pull lmsysorg/sglang:dev-cu13-dev-qwen36-27b-nvfp4
```
For the full Docker setup and other installation methods, please refer to the [official SGLang installation guide](../../../docs/get-started/install).
docs_new/cookbook/autoregressive/StepFun/Step3.5.mdx
+2 -2
View File
@@ -22,14 +22,14 @@ Step-3.5-Flash is currently available in SGLang via Docker image install.
### Docker (NVIDIA)
```bash Command
# Pull the docker image
docker pull lmsysorg/sglang:dev-pr-18084
docker pull lmsysorg/sglang:latest
# Launch the container
docker run -it --gpus all \
--shm-size=32g \
--ipc=host \
--network=host \
lmsysorg/sglang:dev-pr-18084 bash
lmsysorg/sglang:latest bash
```
### Docker (AMD ROCm)
docs_new/cookbook/autoregressive/Tencent/Hunyuan3-Preview.mdx
+3 -7
View File
@@ -65,17 +65,13 @@ Please refer to the [official SGLang installation guide](../../../docs/get-start
</thead>
<tbody>
<tr>
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>NVIDIA H200 / B200</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`lmsysorg/sglang:hy3-preview`</td>
</tr>
<tr>
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>NVIDIA B300 / GB300</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`lmsysorg/sglang:hy3-preview-cu130`</td>
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>NVIDIA H200 / B200 / B300 / GB300</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`lmsysorg/sglang:latest`</td>
</tr>
</tbody>
</table>
The `hy3-preview` tag bundles the HYV3 model code, the `hunyuan` tool-call / reasoning parsers, and the MTP draft-module runtime.
`lmsysorg/sglang:latest` bundles the HYV3 model code, the `hunyuan` tool-call / reasoning parsers, and the MTP draft-module runtime.
For SGLang CPU installation, please refer to the [CPU version installation guide](../../../docs/hardware-platforms/cpu_server#installation).
docs_new/cookbook/autoregressive/Xiaomi/MiMo-V2-Flash.mdx
+2 -2
View File
@@ -23,14 +23,14 @@ MiMo-V2-Flash is currently available in SGLang via Docker image and pip install.
```bash Command
# Pull the docker image
docker pull lmsysorg/sglang:dev-pr-15207
docker pull lmsysorg/sglang:latest
# Launch the container
docker run -it --gpus all \
--shm-size=32g \
--ipc=host \
--network=host \
lmsysorg/sglang:dev-pr-15207 bash
lmsysorg/sglang:latest bash
```
### Pip Installation
docs_new/src/snippets/configs/MiniMaxAI/minimax-m3.jsx
+8 -9
View File
@@ -92,14 +92,13 @@ sgl-eval run mmmu_pro \\
],
dockerImages: {
// M3-specific dev images (multi-arch amd64+arm64). cu13 carries the sm_103
// (B300/GB300) + Grace arm64 builds; cu12 is the Hopper/CUDA-12 build;
// dev-minimax-m3 is the rolling default.
b200: "lmsysorg/sglang:dev-minimax-m3",
b300: "lmsysorg/sglang:dev-cu13-minimax-m3",
gb200: "lmsysorg/sglang:dev-cu13-minimax-m3",
gb300: "lmsysorg/sglang:dev-cu13-minimax-m3",
h200: "lmsysorg/sglang:dev-cu12-minimax-m3",
// lmsysorg/sglang:latest (cu13, multi-arch amd64+arm64) covers H200 + all
// Blackwell (incl. sm_103 B300/GB300 and Grace arm64).
b200: "lmsysorg/sglang:latest",
b300: "lmsysorg/sglang:latest",
gb200: "lmsysorg/sglang:latest",
gb300: "lmsysorg/sglang:latest",
h200: "lmsysorg/sglang:latest",
// AMD ROCm images — published M3 builds, by arch (gfx942 -> mi30x, gfx950 -> mi35x).
mi300x: "aigmkt/minimax-m3-sglang-rocm700-mi30x",
mi325x: "aigmkt/minimax-m3-sglang-rocm700-mi30x",
@@ -199,7 +198,7 @@ sgl-eval run mmmu_pro \\
// B300 / GB200 / GB300, tp8 on B200. fa4 + page 128 + deep_gemm are the M3
// SM100 auto-defaults on current main, so this is also the bare-launch
// behavior; they engage MiniMax's MSA sparse-attention kernel (fmha_sm100,
// pre-installed in the dev-minimax-m3 images; see Configuration Tips), Triton
// pre-installed in lmsysorg/sglang:latest; see Configuration Tips), Triton
// fallback otherwise.
// AMD: tp8. MI350X/MI355X (gfx950) serve MXFP8 natively (backends auto). MI300X/
// MI325X (gfx942) need --attention-backend aiter + --moe-runner-backend triton,
docs_new/src/snippets/configs/poolside/laguna-m1.jsx
+8 -10
View File
@@ -100,16 +100,14 @@ sgl-eval run gsm8k \\
["gsm8k_pct", "GSM8K", "%"],
],
// Pinned nightly with the Laguna-M.1 build (PR #28400 + #28604 + #28649; cu13 covers H200 + all Blackwell).
// dev-cu13-618-nightly was generated after the FP8 g_proj fix (#28649) landed, so it serves FP8 too.
// (Equivalent pip nightly: 0.5.14.dev20260618+g97e3b8998d.) Blackwell FP8 additionally needs the
// --fp8-gemm-backend triton flag (in those cells) until PR #28662 merges.
// lmsysorg/sglang:latest (cu13) covers H200 + all Blackwell and carries the
// Laguna-M.1 build (PR #28400 + #28604 + #28649, incl. the FP8 g_proj fix).
dockerImages: {
h200: "lmsysorg/sglang:dev-cu13-618-nightly",
b200: "lmsysorg/sglang:dev-cu13-618-nightly",
b300: "lmsysorg/sglang:dev-cu13-618-nightly",
gb200: "lmsysorg/sglang:dev-cu13-618-nightly",
gb300: "lmsysorg/sglang:dev-cu13-618-nightly",
h200: "lmsysorg/sglang:latest",
b200: "lmsysorg/sglang:latest",
b300: "lmsysorg/sglang:latest",
gb200: "lmsysorg/sglang:latest",
gb300: "lmsysorg/sglang:latest",
},
github: {
@@ -232,7 +230,7 @@ sgl-eval run gsm8k \\
},
{
// VERIFIED on 8xH200 (FP8, tp8): GSM8K 93.25%. FP8 needs the g_proj fix (PR #28649, MERGED) on
// top of #28400+#28604 — the pinned dev-cu13-618-nightly image has it. Hopper does NOT hit the
// top of #28400+#28604 — lmsysorg/sglang:latest has it. Hopper does NOT hit the
// Blackwell DeepGEMM UE8M0 issue, so no --fp8-gemm-backend flag here.
match: { hw: "h200", variant: "default", quant: "fp8", strategy: "balanced", nodes: "single" },
verified: true,
docs_new/src/snippets/configs/poolside/laguna-xs21.jsx
+4 -4
View File
@@ -142,11 +142,11 @@ sgl-eval run aime25 \\
["aime25_pct", "AIME25", "%"],
],
// Dedicated image built for this cookbook page (PR #29446 + #29761 preinstalled on cu13).
// lmsysorg/sglang:latest (cu13) carries the Laguna-XS.2.1 build (PR #29446 + #29761).
dockerImages: {
h200: "lmsysorg/sglang:dev-cu13-laguna-xs-2-1",
b300: "lmsysorg/sglang:dev-cu13-laguna-xs-2-1",
gb300: "lmsysorg/sglang:dev-cu13-laguna-xs-2-1",
h200: "lmsysorg/sglang:latest",
b300: "lmsysorg/sglang:latest",
gb300: "lmsysorg/sglang:latest",
},
github: {