docs: add --trust-remote-code to Laguna-M.1 / XS.2 cookbook configs (#28693)

This commit is contained in:
Jimmy Shong
2026-06-19 10:21:39 +08:00
committed by GitHub
parent c7397de571
commit d962d18f15
4 changed files with 22 additions and 4 deletions
@@ -10,7 +10,7 @@ tag: NEW
<Accordion title="Install SGLang">
Laguna-M.1 support is already on SGLang `main` — **softplus per-element attention-output gating** ([PR #28400](https://github.com/sgl-project/sglang/pull/28400)) and a **global-attention fix** ([PR #28604](https://github.com/sgl-project/sglang/pull/28604), since M.1 is full-attention `sliding_window: 0`) — but not yet in a tagged release. The two paths below match the **Python / Docker** toggle in the command panel: install from `main` (Python tab), or use the **Docker** image, which bundles the same build (CUDA 13, covers H200 + all Blackwell). The model loads natively, so no `--trust-remote-code` is needed.
Laguna-M.1 support is already on SGLang `main` — **softplus per-element attention-output gating** ([PR #28400](https://github.com/sgl-project/sglang/pull/28400)) and a **global-attention fix** ([PR #28604](https://github.com/sgl-project/sglang/pull/28604), since M.1 is full-attention `sliding_window: 0`) — but not yet in a tagged release. The two paths below match the **Python / Docker** toggle in the command panel: install from `main` (Python tab), or use the **Docker** image, which bundles the same build (CUDA 13, covers H200 + all Blackwell). The model ships custom config code on the Hub, so `--trust-remote-code` is required (it is included in the launch commands).
<Tabs>
@@ -109,6 +109,7 @@ import { Playground } from "/src/snippets/_playground.jsx";
## 2. Configuration Tips
- **Trust remote code** (`--trust-remote-code`): Laguna-M.1 ships custom modeling/config code on the Hugging Face Hub, so this flag is required for the server to load the model.
- **Long-context memory**: M.1 is global-attention (no sliding-window), so the 262,144-token KV cache is large. If you hit OOM at full context, lower `--mem-fraction-static` or cap `--context-length`.
- **FP8**: On **Blackwell** the recipe adds `--fp8-gemm-backend triton` — the compressed-tensors block-FP8 weight scales aren't UE8M0-packed, so the default DeepGEMM path emits garbage on Blackwell (sm_100); the Triton backend is correct (~19% slower). Temporary workaround pending [PR #28662](https://github.com/sgl-project/sglang/pull/28662) (which fixes the scales and restores the DeepGEMM fast path). On **Hopper (H200)** FP8 uses DeepGEMM with no extra flag — pre-warm its multi-session JIT with `python3 -m sglang.compile_deep_gemm --model poolside/Laguna-M.1-FP8` to avoid paying it on each restart.
- **Parsers** (`poolside_v1`): for agentic / tool-using deployments enable the **Reasoning Parser** and **Tool Call Parser** in the Playground above — they emit `--reasoning-parser poolside_v1` (thinking → `reasoning_content`) and `--tool-call-parser poolside_v1` (structured `tool_calls`).
@@ -81,6 +81,7 @@ import { LagunaXS2Deployment } from '/src/snippets/autoregressive/laguna-xs2-dep
### 3.2 Configuration Tips
- **Trust remote code** (`--trust-remote-code`): Laguna-XS.2 ships custom modeling/config code on the Hugging Face Hub, so this flag is required for the server to load the model.
- **Quantization**: NVFP4 requires Blackwell (B200 / B300); BF16 and FP8 run on either H200 or B200. FP8's first launch triggers a multi-session DeepGEMM JIT pre-compile (~10-20 min); pre-warm with `python3 -m sglang.compile_deep_gemm --model poolside/Laguna-XS.2-FP8` to avoid that cost on every restart.
- **Reasoning parser** (`--reasoning-parser poolside_v1`): Splits `<think>...</think>` segments into `reasoning_content` so `content` holds only the final answer. Disable only if you want the raw `<think>` tags in `content`.
- **Tool call parser** (`--tool-call-parser poolside_v1`): Required for OpenAI-compatible tool-call streaming. Disable only for chat-only deployments.
@@ -121,7 +121,8 @@ export const LagunaXS2Deployment = () => {
const lines = [
'sglang serve \\',
` --model-path ${modelId} \\`,
` --tp ${tp}`
` --tp ${tp} \\`,
' --trust-remote-code'
];
if (dpAttention === 'enabled') {
@@ -9,7 +9,8 @@
// and crashes under load. Pin dockerImages + benchmarks.sglang_version to a build at a commit
// ≥ #28604. See /sgl-workspace/laguna-m1-day0-checklist.md (step 2) + laguna-m1-results.md.
//
// Model is now natively supported (#28400) → NO --trust-remote-code needed.
// --trust-remote-code is required: M.1 ships custom config code on the Hub (the transformers-native
// `laguna` config is incompatible). Carried on every cell.
//
// Hardware: H200 (Hopper) + B200/B300/GB200/GB300 (Blackwell).
// - BF16 runs everywhere.
@@ -151,7 +152,7 @@ sgl-eval run gsm8k \\
// One Balanced cell per valid (hw × quant): H200×{BF16,FP8}; each Blackwell×{BF16,FP8,NVFP4}.
// Blackwell FP8 cells add `--fp8-gemm-backend triton` (DeepGEMM UE8M0 workaround, pending #28662);
// H200 FP8 needs no such flag. Baseline recipe (parsers poolside_v1, NO --trust-remote-code) on every cell.
// H200 FP8 needs no such flag. Baseline recipe (parsers poolside_v1 + --trust-remote-code) on every cell.
// TP: H200/B200/B300 = --tp 8; GB200/GB300 = --tp 4 (4-GPU single node).
// verified:true = ran that exact command on that hardware and it served correctly + passed a
// GSM8K-class eval. Absent verified = yellow/unverified badge.
@@ -164,6 +165,7 @@ sgl-eval run gsm8k \\
env: [],
flags: [
"--model-path {{MODEL_NAME}}",
"--trust-remote-code",
"--reasoning-parser poolside_v1",
"--tool-call-parser poolside_v1",
"--tp 8",
@@ -180,6 +182,7 @@ sgl-eval run gsm8k \\
env: [],
flags: [
"--model-path {{MODEL_NAME}}",
"--trust-remote-code",
"--reasoning-parser poolside_v1",
"--tool-call-parser poolside_v1",
"--tp 8",
@@ -195,6 +198,7 @@ sgl-eval run gsm8k \\
env: [],
flags: [
"--model-path {{MODEL_NAME}}",
"--trust-remote-code",
"--reasoning-parser poolside_v1",
"--tool-call-parser poolside_v1",
"--tp 8",
@@ -211,6 +215,7 @@ sgl-eval run gsm8k \\
env: [],
flags: [
"--model-path {{MODEL_NAME}}",
"--trust-remote-code",
"--reasoning-parser poolside_v1",
"--tool-call-parser poolside_v1",
"--tp 8",
@@ -226,6 +231,7 @@ sgl-eval run gsm8k \\
env: [],
flags: [
"--model-path {{MODEL_NAME}}",
"--trust-remote-code",
"--reasoning-parser poolside_v1",
"--tool-call-parser poolside_v1",
"--tp 8",
@@ -239,6 +245,7 @@ sgl-eval run gsm8k \\
env: [],
flags: [
"--model-path {{MODEL_NAME}}",
"--trust-remote-code",
"--reasoning-parser poolside_v1",
"--tool-call-parser poolside_v1",
"--tp 8",
@@ -252,6 +259,7 @@ sgl-eval run gsm8k \\
env: [],
flags: [
"--model-path {{MODEL_NAME}}",
"--trust-remote-code",
"--reasoning-parser poolside_v1",
"--tool-call-parser poolside_v1",
"--tp 8",
@@ -265,6 +273,7 @@ sgl-eval run gsm8k \\
env: [],
flags: [
"--model-path {{MODEL_NAME}}",
"--trust-remote-code",
"--reasoning-parser poolside_v1",
"--tool-call-parser poolside_v1",
"--tp 8",
@@ -278,6 +287,7 @@ sgl-eval run gsm8k \\
env: [],
flags: [
"--model-path {{MODEL_NAME}}",
"--trust-remote-code",
"--reasoning-parser poolside_v1",
"--tool-call-parser poolside_v1",
"--tp 4",
@@ -291,6 +301,7 @@ sgl-eval run gsm8k \\
env: [],
flags: [
"--model-path {{MODEL_NAME}}",
"--trust-remote-code",
"--reasoning-parser poolside_v1",
"--tool-call-parser poolside_v1",
"--tp 4",
@@ -304,6 +315,7 @@ sgl-eval run gsm8k \\
env: [],
flags: [
"--model-path {{MODEL_NAME}}",
"--trust-remote-code",
"--reasoning-parser poolside_v1",
"--tool-call-parser poolside_v1",
"--tp 4",
@@ -317,6 +329,7 @@ sgl-eval run gsm8k \\
env: [],
flags: [
"--model-path {{MODEL_NAME}}",
"--trust-remote-code",
"--reasoning-parser poolside_v1",
"--tool-call-parser poolside_v1",
"--tp 4",
@@ -330,6 +343,7 @@ sgl-eval run gsm8k \\
env: [],
flags: [
"--model-path {{MODEL_NAME}}",
"--trust-remote-code",
"--reasoning-parser poolside_v1",
"--tool-call-parser poolside_v1",
"--tp 4",
@@ -343,6 +357,7 @@ sgl-eval run gsm8k \\
env: [],
flags: [
"--model-path {{MODEL_NAME}}",
"--trust-remote-code",
"--reasoning-parser poolside_v1",
"--tool-call-parser poolside_v1",
"--tp 4",