docs: add --trust-remote-code to Laguna-M.1 / XS.2 cookbook configs (#28693)
This commit is contained in:
@@ -10,7 +10,7 @@ tag: NEW
|
|||||||
|
|
||||||
<Accordion title="Install SGLang">
|
<Accordion title="Install SGLang">
|
||||||
|
|
||||||
Laguna-M.1 support is already on SGLang `main` — **softplus per-element attention-output gating** ([PR #28400](https://github.com/sgl-project/sglang/pull/28400)) and a **global-attention fix** ([PR #28604](https://github.com/sgl-project/sglang/pull/28604), since M.1 is full-attention `sliding_window: 0`) — but not yet in a tagged release. The two paths below match the **Python / Docker** toggle in the command panel: install from `main` (Python tab), or use the **Docker** image, which bundles the same build (CUDA 13, covers H200 + all Blackwell). The model loads natively, so no `--trust-remote-code` is needed.
|
Laguna-M.1 support is already on SGLang `main` — **softplus per-element attention-output gating** ([PR #28400](https://github.com/sgl-project/sglang/pull/28400)) and a **global-attention fix** ([PR #28604](https://github.com/sgl-project/sglang/pull/28604), since M.1 is full-attention `sliding_window: 0`) — but not yet in a tagged release. The two paths below match the **Python / Docker** toggle in the command panel: install from `main` (Python tab), or use the **Docker** image, which bundles the same build (CUDA 13, covers H200 + all Blackwell). The model ships custom config code on the Hub, so `--trust-remote-code` is required (it is included in the launch commands).
|
||||||
|
|
||||||
<Tabs>
|
<Tabs>
|
||||||
|
|
||||||
@@ -109,6 +109,7 @@ import { Playground } from "/src/snippets/_playground.jsx";
|
|||||||
|
|
||||||
## 2. Configuration Tips
|
## 2. Configuration Tips
|
||||||
|
|
||||||
|
- **Trust remote code** (`--trust-remote-code`): Laguna-M.1 ships custom modeling/config code on the Hugging Face Hub, so this flag is required for the server to load the model.
|
||||||
- **Long-context memory**: M.1 is global-attention (no sliding-window), so the 262,144-token KV cache is large. If you hit OOM at full context, lower `--mem-fraction-static` or cap `--context-length`.
|
- **Long-context memory**: M.1 is global-attention (no sliding-window), so the 262,144-token KV cache is large. If you hit OOM at full context, lower `--mem-fraction-static` or cap `--context-length`.
|
||||||
- **FP8**: On **Blackwell** the recipe adds `--fp8-gemm-backend triton` — the compressed-tensors block-FP8 weight scales aren't UE8M0-packed, so the default DeepGEMM path emits garbage on Blackwell (sm_100); the Triton backend is correct (~19% slower). Temporary workaround pending [PR #28662](https://github.com/sgl-project/sglang/pull/28662) (which fixes the scales and restores the DeepGEMM fast path). On **Hopper (H200)** FP8 uses DeepGEMM with no extra flag — pre-warm its multi-session JIT with `python3 -m sglang.compile_deep_gemm --model poolside/Laguna-M.1-FP8` to avoid paying it on each restart.
|
- **FP8**: On **Blackwell** the recipe adds `--fp8-gemm-backend triton` — the compressed-tensors block-FP8 weight scales aren't UE8M0-packed, so the default DeepGEMM path emits garbage on Blackwell (sm_100); the Triton backend is correct (~19% slower). Temporary workaround pending [PR #28662](https://github.com/sgl-project/sglang/pull/28662) (which fixes the scales and restores the DeepGEMM fast path). On **Hopper (H200)** FP8 uses DeepGEMM with no extra flag — pre-warm its multi-session JIT with `python3 -m sglang.compile_deep_gemm --model poolside/Laguna-M.1-FP8` to avoid paying it on each restart.
|
||||||
- **Parsers** (`poolside_v1`): for agentic / tool-using deployments enable the **Reasoning Parser** and **Tool Call Parser** in the Playground above — they emit `--reasoning-parser poolside_v1` (thinking → `reasoning_content`) and `--tool-call-parser poolside_v1` (structured `tool_calls`).
|
- **Parsers** (`poolside_v1`): for agentic / tool-using deployments enable the **Reasoning Parser** and **Tool Call Parser** in the Playground above — they emit `--reasoning-parser poolside_v1` (thinking → `reasoning_content`) and `--tool-call-parser poolside_v1` (structured `tool_calls`).
|
||||||
|
|||||||
@@ -81,6 +81,7 @@ import { LagunaXS2Deployment } from '/src/snippets/autoregressive/laguna-xs2-dep
|
|||||||
|
|
||||||
### 3.2 Configuration Tips
|
### 3.2 Configuration Tips
|
||||||
|
|
||||||
|
- **Trust remote code** (`--trust-remote-code`): Laguna-XS.2 ships custom modeling/config code on the Hugging Face Hub, so this flag is required for the server to load the model.
|
||||||
- **Quantization**: NVFP4 requires Blackwell (B200 / B300); BF16 and FP8 run on either H200 or B200. FP8's first launch triggers a multi-session DeepGEMM JIT pre-compile (~10-20 min); pre-warm with `python3 -m sglang.compile_deep_gemm --model poolside/Laguna-XS.2-FP8` to avoid that cost on every restart.
|
- **Quantization**: NVFP4 requires Blackwell (B200 / B300); BF16 and FP8 run on either H200 or B200. FP8's first launch triggers a multi-session DeepGEMM JIT pre-compile (~10-20 min); pre-warm with `python3 -m sglang.compile_deep_gemm --model poolside/Laguna-XS.2-FP8` to avoid that cost on every restart.
|
||||||
- **Reasoning parser** (`--reasoning-parser poolside_v1`): Splits `<think>...</think>` segments into `reasoning_content` so `content` holds only the final answer. Disable only if you want the raw `<think>` tags in `content`.
|
- **Reasoning parser** (`--reasoning-parser poolside_v1`): Splits `<think>...</think>` segments into `reasoning_content` so `content` holds only the final answer. Disable only if you want the raw `<think>` tags in `content`.
|
||||||
- **Tool call parser** (`--tool-call-parser poolside_v1`): Required for OpenAI-compatible tool-call streaming. Disable only for chat-only deployments.
|
- **Tool call parser** (`--tool-call-parser poolside_v1`): Required for OpenAI-compatible tool-call streaming. Disable only for chat-only deployments.
|
||||||
|
|||||||
@@ -121,7 +121,8 @@ export const LagunaXS2Deployment = () => {
|
|||||||
const lines = [
|
const lines = [
|
||||||
'sglang serve \\',
|
'sglang serve \\',
|
||||||
` --model-path ${modelId} \\`,
|
` --model-path ${modelId} \\`,
|
||||||
` --tp ${tp}`
|
` --tp ${tp} \\`,
|
||||||
|
' --trust-remote-code'
|
||||||
];
|
];
|
||||||
|
|
||||||
if (dpAttention === 'enabled') {
|
if (dpAttention === 'enabled') {
|
||||||
|
|||||||
@@ -9,7 +9,8 @@
|
|||||||
// and crashes under load. Pin dockerImages + benchmarks.sglang_version to a build at a commit
|
// and crashes under load. Pin dockerImages + benchmarks.sglang_version to a build at a commit
|
||||||
// ≥ #28604. See /sgl-workspace/laguna-m1-day0-checklist.md (step 2) + laguna-m1-results.md.
|
// ≥ #28604. See /sgl-workspace/laguna-m1-day0-checklist.md (step 2) + laguna-m1-results.md.
|
||||||
//
|
//
|
||||||
// Model is now natively supported (#28400) → NO --trust-remote-code needed.
|
// --trust-remote-code is required: M.1 ships custom config code on the Hub (the transformers-native
|
||||||
|
// `laguna` config is incompatible). Carried on every cell.
|
||||||
//
|
//
|
||||||
// Hardware: H200 (Hopper) + B200/B300/GB200/GB300 (Blackwell).
|
// Hardware: H200 (Hopper) + B200/B300/GB200/GB300 (Blackwell).
|
||||||
// - BF16 runs everywhere.
|
// - BF16 runs everywhere.
|
||||||
@@ -151,7 +152,7 @@ sgl-eval run gsm8k \\
|
|||||||
|
|
||||||
// One Balanced cell per valid (hw × quant): H200×{BF16,FP8}; each Blackwell×{BF16,FP8,NVFP4}.
|
// One Balanced cell per valid (hw × quant): H200×{BF16,FP8}; each Blackwell×{BF16,FP8,NVFP4}.
|
||||||
// Blackwell FP8 cells add `--fp8-gemm-backend triton` (DeepGEMM UE8M0 workaround, pending #28662);
|
// Blackwell FP8 cells add `--fp8-gemm-backend triton` (DeepGEMM UE8M0 workaround, pending #28662);
|
||||||
// H200 FP8 needs no such flag. Baseline recipe (parsers poolside_v1, NO --trust-remote-code) on every cell.
|
// H200 FP8 needs no such flag. Baseline recipe (parsers poolside_v1 + --trust-remote-code) on every cell.
|
||||||
// TP: H200/B200/B300 = --tp 8; GB200/GB300 = --tp 4 (4-GPU single node).
|
// TP: H200/B200/B300 = --tp 8; GB200/GB300 = --tp 4 (4-GPU single node).
|
||||||
// verified:true = ran that exact command on that hardware and it served correctly + passed a
|
// verified:true = ran that exact command on that hardware and it served correctly + passed a
|
||||||
// GSM8K-class eval. Absent verified = yellow/unverified badge.
|
// GSM8K-class eval. Absent verified = yellow/unverified badge.
|
||||||
@@ -164,6 +165,7 @@ sgl-eval run gsm8k \\
|
|||||||
env: [],
|
env: [],
|
||||||
flags: [
|
flags: [
|
||||||
"--model-path {{MODEL_NAME}}",
|
"--model-path {{MODEL_NAME}}",
|
||||||
|
"--trust-remote-code",
|
||||||
"--reasoning-parser poolside_v1",
|
"--reasoning-parser poolside_v1",
|
||||||
"--tool-call-parser poolside_v1",
|
"--tool-call-parser poolside_v1",
|
||||||
"--tp 8",
|
"--tp 8",
|
||||||
@@ -180,6 +182,7 @@ sgl-eval run gsm8k \\
|
|||||||
env: [],
|
env: [],
|
||||||
flags: [
|
flags: [
|
||||||
"--model-path {{MODEL_NAME}}",
|
"--model-path {{MODEL_NAME}}",
|
||||||
|
"--trust-remote-code",
|
||||||
"--reasoning-parser poolside_v1",
|
"--reasoning-parser poolside_v1",
|
||||||
"--tool-call-parser poolside_v1",
|
"--tool-call-parser poolside_v1",
|
||||||
"--tp 8",
|
"--tp 8",
|
||||||
@@ -195,6 +198,7 @@ sgl-eval run gsm8k \\
|
|||||||
env: [],
|
env: [],
|
||||||
flags: [
|
flags: [
|
||||||
"--model-path {{MODEL_NAME}}",
|
"--model-path {{MODEL_NAME}}",
|
||||||
|
"--trust-remote-code",
|
||||||
"--reasoning-parser poolside_v1",
|
"--reasoning-parser poolside_v1",
|
||||||
"--tool-call-parser poolside_v1",
|
"--tool-call-parser poolside_v1",
|
||||||
"--tp 8",
|
"--tp 8",
|
||||||
@@ -211,6 +215,7 @@ sgl-eval run gsm8k \\
|
|||||||
env: [],
|
env: [],
|
||||||
flags: [
|
flags: [
|
||||||
"--model-path {{MODEL_NAME}}",
|
"--model-path {{MODEL_NAME}}",
|
||||||
|
"--trust-remote-code",
|
||||||
"--reasoning-parser poolside_v1",
|
"--reasoning-parser poolside_v1",
|
||||||
"--tool-call-parser poolside_v1",
|
"--tool-call-parser poolside_v1",
|
||||||
"--tp 8",
|
"--tp 8",
|
||||||
@@ -226,6 +231,7 @@ sgl-eval run gsm8k \\
|
|||||||
env: [],
|
env: [],
|
||||||
flags: [
|
flags: [
|
||||||
"--model-path {{MODEL_NAME}}",
|
"--model-path {{MODEL_NAME}}",
|
||||||
|
"--trust-remote-code",
|
||||||
"--reasoning-parser poolside_v1",
|
"--reasoning-parser poolside_v1",
|
||||||
"--tool-call-parser poolside_v1",
|
"--tool-call-parser poolside_v1",
|
||||||
"--tp 8",
|
"--tp 8",
|
||||||
@@ -239,6 +245,7 @@ sgl-eval run gsm8k \\
|
|||||||
env: [],
|
env: [],
|
||||||
flags: [
|
flags: [
|
||||||
"--model-path {{MODEL_NAME}}",
|
"--model-path {{MODEL_NAME}}",
|
||||||
|
"--trust-remote-code",
|
||||||
"--reasoning-parser poolside_v1",
|
"--reasoning-parser poolside_v1",
|
||||||
"--tool-call-parser poolside_v1",
|
"--tool-call-parser poolside_v1",
|
||||||
"--tp 8",
|
"--tp 8",
|
||||||
@@ -252,6 +259,7 @@ sgl-eval run gsm8k \\
|
|||||||
env: [],
|
env: [],
|
||||||
flags: [
|
flags: [
|
||||||
"--model-path {{MODEL_NAME}}",
|
"--model-path {{MODEL_NAME}}",
|
||||||
|
"--trust-remote-code",
|
||||||
"--reasoning-parser poolside_v1",
|
"--reasoning-parser poolside_v1",
|
||||||
"--tool-call-parser poolside_v1",
|
"--tool-call-parser poolside_v1",
|
||||||
"--tp 8",
|
"--tp 8",
|
||||||
@@ -265,6 +273,7 @@ sgl-eval run gsm8k \\
|
|||||||
env: [],
|
env: [],
|
||||||
flags: [
|
flags: [
|
||||||
"--model-path {{MODEL_NAME}}",
|
"--model-path {{MODEL_NAME}}",
|
||||||
|
"--trust-remote-code",
|
||||||
"--reasoning-parser poolside_v1",
|
"--reasoning-parser poolside_v1",
|
||||||
"--tool-call-parser poolside_v1",
|
"--tool-call-parser poolside_v1",
|
||||||
"--tp 8",
|
"--tp 8",
|
||||||
@@ -278,6 +287,7 @@ sgl-eval run gsm8k \\
|
|||||||
env: [],
|
env: [],
|
||||||
flags: [
|
flags: [
|
||||||
"--model-path {{MODEL_NAME}}",
|
"--model-path {{MODEL_NAME}}",
|
||||||
|
"--trust-remote-code",
|
||||||
"--reasoning-parser poolside_v1",
|
"--reasoning-parser poolside_v1",
|
||||||
"--tool-call-parser poolside_v1",
|
"--tool-call-parser poolside_v1",
|
||||||
"--tp 4",
|
"--tp 4",
|
||||||
@@ -291,6 +301,7 @@ sgl-eval run gsm8k \\
|
|||||||
env: [],
|
env: [],
|
||||||
flags: [
|
flags: [
|
||||||
"--model-path {{MODEL_NAME}}",
|
"--model-path {{MODEL_NAME}}",
|
||||||
|
"--trust-remote-code",
|
||||||
"--reasoning-parser poolside_v1",
|
"--reasoning-parser poolside_v1",
|
||||||
"--tool-call-parser poolside_v1",
|
"--tool-call-parser poolside_v1",
|
||||||
"--tp 4",
|
"--tp 4",
|
||||||
@@ -304,6 +315,7 @@ sgl-eval run gsm8k \\
|
|||||||
env: [],
|
env: [],
|
||||||
flags: [
|
flags: [
|
||||||
"--model-path {{MODEL_NAME}}",
|
"--model-path {{MODEL_NAME}}",
|
||||||
|
"--trust-remote-code",
|
||||||
"--reasoning-parser poolside_v1",
|
"--reasoning-parser poolside_v1",
|
||||||
"--tool-call-parser poolside_v1",
|
"--tool-call-parser poolside_v1",
|
||||||
"--tp 4",
|
"--tp 4",
|
||||||
@@ -317,6 +329,7 @@ sgl-eval run gsm8k \\
|
|||||||
env: [],
|
env: [],
|
||||||
flags: [
|
flags: [
|
||||||
"--model-path {{MODEL_NAME}}",
|
"--model-path {{MODEL_NAME}}",
|
||||||
|
"--trust-remote-code",
|
||||||
"--reasoning-parser poolside_v1",
|
"--reasoning-parser poolside_v1",
|
||||||
"--tool-call-parser poolside_v1",
|
"--tool-call-parser poolside_v1",
|
||||||
"--tp 4",
|
"--tp 4",
|
||||||
@@ -330,6 +343,7 @@ sgl-eval run gsm8k \\
|
|||||||
env: [],
|
env: [],
|
||||||
flags: [
|
flags: [
|
||||||
"--model-path {{MODEL_NAME}}",
|
"--model-path {{MODEL_NAME}}",
|
||||||
|
"--trust-remote-code",
|
||||||
"--reasoning-parser poolside_v1",
|
"--reasoning-parser poolside_v1",
|
||||||
"--tool-call-parser poolside_v1",
|
"--tool-call-parser poolside_v1",
|
||||||
"--tp 4",
|
"--tp 4",
|
||||||
@@ -343,6 +357,7 @@ sgl-eval run gsm8k \\
|
|||||||
env: [],
|
env: [],
|
||||||
flags: [
|
flags: [
|
||||||
"--model-path {{MODEL_NAME}}",
|
"--model-path {{MODEL_NAME}}",
|
||||||
|
"--trust-remote-code",
|
||||||
"--reasoning-parser poolside_v1",
|
"--reasoning-parser poolside_v1",
|
||||||
"--tool-call-parser poolside_v1",
|
"--tool-call-parser poolside_v1",
|
||||||
"--tp 4",
|
"--tp 4",
|
||||||
|
|||||||
Reference in New Issue
Block a user