diff --git a/docs_new/cookbook/autoregressive/Poolside/Laguna-M.1.mdx b/docs_new/cookbook/autoregressive/Poolside/Laguna-M.1.mdx new file mode 100644 index 000000000..cac3e707c --- /dev/null +++ b/docs_new/cookbook/autoregressive/Poolside/Laguna-M.1.mdx @@ -0,0 +1,234 @@ +--- +title: Laguna-M.1 +description: "Deploy poolside's Laguna-M.1 — a 225B-parameter Mixture-of-Experts model (23B active) for agentic coding — with SGLang on NVIDIA H200, B200, B300, GB200, and GB300, across BF16, FP8, and NVFP4." +tag: NEW +--- + +## Deployment + + + + + +Laguna-M.1 support is already on SGLang `main` — **softplus per-element attention-output gating** ([PR #28400](https://github.com/sgl-project/sglang/pull/28400)) and a **global-attention fix** ([PR #28604](https://github.com/sgl-project/sglang/pull/28604), since M.1 is full-attention `sliding_window: 0`) — but not yet in a tagged release. The two paths below match the **Python / Docker** toggle in the command panel: install from `main` (Python tab), or use the **Docker** image, which bundles the same build (CUDA 13, covers H200 + all Blackwell). The model loads natively, so no `--trust-remote-code` is needed. + + + + + +```bash Command +pip install -U uv +uv venv --python 3.12 && source .venv/bin/activate + +# Laguna-M.1 support is on SGLang main (PRs #28400 + #28604, plus #28649 for FP8), not yet in a +# tagged release — install from main. The serving runtime is in the base dependencies, no extra needed: +git clone https://github.com/sgl-project/sglang.git +cd sglang +uv pip install -e python +``` + +Then run the **Python** output of the command panel below in that environment. The **Docker** tab is simpler — its image (`dev-cu13-618-nightly`) bundles the CUDA-13 runtime and the M.1 code. Once M.1 support lands in a tagged release, `uv pip install sglang` will pull it directly. + + + + + +```bash Command +# Pinned nightly with the Laguna-M.1 build (PR #28400 + #28604; CUDA 13 — covers H200 + all Blackwell): +docker pull lmsysorg/sglang:dev-cu13-618-nightly +``` + +For how to launch the image, see [Install → Method 3: Using Docker](../../../docs/get-started/install#method-3-using-docker). Substitute the inner `sglang serve ...` with what the command generator below produces. + + + + + + + +Pick your hardware + quantization to generate the launch command. Laguna-M.1 ships a single **Balanced** recipe per cell — poolside's recommended operating point, a good speed/throughput trade-off for typical multi-user serving. The 8-GPU HGX platforms (H200 / B200 / B300) use `--tp 8`; the 4-GPU Grace-Blackwell single nodes (GB200 / GB300) use `--tp 4`. + +import { Deployment } from "/src/snippets/_deployment.jsx"; +import { config } from "/src/snippets/configs/poolside/laguna-m1.jsx"; +import { benchmarks } from "/src/snippets/configs/poolside/laguna-m1-benchmarks.jsx"; + + + +## Playground + +The Playground is where you experiment with **SGLang features beyond the verified matrix**. The Deploy panel above only emits combinations the SGLang team has signed off on; the Playground lets you turn on additional knobs (parsers, DP-Attention, DeepEP / EP) on top of whichever cell the Deploy panel is currently showing. + +import { Playground } from "/src/snippets/_playground.jsx"; + + + +## 1. Model Introduction + +[Laguna-M.1](https://huggingface.co/poolside/Laguna-M.1) is an open-weight, **225B-parameter** Mixture-of-Experts model (**23B activated per token**) from [poolside](https://poolside.ai), built for agentic coding and long-horizon software-engineering work. It is released under Apache 2.0. + +**Key Features:** + +- **Large sparse MoE**: 70-layer transformer — the first 3 layers are dense SwiGLU, the remaining 67 are sparse MoE with **256 experts, top-16 routing** (+1 shared expert) and auxiliary-loss-free load balancing. +- **Global attention with output gating**: global attention across all layers, 64 Q-heads / 8 KV-heads (head dim 128), with **softplus attention output gating** (requires [PR #28400](https://github.com/sgl-project/sglang/pull/28400)). +- **Long context**: 262,144 tokens, RoPE with YaRN. +- **Agentic coding**: competitive on SWE-bench Verified, SWE-bench Multilingual, SWE-Bench Pro, and Terminal-Bench 2.0. +- **Native reasoning**: interleaved thinking between tool calls, toggled per request via `chat_template_kwargs={"enable_thinking": ...}`. + +**Available Quantizations:** + + + + + + + + + + + + + + + + + + + + + + + + + + +
QuantizationHugging Face path
BF16[`poolside/Laguna-M.1`](https://huggingface.co/poolside/Laguna-M.1)
FP8[`poolside/Laguna-M.1-FP8`](https://huggingface.co/poolside/Laguna-M.1-FP8)
NVFP4[`poolside/Laguna-M.1-NVFP4`](https://huggingface.co/poolside/Laguna-M.1-NVFP4)
+ +**License:** Apache 2.0 + +**Resources:** [Hugging Face](https://huggingface.co/poolside/Laguna-M.1) · [Release blog post](https://poolside.ai/blog/laguna-a-deeper-dive) · [Technical report](https://poolside.ai/assets/laguna/laguna-m1-xs2-technical-report.pdf) · [API platform](https://platform.poolside.ai). + +## 2. Configuration Tips + +- **Long-context memory**: M.1 is global-attention (no sliding-window), so the 262,144-token KV cache is large. If you hit OOM at full context, lower `--mem-fraction-static` or cap `--context-length`. +- **FP8 first launch**: FP8 triggers a multi-session DeepGEMM JIT pre-compile on first launch; pre-warm with `python3 -m sglang.compile_deep_gemm --model poolside/Laguna-M.1-FP8` to avoid paying it on every restart. +- **Parsers** (`poolside_v1`): for agentic / tool-using deployments enable the **Reasoning Parser** and **Tool Call Parser** in the Playground above — they emit `--reasoning-parser poolside_v1` (thinking → `reasoning_content`) and `--tool-call-parser poolside_v1` (structured `tool_calls`). +- **Thinking default**: thinking is **off by default**; opt in per request with `extra_body={"chat_template_kwargs": {"enable_thinking": True}}`. +- **Served model id**: the server registers the model under whatever you pass to `--model-path`, so a client's `model` field must match it — `poolside/Laguna-M.1` (BF16) or `poolside/Laguna-M.1-FP8` / `-NVFP4` for the quantized cells. The §3 examples use the BF16 id; swap in the id you launched. +- **Recommended sampling**: poolside benchmarks M.1 at `temperature=1.0`, `top_k=20` with thinking enabled. These are per-request sampling params (not launch flags) — e.g. `temperature=1.0, extra_body={"top_k": 20}` on the OpenAI client. +## 3. Advanced Usage + +### 3.1 Reasoning + +Launch with `--reasoning-parser poolside_v1` (or toggle **Reasoning Parser** in the **Parsers** card of the Playground above). Reasoning is **opt-in**: the Laguna chat template gates it on `enable_thinking=True` (passed via `chat_template_kwargs`) — the generic `thinking` key is ignored. The `` trace then lands in `message.reasoning_content`, separate from the final answer in `message.content` — no client-side tag stripping needed. + + + +```python Example +from openai import OpenAI + +client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY") + +response = client.chat.completions.create( + model="poolside/Laguna-M.1", + messages=[{"role": "user", "content": "What is 15% of 240? Explain briefly."}], + max_tokens=2048, + extra_body={"chat_template_kwargs": {"enable_thinking": True}}, +) + +message = response.choices[0].message +print("=============== Reasoning ===============") +print(message.reasoning_content) +print("=============== Answer ==================") +print(message.content) +``` + + + + + +```text Output +=============== Reasoning =============== +Okay, so I need to find out what 15% of 240 is. Hmm, percentages can sometimes be +tricky, but let me think. I remember that "percent" means per hundred, right? So 15% +is the same as 15 per 100 or 15/100. Maybe I can convert that percentage into a decimal +first? ... 15 divided by 100 is 0.15. ... Now, to find 15% of 240, I just need to +multiply 240 by 0.15. ... 240 times 0.1 is 24 (10% of 240), and 240 times 0.05 is 12 +(half of that), so 24 + 12 = 36. +[… verifies the same result several more ways: 15/100 × 240, 240 × 15 ÷ 100, +1% × 15, and the fraction 3/20 × 240 — all give 36 …] +So ... all methods are pointing to 36. I'm pretty confident that 15% of 240 is 36. +=============== Answer ================== +To find 15% of 240, convert the percentage to a decimal (0.15) and multiply by 240: +**240 × 0.15 = 36**. + +**Step-by-Step Explanation:** +1. **Convert 15% to a decimal:** 15% = 15/100 = 0.15. +2. **Multiply by 240:** + - Break it down: + - 10% of 240 = 24 (since 240 × 0.1 = 24). + - 5% of 240 = 12 (half of 24). + - Add them: 24 + 12 = **36**. + +**Answer:** 15% of 240 is **36**. +``` + + + + +Laguna-M.1's reasoning traces are long — the model explores and re-verifies an answer +multiple ways. Give it a generous `max_tokens` for harder problems (reasoning regularly +exceeds 3k tokens). The trace above is abbreviated; the model emits it in full. + + +### 3.2 Tool Calling + +Launch with `--tool-call-parser poolside_v1` (or toggle **Tool Call Parser** in the **Parsers** card of the Playground above). The parser converts Laguna's `` output into the standard OpenAI `tool_calls` structure. Tool calling works with reasoning off (`enable_thinking=False`, the default). + + + +```python Example +from openai import OpenAI + +client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY") + +tools = [ + { + "type": "function", + "function": { + "name": "get_weather", + "description": "Get the current weather for a location", + "parameters": { + "type": "object", + "properties": { + "location": {"type": "string", "description": "The city name"}, + "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}, + }, + "required": ["location"], + }, + }, + } +] + +response = client.chat.completions.create( + model="poolside/Laguna-M.1", + messages=[{"role": "user", "content": "What's the weather in Beijing?"}], + tools=tools, +) + +message = response.choices[0].message +if message.tool_calls: + for call in message.tool_calls: + print(f"Tool: {call.function.name}") + print(f"Args: {call.function.arguments}") +``` + + + + + +```text Output +Tool: get_weather +Args: {"location": "Beijing"} +``` + + diff --git a/docs_new/cookbook/autoregressive/Poolside/Laguna-XS.2.mdx b/docs_new/cookbook/autoregressive/Poolside/Laguna-XS.2.mdx index 48549a646..c3237cb1d 100644 --- a/docs_new/cookbook/autoregressive/Poolside/Laguna-XS.2.mdx +++ b/docs_new/cookbook/autoregressive/Poolside/Laguna-XS.2.mdx @@ -2,7 +2,6 @@ title: Laguna-XS.2 metatags: description: "Deploy Poolside's Laguna-XS.2 hybrid SWA + MoE model with SGLang on NVIDIA H200 / B200 — agentic coding with hybrid reasoning and tool calling." -tag: NEW --- ## 1. Model Introduction diff --git a/docs_new/cookbook/autoregressive/intro.mdx b/docs_new/cookbook/autoregressive/intro.mdx index 88d487997..c3d40f375 100644 --- a/docs_new/cookbook/autoregressive/intro.mdx +++ b/docs_new/cookbook/autoregressive/intro.mdx @@ -136,7 +136,7 @@ metatags: diff --git a/docs_new/docs.json b/docs_new/docs.json index 31528ce92..3a11f6353 100644 --- a/docs_new/docs.json +++ b/docs_new/docs.json @@ -1123,6 +1123,7 @@ { "group": "Poolside", "pages": [ + "cookbook/autoregressive/Poolside/Laguna-M.1", "cookbook/autoregressive/Poolside/Laguna-XS.2" ] } diff --git a/docs_new/src/snippets/configs/poolside/laguna-m1-benchmarks.jsx b/docs_new/src/snippets/configs/poolside/laguna-m1-benchmarks.jsx new file mode 100644 index 000000000..e03d63142 --- /dev/null +++ b/docs_new/src/snippets/configs/poolside/laguna-m1-benchmarks.jsx @@ -0,0 +1,81 @@ +// Laguna-M.1 benchmarks — one entry per cell `match` (same 5 keys as laguna-m1.jsx cells). +// +// All numbers below are REAL measured values; cells without measurements are bare `{ match }` +// pending stubs (the card renders "pending"). NO fabricated/dummy numbers remain. +// (cookbook_guide §3 forbids fabricated numbers in a published page.) +// +// FP8 is HOPPER-ONLY (not compatible with Blackwell) → the only FP8 entry is H200; there are no +// Blackwell FP8 entries. Blackwell cells are BF16 / NVFP4. +// +// REAL numbers (sgl-eval; GSM8K non-thinking; AIME via the enable_thinking wrapper, max_tokens=32768): +// H200 BF16 (tp8): GSM8K 93.02% · AIME25 53.33% overall (~0.80 stop-only) · perf (cc1, cc128). +// H200 FP8 (tp8): GSM8K 93.25% · AIME25 50.0% overall (~0.79 stop-only). g_proj FP8 fix validated; no perf (BF16-only scope). +// B200 BF16 (tp8): GSM8K 91.88% · AIME25 66.88% (n_repeats=16) · perf A/B (cc1, cc128). +// B200 NVFP4 (tp8): GSM8K 89.38%. +// ⚠️ AIME OVERALL is depressed by ~33–37% truncation at the 32k cap (M.1 reasoning is long → no boxed +// answer → scored 0); stop-only (~0.80) is the truer signal, and a 48–64k cap would lift the overall. +// (perf tokens_per_sec_per_gpu = measured output tok/s ÷ 8 GPUs; TTFT = median.) +// +// sglang_version reflects the REQUIRED build = PR #28400 (per-element gating) + PR #28604 +// (global-attention SWA fix). The plain #28400 wheel (0.5.14.dev20260618+g343aeeef39) is NOT +// enough — it crashes M.1 under load. H200 numbers were taken on a main build @ 3f668733. + +export const benchmarks = [ + // ===== H200 — BF16 / FP8 ===== + { + // ✅ REAL — 8xH200, BF16, tp8. GSM8K 0.9302; perf bench_serving random 4096/1024. + match: { hw: "h200", variant: "default", quant: "bf16", strategy: "balanced", nodes: "single" }, + verified: true, + sglang_version: "main @ 3f668733 (#28400 + #28604)", + speed: [ + // cc=1: median TTFT 81.89 ms, median TPOT 8.91 ms, output 109.96 tok/s (÷8 ≈ 13.7/GPU). + { workload: { dataset: "random", isl: 4096, osl: 1024, max_concurrency: 1 }, + ttft_ms: 81.9, tpot_ms: 8.91, tokens_per_sec_per_gpu: 13.7 }, + // cc=128: median TTFT 200.11 ms (mean 1221), median TPOT 52.09 ms, output 2266 tok/s (÷8 ≈ 283/GPU); total 11311 tok/s. + { workload: { dataset: "random", isl: 4096, osl: 1024, max_concurrency: 128 }, + ttft_ms: 200.1, tpot_ms: 52.1, tokens_per_sec_per_gpu: 283 }, + ], + accuracy: { gsm8k_pct: 93.02, aime25_pct: 53.33 }, // AIME overall, n_repeats=1 (32k-truncation-limited, ~33% trunc; stop-only ~0.80) + }, + { + // ✅ REAL — 8xH200, FP8, tp8. GSM8K 93.25 + AIME25 0.50 (overall). The g_proj quant fix is + // validated (FP8 now loads past layer 0). Perf not measured on FP8 (BF16-only scope) → no speed row. + // FP8 is Hopper-only — not compatible with Blackwell (no Blackwell FP8 entry below). + match: { hw: "h200", variant: "default", quant: "fp8", strategy: "balanced", nodes: "single" }, + verified: true, + sglang_version: "main @ 3f668733 (#28400 + #28604 + g_proj FP8 fix)", + accuracy: { gsm8k_pct: 93.25, aime25_pct: 50.0 }, // AIME overall (32k-truncation-limited; stop-only ~0.79) + }, + + // ===== B200 (8-GPU HGX) — BF16 / NVFP4 ===== + { + // ✅ REAL — 8xB200, BF16, tp8. GSM8K 91.88 + AIME25 66.88; perf A/B (laguna-m1-results.md). + match: { hw: "b200", variant: "default", quant: "bf16", strategy: "balanced", nodes: "single" }, + verified: true, + sglang_version: "PR #28400 + #28604", + speed: [ + { workload: { dataset: "random", isl: 4096, osl: 1024, max_concurrency: 1 }, + ttft_ms: 108, tpot_ms: 9.0, tokens_per_sec_per_gpu: 13.6 }, + { workload: { dataset: "random", isl: 4096, osl: 1024, max_concurrency: 128 }, + ttft_ms: 170, tpot_ms: 43.3, tokens_per_sec_per_gpu: 331 }, + ], + accuracy: { gsm8k_pct: 91.88, aime25_pct: 66.88 }, + }, + { + // ✅ REAL (GSM8K only) — 8xB200, NVFP4, tp8. GSM8K 89.38 (laguna-m1-results.md). + // AIME + perf not measured yet → omitted (card shows them pending). + match: { hw: "b200", variant: "default", quant: "nvfp4", strategy: "balanced", nodes: "single" }, + verified: true, + sglang_version: "PR #28400 + #28604", + accuracy: { gsm8k_pct: 89.38 }, + }, + + // ===== B300 / GB200 / GB300 — BF16 / NVFP4, UNVERIFIED, no data yet → bare "pending" stubs + // (no fabricated numbers; FP8 is Hopper-only so no Blackwell FP8 entries). ===== + { match: { hw: "b300", variant: "default", quant: "bf16", strategy: "balanced", nodes: "single" } }, + { match: { hw: "b300", variant: "default", quant: "nvfp4", strategy: "balanced", nodes: "single" } }, + { match: { hw: "gb200", variant: "default", quant: "bf16", strategy: "balanced", nodes: "single" } }, + { match: { hw: "gb200", variant: "default", quant: "nvfp4", strategy: "balanced", nodes: "single" } }, + { match: { hw: "gb300", variant: "default", quant: "bf16", strategy: "balanced", nodes: "single" } }, + { match: { hw: "gb300", variant: "default", quant: "nvfp4", strategy: "balanced", nodes: "single" } }, +]; diff --git a/docs_new/src/snippets/configs/poolside/laguna-m1.jsx b/docs_new/src/snippets/configs/poolside/laguna-m1.jsx new file mode 100644 index 000000000..bfb434171 --- /dev/null +++ b/docs_new/src/snippets/configs/poolside/laguna-m1.jsx @@ -0,0 +1,302 @@ +// Laguna-M.1 (poolside) — config-driven cookbook page. +// Consumed by the shared _deployment.jsx + _playground.jsx engines (no model code there). +// +// Build: M.1 needs SGLang PR #28400 (softplus per-element output gating, MERGED) AND PR #28604 +// (global-attention SWA fix — M.1 is sliding_window=0 / all-global; without it M.1 crashes ~1s +// into any concurrent batch with AssertionError: ... swa_lock_ref=0). Both are merged on main +// (verified on a 3f668733 build). The shipped recipe carries NO workaround flag, but the pinned +// build MUST contain BOTH; the #28400-merge wheel 0.5.14.dev20260618+g343aeeef39 is #28400-ONLY +// and crashes under load. Pin dockerImages + benchmarks.sglang_version to a build at a commit +// ≥ #28604. See /sgl-workspace/laguna-m1-day0-checklist.md (step 2) + laguna-m1-results.md. +// +// Model is now natively supported (#28400) → NO --trust-remote-code needed. +// +// Hardware: H200 (Hopper) + B200/B300/GB200/GB300 (Blackwell). +// - BF16 runs everywhere. +// - FP8 is HOPPER-ONLY — not compatible with Blackwell (use NVFP4 for low-precision there). +// - NVFP4 is Blackwell-only. +// So the only quant×hw combos with a cell are: H200×{BF16,FP8} and each Blackwell×{BF16,NVFP4}. +// TP: 8-GPU HGX nodes (H200/B200/B300) → --tp 8 (the maintainer's baseline); GB200/GB300 +// (Grace-Blackwell, typically 4-GPU single node) → --tp 4. Adjust --tp to your node size. +// +// Strategy: a SINGLE "Balanced" operating point (maintainer decision — no LL/HT split; an +// earlier TP=8+DP-Attention "high-throughput" idea was dropped: DP-Attention is ~15% SLOWER +// on this GQA model, see laguna-m1-results.md). + +export const config = { + modelName: "Laguna-M.1", + + supportedHardware: ["h200", "b200", "b300", "gb200", "gb300"], + + variants: [ + { id: "default", label: "Default" }, + ], + + quantizations: [ + { id: "bf16", label: "BF16" }, + { id: "fp8", label: "FP8" }, + { id: "nvfp4", label: "NVFP4" }, + ], + + // Single balanced operating point (maintainer decision — no LL/HT split). + strategies: [ + { id: "balanced", label: "Balanced" }, + ], + + nodesOptions: [ + { id: "single", label: "Single Node" }, + ], + + modelNames: { + "default|bf16": "poolside/Laguna-M.1", + "default|fp8": "poolside/Laguna-M.1-FP8", + "default|nvfp4": "poolside/Laguna-M.1-NVFP4", + }, + + placeholders: { + HOST_IP: { target: "command", label: "Bind host", default: "0.0.0.0" }, + PORT: { target: "command", label: "Bind port", default: "30000" }, + HF_TOKEN: { target: "command", label: "HF token (Docker)", default: "" }, + CURL_HOST: { target: "curl", label: "Server host", default: "localhost" }, + CURL_PORT: { target: "curl", label: "Server port", default: "30000" }, + }, + + curl: `curl http://{{CURL_HOST}}:{{CURL_PORT}}/v1/chat/completions \\ +-H 'Content-Type: application/json' \\ +-d '{ "model": "{{MODEL_NAME}}", "messages": [{"role":"user","content":"Hello"}] }'`, + + benchmarkCommands: { + speed: +`python3 -m sglang.bench_serving \\ + --backend sglang \\ + --host {{CURL_HOST}} --port {{CURL_PORT}} \\ + --model {{MODEL_NAME}} \\ + --dataset-name {{DATASET}} \\ + --random-input-len {{ISL}} --random-output-len {{OSL}} \\ + --num-prompts {{NUM_PROMPTS}} --max-concurrency {{MAX_CONCURRENCY}}`, + // GSM8K sanity is the REQUIRED minimum on every verified cell (cookbook_guide §3); AIME 25 + // (thinking ON) is the harder accuracy check (model_support_guide). All via sgl-eval. + // NOTE: M.1 needs enable_thinking, not sgl-eval's --thinking key (which the template ignores) + // — thinking evals were run via the enable_thinking wrapper (laguna-m1-results.md). + accuracy: { + gsm8k_pct: +`# pip install git+https://github.com/sgl-project/sgl-eval +sgl-eval run gsm8k \\ + --base-url http://{{CURL_HOST}}:{{CURL_PORT}}/v1 \\ + --num-threads 128`, + aime25_pct: +`# pip install git+https://github.com/sgl-project/sgl-eval +sgl-eval run aime25 --thinking \\ + --base-url http://{{CURL_HOST}}:{{CURL_PORT}}/v1 \\ + --num-threads 32`, + }, + numPromptsByConc: { 1: 8, 16: 32, 64: 128, 128: 256, 256: 512, 1024: 2048, 4096: 4096 }, + }, + + // Accuracy here varies by QUANT (BF16 vs NVFP4), so real numbers live per-cell in + // laguna-m1-benchmarks.jsx (which overrides this). Keep null = no variant-wide default. + defaultAccuracy: { + default: { gsm8k_pct: null, aime25_pct: null }, + }, + + accuracyLabels: [ + ["gsm8k_pct", "GSM8K", "%"], + ["aime25_pct", "AIME 25", "%"], + ], + + // Pinned nightly with the Laguna-M.1 build (PR #28400 + #28604 + #28649; cu13 covers H200 + all Blackwell). + // dev-cu13-618-nightly was generated after the FP8 g_proj fix (#28649) landed, so it serves H200 FP8 as + // well as BF16/NVFP4. (Equivalent pip nightly: 0.5.14.dev20260618+g97e3b8998d.) + dockerImages: { + h200: "lmsysorg/sglang:dev-cu13-618-nightly", + b200: "lmsysorg/sglang:dev-cu13-618-nightly", + b300: "lmsysorg/sglang:dev-cu13-618-nightly", + gb200: "lmsysorg/sglang:dev-cu13-618-nightly", + gb300: "lmsysorg/sglang:dev-cu13-618-nightly", + }, + + github: { + cookbookModel: "poolside/Laguna-M.1", + }, + + playgroundFeatures: { + + // M.1 is global-attention (no SWA); expose TP + DP-Attention. No CP. + // DP-Attention is a Playground experiment only — ~15% slower than plain TP on this GQA + // model (8 KV heads), so it is NOT in the shipped Balanced recipe. + attention: { + knobs: [ + { id: "tp", label: "TP", values: [null, 1, 2, 4, 8] }, + { id: "dpAttn", label: "DP-Attention", values: [null, false, 1, 2, 4, 8], + labels: { "auto": "Auto", "false": "Off" } }, + ], + }, + + // 256-expert top-16 MoE. DeepEP all-to-all + EP degree. + moe: { + backend: { + options: [ + { id: null, label: "Inherited" }, + { id: "deepep", label: "DeepEP", flags: ["--moe-a2a-backend deepep"] }, + ], + }, + ep: { label: "EP", values: [null, 1, 2, 4, 8] }, + }, + + // Reasoning + tool-call parsers (poolside_v1, same family as Laguna-XS.2). ALSO baked into + // every Deploy cell below (the maintainer's baseline carries them). + parsers: { + items: [ + { id: "reasoning", label: "Reasoning Parser", flag: "--reasoning-parser poolside_v1" }, + { id: "toolCall", label: "Tool Call Parser", flag: "--tool-call-parser poolside_v1" }, + ], + }, + }, + + // One Balanced cell per valid (hw × quant): H200×{BF16,FP8}, each Blackwell×{BF16,NVFP4}. + // FP8 is Hopper-only (no Blackwell FP8 cell — those combos grey out). Baseline recipe + // (parsers poolside_v1, NO --trust-remote-code) baked into every cell. + // TP: H200/B200/B300 = --tp 8; GB200/GB300 = --tp 4 (4-GPU single node). + // verified:true = ran that exact command on that hardware and it served correctly + passed a + // GSM8K-class eval. Absent verified = yellow/unverified badge. + cells: [ + // ===== NVIDIA Hopper (H200) — BF16 / FP8 (NVFP4 is Blackwell-only) ===== + { + // VERIFIED on 8xH200 (BF16, tp8): GSM8K 93.02% + perf (laguna-m1 H200 results). + match: { hw: "h200", variant: "default", quant: "bf16", strategy: "balanced", nodes: "single" }, + verified: true, + env: [], + flags: [ + "--model-path {{MODEL_NAME}}", + "--reasoning-parser poolside_v1", + "--tool-call-parser poolside_v1", + "--tp 8", + "--host {{HOST_IP}}", + "--port {{PORT}}", + ], + }, + { + // VERIFIED on 8xH200 (FP8, tp8): GSM8K 93.25 + AIME25 0.50; g_proj FP8 quant fix validated. + // FP8 needs the g_proj fix (PR #28649, MERGED) on top of #28400+#28604 — the pinned + // dev-cu13-618-nightly image includes all three. + // FP8 is Hopper-only — it is NOT compatible with Blackwell, so there is no Blackwell FP8 cell. + match: { hw: "h200", variant: "default", quant: "fp8", strategy: "balanced", nodes: "single" }, + verified: true, + env: [], + flags: [ + "--model-path {{MODEL_NAME}}", + "--reasoning-parser poolside_v1", + "--tool-call-parser poolside_v1", + "--tp 8", + "--host {{HOST_IP}}", + "--port {{PORT}}", + ], + }, + // ===== NVIDIA Blackwell B200 (8-GPU HGX) — BF16 / NVFP4 (FP8 is Hopper-only) ===== + { + // VERIFIED on 8xB200 (BF16, tp8): served clean under batched shared-prefix load, + // GSM8K 91.88% + AIME25 66.88% (laguna-m1-results.md). + match: { hw: "b200", variant: "default", quant: "bf16", strategy: "balanced", nodes: "single" }, + verified: true, + env: [], + flags: [ + "--model-path {{MODEL_NAME}}", + "--reasoning-parser poolside_v1", + "--tool-call-parser poolside_v1", + "--tp 8", + "--host {{HOST_IP}}", + "--port {{PORT}}", + ], + }, + { + // VERIFIED on 8xB200 (NVFP4, tp8): GSM8K 89.38% (laguna-m1-results.md). tp8 now matches + // the shipped recipe. + match: { hw: "b200", variant: "default", quant: "nvfp4", strategy: "balanced", nodes: "single" }, + verified: true, + env: [], + flags: [ + "--model-path {{MODEL_NAME}}", + "--reasoning-parser poolside_v1", + "--tool-call-parser poolside_v1", + "--tp 8", + "--host {{HOST_IP}}", + "--port {{PORT}}", + ], + }, + // ===== NVIDIA Blackwell B300 (8-GPU HGX) — BF16 / NVFP4 (UNVERIFIED; FP8 is Hopper-only) ===== + { + match: { hw: "b300", variant: "default", quant: "bf16", strategy: "balanced", nodes: "single" }, + env: [], + flags: [ + "--model-path {{MODEL_NAME}}", + "--reasoning-parser poolside_v1", + "--tool-call-parser poolside_v1", + "--tp 8", + "--host {{HOST_IP}}", + "--port {{PORT}}", + ], + }, + { + match: { hw: "b300", variant: "default", quant: "nvfp4", strategy: "balanced", nodes: "single" }, + env: [], + flags: [ + "--model-path {{MODEL_NAME}}", + "--reasoning-parser poolside_v1", + "--tool-call-parser poolside_v1", + "--tp 8", + "--host {{HOST_IP}}", + "--port {{PORT}}", + ], + }, + // ===== NVIDIA Grace-Blackwell GB200 (4-GPU single node) — BF16 / NVFP4 (UNVERIFIED; FP8 Hopper-only) ===== + { + match: { hw: "gb200", variant: "default", quant: "bf16", strategy: "balanced", nodes: "single" }, + env: [], + flags: [ + "--model-path {{MODEL_NAME}}", + "--reasoning-parser poolside_v1", + "--tool-call-parser poolside_v1", + "--tp 4", + "--host {{HOST_IP}}", + "--port {{PORT}}", + ], + }, + { + match: { hw: "gb200", variant: "default", quant: "nvfp4", strategy: "balanced", nodes: "single" }, + env: [], + flags: [ + "--model-path {{MODEL_NAME}}", + "--reasoning-parser poolside_v1", + "--tool-call-parser poolside_v1", + "--tp 4", + "--host {{HOST_IP}}", + "--port {{PORT}}", + ], + }, + // ===== NVIDIA Grace-Blackwell GB300 (4-GPU single node) — BF16 / NVFP4 (UNVERIFIED; FP8 Hopper-only) ===== + { + match: { hw: "gb300", variant: "default", quant: "bf16", strategy: "balanced", nodes: "single" }, + env: [], + flags: [ + "--model-path {{MODEL_NAME}}", + "--reasoning-parser poolside_v1", + "--tool-call-parser poolside_v1", + "--tp 4", + "--host {{HOST_IP}}", + "--port {{PORT}}", + ], + }, + { + match: { hw: "gb300", variant: "default", quant: "nvfp4", strategy: "balanced", nodes: "single" }, + env: [], + flags: [ + "--model-path {{MODEL_NAME}}", + "--reasoning-parser poolside_v1", + "--tool-call-parser poolside_v1", + "--tp 4", + "--host {{HOST_IP}}", + "--port {{PORT}}", + ], + }, + ], +};