minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

add lfm2.5 to new cookbook. (#27409)

Browse Source
This commit is contained in:
Yi Zhong
2026-06-11 20:08:19 -07:00
committed by GitHub
parent f5c9f88ee2
commit 40894be3c3
6 changed files with 1050 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs_new/cards/logos/liquidai.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 8.7 KiB

docs_new/cookbook/autoregressive/LiquidAI/LFM2.5.mdx
+334
View File
@@ -0,0 +1,334 @@
---
title: LFM2.5
description: "Deploy Liquid AI's LFM2.5 with SGLang — hybrid LIV-convolution + GQA models from 350M to the 8B-A1B MoE, plus LFM2.5-VL vision, with reasoning and Pythonic tool calling."
tag: NEW
---
## Deployment
<a id="install" />
<Accordion title="Install SGLang">
For all methods and hardware platforms, see the [official SGLang installation guide](../../../docs/get-started/install). The two paths below match the **Python / Docker** toggle in the command panel.
<Tabs>
<Tab title="Python (pip / uv)">
```bash Command
pip install --upgrade pip
pip install uv
uv pip install sglang
```
<Note>
LFM2.5 support — the dense / MoE / VL model classes and the `lfm2` tool-call parser — ships on SGLang `main`. If your installed release predates it, install from source or use the Docker dev image.
</Note>
Then run the **Python** output of the command panel below in that environment.
</Tab>
<Tab title="Docker">
LFM2.5 support ships in the pinned SGLang dev image:
```bash Command
docker pull lmsysorg/sglang:dev-cu13
```
For how to launch the image, see [Install → Method 3: Using Docker](../../../docs/get-started/install#method-3-using-docker). A minimal example (substitute the inner `sglang serve ...` with whatever the command generator below produces):
```bash Command
docker run --gpus all \
--shm-size 32g \
-p 30000:30000 \
-v ~/.cache/huggingface:/root/.cache/huggingface \
--env "HF_TOKEN=<your-hf-token>" \
--ipc=host \
lmsysorg/sglang:dev-cu13 \
sglang serve <use args below>
```
</Tab>
</Tabs>
</Accordion>
Every LFM2.5 model runs on a **single GPU (TP=1)** — pick your hardware + model variant to generate the launch command. One recipe covers all operating points per variant; the commands differ only by the parsers a model needs and, on Blackwell, the attention backend. The `lfm2` tool-call parser and each reasoning model's `--reasoning-parser` are already part of the verified command.
import { Deployment } from "/src/snippets/_deployment.jsx";
import { config } from "/src/snippets/configs/LiquidAI/lfm2.5.jsx";
import { benchmarks } from "/src/snippets/configs/LiquidAI/lfm2.5-benchmarks.jsx";
<Deployment config={config} benchmarks={benchmarks} />
<div style={{fontSize: "0.85em", lineHeight: "1.55", color: "#6b7280", margin: "0.5rem 0 1rem 0"}}>
<p style={{margin: "0 0 0.3rem 0"}}><strong>Panel controls</strong> (top of the command box):</p>
<ul style={{margin: 0, paddingLeft: "1.25rem"}}>
<li style={{marginBottom: "0.2rem"}}><strong>Python / Docker</strong> — bare <code>sglang serve …</code> for an existing SGLang env, or a <code>docker run … sglang serve …</code> wrap against the dev image from the <a href="#install">Install SGLang</a> panel above.</li>
<li style={{marginBottom: "0.2rem"}}><strong>⧉ Copy</strong> — copies the current command (with whichever framing is active) to your clipboard.</li>
<li style={{marginBottom: "0.2rem"}}><strong>$ cURL</strong> — a sample request against <code>localhost:30000</code> to confirm the server is up.</li>
<li style={{marginBottom: "0.2rem"}}><strong>⚙ Env</strong> — edits the placeholders (<code>HOST_IP</code>, <code>PORT</code>, <code>HF_TOKEN</code>) the command and cURL share. Persists in localStorage across cookbooks.</li>
<li><strong>Verified / Not Verified</strong> badge — green when the <code>(hw, variant, quant, strategy, nodes)</code> combo has been run end-to-end on real hardware; yellow when auto-derived from a neighbor and not yet re-checked.</li>
</ul>
</div>
## Playground
The Playground is where you experiment with **SGLang features beyond the verified matrix**. The Deploy panel above only emits combinations that have been signed off on; the Playground lets you turn on additional knobs on top of whichever cell the Deploy panel is currently showing. The base is read live from your Deploy selection — only your overrides change.
For LFM2.5 the exposed knob is the **TP override** (every variant is verified at TP=1; TP=2 is available for experimentation on the larger checkpoints). The reasoning and tool-call parsers are not playground toggles here — they are variant-intrinsic and already baked into each verified command.
Lines highlighted **green** are added by your overrides; lines with **red strikethrough** were in the verified base but stripped by an override. When no override differs from the base cell, the playground inherits the base's **Verified** badge; any actual change flips it to **Not Verified** until the new configuration is run end-to-end and submitted back.
import { Playground } from "/src/snippets/_playground.jsx";
<Playground config={config} />
<div style={{fontSize: "0.85em", lineHeight: "1.55", color: "#6b7280", margin: "0.5rem 0 1rem 0"}}>
<p style={{margin: "0 0 0.3rem 0"}}><strong>Panel controls</strong> reuse <strong>Python / Docker</strong> · <strong>⧉ Copy</strong> · <strong>$ cURL</strong> · <strong>⚙ Env</strong> from the Deploy panel, plus one extra:</p>
<ul style={{margin: 0, paddingLeft: "1.25rem"}}>
<li><strong>Submit ↗</strong> — opens a pre-filled GitHub issue so you can land your override combo as a new verified cookbook cell. Shown only while the badge says <strong>Not Verified</strong>; click it once you've actually run the command on your hardware and confirmed it works.</li>
</ul>
</div>
## 1. Model Introduction
LFM2.5 is [Liquid AI](https://www.liquid.ai/)'s family of hybrid models for on-device deployment, built on the LFM2 architecture with extended pre-training and large-scale reinforcement learning, released under the [LFM Open License v1.0](https://huggingface.co/LiquidAI/LFM2.5-8B-A1B/blob/main/LICENSE). The backbone interleaves **double-gated LIV (linear input-varying) convolution blocks** with a small number of **GQA full-attention blocks**: the convolution blocks give linear-time, low-memory sequence mixing while the periodic attention blocks preserve associative recall.
**Key Features:**
- **Hybrid LIV-conv + GQA architecture**: the 1.2B / 350M dense models are 16 layers (10 conv + 6 GQA); the 8B-A1B MoE is 24 layers (18 conv + 6 GQA).
- **Pythonic tool calling**: function calls are emitted as a Python list between `<|tool_call_start|>` and `<|tool_call_end|>` tokens. The `lfm2` tool-call parser surfaces these as standard `message.tool_calls`.
- **Reasoning variants**: the 8B-A1B and 1.2B-Thinking checkpoints emit an explicit `<think>...</think>` chain-of-thought before the answer.
- **Multilingual**: up to 10 languages, with dedicated Japanese chat checkpoints.
- **Vision**: LFM2.5-VL-1.6B pairs the 1.2B language backbone with a SigLIP2 NaFlex 400M encoder for OCR, document understanding, and multilingual vision; LFM2.5-VL-450M pairs the 350M backbone with a SigLIP2 86M encoder for captioning and object detection at edge sizes.
**Available Models:**
<table style={{width: "100%", borderCollapse: "collapse", tableLayout: "fixed"}}>
<colgroup>
<col style={{width: "26%"}} />
<col style={{width: "18%"}} />
<col style={{width: "12%"}} />
<col style={{width: "44%"}} />
</colgroup>
<thead>
<tr style={{borderBottom: "2px solid #d55816"}}>
<th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Model</th>
<th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>Parameters</th>
<th style={{textAlign: "right", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Context</th>
<th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>Role</th>
</tr>
</thead>
<tbody>
<tr>
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}><strong><a href="https://huggingface.co/LiquidAI/LFM2.5-8B-A1B">LFM2.5-8B-A1B</a></strong></td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>8.3B total / 1.5B active (MoE)</td>
<td style={{padding: "9px 12px", textAlign: "right", backgroundColor: "rgba(255,255,255,0.02)"}}>128K</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Reasoning-tuned, agentic / tool use</td>
</tr>
<tr>
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}><strong><a href="https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct">LFM2.5-1.2B-Instruct</a></strong></td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>1.17B (dense)</td>
<td style={{padding: "9px 12px", textAlign: "right", backgroundColor: "rgba(255,255,255,0.02)"}}>32K</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>General instruct, RAG, data extraction</td>
</tr>
<tr>
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}><strong><a href="https://huggingface.co/LiquidAI/LFM2.5-1.2B-Thinking">LFM2.5-1.2B-Thinking</a></strong></td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>1.17B (dense)</td>
<td style={{padding: "9px 12px", textAlign: "right", backgroundColor: "rgba(255,255,255,0.02)"}}>32K</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Reasoning (always-on chain-of-thought)</td>
</tr>
<tr>
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}><strong><a href="https://huggingface.co/LiquidAI/LFM2.5-350M">LFM2.5-350M</a></strong></td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>350M (dense)</td>
<td style={{padding: "9px 12px", textAlign: "right", backgroundColor: "rgba(255,255,255,0.02)"}}>32K</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Compact instruct, structured output</td>
</tr>
<tr>
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}><strong><a href="https://huggingface.co/LiquidAI/LFM2.5-1.2B-JP-202606">LFM2.5-1.2B-JP-202606</a></strong></td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>1.17B (dense)</td>
<td style={{padding: "9px 12px", textAlign: "right", backgroundColor: "rgba(255,255,255,0.02)"}}>32K</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Japanese chat (latest)</td>
</tr>
<tr>
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}><a href="https://huggingface.co/LiquidAI/LFM2.5-1.2B-JP">LFM2.5-1.2B-JP</a></td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>1.17B (dense)</td>
<td style={{padding: "9px 12px", textAlign: "right", backgroundColor: "rgba(255,255,255,0.02)"}}>32K</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Japanese chat (original)</td>
</tr>
<tr>
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}><strong><a href="https://huggingface.co/LiquidAI/LFM2.5-VL-1.6B">LFM2.5-VL-1.6B</a></strong></td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>1.2B LM + SigLIP2 400M</td>
<td style={{padding: "9px 12px", textAlign: "right", backgroundColor: "rgba(255,255,255,0.02)"}}>32K</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Vision-language (OCR, docs, multi-image)</td>
</tr>
<tr>
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}><strong><a href="https://huggingface.co/LiquidAI/LFM2.5-VL-450M">LFM2.5-VL-450M</a></strong></td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>350M LM + SigLIP2 86M</td>
<td style={{padding: "9px 12px", textAlign: "right", backgroundColor: "rgba(255,255,255,0.02)"}}>32K</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Compact vision-language (captioning, object detection)</td>
</tr>
<tr>
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}><a href="https://huggingface.co/LiquidAI/LFM2.5-1.2B-Base">LFM2.5-1.2B-Base</a></td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>1.17B (dense)</td>
<td style={{padding: "9px 12px", textAlign: "right", backgroundColor: "rgba(255,255,255,0.02)"}}>32K</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Pre-trained base (completions only)</td>
</tr>
</tbody>
</table>
The Deploy panel above covers the seven serving variants; **LFM2.5-1.2B-JP** (original — launch without `--tool-call-parser`) and **LFM2.5-1.2B-Base** (no chat template — use the completions endpoint, see [§3.5](#35-base-checkpoint)) launch the same way with the model path swapped.
**License:** [LFM Open License v1.0](https://huggingface.co/LiquidAI/LFM2.5-8B-A1B/blob/main/LICENSE).
**Resources:** [Liquid AI blog](https://www.liquid.ai/blog), [LFM docs](https://docs.liquid.ai/lfm/getting-started/welcome), [LFM2 Technical Report (arXiv:2511.23404)](https://arxiv.org/abs/2511.23404).
## 2. Configuration Tips
- **Reasoning parser**: LFM2.5 reasoning models wrap their chain-of-thought in `<think>...</think>` tags. The command generator passes `--reasoning-parser qwen3` for **8B-A1B** (it emits an explicit opening `<think>`) and `--reasoning-parser qwen3-thinking` for **1.2B-Thinking** (always-on reasoning). This splits the thinking process into `reasoning_content`; without it the chain-of-thought stays inline in `content`.
- **Tool calling**: `--tool-call-parser lfm2` surfaces LFM2.5's Pythonic `<|tool_call_start|>[...]<|tool_call_end|>` calls as standard `message.tool_calls`. The original **1.2B-JP** does not expose tool calling; **Base** has no chat template (use completions).
- **Attention backend on Blackwell (B200/sm100)**: SGLang defaults to the `trtllm_mha` backend on sm100, which is fastest for the dense text models. The **8B-A1B** uses a mamba-style state cache that runs on a page-size-1 backend, so the generator picks `--attention-backend flashinfer` for it. The **VL** language model also uses that state cache and offers two backends: `--attention-backend flashinfer` (keeps prefix/radix caching — what the generator emits), or `--attention-backend trtllm_mha --disable-radix-cache` to run the language model on Blackwell `trtllm_mha` attention (`--disable-radix-cache` lifts the page-size-1 requirement, at the cost of prefix caching). Pair either with `--mm-attention-backend fa4` for the vision tower.
- **VL vision tower (`--mm-attention-backend`)**: on sm100 the `trtllm_mha` default is fastest for text but applies *causal* attention to image tokens. For the VL model, pass `--mm-attention-backend fa4` on B200/B300 (or `fa3` on H100/H200) to restore bidirectional image-token attention and full vision quality.
- **VL multimodal feature transport**: the generator launches the VL models with `SGLANG_USE_CUDA_IPC_TRANSPORT=1 SGLANG_USE_IPC_POOL_HANDLE_CACHE=1`. The first moves the processor→scheduler image-feature handoff onto CUDA IPC instead of serializing tensors between processes; the second ships the pool handle so the scheduler opens it once and caches it, instead of opening a per-item handle on every request. On the image serving workload (1 image @ 720p, measured on VL-1.6B on H100 and B200) this pair is worth roughly 30–50% higher image throughput and 30–40% lower image TTFT vs running without them (measured on VL-1.6B, H100 and B200); decode speed (TPOT) is unaffected.
- **VL-450M memory headroom (`--mem-fraction-static 0.8`)**: with the default memory fraction, the 450M's small weights make SGLang size its static KV/mamba pools to nearly the whole GPU, leaving no headroom for image-feature tensors — under sustained concurrent image load the scheduler can crash with a CUDA OOM in the radix-cache free path. The generator caps `--mem-fraction-static 0.8` for VL-450M; the pool is still far larger than this model ever needs.
- **Mamba scheduling**: LFM2.5 runs on the default `no_buffer` mamba scheduler strategy — no `--mamba-scheduler-strategy` flag is needed. The `extra_buffer` strategy (an overlap-scheduling throughput optimization available for some Gated-DeltaNet hybrids) does not apply to LFM2.5, whose convolution blocks use `mamba_chunk_size=1`.
- **Hardware requirements**: all LFM2.5 models run on a single GPU (TP=1) on either Hopper or Blackwell. The 1.2B / 350M dense models fit in a few GB; the 8B-A1B MoE needs roughly 16 GB for bf16 weights plus KV cache. Multi-GPU tensor parallelism is not required for any variant.
**Recommended sampling parameters** — pass these explicitly on every request. Some LFM2.5 checkpoints do not ship sampling defaults in `generation_config.json`, so the server will not apply them for you. `top_k`, `min_p`, and `repetition_penalty` are not standard OpenAI `chat.completions` fields — pass them through **`extra_body`** and SGLang forwards them to its sampler. Do not set `max_tokens` unless you intend to cap output, as it can truncate a response (or a reasoning model's chain-of-thought) mid-stream.
<table style={{width: "100%", borderCollapse: "collapse", tableLayout: "fixed"}}>
<thead>
<tr style={{borderBottom: "2px solid #d55816"}}>
<th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Model</th>
<th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>temperature</th>
<th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>extra_body (sampler)</th>
</tr>
</thead>
<tbody>
<tr><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>LFM2.5-8B-A1B</td><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>0.2</td><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}><code>{`{"top_k": 80, "repetition_penalty": 1.05}`}</code></td></tr>
<tr><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>LFM2.5-1.2B-Instruct</td><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>0.1</td><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}><code>{`{"top_k": 50, "repetition_penalty": 1.05}`}</code></td></tr>
<tr><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>LFM2.5-1.2B-Thinking</td><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>0.05</td><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}><code>{`{"top_k": 50, "repetition_penalty": 1.05}`}</code></td></tr>
<tr><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>LFM2.5-350M</td><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>0.1</td><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}><code>{`{"top_k": 50, "repetition_penalty": 1.05}`}</code></td></tr>
<tr><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>LFM2.5-1.2B-JP-202606</td><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>0.1</td><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}><code>{`{"top_k": 50, "repetition_penalty": 1.05}`}</code></td></tr>
<tr><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>LFM2.5-1.2B-JP</td><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>0.3</td><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}><code>{`{"min_p": 0.15, "repetition_penalty": 1.05}`}</code></td></tr>
<tr><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>LFM2.5-VL-1.6B (text)</td><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>0.1</td><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}><code>{`{"min_p": 0.15, "repetition_penalty": 1.05}`}</code></td></tr>
<tr><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>LFM2.5-VL-450M (text)</td><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>0.1</td><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}><code>{`{"min_p": 0.15, "repetition_penalty": 1.05}`}</code></td></tr>
<tr><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>LFM2.5-1.2B-Base</td><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>0.3</td><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}><code>{`{"min_p": 0.15, "repetition_penalty": 1.05}`}</code></td></tr>
</tbody>
</table>
## 3. Advanced Usage
### 3.1 Basic Usage
A single client with the recommended sampling presets applied per model (the examples in the following sections reuse this `chat` helper):
```python Example
from openai import OpenAI
client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")
# Non-OpenAI fields (top_k / min_p / repetition_penalty) ride in extra_body.
SAMPLING = {
"LiquidAI/LFM2.5-8B-A1B": dict(temperature=0.2, extra_body={"top_k": 80, "repetition_penalty": 1.05}),
"LiquidAI/LFM2.5-1.2B-Instruct": dict(temperature=0.1, extra_body={"top_k": 50, "repetition_penalty": 1.05}),
"LiquidAI/LFM2.5-1.2B-Thinking": dict(temperature=0.05, extra_body={"top_k": 50, "repetition_penalty": 1.05}),
"LiquidAI/LFM2.5-350M": dict(temperature=0.1, extra_body={"top_k": 50, "repetition_penalty": 1.05}),
"LiquidAI/LFM2.5-1.2B-JP-202606": dict(temperature=0.1, extra_body={"top_k": 50, "repetition_penalty": 1.05}),
"LiquidAI/LFM2.5-VL-1.6B": dict(temperature=0.1, extra_body={"min_p": 0.15, "repetition_penalty": 1.05}),
"LiquidAI/LFM2.5-VL-450M": dict(temperature=0.1, extra_body={"min_p": 0.15, "repetition_penalty": 1.05}),
}
def chat(model, messages, **overrides):
cfg = SAMPLING[model]
body = cfg["extra_body"] | overrides.pop("extra_body", {})
return client.chat.completions.create(
model=model, messages=messages,
temperature=cfg["temperature"], extra_body=body, **overrides,
)
resp = chat(
"LiquidAI/LFM2.5-1.2B-Instruct",
[{"role": "user", "content": "What is C. elegans? Answer in one sentence."}],
)
print(resp.choices[0].message.content)
```
### 3.2 Reasoning
The 8B-A1B and 1.2B-Thinking checkpoints emit chain-of-thought as a built-in behavior. The Deploy panel launches them with the matching `--reasoning-parser`, which separates the thinking process into `reasoning_content`:
```python Example
resp = chat(
"LiquidAI/LFM2.5-8B-A1B",
[{"role": "user", "content": "If a train travels 60 km/h for 2.5 hours, how far does it go?"}],
)
msg = resp.choices[0].message
print("Reasoning:", msg.reasoning_content)
print("Answer:", msg.content)
```
### 3.3 Tool Calling
LFM2.5 writes Pythonic tool calls. With `--tool-call-parser lfm2` (already part of the launch command) they are surfaced as standard `message.tool_calls`:
```python Example
resp = chat(
"LiquidAI/LFM2.5-1.2B-Instruct",
[{"role": "user", "content": "What's the weather in Paris?"}],
tools=[{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get current weather for a location",
"parameters": {
"type": "object",
"properties": {"location": {"type": "string"}},
"required": ["location"],
},
},
}],
)
for call in resp.choices[0].message.tool_calls or []:
print(call.function.name, call.function.arguments)
```
Tool calling is supported on 8B-A1B, 1.2B-Thinking, 1.2B-Instruct, 350M, 1.2B-JP-202606, VL-1.6B, and VL-450M. For the **VL** models it is text-turn-only — do not combine an image and tools in the same turn.
### 3.4 Vision Input
The VL models (VL-1.6B and VL-450M) accept images via standard OpenAI multimodal content blocks. Base64 data URIs (`data:image/jpeg;base64,...`) work in place of a URL:
```python Example
resp = chat(
"LiquidAI/LFM2.5-VL-1.6B",
[{
"role": "user",
"content": [
{"type": "image_url", "image_url": {
"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"}},
{"type": "text", "text": "What is in this image?"},
],
}],
)
print(resp.choices[0].message.content)
```
### 3.5 Base Checkpoint
LFM2.5-1.2B-Base has no chat template — use the completions endpoint:
```python Example
comp = client.completions.create(
model="LiquidAI/LFM2.5-1.2B-Base",
prompt="The capital of France is",
temperature=0.3,
extra_body={"min_p": 0.15, "repetition_penalty": 1.05},
)
print(comp.choices[0].text)
```
docs_new/cookbook/autoregressive/intro.mdx
+6
View File
@@ -37,6 +37,12 @@ metatags:
href="/cookbook/autoregressive/Google/Gemma4"
img="/cards/logos/google.png"
/>
<Card
title="LiquidAI"
mode="card"
href="/cookbook/autoregressive/LiquidAI/LFM2.5"
img="/cards/logos/liquidai.png"
/>
<Card
title="OpenAI"
mode="card"
docs_new/docs.json
+6
View File
@@ -980,6 +980,12 @@
"cookbook/autoregressive/Google/DiffusionGemma"
]
},
{
"group": "LiquidAI",
"pages": [
"cookbook/autoregressive/LiquidAI/LFM2.5"
]
},
{
"group": "OpenAI",
"pages": [
docs_new/src/snippets/configs/LiquidAI/lfm2.5-benchmarks.jsx
+227
View File
@@ -0,0 +1,227 @@
// One entry per cell `match` tuple (same 5 keys as the config cells). Speed
// measured with python3 -m sglang.bench_serving on Modal cloud GPUs (one GPU,
// TP=1): latency = 10 prompts at concurrency 1, throughput = 1000 prompts at
// concurrency 100 (`random` dataset, 1024/1024 token caps). Accuracy comes
// from the config's `defaultAccuracy` (Liquid-AI-reported GPQA / AIME25).
export const benchmarks = [
// ====================================================================
// H100
// ====================================================================
{
match: { hw: "h100", variant: "8b-a1b", quant: "bf16", strategy: "default", nodes: "single" },
sglang_version: "0.0.0.dev1+g631db6c75",
speed: [
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 1, num_prompts: 10 },
ttft_ms: 287.24, tpot_ms: 2.4, tokens_per_sec_per_gpu: 325.11 },
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 100, num_prompts: 1000 },
ttft_ms: 171.72, tpot_ms: 11.87, tokens_per_sec_per_gpu: 7875.37 },
],
},
{
match: { hw: "h100", variant: "instruct", quant: "bf16", strategy: "default", nodes: "single" },
sglang_version: "0.0.0.dev1+g631db6c75",
speed: [
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 1, num_prompts: 10 },
ttft_ms: 18.9, tpot_ms: 2.08, tokens_per_sec_per_gpu: 471.61 },
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 100, num_prompts: 1000 },
ttft_ms: 180.55, tpot_ms: 7, tokens_per_sec_per_gpu: 13049.7 },
],
},
{
match: { hw: "h100", variant: "thinking", quant: "bf16", strategy: "default", nodes: "single" },
sglang_version: "0.0.0.dev1+g631db6c75",
speed: [
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 1, num_prompts: 10 },
ttft_ms: 16.23, tpot_ms: 2.19, tokens_per_sec_per_gpu: 449.06 },
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 100, num_prompts: 1000 },
ttft_ms: 127.45, tpot_ms: 5.31, tokens_per_sec_per_gpu: 17430.9 },
],
},
{
match: { hw: "h100", variant: "350m", quant: "bf16", strategy: "default", nodes: "single" },
sglang_version: "0.0.0.dev1+g631db6c75",
speed: [
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 1, num_prompts: 10 },
ttft_ms: 18.8, tpot_ms: 1.65, tokens_per_sec_per_gpu: 590.39 },
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 100, num_prompts: 1000 },
ttft_ms: 476.85, tpot_ms: 4.26, tokens_per_sec_per_gpu: 18745.3 },
],
},
{
match: { hw: "h100", variant: "jp", quant: "bf16", strategy: "default", nodes: "single" },
sglang_version: "0.0.0.dev1+g631db6c75",
speed: [
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 1, num_prompts: 10 },
ttft_ms: 17.04, tpot_ms: 2.1, tokens_per_sec_per_gpu: 468.74 },
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 100, num_prompts: 1000 },
ttft_ms: 195.67, tpot_ms: 5.05, tokens_per_sec_per_gpu: 17694.7 },
],
},
{
match: { hw: "h100", variant: "vl", quant: "bf16", strategy: "default", nodes: "single" },
sglang_version: "0.0.0.dev1+g631db6c75",
speed: [
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 1, num_prompts: 10 },
ttft_ms: 22.01, tpot_ms: 1.54, tokens_per_sec_per_gpu: 630.21 },
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 100, num_prompts: 1000 },
ttft_ms: 1676.37, tpot_ms: 3.38, tokens_per_sec_per_gpu: 14483.4 },
],
},
{
match: { hw: "h100", variant: "vl-450m", quant: "bf16", strategy: "default", nodes: "single" },
sglang_version: "0.0.0.dev1+g631db6c75",
speed: [
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 1, num_prompts: 10 },
ttft_ms: 26.01, tpot_ms: 1.34, tokens_per_sec_per_gpu: 713.36 },
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 100, num_prompts: 1000 },
ttft_ms: 1604.2, tpot_ms: 3.38, tokens_per_sec_per_gpu: 14852.1 },
],
},
// ====================================================================
// H200
// ====================================================================
{
match: { hw: "h200", variant: "8b-a1b", quant: "bf16", strategy: "default", nodes: "single" },
sglang_version: "0.0.0.dev1+g631db6c75",
speed: [
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 1, num_prompts: 10 },
ttft_ms: 48.8, tpot_ms: 2.23, tokens_per_sec_per_gpu: 426.61 },
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 100, num_prompts: 1000 },
ttft_ms: 119.9, tpot_ms: 11.96, tokens_per_sec_per_gpu: 7913.04 },
],
},
{
match: { hw: "h200", variant: "instruct", quant: "bf16", strategy: "default", nodes: "single" },
sglang_version: "0.0.0.dev1+g631db6c75",
speed: [
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 1, num_prompts: 10 },
ttft_ms: 20.97, tpot_ms: 2.2, tokens_per_sec_per_gpu: 445.43 },
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 100, num_prompts: 1000 },
ttft_ms: 601.53, tpot_ms: 5.37, tokens_per_sec_per_gpu: 14874 },
],
},
{
match: { hw: "h200", variant: "thinking", quant: "bf16", strategy: "default", nodes: "single" },
sglang_version: "0.0.0.dev1+g631db6c75",
speed: [
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 1, num_prompts: 10 },
ttft_ms: 21.39, tpot_ms: 2.22, tokens_per_sec_per_gpu: 440.08 },
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 100, num_prompts: 1000 },
ttft_ms: 398.87, tpot_ms: 5.58, tokens_per_sec_per_gpu: 15212.9 },
],
},
{
match: { hw: "h200", variant: "350m", quant: "bf16", strategy: "default", nodes: "single" },
sglang_version: "0.0.0.dev1+g631db6c75",
speed: [
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 1, num_prompts: 10 },
ttft_ms: 22.51, tpot_ms: 1.72, tokens_per_sec_per_gpu: 564.53 },
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 100, num_prompts: 1000 },
ttft_ms: 880.23, tpot_ms: 4.37, tokens_per_sec_per_gpu: 15765.2 },
],
},
{
match: { hw: "h200", variant: "jp", quant: "bf16", strategy: "default", nodes: "single" },
sglang_version: "0.0.0.dev1+g631db6c75",
speed: [
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 1, num_prompts: 10 },
ttft_ms: 20.85, tpot_ms: 2.09, tokens_per_sec_per_gpu: 468.85 },
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 100, num_prompts: 1000 },
ttft_ms: 781.82, tpot_ms: 5.23, tokens_per_sec_per_gpu: 14492.3 },
],
},
{
match: { hw: "h200", variant: "vl", quant: "bf16", strategy: "default", nodes: "single" },
sglang_version: "0.0.0.dev1+g631db6c75",
speed: [
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 1, num_prompts: 10 },
ttft_ms: 20.88, tpot_ms: 1.32, tokens_per_sec_per_gpu: 732.43 },
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 100, num_prompts: 1000 },
ttft_ms: 1550.27, tpot_ms: 3.26, tokens_per_sec_per_gpu: 15472 },
],
},
{
match: { hw: "h200", variant: "vl-450m", quant: "bf16", strategy: "default", nodes: "single" },
sglang_version: "0.0.0.dev1+g631db6c75",
speed: [
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 1, num_prompts: 10 },
ttft_ms: 23.48, tpot_ms: 1.2, tokens_per_sec_per_gpu: 798.74 },
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 100, num_prompts: 1000 },
ttft_ms: 1544.41, tpot_ms: 3.14, tokens_per_sec_per_gpu: 15617.7 },
],
},
// ====================================================================
// B200
// ====================================================================
{
match: { hw: "b200", variant: "8b-a1b", quant: "bf16", strategy: "default", nodes: "single" },
sglang_version: "0.0.0.dev1+g631db6c75",
speed: [
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 1, num_prompts: 10 },
ttft_ms: 124.36, tpot_ms: 2, tokens_per_sec_per_gpu: 436.42 },
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 100, num_prompts: 1000 },
ttft_ms: 154.77, tpot_ms: 7.54, tokens_per_sec_per_gpu: 12343.9 },
],
},
{
match: { hw: "b200", variant: "instruct", quant: "bf16", strategy: "default", nodes: "single" },
sglang_version: "0.0.0.dev1+g631db6c75",
speed: [
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 1, num_prompts: 10 },
ttft_ms: 11.22, tpot_ms: 1.19, tokens_per_sec_per_gpu: 818.26 },
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 100, num_prompts: 1000 },
ttft_ms: 1223.9, tpot_ms: 2.19, tokens_per_sec_per_gpu: 21137.2 },
],
},
{
match: { hw: "b200", variant: "thinking", quant: "bf16", strategy: "default", nodes: "single" },
sglang_version: "0.0.0.dev1+g631db6c75",
speed: [
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 1, num_prompts: 10 },
ttft_ms: 11.12, tpot_ms: 1.19, tokens_per_sec_per_gpu: 818.62 },
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 100, num_prompts: 1000 },
ttft_ms: 1230.34, tpot_ms: 2.18, tokens_per_sec_per_gpu: 21121.5 },
],
},
{
match: { hw: "b200", variant: "350m", quant: "bf16", strategy: "default", nodes: "single" },
sglang_version: "0.0.0.dev1+g631db6c75",
speed: [
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 1, num_prompts: 10 },
ttft_ms: 12.18, tpot_ms: 0.91, tokens_per_sec_per_gpu: 1065.73 },
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 100, num_prompts: 1000 },
ttft_ms: 1177.6, tpot_ms: 1.92, tokens_per_sec_per_gpu: 22636.7 },
],
},
{
match: { hw: "b200", variant: "jp", quant: "bf16", strategy: "default", nodes: "single" },
sglang_version: "0.0.0.dev1+g631db6c75",
speed: [
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 1, num_prompts: 10 },
ttft_ms: 11.98, tpot_ms: 1.19, tokens_per_sec_per_gpu: 817.6 },
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 100, num_prompts: 1000 },
ttft_ms: 1367.79, tpot_ms: 2.27, tokens_per_sec_per_gpu: 19794.3 },
],
},
{
match: { hw: "b200", variant: "vl", quant: "bf16", strategy: "default", nodes: "single" },
sglang_version: "0.0.0.dev1+g631db6c75",
speed: [
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 1, num_prompts: 10 },
ttft_ms: 11.55, tpot_ms: 1.22, tokens_per_sec_per_gpu: 807.13 },
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 100, num_prompts: 1000 },
ttft_ms: 935.24, tpot_ms: 2.34, tokens_per_sec_per_gpu: 23135.8 },
],
},
{
match: { hw: "b200", variant: "vl-450m", quant: "bf16", strategy: "default", nodes: "single" },
sglang_version: "0.0.0.dev1+g631db6c75",
speed: [
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 1, num_prompts: 10 },
ttft_ms: 12.09, tpot_ms: 0.92, tokens_per_sec_per_gpu: 1053.2 },
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 100, num_prompts: 1000 },
ttft_ms: 939.41, tpot_ms: 2.25, tokens_per_sec_per_gpu: 23880.7 },
],
},
];
docs_new/src/snippets/configs/LiquidAI/lfm2.5.jsx
+477
View File
@@ -0,0 +1,477 @@
// Single `export const config` literal — no spreads/calls/IIFE (Mintlify re-evals at hydration).
// Cells are denormalized: no `--nnodes`/`--node-rank`/`--dist-init-addr`/`--host`/`--port` literals — engine injects them.
//
// LFM2.5 note: every variant runs on ONE GPU (TP=1), so the matrix is
// hw × variant only (single quant / strategy / nodes). The reasoning parser is
// variant-intrinsic (`qwen3` for 8B-A1B, `qwen3-thinking` for 1.2B-Thinking) and
// the `lfm2` tool-call parser is part of the recommended launch, so both are baked
// into the verified cells rather than exposed as a Parsers playground axis (a
// single axis item cannot carry a per-variant flag).
export const config = {
modelName: "LFM2.5",
supportedHardware: ["h100", "h200", "b200"],
variants: [
{ id: "8b-a1b", label: "8B-A1B", subtitle: "8.3B MoE · reasoning" },
{ id: "instruct", label: "1.2B Instruct", subtitle: "1.17B dense" },
{ id: "thinking", label: "1.2B Thinking", subtitle: "1.17B · reasoning" },
{ id: "350m", label: "350M", subtitle: "dense" },
{ id: "jp", label: "1.2B JP", subtitle: "Japanese" },
{ id: "vl", label: "VL 1.6B", subtitle: "vision" },
{ id: "vl-450m", label: "VL 450M", subtitle: "vision · compact" },
],
quantizations: [
{ id: "bf16", label: "BF16" },
],
strategies: [
{ id: "default", label: "Default" },
],
nodesOptions: [
{ id: "single", label: "Single Node" },
],
modelNames: {
"8b-a1b|bf16": "LiquidAI/LFM2.5-8B-A1B",
"instruct|bf16": "LiquidAI/LFM2.5-1.2B-Instruct",
"thinking|bf16": "LiquidAI/LFM2.5-1.2B-Thinking",
"350m|bf16": "LiquidAI/LFM2.5-350M",
"jp|bf16": "LiquidAI/LFM2.5-1.2B-JP-202606",
"vl|bf16": "LiquidAI/LFM2.5-VL-1.6B",
"vl-450m|bf16": "LiquidAI/LFM2.5-VL-450M",
},
placeholders: {
HOST_IP: { target: "command", label: "Bind host", default: "0.0.0.0" },
PORT: { target: "command", label: "Bind port", default: "30000" },
HF_TOKEN: { target: "command", label: "HF token (Docker)", default: "<your-hf-token>" },
CURL_HOST: { target: "curl", label: "Server host", default: "localhost" },
CURL_PORT: { target: "curl", label: "Server port", default: "30000" },
},
curl: `curl http://{{CURL_HOST}}:{{CURL_PORT}}/v1/chat/completions \\
-H 'Content-Type: application/json' \\
-d '{ "model": "{{MODEL_NAME}}", "messages": [{"role":"user","content":"Hello"}] }'`,
// Reproduce commands for the Benchmark card's "⚡ Reproduce" modal.
benchmarkCommands: {
speed:
`python3 -m sglang.bench_serving \\
--backend sglang \\
--host {{CURL_HOST}} --port {{CURL_PORT}} \\
--model {{MODEL_NAME}} \\
--dataset-name {{DATASET}} \\
--random-input-len {{ISL}} --random-output-len {{OSL}} \\
--num-prompts {{NUM_PROMPTS}} --max-concurrency {{MAX_CONCURRENCY}}`,
accuracy: {
gsm8k_pct:
`# To install sgl-eval: pip install git+https://github.com/sgl-project/sgl-eval
sgl-eval run gsm8k \\
--base-url http://{{CURL_HOST}}:{{CURL_PORT}}/v1 \\
--model {{MODEL_NAME}} \\
--num-threads 128`,
gpqa_pct:
`# To install sgl-eval: pip install git+https://github.com/sgl-project/sgl-eval
# GPQA's HF dataset (Idavidrein/gpqa) is gated — accept its terms with your HF account first.
sgl-eval run gpqa \\
--base-url http://{{CURL_HOST}}:{{CURL_PORT}}/v1 \\
--model {{MODEL_NAME}} \\
--num-threads 128`,
mmlu_pct:
`# To install sgl-eval: pip install git+https://github.com/sgl-project/sgl-eval
sgl-eval run mmlu \\
--base-url http://{{CURL_HOST}}:{{CURL_PORT}}/v1 \\
--model {{MODEL_NAME}} \\
--num-threads 128`,
aime25_pct:
`# To install sgl-eval: pip install git+https://github.com/sgl-project/sgl-eval
sgl-eval run aime25 \\
--base-url http://{{CURL_HOST}}:{{CURL_PORT}}/v1 \\
--model {{MODEL_NAME}} \\
--num-threads 128`,
mmmu_pct:
`python3 -m sglang.test.run_eval --eval-name mmmu \\
--host {{CURL_HOST}} --port {{CURL_PORT}} \\
--model {{MODEL_NAME}} \\
--num-examples 900 --num-threads 128 --max-tokens 2048 \\
--temperature 0.1 --min-p 0.15`,
},
numPromptsByConc: { 1: 10, 16: 32, 64: 128, 100: 1000, 256: 512 },
},
// The eval set rendered in the benchmark card + "⚡ Reproduce" (the engine
// ships no default — every config declares its own).
accuracyLabels: [
["gpqa_pct", "GPQA Diamond", "%"],
["aime25_pct", "AIME25", "%"],
["gsm8k_pct", "GSM8K (1-shot)", "%"],
["mmlu_pct", "MMLU", "%"],
["mmmu_pct", "MMMU (val)", "%"],
],
// Per-variant accuracy applied to every cell. ALL values are MEASURED through
// SGLang (B200, dev-cu13) with the exact commands in the Reproduce modal:
// gsm8k / gpqa / aime25 / mmlu via sgl-eval (registry defaults — gpqa pass@1
// avg-of-8, aime25 avg-of-16, gsm8k + mmlu single-shot), mmmu via
// sglang.test.run_eval (900 examples, card sampling). They agree with the
// LiquidAI model-card numbers within a few points where both exist; see the
// model cards for Liquid's own reported suite (IFEval / MATH500 / BFCL / ...).
defaultAccuracy: {
"8b-a1b": { mmlu_pct: 76.61, gsm8k_pct: 91.96, gpqa_pct: 52.27, aime25_pct: 45.21 },
thinking: { mmlu_pct: 63.2, gsm8k_pct: 86.35, gpqa_pct: 39.08, aime25_pct: 27.08 },
instruct: { mmlu_pct: 60.33, gsm8k_pct: 75.13, gpqa_pct: 34.41, aime25_pct: 9.58 },
"350m": { mmlu_pct: 40.69, gsm8k_pct: 30.63, gpqa_pct: 28.35 },
vl: { mmmu_pct: 39.12 },
"vl-450m": { mmmu_pct: 30.56 },
},
// LFM2.5 support (model classes + the `lfm2` tool-call parser) ships in the
// SGLang dev image; not yet in a tagged release.
dockerImages: {
h100: "lmsysorg/sglang:dev-cu13",
h200: "lmsysorg/sglang:dev-cu13",
b200: "lmsysorg/sglang:dev-cu13",
},
// Pre-selects the issue template's `model` dropdown on "Submit verified cell".
github: {
cookbookModel: "LiquidAI/lfm2.5",
},
playgroundFeatures: {
// TP override only: every variant fits on (and is verified at) TP=1; TP=2 is
// exposed for experimentation on the larger checkpoints. No Parsers axis —
// see the header note (parsers are variant-intrinsic and live in the cells).
attention: {
knobs: [
{ id: "tp", label: "TP", values: [null, 1, 2] },
],
},
},
cells: [
// ====================================================================
// H100 (sm90) — default attention backend; parsers per variant
// ====================================================================
{
match: { hw: "h100", variant: "8b-a1b", quant: "bf16", strategy: "default", nodes: "single" },
verified: true,
env: [],
flags: [
"--trust-remote-code",
"--model-path {{MODEL_NAME}}",
"--tp 1",
"--reasoning-parser qwen3",
"--tool-call-parser lfm2",
"--host {{HOST_IP}}",
"--port {{PORT}}",
],
},
{
match: { hw: "h100", variant: "instruct", quant: "bf16", strategy: "default", nodes: "single" },
verified: true,
env: [],
flags: [
"--trust-remote-code",
"--model-path {{MODEL_NAME}}",
"--tp 1",
"--tool-call-parser lfm2",
"--host {{HOST_IP}}",
"--port {{PORT}}",
],
},
{
match: { hw: "h100", variant: "thinking", quant: "bf16", strategy: "default", nodes: "single" },
verified: true,
env: [],
flags: [
"--trust-remote-code",
"--model-path {{MODEL_NAME}}",
"--tp 1",
"--reasoning-parser qwen3-thinking",
"--tool-call-parser lfm2",
"--host {{HOST_IP}}",
"--port {{PORT}}",
],
},
{
match: { hw: "h100", variant: "350m", quant: "bf16", strategy: "default", nodes: "single" },
verified: true,
env: [],
flags: [
"--trust-remote-code",
"--model-path {{MODEL_NAME}}",
"--tp 1",
"--tool-call-parser lfm2",
"--host {{HOST_IP}}",
"--port {{PORT}}",
],
},
{
match: { hw: "h100", variant: "jp", quant: "bf16", strategy: "default", nodes: "single" },
verified: true,
env: [],
flags: [
"--trust-remote-code",
"--model-path {{MODEL_NAME}}",
"--tp 1",
"--tool-call-parser lfm2",
"--host {{HOST_IP}}",
"--port {{PORT}}",
],
},
{
match: { hw: "h100", variant: "vl", quant: "bf16", strategy: "default", nodes: "single" },
verified: true,
env: [
"SGLANG_USE_CUDA_IPC_TRANSPORT=1",
"SGLANG_USE_IPC_POOL_HANDLE_CACHE=1",
],
flags: [
"--trust-remote-code",
"--model-path {{MODEL_NAME}}",
"--tp 1",
"--tool-call-parser lfm2",
"--host {{HOST_IP}}",
"--port {{PORT}}",
],
},
{
match: { hw: "h100", variant: "vl-450m", quant: "bf16", strategy: "default", nodes: "single" },
verified: true,
env: [
"SGLANG_USE_CUDA_IPC_TRANSPORT=1",
"SGLANG_USE_IPC_POOL_HANDLE_CACHE=1",
],
flags: [
"--trust-remote-code",
"--model-path {{MODEL_NAME}}",
"--tp 1",
"--tool-call-parser lfm2",
"--mem-fraction-static 0.8",
"--host {{HOST_IP}}",
"--port {{PORT}}",
],
},
// ====================================================================
// H200 (sm90) — same hopper recipes as H100
// ====================================================================
{
match: { hw: "h200", variant: "8b-a1b", quant: "bf16", strategy: "default", nodes: "single" },
verified: true,
env: [],
flags: [
"--trust-remote-code",
"--model-path {{MODEL_NAME}}",
"--tp 1",
"--reasoning-parser qwen3",
"--tool-call-parser lfm2",
"--host {{HOST_IP}}",
"--port {{PORT}}",
],
},
{
match: { hw: "h200", variant: "instruct", quant: "bf16", strategy: "default", nodes: "single" },
verified: true,
env: [],
flags: [
"--trust-remote-code",
"--model-path {{MODEL_NAME}}",
"--tp 1",
"--tool-call-parser lfm2",
"--host {{HOST_IP}}",
"--port {{PORT}}",
],
},
{
match: { hw: "h200", variant: "thinking", quant: "bf16", strategy: "default", nodes: "single" },
verified: true,
env: [],
flags: [
"--trust-remote-code",
"--model-path {{MODEL_NAME}}",
"--tp 1",
"--reasoning-parser qwen3-thinking",
"--tool-call-parser lfm2",
"--host {{HOST_IP}}",
"--port {{PORT}}",
],
},
{
match: { hw: "h200", variant: "350m", quant: "bf16", strategy: "default", nodes: "single" },
verified: true,
env: [],
flags: [
"--trust-remote-code",
"--model-path {{MODEL_NAME}}",
"--tp 1",
"--tool-call-parser lfm2",
"--host {{HOST_IP}}",
"--port {{PORT}}",
],
},
{
match: { hw: "h200", variant: "jp", quant: "bf16", strategy: "default", nodes: "single" },
verified: true,
env: [],
flags: [
"--trust-remote-code",
"--model-path {{MODEL_NAME}}",
"--tp 1",
"--tool-call-parser lfm2",
"--host {{HOST_IP}}",
"--port {{PORT}}",
],
},
{
match: { hw: "h200", variant: "vl", quant: "bf16", strategy: "default", nodes: "single" },
verified: true,
env: [
"SGLANG_USE_CUDA_IPC_TRANSPORT=1",
"SGLANG_USE_IPC_POOL_HANDLE_CACHE=1",
],
flags: [
"--trust-remote-code",
"--model-path {{MODEL_NAME}}",
"--tp 1",
"--tool-call-parser lfm2",
"--host {{HOST_IP}}",
"--port {{PORT}}",
],
},
{
match: { hw: "h200", variant: "vl-450m", quant: "bf16", strategy: "default", nodes: "single" },
verified: true,
env: [
"SGLANG_USE_CUDA_IPC_TRANSPORT=1",
"SGLANG_USE_IPC_POOL_HANDLE_CACHE=1",
],
flags: [
"--trust-remote-code",
"--model-path {{MODEL_NAME}}",
"--tp 1",
"--tool-call-parser lfm2",
"--mem-fraction-static 0.8",
"--host {{HOST_IP}}",
"--port {{PORT}}",
],
},
// ====================================================================
// B200 (sm100) — explicit attention backend per variant:
// dense text → trtllm_mha; 8B-A1B + VL use a mamba-style conv state cache
// that needs a page-size-1 backend → flashinfer (VL adds fa4 vision tower)
// ====================================================================
{
match: { hw: "b200", variant: "8b-a1b", quant: "bf16", strategy: "default", nodes: "single" },
verified: true,
env: [],
flags: [
"--trust-remote-code",
"--model-path {{MODEL_NAME}}",
"--tp 1",
"--attention-backend flashinfer",
"--reasoning-parser qwen3",
"--tool-call-parser lfm2",
"--host {{HOST_IP}}",
"--port {{PORT}}",
],
},
{
match: { hw: "b200", variant: "instruct", quant: "bf16", strategy: "default", nodes: "single" },
verified: true,
env: [],
flags: [
"--trust-remote-code",
"--model-path {{MODEL_NAME}}",
"--tp 1",
"--attention-backend trtllm_mha",
"--tool-call-parser lfm2",
"--host {{HOST_IP}}",
"--port {{PORT}}",
],
},
{
match: { hw: "b200", variant: "thinking", quant: "bf16", strategy: "default", nodes: "single" },
verified: true,
env: [],
flags: [
"--trust-remote-code",
"--model-path {{MODEL_NAME}}",
"--tp 1",
"--attention-backend trtllm_mha",
"--reasoning-parser qwen3-thinking",
"--tool-call-parser lfm2",
"--host {{HOST_IP}}",
"--port {{PORT}}",
],
},
{
match: { hw: "b200", variant: "350m", quant: "bf16", strategy: "default", nodes: "single" },
verified: true,
env: [],
flags: [
"--trust-remote-code",
"--model-path {{MODEL_NAME}}",
"--tp 1",
"--attention-backend trtllm_mha",
"--tool-call-parser lfm2",
"--host {{HOST_IP}}",
"--port {{PORT}}",
],
},
{
match: { hw: "b200", variant: "jp", quant: "bf16", strategy: "default", nodes: "single" },
verified: true,
env: [],
flags: [
"--trust-remote-code",
"--model-path {{MODEL_NAME}}",
"--tp 1",
"--attention-backend trtllm_mha",
"--tool-call-parser lfm2",
"--host {{HOST_IP}}",
"--port {{PORT}}",
],
},
{
match: { hw: "b200", variant: "vl", quant: "bf16", strategy: "default", nodes: "single" },
verified: true,
env: [
"SGLANG_USE_CUDA_IPC_TRANSPORT=1",
"SGLANG_USE_IPC_POOL_HANDLE_CACHE=1",
],
flags: [
"--trust-remote-code",
"--model-path {{MODEL_NAME}}",
"--tp 1",
"--attention-backend flashinfer",
"--mm-attention-backend fa4",
"--tool-call-parser lfm2",
"--host {{HOST_IP}}",
"--port {{PORT}}",
],
},
{
match: { hw: "b200", variant: "vl-450m", quant: "bf16", strategy: "default", nodes: "single" },
verified: true,
env: [
"SGLANG_USE_CUDA_IPC_TRANSPORT=1",
"SGLANG_USE_IPC_POOL_HANDLE_CACHE=1",
],
flags: [
"--trust-remote-code",
"--model-path {{MODEL_NAME}}",
"--tp 1",
"--attention-backend flashinfer",
"--mm-attention-backend fa4",
"--tool-call-parser lfm2",
"--mem-fraction-static 0.8",
"--host {{HOST_IP}}",
"--port {{PORT}}",
],
},
],
};