minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

Add LFM2.5-230M to the LFM2.5 cookbook (#29321)

Browse Source
This commit is contained in:
Yi Zhong
2026-06-26 08:27:57 +08:00
committed by GitHub
parent 4b0b0ea55f
commit f83cbc2516
4 changed files with 86 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs_new/cookbook/autoregressive/LiquidAI/LFM2.5.mdx
+12 -3
View File
@@ -156,6 +156,12 @@ This minimal hybrid layout was selected by a hardware-in-the-loop architecture s
<td style={{padding: "9px 12px", textAlign: "right", backgroundColor: "rgba(255,255,255,0.02)"}}>32K</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Compact instruct, structured output</td>
</tr>
<tr>
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}><strong><a href="https://huggingface.co/LiquidAI/LFM2.5-230M">LFM2.5-230M</a></strong></td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>230M (dense)</td>
<td style={{padding: "9px 12px", textAlign: "right", backgroundColor: "rgba(255,255,255,0.02)"}}>32K</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Most compact; data extraction, structured output</td>
</tr>
<tr>
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}><strong><a href="https://huggingface.co/LiquidAI/LFM2.5-1.2B-JP-202606">LFM2.5-1.2B-JP-202606</a></strong></td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>1.17B (dense)</td>
@@ -189,7 +195,7 @@ This minimal hybrid layout was selected by a hardware-in-the-loop architecture s
</tbody>
</table>
The Deploy panel above covers the seven serving variants; **LFM2.5-1.2B-JP** (original — launch without `--tool-call-parser`) and the **Base** repos (pre-trained only, no post-training — see [§3.5](#35-base-checkpoints)) launch the same way with the model path swapped.
The Deploy panel above covers the eight serving variants; **LFM2.5-1.2B-JP** (original — launch without `--tool-call-parser`) and the **Base** repos (pre-trained only, no post-training — see [§3.5](#35-base-checkpoints)) launch the same way with the model path swapped.
**Choosing a variant:**
@@ -197,6 +203,7 @@ The Deploy panel above covers the seven serving variants; **LFM2.5-1.2B-JP** (or
- **1.2B-Thinking** — reasoning-heavy tasks: math, tool use, programming.
- **1.2B-Instruct** — the recommended pick for chat and creative writing.
- **350M** — tool use, data extraction, and structured output; not recommended for math, code, or creative writing.
- **230M** — the most compact checkpoint; same use as the 350M, not for math, code, or creative writing.
**License:** [LFM Open License v1.0](https://huggingface.co/LiquidAI/LFM2.5-8B-A1B/blob/main/LICENSE).
@@ -228,6 +235,7 @@ The Deploy panel above covers the seven serving variants; **LFM2.5-1.2B-JP** (or
<tr><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>LFM2.5-1.2B-Instruct</td><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>0.1</td><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}><code>{`{"top_k": 50, "repetition_penalty": 1.05}`}</code></td></tr>
<tr><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>LFM2.5-1.2B-Thinking</td><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>0.05</td><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}><code>{`{"top_k": 50, "repetition_penalty": 1.05}`}</code></td></tr>
<tr><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>LFM2.5-350M</td><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>0.1</td><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}><code>{`{"top_k": 50, "repetition_penalty": 1.05}`}</code></td></tr>
<tr><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>LFM2.5-230M</td><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>0.1</td><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}><code>{`{"top_k": 50, "repetition_penalty": 1.05}`}</code></td></tr>
<tr><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>LFM2.5-1.2B-JP-202606</td><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>0.1</td><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}><code>{`{"top_k": 50, "repetition_penalty": 1.05}`}</code></td></tr>
<tr><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>LFM2.5-1.2B-JP</td><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>0.3</td><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}><code>{`{"min_p": 0.15, "repetition_penalty": 1.05}`}</code></td></tr>
<tr><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>LFM2.5-VL-1.6B (text)</td><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>0.1</td><td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}><code>{`{"min_p": 0.15, "repetition_penalty": 1.05}`}</code></td></tr>
@@ -253,6 +261,7 @@ SAMPLING = {
"LiquidAI/LFM2.5-1.2B-Instruct": dict(temperature=0.1, extra_body={"top_k": 50, "repetition_penalty": 1.05}),
"LiquidAI/LFM2.5-1.2B-Thinking": dict(temperature=0.05, extra_body={"top_k": 50, "repetition_penalty": 1.05}),
"LiquidAI/LFM2.5-350M": dict(temperature=0.1, extra_body={"top_k": 50, "repetition_penalty": 1.05}),
"LiquidAI/LFM2.5-230M": dict(temperature=0.1, extra_body={"top_k": 50, "repetition_penalty": 1.05}),
"LiquidAI/LFM2.5-1.2B-JP-202606": dict(temperature=0.1, extra_body={"top_k": 50, "repetition_penalty": 1.05}),
"LiquidAI/LFM2.5-VL-1.6B": dict(temperature=0.1, extra_body={"min_p": 0.15, "repetition_penalty": 1.05}),
"LiquidAI/LFM2.5-VL-450M": dict(temperature=0.1, extra_body={"min_p": 0.15, "repetition_penalty": 1.05}),
@@ -312,7 +321,7 @@ for call in resp.choices[0].message.tool_calls or []:
print(call.function.name, call.function.arguments)
```
Tool calling is supported on 8B-A1B, 1.2B-Thinking, 1.2B-Instruct, 350M, 1.2B-JP-202606, VL-1.6B, and VL-450M. For the **VL** models it is text-turn-only — do not combine an image and tools in the same turn.
Tool calling is supported on 8B-A1B, 1.2B-Thinking, 1.2B-Instruct, 350M, 230M, 1.2B-JP-202606, VL-1.6B, and VL-450M. For the **VL** models it is text-turn-only — do not combine an image and tools in the same turn.
### 3.4 Vision Input
@@ -335,7 +344,7 @@ print(resp.choices[0].message.content)
### 3.5 Base Checkpoints
Each size ships a pre-trained Base repo — [LFM2.5-1.2B-Base](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Base), [LFM2.5-350M-Base](https://huggingface.co/LiquidAI/LFM2.5-350M-Base), and [LFM2.5-8B-A1B-Base](https://huggingface.co/LiquidAI/LFM2.5-8B-A1B-Base) — intended for fine-tuning and continued pre-training.
Each size ships a pre-trained Base repo — [LFM2.5-230M-Base](https://huggingface.co/LiquidAI/LFM2.5-230M-Base), [LFM2.5-1.2B-Base](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Base), [LFM2.5-350M-Base](https://huggingface.co/LiquidAI/LFM2.5-350M-Base), and [LFM2.5-8B-A1B-Base](https://huggingface.co/LiquidAI/LFM2.5-8B-A1B-Base) — intended for fine-tuning and continued pre-training.
The repos ship a ChatML-style chat template, so `chat.completions` requests format normally. The checkpoints have no post-training, though — don't expect instruction following. For raw text continuation:
docs_new/docs/supported-models/generative_models.mdx
+1 -1
View File
@@ -254,7 +254,7 @@ in the GitHub search bar.
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>Arcee's foundational MoE Trinity family of models, open weights under Apache 2.0.</td>
</tr>
<tr>
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}><strong>LFM2</strong> (350M, 1.2B)</td>
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}><strong>LFM2</strong> (230M, 350M, 1.2B)</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}><code>LiquidAI/LFM2.5-1.2B-Instruct</code></td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>Liquid AI's hybrid language model combining gated short convolutions with a small number of grouped query attention (GQA) blocks.</td>
</tr>
docs_new/src/snippets/configs/LiquidAI/lfm2.5-benchmarks.jsx
+30
View File
@@ -48,6 +48,16 @@ export const benchmarks = [
ttft_ms: 476.85, tpot_ms: 4.26, tokens_per_sec_per_gpu: 18745.3 },
],
},
{
match: { hw: "h100", variant: "230m", quant: "bf16", strategy: "default", nodes: "single" },
sglang_version: "0.0.0.dev1+g631db6c75",
speed: [
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 1, num_prompts: 10 },
ttft_ms: 23.74, tpot_ms: 1.77, tokens_per_sec_per_gpu: 546.07 },
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 100, num_prompts: 1000 },
ttft_ms: 1128.14, tpot_ms: 4.54, tokens_per_sec_per_gpu: 14280.32 },
],
},
{
match: { hw: "h100", variant: "jp", quant: "bf16", strategy: "default", nodes: "single" },
sglang_version: "0.0.0.dev1+g631db6c75",
@@ -121,6 +131,16 @@ export const benchmarks = [
ttft_ms: 880.23, tpot_ms: 4.37, tokens_per_sec_per_gpu: 15765.2 },
],
},
{
match: { hw: "h200", variant: "230m", quant: "bf16", strategy: "default", nodes: "single" },
sglang_version: "0.0.0.dev1+g631db6c75",
speed: [
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 1, num_prompts: 10 },
ttft_ms: 18.74, tpot_ms: 1.74, tokens_per_sec_per_gpu: 561.43 },
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 100, num_prompts: 1000 },
ttft_ms: 458.07, tpot_ms: 4.47, tokens_per_sec_per_gpu: 17892.27 },
],
},
{
match: { hw: "h200", variant: "jp", quant: "bf16", strategy: "default", nodes: "single" },
sglang_version: "0.0.0.dev1+g631db6c75",
@@ -194,6 +214,16 @@ export const benchmarks = [
ttft_ms: 1177.6, tpot_ms: 1.92, tokens_per_sec_per_gpu: 22636.7 },
],
},
{
match: { hw: "b200", variant: "230m", quant: "bf16", strategy: "default", nodes: "single" },
sglang_version: "0.0.0.dev1+g631db6c75",
speed: [
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 1, num_prompts: 10 },
ttft_ms: 12.28, tpot_ms: 0.84, tokens_per_sec_per_gpu: 1158.08 },
{ workload: { dataset: "random", isl: 1024, osl: 1024, max_concurrency: 100, num_prompts: 1000 },
ttft_ms: 1550.61, tpot_ms: 1.93, tokens_per_sec_per_gpu: 19205.62 },
],
},
{
match: { hw: "b200", variant: "jp", quant: "bf16", strategy: "default", nodes: "single" },
sglang_version: "0.0.0.dev1+g631db6c75",
docs_new/src/snippets/configs/LiquidAI/lfm2.5.jsx
+43
View File
@@ -18,6 +18,7 @@ export const config = {
{ id: "instruct", label: "1.2B Instruct", subtitle: "1.17B dense" },
{ id: "thinking", label: "1.2B Thinking", subtitle: "1.17B · reasoning" },
{ id: "350m", label: "350M", subtitle: "dense" },
{ id: "230m", label: "230M", subtitle: "dense · compact" },
{ id: "jp", label: "1.2B JP", subtitle: "Japanese" },
{ id: "vl", label: "VL 1.6B", subtitle: "vision" },
{ id: "vl-450m", label: "VL 450M", subtitle: "vision · compact" },
@@ -37,6 +38,7 @@ export const config = {
"instruct|bf16": "LiquidAI/LFM2.5-1.2B-Instruct",
"thinking|bf16": "LiquidAI/LFM2.5-1.2B-Thinking",
"350m|bf16": "LiquidAI/LFM2.5-350M",
"230m|bf16": "LiquidAI/LFM2.5-230M",
"jp|bf16": "LiquidAI/LFM2.5-1.2B-JP-202606",
"vl|bf16": "LiquidAI/LFM2.5-VL-1.6B",
"vl-450m|bf16": "LiquidAI/LFM2.5-VL-450M",
@@ -122,6 +124,7 @@ sgl-eval run aime25 \\
thinking: { mmlu_pct: 63.2, gsm8k_pct: 86.35, gpqa_pct: 39.08, aime25_pct: 27.08 },
instruct: { mmlu_pct: 60.33, gsm8k_pct: 75.13, gpqa_pct: 34.41, aime25_pct: 9.58 },
"350m": { mmlu_pct: 40.69, gsm8k_pct: 30.63, gpqa_pct: 28.35 },
"230m": { mmlu_pct: 38.45, gsm8k_pct: 31.84, gpqa_pct: 27.78 },
vl: { mmmu_pct: 39.12 },
"vl-450m": { mmmu_pct: 30.56 },
},
@@ -208,6 +211,19 @@ sgl-eval run aime25 \\
"--port {{PORT}}",
],
},
{
match: { hw: "h100", variant: "230m", quant: "bf16", strategy: "default", nodes: "single" },
verified: true,
env: [],
flags: [
"--trust-remote-code",
"--model-path {{MODEL_NAME}}",
"--tp 1",
"--tool-call-parser lfm2",
"--host {{HOST_IP}}",
"--port {{PORT}}",
],
},
{
match: { hw: "h100", variant: "jp", quant: "bf16", strategy: "default", nodes: "single" },
verified: true,
@@ -312,6 +328,19 @@ sgl-eval run aime25 \\
"--port {{PORT}}",
],
},
{
match: { hw: "h200", variant: "230m", quant: "bf16", strategy: "default", nodes: "single" },
verified: true,
env: [],
flags: [
"--trust-remote-code",
"--model-path {{MODEL_NAME}}",
"--tp 1",
"--tool-call-parser lfm2",
"--host {{HOST_IP}}",
"--port {{PORT}}",
],
},
{
match: { hw: "h200", variant: "jp", quant: "bf16", strategy: "default", nodes: "single" },
verified: true,
@@ -422,6 +451,20 @@ sgl-eval run aime25 \\
"--port {{PORT}}",
],
},
{
match: { hw: "b200", variant: "230m", quant: "bf16", strategy: "default", nodes: "single" },
verified: true,
env: [],
flags: [
"--trust-remote-code",
"--model-path {{MODEL_NAME}}",
"--tp 1",
"--attention-backend trtllm_mha",
"--tool-call-parser lfm2",
"--host {{HOST_IP}}",
"--port {{PORT}}",
],
},
{
match: { hw: "b200", variant: "jp", quant: "bf16", strategy: "default", nodes: "single" },
verified: true,