model: support baidu unlimited-ocr (#29186)

Co-authored-by: Mick <mickjagger19@icloud.com>
This commit is contained in:
Aditya Kamat
2026-06-27 23:36:19 +08:00
committed by GitHub
co-authored by Mick
parent b030b1a5f3
commit 1589603114
32 changed files with 2237 additions and 25 deletions
+6
View File
@@ -0,0 +1,6 @@
<svg width="940" height="525" viewBox="0 0 940 525" fill="none" xmlns="http://www.w3.org/2000/svg">
<rect width="940" height="525" fill="none"/>
<text x="470" y="286" text-anchor="middle" font-family="Arial, Helvetica, sans-serif" font-size="128" font-weight="700" letter-spacing="0">
<tspan fill="#2B5BFF">Bai</tspan><tspan fill="#D5001C">du</tspan>
</text>
</svg>

After

Width:  |  Height:  |  Size: 376 B

@@ -0,0 +1,155 @@
---
title: Unlimited-OCR
description: "Deploy Baidu Unlimited-OCR with SGLang for long document OCR using prefill-aware sliding-window attention."
tag: NEW
---
## Deployment
<a id="install" />
<Accordion title="Install SGLang">
Unlimited-OCR support is in [SGLang PR #29186](https://github.com/sgl-project/sglang/pull/29186). Until that PR is included in a tagged SGLang release, install from a build that contains the PR.
<Tabs>
<Tab title="Python (pip / uv)">
```bash Command
pip install -U uv
uv venv --python 3.12 && source .venv/bin/activate
git clone https://github.com/sgl-project/sglang.git
cd sglang
git fetch origin pull/29186/head && git checkout FETCH_HEAD
uv pip install -e python
```
Then run the **Python** output of the command panel below in that environment.
</Tab>
<Tab title="Docker">
```bash Command
docker pull lmsysorg/sglang:dev
```
For how to launch the image, see [Install → Method 3: Using Docker](../../../docs/get-started/install#method-3-using-docker). Substitute the inner `sglang serve ...` with what the command generator below produces.
</Tab>
</Tabs>
</Accordion>
Pick your hardware to generate the launch command. The recipe uses FlashAttention-3 with `--page-size 1`, which is required by the current prefill-aware sliding-window attention path. It also disables radix cache by default, which is the better fit for batch OCR workloads where each request usually contains a different image.
import { Deployment } from "/src/snippets/_deployment.jsx";
import { config } from "/src/snippets/configs/baidu/unlimited-ocr.jsx";
<Deployment config={config} />
## Playground
Use the Playground to adjust tensor parallelism on top of the selected deployment cell.
import { Playground } from "/src/snippets/_playground.jsx";
<Playground config={config} />
## 1. Model Introduction
[Unlimited-OCR](https://huggingface.co/baidu/Unlimited-OCR) is Baidu's multimodal OCR model for document parsing. It uses a sliding-window language backbone, but SGLang serves it with a prefill-aware sliding-window path so image and prompt tokens remain visible during long decode.
The SGLang integration loads the standalone Unlimited-OCR architecture with SAM and CLIP vision encoders plus a DeepSeek-style language backbone. It supports OpenAI-compatible image requests and model-specific image processing options through `images_config`.
**Resources:** [Hugging Face](https://huggingface.co/baidu/Unlimited-OCR) · [SGLang PR #29186](https://github.com/sgl-project/sglang/pull/29186)
## 2. Configuration Tips
- **Attention backend**: use `--attention-backend fa3 --page-size 1`. The prefill-aware SWA page table is built with token-level locations, so page size 1 is required.
- **Radix cache**: keep `--disable-radix-cache` for batch OCR over different documents. If your workload repeatedly asks about the same image and prompt, remove this flag to allow prefix reuse through `PureSWARadixCache`.
- **Long OCR generations**: keep the default prefill-aware SWA path enabled. It retains prompt and image KV while still applying a sliding window to generated text.
- **Custom logit processor**: keep `--enable-custom-logit-processor` in the launch command.
- **Image modes**: pass `images_config.image_mode` per request. Supported modes are `tiny`, `small`, `base`, `large`, and `gundam`. Multiple images are supported only for `tiny`, `small`, and `base`.
- **Default image mode**: when `images_config.image_mode` is omitted, SGLang uses `gundam`.
## 3. Advanced Usage
### 3.1 OCR request
<Accordion title="OCR Example (Python)">
```python Example
from openai import OpenAI
client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")
response = client.chat.completions.create(
model="baidu/Unlimited-OCR",
messages=[
{
"role": "user",
"content": [
{"type": "text", "text": "document parsing."},
{
"type": "image_url",
"image_url": {
"url": "https://example.com/your_document.png"
},
},
],
}
],
max_tokens=2048,
temperature=0,
extra_body={"images_config": {"image_mode": "gundam"}},
)
print(response.choices[0].message.content)
```
</Accordion>
### 3.2 Choosing an image mode
Use lower modes to reduce prefill cost for simple images, and use `gundam` for high-detail document parsing.
<table style={{width: "100%", borderCollapse: "collapse", tableLayout: "fixed"}}>
<thead>
<tr style={{borderBottom: "2px solid #d55816"}}>
<th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700}}>Mode</th>
<th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700}}>Use</th>
<th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700}}>Multiple images</th>
</tr>
</thead>
<tbody>
<tr>
<td style={{padding: "9px 12px"}}><code>tiny</code></td>
<td style={{padding: "9px 12px"}}>Lowest prefill cost.</td>
<td style={{padding: "9px 12px"}}>Yes</td>
</tr>
<tr>
<td style={{padding: "9px 12px"}}><code>small</code></td>
<td style={{padding: "9px 12px"}}>Lightweight OCR requests.</td>
<td style={{padding: "9px 12px"}}>Yes</td>
</tr>
<tr>
<td style={{padding: "9px 12px"}}><code>base</code></td>
<td style={{padding: "9px 12px"}}>Balanced quality and cost.</td>
<td style={{padding: "9px 12px"}}>Yes</td>
</tr>
<tr>
<td style={{padding: "9px 12px"}}><code>large</code></td>
<td style={{padding: "9px 12px"}}>Higher resolution single-image OCR.</td>
<td style={{padding: "9px 12px"}}>No</td>
</tr>
<tr>
<td style={{padding: "9px 12px"}}><code>gundam</code></td>
<td style={{padding: "9px 12px"}}>Default high-detail document parsing mode.</td>
<td style={{padding: "9px 12px"}}>No</td>
</tr>
</tbody>
</table>
@@ -67,6 +67,12 @@ metatags:
href="/cookbook/autoregressive/NVIDIA/Nemotron3-Ultra"
img="/cards/logos/nvidia.png"
/>
<Card
title="Baidu"
mode="card"
href="/cookbook/autoregressive/Baidu/Unlimited-OCR"
img="/cards/logos/baidu.svg"
/>
<Card
title="Ernie"
mode="card"
+10
View File
@@ -201,6 +201,10 @@
"source": "/advanced_features/vlm_query.html",
"destination": "/docs/advanced_features/vlm_query"
},
{
"source": "/basic_usage/unlimited_ocr.html",
"destination": "/cookbook/autoregressive/Baidu/Unlimited-OCR"
},
{
"source": "/basic_usage/deepseek_ocr.html",
"destination": "/cookbook/autoregressive/DeepSeek/DeepSeek-OCR"
@@ -1058,6 +1062,12 @@
"cookbook/autoregressive/NVIDIA/Nemotron3-Super"
]
},
{
"group": "Baidu",
"pages": [
"cookbook/autoregressive/Baidu/Unlimited-OCR"
]
},
{
"group": "Ernie",
"pages": [
@@ -0,0 +1,188 @@
// Unlimited-OCR cookbook config. Consumed by _deployment.jsx + _playground.jsx.
export const config = {
modelName: "Unlimited-OCR",
supportedHardware: ["h100", "h200", "b200", "b300", "gb200", "gb300"],
variants: [{ id: "default", label: "Default" }],
quantizations: [{ id: "default", label: "Default" }],
strategies: [{ id: "balanced", label: "Balanced" }],
nodesOptions: [{ id: "single", label: "Single Node" }],
modelNames: {
"default|default": "baidu/Unlimited-OCR",
},
placeholders: {
HOST_IP: { target: "command", label: "Bind host", default: "0.0.0.0" },
PORT: { target: "command", label: "Bind port", default: "30000" },
HF_TOKEN: {
target: "command",
label: "HF token (Docker)",
default: "<your-hf-token>",
},
CURL_HOST: { target: "curl", label: "Server host", default: "localhost" },
CURL_PORT: { target: "curl", label: "Server port", default: "30000" },
},
curl: `curl http://{{CURL_HOST}}:{{CURL_PORT}}/v1/chat/completions \\
-H 'Content-Type: application/json' \\
-d '{
"model": "{{MODEL_NAME}}",
"messages": [{
"role": "user",
"content": [
{"type": "text", "text": "document parsing."},
{"type": "image_url", "image_url": {"url": "https://example.com/your_document.png"}}
]
}],
"images_config": {"image_mode": "gundam"},
"temperature": 0,
"max_tokens": 2048
}'`,
dockerImages: {
h100: "lmsysorg/sglang:dev",
h200: "lmsysorg/sglang:dev",
b200: "lmsysorg/sglang:dev",
b300: "lmsysorg/sglang:dev",
gb200: "lmsysorg/sglang:dev",
gb300: "lmsysorg/sglang:dev",
},
github: {
cookbookModel: "baidu/Unlimited-OCR",
},
playgroundFeatures: {
attention: {
knobs: [
{ id: "tp", label: "TP", values: [null, 1, 2, 4, 8] },
],
},
},
cells: [
{
match: {
hw: "h100",
variant: "default",
quant: "default",
strategy: "balanced",
nodes: "single",
},
env: [],
flags: [
"--model-path {{MODEL_NAME}}",
"--attention-backend fa3",
"--page-size 1",
"--context-length 32768",
"--enable-custom-logit-processor",
"--disable-radix-cache",
"--host {{HOST_IP}}",
"--port {{PORT}}",
],
},
{
match: {
hw: "h200",
variant: "default",
quant: "default",
strategy: "balanced",
nodes: "single",
},
env: [],
flags: [
"--model-path {{MODEL_NAME}}",
"--attention-backend fa3",
"--page-size 1",
"--context-length 32768",
"--enable-custom-logit-processor",
"--disable-radix-cache",
"--host {{HOST_IP}}",
"--port {{PORT}}",
],
},
{
match: {
hw: "b200",
variant: "default",
quant: "default",
strategy: "balanced",
nodes: "single",
},
env: [],
flags: [
"--model-path {{MODEL_NAME}}",
"--attention-backend fa3",
"--page-size 1",
"--context-length 32768",
"--enable-custom-logit-processor",
"--disable-radix-cache",
"--host {{HOST_IP}}",
"--port {{PORT}}",
],
},
{
match: {
hw: "b300",
variant: "default",
quant: "default",
strategy: "balanced",
nodes: "single",
},
env: [],
flags: [
"--model-path {{MODEL_NAME}}",
"--attention-backend fa3",
"--page-size 1",
"--context-length 32768",
"--enable-custom-logit-processor",
"--disable-radix-cache",
"--host {{HOST_IP}}",
"--port {{PORT}}",
],
},
{
match: {
hw: "gb200",
variant: "default",
quant: "default",
strategy: "balanced",
nodes: "single",
},
env: [],
flags: [
"--model-path {{MODEL_NAME}}",
"--attention-backend fa3",
"--page-size 1",
"--context-length 32768",
"--enable-custom-logit-processor",
"--disable-radix-cache",
"--host {{HOST_IP}}",
"--port {{PORT}}",
],
},
{
match: {
hw: "gb300",
variant: "default",
quant: "default",
strategy: "balanced",
nodes: "single",
},
env: [],
flags: [
"--model-path {{MODEL_NAME}}",
"--attention-backend fa3",
"--page-size 1",
"--context-length 32768",
"--enable-custom-logit-processor",
"--disable-radix-cache",
"--host {{HOST_IP}}",
"--port {{PORT}}",
],
},
],
};