model: support baidu unlimited-ocr (#29186)

Co-authored-by: Mick <mickjagger19@icloud.com>
2026-06-27 23:36:19 +08:00
parent b030b1a5f3
commit 1589603114
32 changed files with 2237 additions and 25 deletions
@@ -0,0 +1,155 @@
+---
+title: Unlimited-OCR
+description: "Deploy Baidu Unlimited-OCR with SGLang for long document OCR using prefill-aware sliding-window attention."
+tag: NEW
+---
+
+## Deployment
+
+<a id="install" />
+
+<Accordion title="Install SGLang">
+
+Unlimited-OCR support is in [SGLang PR #29186](https://github.com/sgl-project/sglang/pull/29186). Until that PR is included in a tagged SGLang release, install from a build that contains the PR.
+
+<Tabs>
+
+<Tab title="Python (pip / uv)">
+
+```bash Command
+pip install -U uv
+uv venv --python 3.12 && source .venv/bin/activate
+
+git clone https://github.com/sgl-project/sglang.git
+cd sglang
+git fetch origin pull/29186/head && git checkout FETCH_HEAD
+uv pip install -e python
+```
+
+Then run the **Python** output of the command panel below in that environment.
+
+</Tab>
+
+<Tab title="Docker">
+
+```bash Command
+docker pull lmsysorg/sglang:dev
+```
+
+For how to launch the image, see [Install → Method 3: Using Docker](../../../docs/get-started/install#method-3-using-docker). Substitute the inner `sglang serve ...` with what the command generator below produces.
+
+</Tab>
+
+</Tabs>
+
+</Accordion>
+
+Pick your hardware to generate the launch command. The recipe uses FlashAttention-3 with `--page-size 1`, which is required by the current prefill-aware sliding-window attention path. It also disables radix cache by default, which is the better fit for batch OCR workloads where each request usually contains a different image.
+
+import { Deployment } from "/src/snippets/_deployment.jsx";
+import { config } from "/src/snippets/configs/baidu/unlimited-ocr.jsx";
+
+<Deployment config={config} />
+
+## Playground
+
+Use the Playground to adjust tensor parallelism on top of the selected deployment cell.
+
+import { Playground } from "/src/snippets/_playground.jsx";
+
+<Playground config={config} />
+
+## 1. Model Introduction
+
+[Unlimited-OCR](https://huggingface.co/baidu/Unlimited-OCR) is Baidu's multimodal OCR model for document parsing. It uses a sliding-window language backbone, but SGLang serves it with a prefill-aware sliding-window path so image and prompt tokens remain visible during long decode.
+
+The SGLang integration loads the standalone Unlimited-OCR architecture with SAM and CLIP vision encoders plus a DeepSeek-style language backbone. It supports OpenAI-compatible image requests and model-specific image processing options through `images_config`.
+
+**Resources:** [Hugging Face](https://huggingface.co/baidu/Unlimited-OCR) · [SGLang PR #29186](https://github.com/sgl-project/sglang/pull/29186)
+
+## 2. Configuration Tips
+
+- **Attention backend**: use `--attention-backend fa3 --page-size 1`. The prefill-aware SWA page table is built with token-level locations, so page size 1 is required.
+- **Radix cache**: keep `--disable-radix-cache` for batch OCR over different documents. If your workload repeatedly asks about the same image and prompt, remove this flag to allow prefix reuse through `PureSWARadixCache`.
+- **Long OCR generations**: keep the default prefill-aware SWA path enabled. It retains prompt and image KV while still applying a sliding window to generated text.
+- **Custom logit processor**: keep `--enable-custom-logit-processor` in the launch command.
+- **Image modes**: pass `images_config.image_mode` per request. Supported modes are `tiny`, `small`, `base`, `large`, and `gundam`. Multiple images are supported only for `tiny`, `small`, and `base`.
+- **Default image mode**: when `images_config.image_mode` is omitted, SGLang uses `gundam`.
+
+## 3. Advanced Usage
+
+### 3.1 OCR request
+
+<Accordion title="OCR Example (Python)">
+
+```python Example
+from openai import OpenAI
+
+client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")
+
+response = client.chat.completions.create(
+    model="baidu/Unlimited-OCR",
+    messages=[
+        {
+            "role": "user",
+            "content": [
+                {"type": "text", "text": "document parsing."},
+                {
+                    "type": "image_url",
+                    "image_url": {
+                        "url": "https://example.com/your_document.png"
+                    },
+                },
+            ],
+        }
+    ],
+    max_tokens=2048,
+    temperature=0,
+    extra_body={"images_config": {"image_mode": "gundam"}},
+)
+
+print(response.choices[0].message.content)
+```
+
+</Accordion>
+
+### 3.2 Choosing an image mode
+
+Use lower modes to reduce prefill cost for simple images, and use `gundam` for high-detail document parsing.
+
+<table style={{width: "100%", borderCollapse: "collapse", tableLayout: "fixed"}}>
+  <thead>
+    <tr style={{borderBottom: "2px solid #d55816"}}>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700}}>Mode</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700}}>Use</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700}}>Multiple images</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td style={{padding: "9px 12px"}}><code>tiny</code></td>
+      <td style={{padding: "9px 12px"}}>Lowest prefill cost.</td>
+      <td style={{padding: "9px 12px"}}>Yes</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px"}}><code>small</code></td>
+      <td style={{padding: "9px 12px"}}>Lightweight OCR requests.</td>
+      <td style={{padding: "9px 12px"}}>Yes</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px"}}><code>base</code></td>
+      <td style={{padding: "9px 12px"}}>Balanced quality and cost.</td>
+      <td style={{padding: "9px 12px"}}>Yes</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px"}}><code>large</code></td>
+      <td style={{padding: "9px 12px"}}>Higher resolution single-image OCR.</td>
+      <td style={{padding: "9px 12px"}}>No</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px"}}><code>gundam</code></td>
+      <td style={{padding: "9px 12px"}}>Default high-detail document parsing mode.</td>
+      <td style={{padding: "9px 12px"}}>No</td>
+    </tr>
+  </tbody>
+</table>
@@ -67,6 +67,12 @@ metatags:
    href="/cookbook/autoregressive/NVIDIA/Nemotron3-Ultra"
    img="/cards/logos/nvidia.png"
  />
+  <Card
+    title="Baidu"
+    mode="card"
+    href="/cookbook/autoregressive/Baidu/Unlimited-OCR"
+    img="/cards/logos/baidu.svg"
+  />
  <Card
    title="Ernie"
    mode="card"