[Doc] Add LongCat 2.0 FP8 cookbook (#30320)

Co-authored-by: Zijie Xia <zijie.xia@radixark.ai>

docs_new/cookbook/autoregressive/Meituan/LongCat-2.0.mdx

@@ -0,0 +1,197 @@
+---
+title: LongCat-2.0
+description: "Deploy LongCat-2.0-FP8 with SGLang - config-driven recipes for Meituan's 1.6T sparse MoE model on B300, B200, H200, and H20 GPUs."
+tag: NEW
+---
+
+## Deployment
+
+<a id="install" />
+
+<Accordion title="Install SGLang">
+
+For all methods and hardware platforms, see the [official SGLang installation guide](../../../docs/get-started/install). LongCat-2.0 support is on SGLang `main`; use a nightly wheel or rolling nightly Docker image until the next tagged release includes it. The two paths below match the **Python / Docker** toggle in the command panel.
+
+<Tabs>
+
+<Tab title="Python (pip / uv)">
+
+```bash Command
+pip install --upgrade pip
+pip install uv
+
+# Choose the nightly wheel index for your CUDA runtime.
+SGLANG_WHL_INDEX=https://docs.sglang.ai/whl/cu130  # B300 / CUDA 13
+# SGLANG_WHL_INDEX=https://docs.sglang.ai/whl/cu129  # CUDA 12.9
+
+uv pip install --prerelease=allow --extra-index-url "${SGLANG_WHL_INDEX}" "sglang[all]"
+```
+
+Then run the **Python** output of the command panel below in that environment.
+
+</Tab>
+
+<Tab title="Docker">
+
+```bash Command
+# Choose the rolling nightly image for your hardware.
+SGLANG_DOCKER_IMAGE=lmsysorg/sglang:dev-cu13  # B300 / CUDA 13
+# SGLANG_DOCKER_IMAGE=lmsysorg/sglang:dev  # Other supported hardware
+
+docker pull "${SGLANG_DOCKER_IMAGE}"
+```
+
+For how to launch the image, see [Install -> Method 3: Using Docker](../../../docs/get-started/install#method-3-using-docker). Substitute the inner `sglang serve ...` with what the command generator below produces.
+
+</Tab>
+
+</Tabs>
+
+</Accordion>
+
+Pick your hardware + recipe to generate the launch command. LongCat-2.0 currently exposes one model-card-aligned serving strategy:
+
+- **Balanced** - the validated B300 recipe and the 2-node H200/B200/H20 topology use TP/EP parallelism with LongCat sparse attention prefill.
+
+import { Deployment } from "/src/snippets/_deployment.jsx";
+import { config }     from "/src/snippets/configs/meituan-longcat/longcat-2.0.jsx";
+import { benchmarks } from "/src/snippets/configs/meituan-longcat/longcat-2.0-benchmarks.jsx";
+
+<Deployment config={config} benchmarks={benchmarks} />
+
+<Note>
+  The B300 single-node recipe was validated end-to-end with CUDA graph capture enabled. H200, B200, and H20 are shown as 2-node recipes because LongCat-2.0-FP8 needs 16 ranks for those GPU memory profiles.
+</Note>
+
+## Playground
+
+The Playground is where you experiment with **SGLang features beyond the verified matrix**. The Deploy panel above only emits combinations the SGLang team has signed off on; the Playground lets you turn on additional knobs on top of whichever cell the Deploy panel is currently showing.
+
+import { Playground } from "/src/snippets/_playground.jsx";
+
+<Playground config={config} />
+
+## 1. Model Introduction
+
+[LongCat-2.0-FP8](https://huggingface.co/meituan-longcat/LongCat-2.0-FP8) is the FP8 checkpoint of Meituan LongCat-2.0, a large sparse Mixture-of-Experts language model with 1.6T total parameters and about 48B activated parameters per token. It combines LongCat Sparse Attention (LSA), expert parallel MoE layers, and an n-gram/token-table embedding path for serving long-context workloads efficiently.
+
+<table style={{width: "100%", borderCollapse: "collapse", tableLayout: "fixed"}}>
+  <thead>
+    <tr style={{borderBottom: "2px solid #d55816"}}>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700}}>Model</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700}}>Architecture</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700}}>Serving precision</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td style={{padding: "9px 12px"}}><strong><a href="https://huggingface.co/meituan-longcat/LongCat-2.0-FP8">LongCat-2.0-FP8</a></strong></td>
+      <td style={{padding: "9px 12px"}}>Sparse MoE · LongCat Sparse Attention · n-gram embedding</td>
+      <td style={{padding: "9px 12px"}}>FP8 weights, BF16 KV cache</td>
+    </tr>
+  </tbody>
+</table>
+
+**Resources:** [LongCat-2.0-FP8](https://huggingface.co/meituan-longcat/LongCat-2.0-FP8).
+
+## 2. Configuration Tips
+
+- **Remote code.** Use `--trust-remote-code` for the Hugging Face checkpoint.
+- **Topology.** The 8x B300 recipe uses TP=8 and EP=8. H200, B200, and H20 use a 2-node 16 GPU layout with TP=16 and EP=16; the command panel injects the multi-node rank flags for you.
+- **LongCat sparse attention.** Keep `--nsa-prefill-backend fa3` with `--chunked-prefill-size 2048` for the model-card-aligned prefill path.
+- **Memory.** The recipe uses `--kv-cache-dtype bfloat16` and starts at `--mem-fraction-static 0.92`. Tune memory only after the generated command launches cleanly on your cluster.
+- **Weight loading.** `--model-loader-extra-config '{"enable_multithread_load":true,"num_threads":12}'` loads checkpoint shards in parallel and reduces startup time.
+- **FP8 backend selection.** Do not pass `--fp8-gemm-runner-backend` manually. SGLang selects the correct backend for the LongCat FP8 scale layout.
+- **Host, port, and ranks.** Use the command panel environment fields for `HOST_IP`, `PORT`, `NODE0_IP`, and `NODE_RANK` instead of hardcoding them in the recipe.
+
+## 3. Advanced Usage
+
+### 3.1 Test the deployment
+
+<Accordion title="Chat completion example (cURL)">
+
+```bash Command
+curl http://localhost:30000/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -d '{
+    "model": "meituan-longcat/LongCat-2.0-FP8",
+    "messages": [
+      {"role": "user", "content": "A shop has 17 apples and sells 8. Then it buys 6 more. How many apples are there? Answer with only the final number."}
+    ],
+    "max_tokens": 32,
+    "chat_template_kwargs": {"enable_thinking": false}
+  }'
+```
+
+</Accordion>
+
+<Accordion title="Expected output">
+
+```text Output
+15
+```
+
+</Accordion>
+
+<Accordion title="OpenAI-compatible client (Python)">
+
+```python Example
+from openai import OpenAI
+
+client = OpenAI(
+    base_url="http://localhost:30000/v1",
+    api_key="EMPTY",
+)
+
+response = client.chat.completions.create(
+    model="meituan-longcat/LongCat-2.0-FP8",
+    messages=[
+        {
+            "role": "user",
+            "content": "Solve: A shop has 17 apples and sells 8, then buys 6 more. Answer with only the final number.",
+        }
+    ],
+    max_tokens=32,
+    extra_body={"chat_template_kwargs": {"enable_thinking": False}},
+)
+
+print(response.choices[0].message.content)
+```
+
+</Accordion>
+
+<Accordion title="Example output">
+
+```text Output
+15
+```
+
+</Accordion>
+
+## 4. Validation
+
+The B300 recipe was validated with `meituan-longcat/LongCat-2.0-FP8` on 8x B300 using the command generated above.
+
+<table style={{width: "100%", borderCollapse: "collapse", tableLayout: "fixed"}}>
+  <thead>
+    <tr style={{borderBottom: "2px solid #d55816"}}>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700}}>Evaluation</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700}}>Examples</th>
+      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700}}>Accuracy</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td style={{padding: "9px 12px"}}>GSM8K</td>
+      <td style={{padding: "9px 12px"}}>200</td>
+      <td style={{padding: "9px 12px"}}>98.0%</td>
+    </tr>
+    <tr>
+      <td style={{padding: "9px 12px"}}>GSM8K</td>
+      <td style={{padding: "9px 12px"}}>1314</td>
+      <td style={{padding: "9px 12px"}}>95.8904109589041%</td>
+    </tr>
+  </tbody>
+</table>
+
+CUDA graph was enabled, and decode CUDA graph capture completed successfully during serving validation.

docs_new/cookbook/autoregressive/intro.mdx

@@ -31,6 +31,12 @@ metatags:
     href="/cookbook/autoregressive/GLM/GLM-5.2"
     img="/cards/logos/glm.png"
   />
+  <Card
+    title="Meituan"
+    mode="card"
+    href="/cookbook/autoregressive/Meituan/LongCat-2.0"
+    img="/cards/logos/meituan.png"
+  />
   <Card
     title="Google"
     mode="card"

docs_new/docs.json

@@ -1018,6 +1018,12 @@
                       "cookbook/autoregressive/GLM/GLM-4.5V"
                     ]
                   },
+                  {
+                    "group": "Meituan",
+                    "pages": [
+                      "cookbook/autoregressive/Meituan/LongCat-2.0"
+                    ]
+                  },
                   {
                     "group": "Google",
                     "pages": [

docs_new/src/snippets/configs/meituan-longcat/longcat-2.0-benchmarks.jsx

@@ -0,0 +1,10 @@
+// LongCat-2.0 per-cell benchmark numbers, keyed by the same `match` tuple as longcat-2.0.jsx cells.
+// See _deployment.jsx for the speed/accuracy schema.
+export const benchmarks = [
+  {
+    match: { hw: "b300", variant: "default", quant: "fp8", strategy: "balanced", nodes: "single" },
+    sglang_version: "SGLang nightly",
+    accuracy: { gsm8k_pct: 95.8904109589041 },
+    notes: "GSM8K was also spot-checked on 200 examples at 98.0%.",
+  },
+];

docs_new/src/snippets/configs/meituan-longcat/longcat-2.0.jsx

@@ -0,0 +1,203 @@
+// Single `export const config` literal - no spreads/calls/IIFE (Mintlify re-evals at hydration).
+// Cells are denormalized: no `--nnodes`/`--node-rank`/`--dist-init-addr` literals - engine injects them.
+
+export const config = {
+  modelName: "LongCat-2.0",
+
+  supportedHardware: ["b300", "b200", "h200", "h20"],
+
+  // Model-specific GPUs the shared HARDWARE_CATALOG does not carry.
+  hardware: [
+    { id: "h20", label: "H20", vram: "96GB", vendor: "nvidia" },
+  ],
+
+  variants: [
+    { id: "default", label: "LongCat-2.0", subtitle: "1.6T MoE · LSA" },
+  ],
+  quantizations: [
+    { id: "fp8", label: "FP8" },
+  ],
+  strategies: [
+    { id: "balanced", label: "Balanced" },
+  ],
+  nodesOptions: [
+    { id: "single",  label: "Single Node" },
+    { id: "multi-2", label: "Multi-Nodes" },
+  ],
+
+  modelNames: {
+    "default|fp8": "meituan-longcat/LongCat-2.0-FP8",
+  },
+
+  placeholders: {
+    HOST_IP:   { target: "command", label: "Bind host",         default: "0.0.0.0"  },
+    PORT:      { target: "command", label: "Bind port",         default: "30000"    },
+    NODE0_IP:  { target: "command", label: "Head node IP",      default: "<node0-ip>"   },
+    NODE_RANK: { target: "command", label: "This node rank",    default: "<node-rank>"  },
+    HF_TOKEN:  { target: "command", label: "HF token (Docker)", default: "<your-hf-token>" },
+    CURL_HOST: { target: "curl",    label: "Server host",       default: "localhost" },
+    CURL_PORT: { target: "curl",    label: "Server port",       default: "30000"     },
+  },
+
+  curl: `curl http://{{CURL_HOST}}:{{CURL_PORT}}/v1/chat/completions \\
+-H 'Content-Type: application/json' \\
+-d '{ "model": "{{MODEL_NAME}}", "messages": [{"role":"user","content":"Hello"}] }'`,
+
+  // Reproduce commands for the Benchmark card's "Reproduce" modal.
+  benchmarkCommands: {
+    speed:
+`python3 -m sglang.bench_serving \\
+  --backend sglang \\
+  --host {{CURL_HOST}} --port {{CURL_PORT}} \\
+  --model {{MODEL_NAME}} \\
+  --dataset-name {{DATASET}} \\
+  --random-input-len {{ISL}} --random-output-len {{OSL}} \\
+  --random-range-ratio 1.0 \\
+  --num-prompts {{NUM_PROMPTS}} --max-concurrency {{MAX_CONCURRENCY}} \\
+  --warmup-requests 64 --flush-cache`,
+    accuracy: {
+      gsm8k_pct:
+`# To install sgl-eval: pip install git+https://github.com/sgl-project/sgl-eval
+sgl-eval run gsm8k \\
+  --base-url http://{{CURL_HOST}}:{{CURL_PORT}}/v1 \\
+  --num-threads 32`,
+    },
+    numPromptsByConc: { 1: 8, 16: 64, 64: 128, 256: 512, 1024: 2048 },
+  },
+
+  accuracyLabels: [
+    ["gsm8k_pct", "GSM8K", "%"],
+  ],
+
+  dockerImages: {
+    b300: "lmsysorg/sglang:dev-cu13",
+    b200: "lmsysorg/sglang:dev",
+    h200: "lmsysorg/sglang:dev",
+    h20:  "lmsysorg/sglang:dev",
+  },
+
+  github: {
+    cookbookModel: "meituan-longcat/LongCat-2.0-FP8",
+  },
+
+  playgroundFeatures: {
+    attention: {
+      knobs: [
+        { id: "tp", label: "TP", values: [
+          null,
+          8,
+          { value: 16, disable: { nodes: ["single"] },
+            disableReason: "TP=16 requires 16 ranks - switch the Deploy panel's Nodes to Multi-Nodes first." },
+        ]},
+      ],
+    },
+
+    moe: {
+      backend: {
+        options: [
+          { id: null,     label: "Inherited" },
+          { id: "deepep", label: "DeepEP", flags: ["--moe-a2a-backend deepep"] },
+        ],
+      },
+      ep: { label: "EP", values: [
+        null,
+        8,
+        { value: 16, disable: { nodes: ["single"] },
+          disableReason: "EP=16 requires 16 ranks - switch the Deploy panel's Nodes to Multi-Nodes first." },
+      ]},
+    },
+
+    hicache: {
+      backends: [
+        { id: null,       label: "Auto" },
+        { id: "file",     label: "File" },
+        { id: "mooncake", label: "Mooncake" },
+      ],
+      writePolicies: [
+        { id: "auto",          label: "Auto" },
+        { id: "write_through", label: "Write-through" },
+        { id: "write_back",    label: "Write-back" },
+      ],
+    },
+  },
+
+  cells: [
+    {
+      match: { hw: "b300", variant: "default", quant: "fp8", strategy: "balanced", nodes: "single" },
+      verified: true,
+      env: [],
+      flags: [
+        "--trust-remote-code",
+        "--model-path {{MODEL_NAME}}",
+        "--tp 8",
+        "--ep 8",
+        "--max-running-requests 64",
+        "--mem-fraction-static 0.92",
+        "--chunked-prefill-size 2048",
+        "--nsa-prefill-backend fa3",
+        "--kv-cache-dtype bfloat16",
+        "--model-loader-extra-config '{\"enable_multithread_load\":true,\"num_threads\":12}'",
+        "--host {{HOST_IP}}",
+        "--port {{PORT}}",
+      ],
+    },
+
+    {
+      match: { hw: "b200", variant: "default", quant: "fp8", strategy: "balanced", nodes: "multi-2" },
+      verified: false,
+      env: [],
+      flags: [
+        "--trust-remote-code",
+        "--model-path {{MODEL_NAME}}",
+        "--tp 16",
+        "--ep 16",
+        "--max-running-requests 64",
+        "--mem-fraction-static 0.92",
+        "--chunked-prefill-size 2048",
+        "--nsa-prefill-backend fa3",
+        "--kv-cache-dtype bfloat16",
+        "--model-loader-extra-config '{\"enable_multithread_load\":true,\"num_threads\":12}'",
+        "--host {{HOST_IP}}",
+        "--port {{PORT}}",
+      ],
+    },
+    {
+      match: { hw: "h200", variant: "default", quant: "fp8", strategy: "balanced", nodes: "multi-2" },
+      verified: false,
+      env: [],
+      flags: [
+        "--trust-remote-code",
+        "--model-path {{MODEL_NAME}}",
+        "--tp 16",
+        "--ep 16",
+        "--max-running-requests 64",
+        "--mem-fraction-static 0.92",
+        "--chunked-prefill-size 2048",
+        "--nsa-prefill-backend fa3",
+        "--kv-cache-dtype bfloat16",
+        "--model-loader-extra-config '{\"enable_multithread_load\":true,\"num_threads\":12}'",
+        "--host {{HOST_IP}}",
+        "--port {{PORT}}",
+      ],
+    },
+    {
+      match: { hw: "h20", variant: "default", quant: "fp8", strategy: "balanced", nodes: "multi-2" },
+      verified: false,
+      env: [],
+      flags: [
+        "--trust-remote-code",
+        "--model-path {{MODEL_NAME}}",
+        "--tp 16",
+        "--ep 16",
+        "--max-running-requests 64",
+        "--mem-fraction-static 0.92",
+        "--chunked-prefill-size 2048",
+        "--nsa-prefill-backend fa3",
+        "--kv-cache-dtype bfloat16",
+        "--model-loader-extra-config '{\"enable_multithread_load\":true,\"num_threads\":12}'",
+        "--host {{HOST_IP}}",
+        "--port {{PORT}}",
+      ],
+    },
+  ],
+};