[diffusion] refactor: scope model-specific API parameters (#35613)

2026-08-28 19:08:30 +08:00
parent d56706459c
commit 803b4fb31c
21 changed files with 732 additions and 317 deletions
@@ -123,7 +123,7 @@ curl -sS -X POST http://127.0.0.1:30010/v1/images/generations \
    "guidance_scale": 6.0,
    "flow_shift": 3.0,
    "seed": 0,
-    "extra_args": {
+    "extra_body": {
      "use_resolution_template": false,
      "guardrails": true
    }
@@ -141,7 +141,7 @@ curl -sS -X POST http://127.0.0.1:30010/v1/images/generations \
    "n": 1,
    "guidance_scale": 1.0,
    "seed": 0,
-    "extra_args": {
+    "extra_body": {
      "use_resolution_template": false,
      "guardrails": true
    }
@@ -390,7 +390,10 @@ Cosmos3 omnimodal fields are accepted as extra JSON fields or multipart form fie
 - `action_view_point`: viewpoint used in the structured action caption.
 - `action_normalization`: dataset normalization mode, such as `quantile`, `meanstd`, or `minmax`.

-Put model-specific compatibility knobs in `extra_params` for video requests, or `extra_args` for image requests:
+Pass model-specific controls through `extra_body` with the OpenAI Python SDK.
+Raw JSON may keep them at the top level; multipart video requests should put
+them in the `extra_params` JSON object. The legacy image `extra_args` container
+remains accepted for compatibility, but new clients should use `extra_body`:

 - `use_duration_template`: whether to append SGLang's generated duration suffix to video prompts.
 - `use_resolution_template`: accepted for vLLM-Omni request compatibility.
@@ -215,6 +215,23 @@ The prediction is clamped to `--auto-duration-min-seconds` /
 `--auto-duration-max-seconds` (default 1–20 s) and snapped to the VAE's temporal
 grid, so the result is always a valid frame count. It overrides `--num-frames`.

+For an online server, pass the same LTX-2.5-only controls through `extra_body`:
+
+```python Python
+from openai import OpenAI
+
+client = OpenAI(api_key="EMPTY", base_url="http://localhost:30010/v1")
+video = client.videos.create(
+    model="Lightricks/LTX-2.5-Diffusers",
+    prompt="A red fox walking through a snowy forest at dawn.",
+    extra_body={
+        "auto_duration": True,
+        "auto_duration_min_seconds": 2.0,
+        "auto_duration_max_seconds": 8.0,
+    },
+)
+```
+
 ### 4.4 Two-stage (higher quality)

 Stage 1 runs at half the requested resolution, the latents are upsampled 2x, and
@@ -290,6 +307,14 @@ sglang serve \
  --load-diffusion-decoder
 ```

+```python Python
+video = client.videos.create(
+    model="Lightricks/LTX-2.5-Diffusers",
+    prompt="A red fox walking through a snowy forest at dawn.",
+    extra_body={"use_diffusion_decoder": True},
+)
+```
+
 This keeps the default server footprint unchanged while still allowing VAE and
 diffusion-decoder requests to share one server. When GPU memory is constrained,
 `--cpu-offload-components diffusion_decoder` keeps the optional decoder on CPU
@@ -0,0 +1,92 @@
+---
+title: LongCat-Image
+metatags:
+    description: "Deploy LongCat-Image with SGLang Diffusion and its native in-process Qwen2.5-VL prompt rewriter."
+---
+
+import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';
+
+<DiffusionModelTags tags={["image", "text-to-image", "prompt rewriting", "Qwen2.5-VL"]} />
+
+## 1. Model Introduction
+
+[LongCat-Image](https://huggingface.co/meituan-longcat/LongCat-Image) is a
+text-to-image model from Meituan. SGLang runs its Qwen2.5-VL prompt rewriter
+in process with the native SGLang runtime before text encoding and denoising.
+
+The native pipeline keeps prompt rewriting and diffusion behind one OpenAI-compatible
+image endpoint. Rewriting is enabled by default for stronger prompt expansion, but
+each request can disable it when lower latency matters more than the rewritten prompt.
+
+## 2. Installation
+
+Install SGLang with the diffusion dependencies:
+
+```bash Command
+pip install -e "python[diffusion]"
+```
+
+For other installation options, see the
+[SGLang Diffusion installation guide](/docs/sglang-diffusion/installation).
+
+## 3. Serve the model
+
+```bash Command
+sglang serve \
+  --model-path meituan-longcat/LongCat-Image \
+  --performance-mode auto \
+  --port 30010
+```
+
+Prompt rewriting is enabled by default for LongCat-Image. It adds an
+autoregressive Qwen2.5-VL pass before diffusion; set
+`enable_prompt_rewrite=false` on a request when lower latency is more important
+than rewritten prompt quality.
+
+## 4. Generate an image
+
+```python Python
+import base64
+from openai import OpenAI
+
+client = OpenAI(api_key="EMPTY", base_url="http://127.0.0.1:30010/v1")
+
+response = client.images.generate(
+    model="meituan-longcat/LongCat-Image",
+    prompt="A quiet bookshop on a rainy evening, warm light in the windows",
+    n=1,
+    response_format="b64_json",
+)
+
+image_bytes = base64.b64decode(response.data[0].b64_json)
+with open("longcat_image.png", "wb") as f:
+    f.write(image_bytes)
+```
+
+To skip prompt rewriting with the OpenAI client, pass the model-specific request
+field through `extra_body`:
+
+```python Python
+response = client.images.generate(
+    model="meituan-longcat/LongCat-Image",
+    prompt="A quiet bookshop on a rainy evening",
+    extra_body={"enable_prompt_rewrite": False},
+)
+```
+
+## 5. Memory placement
+
+Use the unified component-residency selector when the complete pipeline does
+not fit on the accelerator. For example, keep the repeatedly used DiT resident
+while moving auxiliary components to CPU between stages:
+
+```bash Command
+sglang serve \
+  --model-path meituan-longcat/LongCat-Image \
+  --component-residency dit=resident text_encoder=component-offload vae=component-offload \
+  --pin-cpu-memory \
+  --port 30010
+```
+
+See [Component Residency](/docs/sglang-diffusion/api/cli#component-residency)
+for mode semantics and compatibility with the existing CPU-offload flags.
@@ -31,6 +31,12 @@ Image models generate one image request as a bounded denoising job, usually with
    href="/cookbook/diffusion/Qwen-Image/Qwen-Image"
    img="/cards/logos/qwen.png"
  />
+  <Card
+    title="LongCat-Image"
+    mode="card"
+    href="/cookbook/diffusion/LongCat/LongCat-Image"
+    img="/cards/logos/meituan.png"
+  />
  <Card
    title="Z-Image"
    mode="card"