[diffusion] feat: support --served-model-name in sglang serve (#34228)

Co-authored-by: TobyMint <tobymint@users.noreply.github.com>
Co-authored-by: Mick <mickjagger19@icloud.com>
This commit is contained in:
TobyMint
2026-08-10 22:27:09 +08:00
committed by GitHub
co-authored by TobyMint Mick
parent f5f0c3ee7a
commit d07ac32d05
13 changed files with 198 additions and 25 deletions
+1
View File
@@ -75,6 +75,7 @@ Use `sglang generate --help` and `sglang serve --help` for the full argument lis
### Model and runtime
- `--model-path {MODEL}`: model path or Hugging Face model ID
- `--served-model-name {NAME}`: stable model name exposed by serving APIs. Defaults to `--model-id` when set, otherwise `--model-path`.
- `--model-variant {NAME}`: semantic checkpoint variant to load when one model repository contains multiple weight partitions. The pipeline maps this stable name to the repository layout before loading; for example, MiniMax-H3 accepts `fl2va` and `ref2va`. This is a server/load-time choice, unlike a request's `task`.
- `--model-subfolder {PATH}`: advanced direct override for a component subfolder inside the model repository. Prefer `--model-variant` when the pipeline exposes semantic routing. If both are supplied, they must resolve to the same weight partition.
- `--lora-path {PATH}` and `--lora-nickname {NAME}`: load a LoRA adapter
+45 -11
View File
@@ -18,6 +18,7 @@ Launch the server using the `sglang serve` command.
```bash
SERVER_ARGS=(
--model-path Wan-AI/Wan2.1-T2V-1.3B-Diffusers
--served-model-name wan-t2v
--text-encoder-cpu-offload
--pin-cpu-memory
--num-gpus 4
@@ -30,34 +31,67 @@ sglang serve "${SERVER_ARGS[@]}"
```
- **--model-path**: Path to the model or model ID.
- **--served-model-name**: Stable model name exposed by the serving APIs. It defaults to `--model-id` when set, otherwise `--model-path`.
- **--port**: HTTP port to listen on (default: `30000`).
**Get Model Information**
### Served model name
**Endpoint:** `GET /models`
`--served-model-name` separates the public API identity from the checkpoint location. This is useful when replicas use different local mount paths or when a gateway needs one stable model name:
Returns information about the model served by this server, including model path, task type, pipeline configuration, and precision settings.
```bash
sglang serve \
--model-path /models/Wan2.1-T2V-1.3B-Diffusers \
--served-model-name wan-t2v \
--port 30010
```
`--model-id` is not a free-form deployment alias: it selects the registered model configuration for checkpoints whose local path cannot be identified. `--served-model-name` only controls the name exposed by serving APIs. When both are set, the served name takes precedence for API responses.
### Discover the served model
**Endpoint:** `GET /v1/models`
Returns the public model name together with diffusion-specific runtime information.
**Curl Example:**
```bash curl
curl -sS -X GET "http://localhost:30010/models"
curl -sS "http://localhost:30010/v1/models"
```
**Response Example:**
```json
{
"model_path": "Wan-AI/Wan2.1-T2V-1.3B-Diffusers",
"task_type": "T2V",
"pipeline_name": "wan_pipeline",
"pipeline_class": "WanPipeline",
"num_gpus": 4,
"dit_precision": "bf16",
"vae_precision": "fp16"
"object": "list",
"data": [
{
"id": "wan-t2v",
"object": "model",
"created": 1786348800,
"owned_by": "sglang",
"root": "wan-t2v",
"parent": null,
"max_model_len": null,
"num_gpus": 4,
"task_type": "T2V",
"dit_precision": "bf16",
"vae_precision": "fp16",
"pipeline_name": "WanPipeline",
"pipeline_class": "WanPipeline"
}
]
}
```
Retrieve the same model by its served name:
```bash curl
curl -sS "http://localhost:30010/v1/models/wan-t2v"
```
`GET /server_info` also reports `served_model_name` for gateway discovery. Video and action responses use this name when the request does not provide a model explicitly.
---
## Endpoints
@@ -77,6 +77,22 @@ livenessProbe:
See [Health endpoints](/docs/sglang-diffusion/api/cli#health-endpoints) for the
status-code contract and warmup-mode behavior.
## Stable Model Identity
Use `--served-model-name` when the public model name must remain stable across replicas, hosts, or checkpoint mount paths:
```bash
sglang serve \
--model-path /mnt/checkpoints/Qwen-Image \
--model-id Qwen-Image \
--served-model-name image-production \
--port 30010
```
The resolved public name follows `--served-model-name`, then `--model-id`, then `--model-path`. `--model-id` remains an internal model registry and configuration-resolution hint; it is not a replacement for a deployment alias. The resolved name is exposed through `/server_info` and `/v1/models` and is used by video and action responses when a request does not supply its own model.
See [OpenAI API: Served model name](/docs/sglang-diffusion/api/openai_api#served-model-name) for discovery and retrieval examples.
## Performance Modes
`--performance-mode` applies safe presets without overriding explicit offload, FSDP, or parallelism flags. `auto` is the default. Use `manual` when you need to keep performance-related server args under explicit user control. `--mode` is a short alias.