[diffusion] chore: use native ernie prompt enhancer (#34951)
This commit is contained in:
@@ -77,5 +77,5 @@ with open("ernie_image.png", "wb") as f:
|
||||
|
||||
- ERNIE-Image is a text-to-image pipeline; do not pass `--image-path`.
|
||||
- `--performance-mode auto` keeps conservative defaults while preserving explicit user flags.
|
||||
- If the checkpoint includes a PE component, SGLang loads it automatically from `model_index.json`.
|
||||
- If the checkpoint includes a PE component, SGLang loads it automatically with the native Ministral3 runtime. Use `--layerwise-offload-components pe` when the local PE decoder needs to trade latency for lower GPU memory usage.
|
||||
- Treat FSDP, SP/Ulysses/Ring, and TP as explicit benchmark knobs. Measure the target resolution, step count, and GPU type before making them production defaults.
|
||||
|
||||
@@ -5,9 +5,9 @@ description: "Run ERNIE-Image with built-in prompt enhancement or a separate SGL
|
||||
|
||||
## Quick Start
|
||||
|
||||
By default, the PE model is loaded by the diffusion model server, which may not provide optimal performance. For higher performance, the PE model can be deployed as a separate SGLang server. This document uses `baidu/ERNIE-Image` as an example.
|
||||
By default, the diffusion server loads the PE model in-process with SGLang's native Ministral3 implementation. Deploy the PE model as a separate SGLang server when it needs independent resources or scaling. This document uses `baidu/ERNIE-Image` as an example.
|
||||
|
||||
Run the model with the built-in Transformers PE implementation (default):
|
||||
Run the model with the built-in native PE implementation (default):
|
||||
|
||||
```bash
|
||||
# Terminal 1: launch server
|
||||
@@ -27,7 +27,7 @@ curl -X POST http://${HOST}:${PORT}/v1/images/generations \
|
||||
}'
|
||||
```
|
||||
|
||||
Run the model with an SGLang-served PE model (high performance):
|
||||
Run the model with an SGLang-served PE model (high performance):
|
||||
|
||||
```bash
|
||||
# Terminal 1: launch SGLang PE model server
|
||||
@@ -51,9 +51,21 @@ curl -X POST http://${HOST}:${PORT}/v1/images/generations \
|
||||
"guidance_scale": 4.0,
|
||||
"use_pe": true
|
||||
}'
|
||||
```
|
||||
|
||||
## Support matrix
|
||||
```
|
||||
|
||||
For a memory-constrained in-process deployment, the native PE decoder supports
|
||||
layerwise offload:
|
||||
|
||||
```bash
|
||||
sglang serve --model-path baidu/ERNIE-Image \
|
||||
--layerwise-offload-components pe \
|
||||
--port ${PORT}
|
||||
```
|
||||
|
||||
This option streams PE decoder layers from CPU and can increase prompt-enhancement
|
||||
latency. It does not apply when `--pe-server-url` selects an external PE server.
|
||||
|
||||
## Support matrix
|
||||
|
||||
| Model | Built-in PE | SGLang PE Server |
|
||||
|-------|-------------|------------------|
|
||||
|
||||
Reference in New Issue
Block a user