faceless voidandronnie_zheng
44bd359082
Signed-off-by: syd520zy <529477025@qq.com> Co-authored-by: ronnie_zheng <zl19940307@163.com>
185 lines
7.1 KiB
Plaintext
185 lines
7.1 KiB
Plaintext
---
|
|
title: SenseNova-U1.5-8B-MoT
|
|
metatags:
|
|
description: "Run SenseNova-U1.5-8B-MoT text-to-image generation with SGLang's native multimodal generation runtime."
|
|
---
|
|
|
|
import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';
|
|
|
|
<DiffusionModelTags tags={["image", "text-to-image", "Chinese typography", "poster layouts", "native pipeline"]} />
|
|
|
|
## 1. Model Introduction
|
|
|
|
[SenseNova-U1.5-8B-MoT](https://huggingface.co/sensenova/SenseNova-U1.5-8B-MoT) is a text-to-image model in the SenseNova-U1 family. It is useful for high-resolution poster, infographic, and enterprise visual-generation workloads where prompt following, layout structure, and Chinese text rendering are important.
|
|
|
|
Built on [NEO-Unify](https://huggingface.co/blog/sensenova/neo-unify), this release improves composition, color harmony, material rendering, and local visual detail for native high-resolution generation. It also focuses on clearer Chinese and English text rendering, stronger infographic layout structure, and more reliable execution of complex prompts with object counts, spatial relationships, styles, and multiple constraints.
|
|
|
|
## 2. SGLang-diffusion Installation
|
|
|
|
SGLang-diffusion offers multiple installation methods. You can choose the most suitable installation method based on your hardware platform and requirements.
|
|
|
|
Please refer to the [official SGLang-diffusion installation guide](../../../docs/sglang-diffusion/installation) for installation instructions.
|
|
|
|
## 3. Model Deployment
|
|
|
|
### 3.1 Offline Image Generation
|
|
|
|
SenseNova-U1.5-8B-MoT is a text-to-image model. You can run a single offline generation job with `sglang generate`:
|
|
|
|
```bash Command
|
|
sglang generate \
|
|
--model-path sensenova/SenseNova-U1.5-8B-MoT \
|
|
--prompt "A cinematic mountain lake at sunrise, realistic photography." \
|
|
--width 2048 \
|
|
--height 2048 \
|
|
--guidance-scale 4.0 \
|
|
--num-inference-steps 50 \
|
|
--seed 42 \
|
|
--output-file-path output_sensenova_u15.png
|
|
```
|
|
|
|
### 3.2 Recommended Resolutions
|
|
|
|
SenseNova-U1 was trained around the 2K-pixel resolution tier. The SGLang pipeline only supports positive `--width` and `--height` values that are divisible by 32, and generation quality may degrade for untrained size combinations.
|
|
|
|
Recommended resolution tiers:
|
|
|
|
| Aspect ratio | Resolution |
|
|
| --- | --- |
|
|
| 1:1 | `2048 x 2048` |
|
|
| 16:9 / 9:16 | `2720 x 1536` / `1536 x 2720` |
|
|
| 3:2 / 2:3 | `2496 x 1664` / `1664 x 2496` |
|
|
| 4:3 / 3:4 | `2368 x 1760` / `1760 x 2368` |
|
|
| 2:1 / 1:2 | `2880 x 1440` / `1440 x 2880` |
|
|
| 3:1 / 1:3 | `3456 x 1152` / `1152 x 3456` |
|
|
| 9:16 4K | `2304 x 4096` |
|
|
|
|
## 4. API Usage
|
|
|
|
For complete API documentation, please refer to the [official API usage guide](../../../docs/sglang-diffusion/api/openai_api).
|
|
|
|
### 4.1 Generate an Image
|
|
|
|
```python Example
|
|
import base64
|
|
from openai import OpenAI
|
|
|
|
client = OpenAI(api_key="EMPTY", base_url="http://localhost:30000/v1")
|
|
|
|
response = client.images.generate(
|
|
model="sensenova/SenseNova-U1.5-8B-MoT",
|
|
prompt="A clean technology poster with structured layout and clear typography.",
|
|
n=1,
|
|
response_format="b64_json",
|
|
)
|
|
|
|
image_bytes = base64.b64decode(response.data[0].b64_json)
|
|
with open("output_sensenova_u15.png", "wb") as f:
|
|
f.write(image_bytes)
|
|
```
|
|
|
|
## 5. Advanced usage
|
|
|
|
### 5.1 Dynamic request batching
|
|
|
|
Start the server with dynamic batching enabled:
|
|
|
|
```bash Command
|
|
sglang serve \
|
|
--model-path sensenova/SenseNova-U1.5-8B-MoT \
|
|
--port 30000 \
|
|
--batching-max-size 2 \
|
|
--batching-delay-ms 100
|
|
```
|
|
|
|
Submit requests concurrently so they arrive within the batching delay. Requests
|
|
in one batch must use the same resolution, inference steps, guidance settings,
|
|
and output options. Requests with multiple outputs or `think_mode=true` are
|
|
executed sequentially.
|
|
|
|
### 5.2 Ascend NPU optimizations
|
|
|
|
On Ascend NPU, SenseNova-U1 automatically uses fused inference attention,
|
|
RMSNorm, and SwiGLU MLP operators for supported inputs. FIA and SwiGLU fall back
|
|
when their requirements are not met. Run one complete warmup request before
|
|
benchmarking because the fused MLP packs its gate and up projection weights on
|
|
first use.
|
|
|
|
## 6. Benchmark
|
|
|
|
### 6.1 Speedup benchmark
|
|
|
|
#### 6.1.1 Single-run profile
|
|
|
|
<Tabs>
|
|
<Tab title="NVIDIA A800">
|
|
|
|
This is a single-run profile for smoke validation rather than a formal benchmark. Environment: NVIDIA A800; workload: vbench text-to-image, 1 prompt, concurrency 1, 1 output per prompt; warmup: none; repetitions: 1; dispersion: not reported; baseline: none; revisions: PR checkout and resolved model snapshot at run time; output validation: request completed and returned one image.
|
|
|
|
**Server Command**:
|
|
|
|
```shell Command
|
|
sglang serve \
|
|
--model-path sensenova/SenseNova-U1.5-8B-MoT \
|
|
--port 30000
|
|
```
|
|
|
|
**Benchmark Command**:
|
|
|
|
```shell Command
|
|
python3 -m sglang.multimodal_gen.benchmarks.bench_serving \
|
|
--dataset vbench \
|
|
--task text-to-image \
|
|
--num-prompts 1 \
|
|
--max-concurrency 1 \
|
|
--port 30000
|
|
```
|
|
|
|
**Result**:
|
|
|
|
```text Output
|
|
================= Serving Benchmark Result =================
|
|
Task: text-to-image
|
|
Model: /workspace/models/ModelScope/SenseNova/SenseNova-U1.5-8B-MoT
|
|
Dataset: vbench
|
|
--------------------------------------------------
|
|
Benchmark duration (s): 13.10
|
|
Request rate: inf
|
|
Max request concurrency: 1
|
|
Successful requests: 1/1
|
|
Completed outputs: 1
|
|
Outputs per prompt: 1
|
|
--------------------------------------------------
|
|
Request throughput (req/s): 0.08
|
|
Output throughput (outputs/s): 0.08
|
|
Latency Mean (s): 13.10
|
|
Latency Median (s): 13.10
|
|
Latency P90 (s): 13.10
|
|
Latency P95 (s): 13.10
|
|
Latency P99 (s): 13.10
|
|
--------------------------------------------------
|
|
Peak Memory Max (MB): 34326.00
|
|
Peak Memory Mean (MB): 34326.00
|
|
Peak Memory Median (MB): 34326.00
|
|
------------------------------------------------------------
|
|
```
|
|
</Tab>
|
|
<Tab title="Ascend 910C">
|
|
|
|
Environment: one Ascend 910C; workload: four concurrent 2048 x 2048
|
|
text-to-image requests, 50 denoising steps, CFG 4, BF16.
|
|
|
|
| Batch size | NPU operators | Duration (s) | Throughput (images/s) | Mean latency (s) | Peak reserved memory (MB) |
|
|
|---:|---|---:|---:|---:|---:|
|
|
| 1 | Disabled | 234.28 | 0.01707 | 146.47 | 35698 |
|
|
| 2 | Disabled | 234.83 | 0.01703 | 176.06 | 37870 |
|
|
| 1 | FIA + RMSNorm + SwiGLU | 205.94 | 0.01942 | 128.80 | 35718 |
|
|
| 2 | FIA + RMSNorm + SwiGLU | 195.02 | 0.02051 | 146.33 | 37850 |
|
|
|
|
Optimized B1 improves throughput by 13.76% over unoptimized B1. Optimized
|
|
B2 improves throughput by 20.42% over unoptimized B2 and by 5.60% over
|
|
optimized B1.
|
|
|
|
</Tab>
|
|
</Tabs>
|