[NPU][Diffusion] Optimize SenseNova-U1 batched generation (#39382)
Signed-off-by: syd520zy <529477025@qq.com> Co-authored-by: ronnie_zheng <zl19940307@163.com>
This commit is contained in:
co-authored by
ronnie_zheng
parent
3401b75240
commit
44bd359082
@@ -78,11 +78,38 @@ with open("output_sensenova_u15.png", "wb") as f:
|
||||
f.write(image_bytes)
|
||||
```
|
||||
|
||||
## 5. Benchmark
|
||||
## 5. Advanced usage
|
||||
|
||||
### 5.1 Speedup Benchmark
|
||||
### 5.1 Dynamic request batching
|
||||
|
||||
#### 5.1.1 Single-run Profile
|
||||
Start the server with dynamic batching enabled:
|
||||
|
||||
```bash Command
|
||||
sglang serve \
|
||||
--model-path sensenova/SenseNova-U1.5-8B-MoT \
|
||||
--port 30000 \
|
||||
--batching-max-size 2 \
|
||||
--batching-delay-ms 100
|
||||
```
|
||||
|
||||
Submit requests concurrently so they arrive within the batching delay. Requests
|
||||
in one batch must use the same resolution, inference steps, guidance settings,
|
||||
and output options. Requests with multiple outputs or `think_mode=true` are
|
||||
executed sequentially.
|
||||
|
||||
### 5.2 Ascend NPU optimizations
|
||||
|
||||
On Ascend NPU, SenseNova-U1 automatically uses fused inference attention,
|
||||
RMSNorm, and SwiGLU MLP operators for supported inputs. FIA and SwiGLU fall back
|
||||
when their requirements are not met. Run one complete warmup request before
|
||||
benchmarking because the fused MLP packs its gate and up projection weights on
|
||||
first use.
|
||||
|
||||
## 6. Benchmark
|
||||
|
||||
### 6.1 Speedup benchmark
|
||||
|
||||
#### 6.1.1 Single-run profile
|
||||
|
||||
<Tabs>
|
||||
<Tab title="NVIDIA A800">
|
||||
@@ -137,4 +164,21 @@ with open("output_sensenova_u15.png", "wb") as f:
|
||||
------------------------------------------------------------
|
||||
```
|
||||
</Tab>
|
||||
<Tab title="Ascend 910C">
|
||||
|
||||
Environment: one Ascend 910C; workload: four concurrent 2048 x 2048
|
||||
text-to-image requests, 50 denoising steps, CFG 4, BF16.
|
||||
|
||||
| Batch size | NPU operators | Duration (s) | Throughput (images/s) | Mean latency (s) | Peak reserved memory (MB) |
|
||||
|---:|---|---:|---:|---:|---:|
|
||||
| 1 | Disabled | 234.28 | 0.01707 | 146.47 | 35698 |
|
||||
| 2 | Disabled | 234.83 | 0.01703 | 176.06 | 37870 |
|
||||
| 1 | FIA + RMSNorm + SwiGLU | 205.94 | 0.01942 | 128.80 | 35718 |
|
||||
| 2 | FIA + RMSNorm + SwiGLU | 195.02 | 0.02051 | 146.33 | 37850 |
|
||||
|
||||
Optimized B1 improves throughput by 13.76% over unoptimized B1. Optimized
|
||||
B2 improves throughput by 20.42% over unoptimized B2 and by 5.60% over
|
||||
optimized B1.
|
||||
|
||||
</Tab>
|
||||
</Tabs>
|
||||
|
||||
Reference in New Issue
Block a user