[NPU][Diffusion] Optimize SenseNova-U1 batched generation (#39382)

Signed-off-by: syd520zy <529477025@qq.com>
Co-authored-by: ronnie_zheng <zl19940307@163.com>
This commit is contained in:
faceless void
2026-09-17 10:08:20 +03:00
committed by GitHub
co-authored by ronnie_zheng
parent 3401b75240
commit 44bd359082
11 changed files with 1333 additions and 204 deletions
@@ -78,11 +78,38 @@ with open("output_sensenova_u15.png", "wb") as f:
f.write(image_bytes)
```
## 5. Benchmark
## 5. Advanced usage
### 5.1 Speedup Benchmark
### 5.1 Dynamic request batching
#### 5.1.1 Single-run Profile
Start the server with dynamic batching enabled:
```bash Command
sglang serve \
--model-path sensenova/SenseNova-U1.5-8B-MoT \
--port 30000 \
--batching-max-size 2 \
--batching-delay-ms 100
```
Submit requests concurrently so they arrive within the batching delay. Requests
in one batch must use the same resolution, inference steps, guidance settings,
and output options. Requests with multiple outputs or `think_mode=true` are
executed sequentially.
### 5.2 Ascend NPU optimizations
On Ascend NPU, SenseNova-U1 automatically uses fused inference attention,
RMSNorm, and SwiGLU MLP operators for supported inputs. FIA and SwiGLU fall back
when their requirements are not met. Run one complete warmup request before
benchmarking because the fused MLP packs its gate and up projection weights on
first use.
## 6. Benchmark
### 6.1 Speedup benchmark
#### 6.1.1 Single-run profile
<Tabs>
<Tab title="NVIDIA A800">
@@ -137,4 +164,21 @@ with open("output_sensenova_u15.png", "wb") as f:
------------------------------------------------------------
```
</Tab>
<Tab title="Ascend 910C">
Environment: one Ascend 910C; workload: four concurrent 2048 x 2048
text-to-image requests, 50 denoising steps, CFG 4, BF16.
| Batch size | NPU operators | Duration (s) | Throughput (images/s) | Mean latency (s) | Peak reserved memory (MB) |
|---:|---|---:|---:|---:|---:|
| 1 | Disabled | 234.28 | 0.01707 | 146.47 | 35698 |
| 2 | Disabled | 234.83 | 0.01703 | 176.06 | 37870 |
| 1 | FIA + RMSNorm + SwiGLU | 205.94 | 0.01942 | 128.80 | 35718 |
| 2 | FIA + RMSNorm + SwiGLU | 195.02 | 0.02051 | 146.33 | 37850 |
Optimized B1 improves throughput by 13.76% over unoptimized B1. Optimized
B2 improves throughput by 20.42% over unoptimized B2 and by 5.60% over
optimized B1.
</Tab>
</Tabs>