minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 13 Packages Projects Releases Wiki Activity

[Diffusion][Docs] Ascend A2, A3 add basic usage and benchmark results in diffusion cookbook (#30614)

Browse Source 
Co-authored-by: ANDREW_K <andrewsha3@DESKTOP-KNDINTT.localdomain>
This commit is contained in:
Andrew Kuksa
2026-07-29 11:50:15 +03:00
committed by GitHub
co-authored by ANDREW_K
parent 7dcebca255
commit 0caf0fc01d
10 changed files with 1066 additions and 338 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs_new/cookbook/diffusion/FLUX/FLUX.mdx
+95 -1
View File
@@ -44,7 +44,7 @@ This section provides deployment configurations optimized for different hardware
FLUX models are optimized for high-quality image generation. The recommended launch configurations vary by hardware and model version.
**Interactive Command Generator**: Use the configuration selector below to automatically generate the appropriate deployment command for your hardware platform and model version. SGLang supports serving FLUX on NVIDIA B200, H200, H100, and AMD MI355X, MI325X, MI300X GPUs.
**Interactive Command Generator**: Use the configuration selector below to automatically generate the appropriate deployment command for your hardware platform and model version. SGLang supports serving FLUX on NVIDIA B200, H200, H100, and AMD MI355X, MI325X, MI300X GPUs and Ascend A2, A3 NPUs.
<FluxDeployment />
@@ -215,6 +215,8 @@ Test Environment:
- Model: black-forest-labs/FLUX.1-dev
- sglang diffusion version: 0.5.6.post2
<Tabs>
<Tab title="NVIDIA B200">
**Server Command**:
```shell Command
@@ -252,9 +254,56 @@ Peak Memory Mean (MB): 27905.19
Peak Memory Median (MB): 27905.19
============================================================
```
</Tab>
<Tab title="Ascend A3">
**Server Command**:
```shell Command
#One A3 card has 2 npu chips
sglang serve --tp-size 2 --sp-degree 1 --model-path black-forest-labs/FLUX.1-dev --num-gpus 2
```
**Benchmark Command**:
```shell Command
python -m sglang.multimodal_gen.benchmarks.bench_serving --dataset vbench --task text-to-image --num-prompts 1 --max-concurrency 1
```
**Result**:
```text Output
================= Serving Benchmark Result =================
Task: text-to-image
Model: black-forest-labs/FLUX.1-dev
Dataset: vbench
--------------------------------------------------
Benchmark duration (s): 16.30
Request rate: inf
Max request concurrency: 1
Successful requests: 1/1
Completed outputs: 1
Outputs per prompt: 1
--------------------------------------------------
Request throughput (req/s): 0.06
Output throughput (outputs/s): 0.06
Latency Mean (s): 16.30
Latency Median (s): 16.30
Latency P90 (s): 16.30
Latency P95 (s): 16.30
Latency P99 (s): 16.30
--------------------------------------------------
Peak Memory Max (MB): 19972.00
Peak Memory Mean (MB): 19972.00
Peak Memory Median (MB): 19972.00
------------------------------------------------------------
```
</Tab>
</Tabs>
#### 5.1.2 Generate images with high concurrency
<Tabs>
<Tab title="NVIDIA B200">
**Server Command** :
```shell Command
@@ -292,3 +341,48 @@ Peak Memory Mean (MB): 27916.59
Peak Memory Median (MB): 27917.19
============================================================
```
</Tab>
<Tab title="Ascend A3">
**Server Command** :
```shell Command
#One A3 card has 2 npu chips
sglang serve --tp-size 2 --sp-degree 1 --model-path black-forest-labs/FLUX.1-dev --num-gpus 2
```
**Benchmark Command** :
```shell Command
python -m sglang.multimodal_gen.benchmarks.bench_serving --dataset vbench --task text-to-image --num-prompts 20 --max-concurrency 20
```
**Result** :
```text Output
================= Serving Benchmark Result =================
Task: text-to-image
Model: black-forest-labs/FLUX.1-dev
Dataset: vbench
--------------------------------------------------
Benchmark duration (s): 300.85
Request rate: inf
Max request concurrency: 20
Successful requests: 18/20
Completed outputs: 18
Outputs per prompt: 1
--------------------------------------------------
Request throughput (req/s): 0.06
Output throughput (outputs/s): 0.06
Latency Mean (s): 155.16
Latency Median (s): 155.11
Latency P90 (s): 266.30
Latency P95 (s): 280.15
Latency P99 (s): 291.23
--------------------------------------------------
Peak Memory Max (MB): 19972.00
Peak Memory Mean (MB): 19972.00
Peak Memory Median (MB): 19972.00
------------------------------------------------------------
```
</Tab>
</Tabs>
docs_new/cookbook/diffusion/Qwen-Image/Qwen-Image.mdx
+90 -3
View File
@@ -24,7 +24,7 @@ This section provides deployment configurations optimized for different hardware
### 3.1 Basic Configuration
Qwen-Image is a text-to-image model. The recommended launch configurations vary by hardware.
Qwen-Image is a text-to-image model. The recommended launch configurations vary by hardware. SGLang supports serving Qwen-Image on NVIDIA B200, B300, H200, H100, AMD MI300X, MI325X, MI355X GPUs and Ascend A2, A3 NPUs.
**Interactive Command Generator**: Use the configuration selector below to automatically generate the appropriate deployment command for your hardware platform.
@@ -221,6 +221,8 @@ Test Environment:
#### 5.1.1 Generate an image
<Tabs>
<Tab title="AMD MI300X">
**Server Command**:
```shell Command
@@ -258,14 +260,61 @@ Peak Memory Mean (MB): 48018.83
Peak Memory Median (MB): 48018.83
============================================================
```
</Tab>
<Tab title="Ascend A3">
**Server Command**:
#### 5.1.2 Generate images with high concurrency
```shell Command
#One A3 card has 2 npu chips
sglang serve --tp-size 2 --sp-degree 1 --model-path Qwen/Qwen-Image --num-gpus 2
```
**Benchmark Command**:
```shell Command
python -m sglang.multimodal_gen.benchmarks.bench_serving --dataset vbench --task text-to-image --num-prompts 1 --max-concurrency 1
```
**Result**:
```text Output
================= Serving Benchmark Result =================
Task: text-to-image
Model: Qwen/Qwen-Image
Dataset: vbench
--------------------------------------------------
Benchmark duration (s): 36.26
Request rate: inf
Max request concurrency: 1
Successful requests: 1/1
Completed outputs: 1
Outputs per prompt: 1
--------------------------------------------------
Request throughput (req/s): 0.03
Output throughput (outputs/s): 0.03
Latency Mean (s): 36.26
Latency Median (s): 36.26
Latency P90 (s): 36.26
Latency P95 (s): 36.26
Latency P99 (s): 36.26
--------------------------------------------------
Peak Memory Max (MB): 36984.00
Peak Memory Mean (MB): 36984.00
Peak Memory Median (MB): 36984.00
------------------------------------------------------------
```
</Tab>
</Tabs>
#### 5.1.2 Generate images with high concurrency
<Tabs>
<Tab title="AMD MI300X">
**Benchmark Command**:
```shell Command
python3 -m sglang.multimodal_gen.benchmarks.bench_serving \
--backend sglang-image --dataset vbench --task text-to-image --num-prompts 20 --max-concurrency 20
--backend sglang-image --dataset vbench --task text-to-image --num-prompts 20 --max-concurrency 20 --port 30000
```
**Result**:
@@ -291,3 +340,41 @@ Peak Memory Mean (MB): 48030.30
Peak Memory Median (MB): 48030.29
============================================================
```
</Tab>
<Tab title="Ascend A3">
**Benchmark Command**:
```shell Command
python -m sglang.multimodal_gen.benchmarks.bench_serving --dataset vbench --task text-to-image --num-prompts 20 --max-concurrency 20
```
**Result**:
```text Output
================= Serving Benchmark Result =================
Task: text-to-image
Model: Qwen/Qwen-Image
Dataset: vbench
--------------------------------------------------
Benchmark duration (s): 300.81
Request rate: inf
Max request concurrency: 20
Successful requests: 8/20
Completed outputs: 8
Outputs per prompt: 1
--------------------------------------------------
Request throughput (req/s): 0.03
Output throughput (outputs/s): 0.03
Latency Mean (s): 166.61
Latency Median (s): 167.02
Latency P90 (s): 270.80
Latency P95 (s): 283.48
Latency P99 (s): 293.64
--------------------------------------------------
Peak Memory Max (MB): 36984.00
Peak Memory Mean (MB): 36984.00
Peak Memory Median (MB): 36984.00
------------------------------------------------------------
```
</Tab>
</Tabs>
docs_new/cookbook/diffusion/Wan/Wan2.1.mdx
+122 -1
View File
@@ -35,7 +35,7 @@ This section provides deployment configurations optimized for different hardware
### 3.1 Basic Configuration
The Wan2.1 series offers models in multiple sizes and resolutions. SGLang supports Wan2.1 deployment on NVIDIA B200, B300, H200, H100, and AMD MI300X, MI325X, MI355X GPUs. The recommended launch configurations vary by hardware, model size, and memory headroom.
The Wan2.1 series offers models in multiple sizes and resolutions. SGLang supports Wan2.1 deployment on NVIDIA B200, B300, H200, H100, and AMD MI300X, MI325X, MI355X GPUs and Ascend A2, A3 NPUs. The recommended launch configurations vary by hardware, model size, and memory headroom.
**Interactive Command Generator**: Use the configuration selector below to automatically generate an appropriate deployment command for your model variant and options.
@@ -185,6 +185,8 @@ You can use the built-in SGLang diffusion benchmark script to evaluate Wan2.1 pe
#### 5.1.1 Generate a single video
<Tabs>
<Tab title="NVIDIA B200">
**Server Command**:
```bash Command
@@ -221,9 +223,66 @@ Peak Memory Mean (MB): 59662.00
Peak Memory Median (MB): 59662.00
============================================================
```
</Tab>
<Tab title="Ascend A3">
**Server Command**:
```bash Command
#One A3 card has 2 npu chips. Benchmark was did with two A3 cards
sglang serve \
--model-path /models/Wan-AI/Wan2.1-T2V-14B-Diffusers/ \
--tp-size 2 \
--sp-degree 2 \
--num-gpus 4 \
--attention-backend laser_attn
```
**Benchmark Command**:
```bash Command
python -m sglang.multimodal_gen.benchmarks.bench_serving \
--dataset vbench \
--task text-to-video \
--num-prompts 1 \
--max-concurrency 1
```
**Result**:
```text Output
================= Serving Benchmark Result =================
Task: text-to-video
Model: Wan-AI/Wan2.1-T2V-14B-Diffusers/
Dataset: vbench
--------------------------------------------------
Benchmark duration (s): 1282.90
Request rate: inf
Max request concurrency: 1
Successful requests: 1/1
Completed outputs: 1
Outputs per prompt: 1
--------------------------------------------------
Request throughput (req/s): 0.00
Output throughput (outputs/s): 0.00
Latency Mean (s): 1282.90
Latency Median (s): 1282.90
Latency P90 (s): 1282.90
Latency P95 (s): 1282.90
Latency P99 (s): 1282.90
--------------------------------------------------
Peak Memory Max (MB): 31938.00
Peak Memory Mean (MB): 31938.00
Peak Memory Median (MB): 31938.00
============================================================
```
</Tab>
</Tabs>
#### 5.1.2 Generate videos with Cache-DiT acceleration
<Tabs>
<Tab title="NVIDIA B200">
**Server Command**:
```bash Command
@@ -268,3 +327,65 @@ Peak Memory Mean (MB): 69306.00
Peak Memory Median (MB): 69306.00
============================================================
```
</Tab>
<Tab title="Ascend A3">
**Server Command**:
```bash Command
#One A3 card has 2 npu chips. Benchmark was did with two Atlas 3 cards
SGLANG_CACHE_DIT_FN=2 \
SGLANG_CACHE_DIT_BN=1 \
SGLANG_CACHE_DIT_WARMUP=4 \
SGLANG_CACHE_DIT_RDT=0.4 \
SGLANG_CACHE_DIT_MC=4 \
SGLANG_CACHE_DIT_TAYLORSEER=true \
SGLANG_CACHE_DIT_TS_ORDER=2 \
SGLANG_CACHE_DIT_ENABLED=true sglang serve \
--model-path /models/Wan-AI/Wan2.1-T2V-14B-Diffusers/ \
--tp-size 2 \
--sp-degree 2 \
--num-gpus 4 \
--attention-backend laser_attn
```
**Benchmark Command**:
```bash Command
python -m sglang.multimodal_gen.benchmarks.bench_serving \
--dataset vbench \
--task text-to-video \
--num-prompts 1 \
--max-concurrency 1
```
**Result**:
```text Output
================= Serving Benchmark Result =================
Task: text-to-video
Model: Wan-AI/Wan2.1-T2V-14B-Diffusers/
Dataset: vbench
--------------------------------------------------
Benchmark duration (s): 413.88
Request rate: inf
Max request concurrency: 1
Successful requests: 1/1
Completed outputs: 1
Outputs per prompt: 1
--------------------------------------------------
Request throughput (req/s): 0.00
Output throughput (outputs/s): 0.00
Latency Mean (s): 413.88
Latency Median (s): 413.88
Latency P90 (s): 413.88
Latency P95 (s): 413.88
Latency P99 (s): 413.88
--------------------------------------------------
Peak Memory Max (MB): 32782.00
Peak Memory Mean (MB): 32782.00
Peak Memory Median (MB): 32782.00
============================================================
```
</Tab>
</Tabs>
docs_new/cookbook/diffusion/Wan/Wan2.2.mdx
+121 -6
View File
@@ -33,7 +33,7 @@ This section provides deployment configurations optimized for different hardware
The Wan2.2 series offers models in various sizes, architectures and input types, optimized for different hardware platforms. The recommended launch configurations vary by hardware and model size.
**Interactive Command Generator**: Use the configuration selector below to automatically generate the appropriate deployment command for your hardware platform, model size. SGLang supports serving Wan2.2 on NVIDIA B200, H200 and AMD MI300X, MI325X and MI355X GPUs.
**Interactive Command Generator**: Use the configuration selector below to automatically generate the appropriate deployment command for your hardware platform, model size. SGLang supports serving Wan2.2 on NVIDIA B200, H200, AMD MI300X, MI325X, MI355X GPUs and Ascend A2, A3 NPUs.
<Wan22Deployment />
@@ -65,7 +65,7 @@ curl http://127.0.0.1:3000/v1/images/generations \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-d '{
"model": "black-forest-labs/FLUX.1-dev",
"model": "Wan-AI/Wan2.2-T2V-A14B-Diffusers",
"prompt": "A cute baby sea otter",
"n": 1,
"size": "1024x1024",
@@ -255,23 +255,22 @@ Test Environment:
### 5.1 Speedup Benchmark
#### 5.1.1 Generate a video
### 5.1.1 Generate a video
<Tabs>
<Tab title="NVIDIA B200">
**Server Command**:
```shell Command
sglang serve --model-path Wan-AI/Wan2.2-T2V-A14B-Diffusers
```
**Benchmark Command**:
```shell Command
python3 -m sglang.multimodal_gen.benchmarks.bench_serving \
--backend sglang-video --dataset vbench --task t2v --num-prompts 1 --max-concurrency 1
```
**Result**:
```text Output
================= Serving Benchmark Result =================
Backend: sglang-video
@@ -295,9 +294,63 @@ Peak Memory Median (MB): 62627.41
============================================================
```
</Tab>
<Tab title="Ascend A3">
**Server Command**:
```shell Command
#One A3 card has 2 npu chips. Using four A3 cards in benchmarking
sglang serve \
--model-path /models/Wan-AI/Wan2.2-T2V-A14B-Diffusers/ \
--tp-size 2 \
--sp-degree 4 \
--num-gpus 8 \
--attention-backend laser_attn
```
**Benchmark Command**:
```shell Command
python -m sglang.multimodal_gen.benchmarks.bench_serving \
--dataset vbench \
--task text-to-video \
--num-prompts 1 \
--max-concurrency 1
```
**Result**:
```text Output
================= Serving Benchmark Result =================
Task: text-to-video
Model: Wan-AI/Wan2.2-T2V-A14B-Diffusers/
Dataset: vbench
--------------------------------------------------
Benchmark duration (s): 214.50
Request rate: inf
Max request concurrency: 1
Successful requests: 1/1
Completed outputs: 1
Outputs per prompt: 1
--------------------------------------------------
Request throughput (req/s): 0.00
Output throughput (outputs/s): 0.00
Latency Mean (s): 214.50
Latency Median (s): 214.50
Latency P90 (s): 214.50
Latency P95 (s): 214.50
Latency P99 (s): 214.50
--------------------------------------------------
Peak Memory Max (MB): 46692.00
Peak Memory Mean (MB): 46692.00
Peak Memory Median (MB): 46692.00
------------------------------------------------------------
```
</Tab>
</Tabs>
#### 5.1.2 Generate videos with high concurrency
<Tabs>
<Tab title="NVIDIA B200">
**Server Command**:
```shell Command
@@ -344,3 +397,65 @@ Peak Memory Median (MB): 70824.46
============================================================
```
</Tab>
<Tab title="Ascend A3">
**Server Command**:
```shell Command
#One A3 card has 2 npu chips. Using four A3 cards in benchmarking
SGLANG_CACHE_DIT_FN=2 \
SGLANG_CACHE_DIT_BN=1 \
SGLANG_CACHE_DIT_WARMUP=4 \
SGLANG_CACHE_DIT_RDT=0.4 \
SGLANG_CACHE_DIT_MC=4 \
SGLANG_CACHE_DIT_TAYLORSEER=true \
SGLANG_CACHE_DIT_TS_ORDER=2 \
SGLANG_CACHE_DIT_ENABLED=true sglang serve \
--model-path /models/Wan-AI/Wan2.2-T2V-A14B-Diffusers/ \
--tp-size 2 \
--sp-degree 4 \
--num-gpus 8 \
--attention-backend laser_attn
```
**Benchmark Command**:
```shell Command
python -m sglang.multimodal_gen.benchmarks.bench_serving \
--dataset vbench \
--task text-to-video \
--num-prompts 20 \
--max-concurrency 20
```
**Result**:
```text Output
================= Serving Benchmark Result =================
Task: text-to-video
Model: Wan-AI/Wan2.2-T2V-A14B-Diffusers/
Dataset: vbench
--------------------------------------------------
Benchmark duration (s): 4384.65
Request rate: inf
Max request concurrency: 20
Successful requests: 20/20
Completed outputs: 20
Outputs per prompt: 1
--------------------------------------------------
Request throughput (req/s): 0.00
Output throughput (outputs/s): 0.00
Latency Mean (s): 2304.17
Latency Median (s): 2297.69
Latency P90 (s): 3972.32
Latency P95 (s): 4178.99
Latency P99 (s): 4343.52
--------------------------------------------------
Peak Memory Max (MB): 46692.00
Peak Memory Mean (MB): 46691.90
Peak Memory Median (MB): 46692.00
------------------------------------------------------------
```
</Tab>
</Tabs>
docs_new/cookbook/diffusion/Z-Image/Z-Image-Turbo.mdx
+87
View File
@@ -209,6 +209,8 @@ Test Environment:
#### 5.1.1 Generate an image
<Tabs>
<Tab title="AMD MI300X">
**Server Command**:
```shell Command
@@ -246,9 +248,56 @@ Peak Memory Mean (MB): 30689.20
Peak Memory Median (MB): 30689.20
============================================================
```
</Tab>
<Tab title="Ascend A3">
**Server Command**:
```shell Command
#One A3 card has 2 npu chips
sglang serve --model-path Tongyi-MAI/Z-Image-Turbo --tp-size 2 --sp-degree 1 --num-gpus 2
```
**Benchmark Command**:
```shell Command
python -m sglang.multimodal_gen.benchmarks.bench_serving --dataset vbench --task text-to-image --num-prompts 1 --max-concurrency 1
```
**Result**:
```text Output
================= Serving Benchmark Result =================
Task: text-to-image
Model: Tongyi-MAI/Z-Image-Turbo
Dataset: vbench
--------------------------------------------------
Benchmark duration (s): 2.43
Request rate: inf
Max request concurrency: 1
Successful requests: 1/1
Completed outputs: 1
Outputs per prompt: 1
--------------------------------------------------
Request throughput (req/s): 0.41
Output throughput (outputs/s): 0.41
Latency Mean (s): 2.43
Latency Median (s): 2.43
Latency P90 (s): 2.43
Latency P95 (s): 2.43
Latency P99 (s): 2.43
--------------------------------------------------
Peak Memory Max (MB): 11052.00
Peak Memory Mean (MB): 11052.00
Peak Memory Median (MB): 11052.00
------------------------------------------------------------
```
</Tab>
</Tabs>
#### 5.1.2 Generate images with high concurrency
<Tabs>
<Tab title="AMD MI300X">
**Benchmark Command**:
```shell Command
@@ -279,3 +328,41 @@ Peak Memory Mean (MB): 30689.21
Peak Memory Median (MB): 30689.21
============================================================
```
</Tab>
<Tab title="Ascend A3">
**Benchmark Command**:
```shell Command
python -m sglang.multimodal_gen.benchmarks.bench_serving --dataset vbench --task text-to-image --num-prompts 20 --max-concurrency 20
```
**Result**:
```text Output
================= Serving Benchmark Result =================
Task: text-to-image
Model: /models/Tongyi-MAI/Z-Image-Turbo/Z-Image-Turbo
Dataset: vbench
--------------------------------------------------
Benchmark duration (s): 49.08
Request rate: inf
Max request concurrency: 20
Successful requests: 20/20
Completed outputs: 20
Outputs per prompt: 1
--------------------------------------------------
Request throughput (req/s): 0.41
Output throughput (outputs/s): 0.41
Latency Mean (s): 25.78
Latency Median (s): 25.77
Latency P90 (s): 44.42
Latency P95 (s): 46.75
Latency P99 (s): 48.61
--------------------------------------------------
Peak Memory Max (MB): 11054.00
Peak Memory Mean (MB): 11054.00
Peak Memory Median (MB): 11054.00
------------------------------------------------------------
```
</Tab>
</Tabs>
docs_new/src/snippets/diffusion/flux-deployment.jsx
+37 -1
View File
@@ -14,6 +14,8 @@ export const FluxDeployment = () => {
{ id: 'mi355x', label: 'MI355X', default: false },
{ id: 'mi325x', label: 'MI325X', default: false },
{ id: 'mi300x', label: 'MI300X', default: false },
{ id: 'a2', label: 'A2', default: false },
{ id: 'a3', label: 'A3', default: false }
]
},
version: {
@@ -32,9 +34,30 @@ export const FluxDeployment = () => {
},
generateCommand: function(values) {
const { version } = values;
const { hardware, version } = values;
const config = this.modelConfigs[version];
if (hardware === 'a2') {
if (version === 'flux1-dev') {
return `sglang serve \\
--model-path ${config.repoId} \\
--num-gpus 1`;
}
return `sglang serve \\
--model-path ${config.repoId} \\
--tp-size 2 \\
--num-gpus 2`;
}
if (hardware === 'a3') {
return `#One A3 card has 2 npu chips
sglang serve \\
--tp-size 2 \\
--model-path ${config.repoId} \\
--num-gpus 2`;
}
return `sglang serve \\
--model-path ${config.repoId} \\
--ulysses-degree=1 \\
@@ -108,6 +131,19 @@ export const FluxDeployment = () => {
return () => observer.disconnect();
}, []);
useEffect(() => {
const isAscend = values.hardware === 'a2' || values.hardware === 'a3';
const targetTabName = isAscend ? 'Ascend A3' : 'NVIDIA B200';
const allTabs = document.querySelectorAll('button, [role="tab"]');
allTabs.forEach((tab) => {
const text = tab.textContent.trim();
if (text === targetTabName && tab.getAttribute('aria-selected') !== 'true') {
tab.click();
}
});
}, [values.hardware]);
const handleRadioChange = (optionName, value) => {
setValues((prev) => ({ ...prev, [optionName]: value }));
};
docs_new/src/snippets/diffusion/qwen-image-deployment.jsx
+34 -1
View File
@@ -13,7 +13,9 @@ export const QwenImageDeployment = () => {
{ id: 'h100', label: 'H100', default: false },
{ id: 'mi300x', label: 'MI300X', default: false },
{ id: 'mi325x', label: 'MI325X', default: false },
{ id: 'mi355x', label: 'MI355X', default: false }
{ id: 'mi355x', label: 'MI355X', default: false },
{ id: 'a2', label: 'A2', default: false },
{ id: 'a3', label: 'A3', default: false }
]
},
precision: {
@@ -33,6 +35,21 @@ export const QwenImageDeployment = () => {
},
generateCommand: function(values) {
if (values.hardware === 'a2') {
return `sglang serve \\
--model-path Qwen/Qwen-Image \\
--num-gpus 1`;
}
if (values.hardware === 'a3') {
return `#One A3 card has 2 npu chips
sglang serve \\
--model-path Qwen/Qwen-Image \\
--tp-size 1 \\
--sp-degree 2 \\
--num-gpus 2`;
}
const isBlackwell = ['b200', 'b300'].includes(values.hardware);
const isNvfp4 = values.precision === 'nvfp4' && isBlackwell;
const modelPath = isNvfp4
@@ -112,6 +129,22 @@ export const QwenImageDeployment = () => {
return () => observer.disconnect();
}, []);
useEffect(() => {
let targetTabName = 'AMD MI300X';
if (values.hardware === 'a2') targetTabName = 'Ascend A2';
if (values.hardware === 'a3') targetTabName = 'Ascend A3';
const allTabs = document.querySelectorAll('button, [role="tab"]');
allTabs.forEach((tab) => {
const text = tab.textContent.trim();
if (text === targetTabName && tab.getAttribute('aria-selected') !== 'true') {
tab.click();
}
});
}, [values.hardware]);
const handleRadioChange = (optionName, value) => {
setValues((prev) => {
const next = { ...prev, [optionName]: value };
docs_new/src/snippets/diffusion/wan21-deployment.jsx
+58 -3
View File
@@ -47,6 +47,8 @@ export const Wan21Deployment = () => {
{ id: 'mi300x', label: 'MI300X', default: false },
{ id: 'mi325x', label: 'MI325X', default: false },
{ id: 'mi355x', label: 'MI355X', default: false },
{ id: 'a2', label: 'A2', default: false },
{ id: 'a3', label: 'A3', default: false }
],
},
task: {
@@ -64,10 +66,10 @@ export const Wan21Deployment = () => {
},
bestPractice: {
name: 'bestPractice',
title: 'Sequence Parallelism',
title: 'Optimization',
items: [
{ id: 'off', label: 'Standard', default: true },
{ id: 'on', label: 'Best Practice (4 GPUs)', default: false },
{ id: 'on', label: 'Best Practice', default: false },
],
},
};
@@ -114,6 +116,22 @@ export const Wan21Deployment = () => {
return () => observer.disconnect();
}, []);
useEffect(() => {
const isAscend = values.hardware === 'a2' || values.hardware === 'a3';
const targetTabName = isAscend ? 'Ascend A3' : 'NVIDIA B200';
const allTabs = document.querySelectorAll('button, [role="tab"]');
allTabs.forEach((tab) => {
const text = tab.textContent.trim();
if (text === targetTabName && tab.getAttribute('aria-selected') !== 'true') {
tab.click();
}
});
}, [values.hardware]);
const handleRadioChange = (optionName, itemId) => {
setValues((prev) => {
let next = { ...prev, [optionName]: itemId };
@@ -154,7 +172,7 @@ export const Wan21Deployment = () => {
};
const generateCommand = () => {
const { task, modelsize, selectedLoraPath, bestPractice } = values;
const { hardware, task, modelsize, selectedLoraPath, bestPractice } = values;
const configKey = `${task}-${modelsize}`;
const config = modelConfigs[configKey];
@@ -162,6 +180,43 @@ export const Wan21Deployment = () => {
return '# Error: Invalid configuration';
}
if (hardware === 'a2' || hardware === 'a3') {
const comment = hardware === 'a3'
? '#One A3 card has 2 npu chips\n'
: '';
const isBestPractice = bestPractice === 'on';
let command;
if (task === 't2v' && modelsize === '1_3b' && hardware === 'a2' && !isBestPractice) {
command = `${comment}sglang serve \\
--model-path ${config.repoId} \\
--num-gpus 1`;
} else {
const tpSize = modelsize === '1_3b' ? (isBestPractice ? 4 : 1) : 2;
const spDegree = modelsize === '14b' && isBestPractice ? 4 : 1;
const numGpus = isBestPractice ? 8 : (hardware === 'a3' ? 2 : 4);
command = `${comment}sglang serve \\
--model-path ${config.repoId} \\
--tp-size ${tpSize} \\
--sp-degree ${spDegree} \\
--num-gpus ${numGpus}`;
}
if (isBestPractice) {
command += ` \\\n --attention-backend laser_attn`;
}
if (
selectedLoraPath === 'NIVEDAN/wan2.1-lora' ||
selectedLoraPath === 'valiantcat/Wan2.1-Fight-LoRA'
) {
command += ` \\\n --lora-path ${selectedLoraPath}`;
}
return command;
}
let command = `sglang serve \\\n --model-path ${config.repoId} \\\n --dit-layerwise-offload true`;
if (bestPractice === 'on') {
docs_new/src/snippets/diffusion/wan22-deployment.jsx
+70 -2
View File
@@ -11,6 +11,8 @@
{ id: 'mi300x', label: 'MI300X', default: false },
{ id: 'mi325x', label: 'MI325X', default: false },
{ id: 'mi355x', label: 'MI355X', default: false },
{ id: 'a2', label: 'A2', default: false },
{ id: 'a3', label: 'A3', default: false }
],
},
task: {
@@ -32,10 +34,10 @@
},
bestPractice: {
name: 'bestPractice',
title: 'Sequence Parallelism',
title: 'Optimization',
items: [
{ id: 'off', label: 'Standard', default: true },
{ id: 'on', label: 'Best Practice (4 GPUs)', default: false },
{ id: 'on', label: 'Best Practice', default: false },
],
},
};
@@ -85,6 +87,22 @@
return modelConfigs[configKey]?.supportedLoras || [];
})();
useEffect(() => {
const isAscend = values.hardware === 'a2' || values.hardware === 'a3';
const targetTabName = isAscend ? 'Ascend A3' : 'NVIDIA B200';
const allTabs = document.querySelectorAll('button, [role="tab"]');
allTabs.forEach((tab) => {
const text = tab.textContent.trim();
if (text === targetTabName && tab.getAttribute('aria-selected') !== 'true') {
tab.click();
}
});
}, [values.hardware]);
const handleRadioChange = (optionName, itemId) => {
setValues((prev) => {
const next = { ...prev, [optionName]: itemId };
@@ -117,6 +135,56 @@
return '# Error: Invalid configuration';
}
if (hardware === 'a2' || hardware === 'a3') {
const comment = hardware === 'a3'
? '#One A3 card has 2 npu chips\n'
: '';
const isBestPractice = bestPractice === 'on';
let command;
if (task === 'ti2v') {
if (isBestPractice) {
command = `${comment}sglang serve \\
--model-path ${config.repoId} \\
--sp-degree 8 \\
--num-gpus 8`;
} else if (hardware === 'a2') {
command = `${comment}sglang serve \\
--model-path ${config.repoId} \\
--num-gpus 1`;
} else {
command = `${comment}sglang serve \\
--model-path ${config.repoId} \\
--tp-size 1 \\
--sp-degree 2 \\
--num-gpus 2`;
}
} else {
const spDegree = isBestPractice ? 4 : 1;
const numGpus = isBestPractice ? 8 : (hardware === 'a3' ? 2 : 4);
command = `${comment}sglang serve \\
--model-path ${config.repoId} \\
--tp-size 2 \\
--sp-degree ${spDegree} \\
--num-gpus ${numGpus}`;
}
if (isBestPractice) {
command += ` \\\n --attention-backend laser_attn`;
}
if (
selectedLoraPath === 'lightx2v/Wan2.2-Distill-Loras' ||
selectedLoraPath === 'Cseti/wan2.2-14B-Arcane_Jinx-lora-v1'
) {
command += ` \\\n --lora-path ${selectedLoraPath}`;
}
return command;
}
let command = `sglang serve \\\n --model-path ${config.repoId} \\\n --dit-layerwise-offload true`;
if (bestPractice === 'on') {
if (hardware === 'b300') {
docs_new/src/snippets/diffusion/zimage-turbo-deployment.jsx
+33 -1
View File
@@ -12,12 +12,31 @@ export const ZImageTurboDeployment = () => {
{ id: 'mi355x', label: 'MI355X', default: false },
{ id: 'b200', label: 'B200', default: true },
{ id: 'h200', label: 'H200', default: false },
{ id: 'h100', label: 'H100', default: false }
{ id: 'h100', label: 'H100', default: false },
{ id: 'a2', label: 'A2', default: false },
{ id: 'a3', label: 'A3', default: false }
]
}
},
generateCommand: function(values) {
const { hardware } = values;
if (hardware === 'a2') {
return `sglang serve \\
--model-path Tongyi-MAI/Z-Image-Turbo \\
--num-gpus 1`;
}
if (hardware === 'a3') {
return `#One A3 card has 2 npu chips
sglang serve \\
--model-path Tongyi-MAI/Z-Image-Turbo \\
--tp-size 2 \\
--sp-degree 1 \\
--num-gpus 2`;
}
return `sglang serve \\
--model-path Tongyi-MAI/Z-Image-Turbo \\
--ulysses-degree=1 \\
@@ -91,6 +110,19 @@ export const ZImageTurboDeployment = () => {
return () => observer.disconnect();
}, []);
useEffect(() => {
const isAscend = values.hardware === 'a2' || values.hardware === 'a3';
const targetTabName = isAscend ? 'Ascend A2 / A3' : 'AMD MI300X';
const allTabs = document.querySelectorAll('button, [role="tab"]');
allTabs.forEach((tab) => {
const text = tab.textContent.trim();
if (text === targetTabName && tab.getAttribute('aria-selected') !== 'true') {
tab.click();
}
});
}, [values.hardware]);
const handleRadioChange = (optionName, value) => {
setValues((prev) => ({ ...prev, [optionName]: value }));
};