Fix inaccuracies and add NPU constraints in ascend_npu_profiling.mdx. (#28283)

This commit is contained in:
qinsir5522
2026-06-15 21:12:18 +08:00
committed by GitHub
parent f8d1d397b6
commit 81166f382d
@@ -44,12 +44,6 @@ sglang serve \
--max-running-requests 128
```
<Note>
On Ascend NPU, SGLang uses `torch_npu._apply_patches()` to automatically
redirect PyTorch Profiler's CUDA activity to NPU, so
`activities: ["CPU", "GPU"]` actually captures NPU operator events.
</Note>
**Profiling-related environment variables:**
<table>
@@ -79,6 +73,10 @@ redirect PyTorch Profiler's CUDA activity to NPU, so
</tbody>
</table>
<Note>
Priority order: **API parameter > Environment variable > Default value**.
</Note>
### 2. Collection Methods
SGLang provides four collection methods. The core differences are **whether you
@@ -227,6 +225,12 @@ sending `/stop_profile`.
shown in the server log as `Profiling done. Traces are saved to: <path>`.
</Note>
<Note>
On Ascend NPU, SGLang uses `torch_npu._apply_patches()` to automatically
redirect PyTorch Profiler's CUDA activity to NPU, so
`activities: ["CPU", "GPU"]` actually captures NPU operator events.
</Note>
**`bench_serving --profile` parameters:**
<table>
@@ -256,7 +260,9 @@ python3 -m sglang.profiler \
--cpu --gpu &
```
```bash Command
# Terminal 2: Immediately send inference requests to provide steps for profiling
# Terminal 2: Wait for "Waiting for N steps" output from Terminal 1, then send requests.
# The profiler starts recording once /start_profile is received by the server.
# Requests sent before the server receives /start_profile will not be captured.
curl http://127.0.0.1:30000/generate \
-H "Content-Type: application/json" \
-d '{"text": "Hello", "sampling_params": {"max_new_tokens": 32}}'
@@ -352,7 +358,8 @@ set of supported parameters:
<tr>
<td><code>activities</code></td>
<td>
Activity types: CPU, GPU, MEM, RPD. On Ascend NPU, primarily CPU and GPU
Activity types: CPU, GPU, MEM, RPD. On Ascend NPU, only CPU and GPU are supported.
MEM depends on CUDA memory APIs and will be silently ignored. RPD requires ROCm and will cause an error.
</td>
<td><code>["CPU", "GPU"]</code></td>
</tr>
@@ -426,7 +433,7 @@ Waiting for 10 steps and the trace to be flushed.... (profile_by_stage=False)
The directory structure is
`<output_dir>/<hostname>_<pid>_<timestamp>_ascend_pt/`. When using Method C
(`bench_serving --profile`), a timestamp subdirectory is added:
`<output_dir>/<timestamp>/`. Always check the server log for the exact path:
`<output_dir>/<timestamp>/<hostname>_<pid>_<timestamp>_ascend_pt/`. Always check the server log for the exact path:
`Profiling done. Traces are saved to: <path>`.
### 5. Viewing Results
@@ -465,12 +472,11 @@ files — **no need to manually call `analyse()`**:
</Note>
<Note>
If you need to merge distributed trace files in a multi-node deployment, set
`"merge_profiles": true` in the `/start_profile` request. Note: on Ascend NPU,
the merger has limited support for the `*_ascend_pt` format — check
`trace_view.json` on each node individually. See
[Benchmark and Profiling](/docs/developer_guide/benchmark_and_profiling#profiler-trace-merger-for-distributed-traces)
for details.
In a multi-node deployment, each node generates its own trace files.
To merge them into a single unified trace, set `"merge_profiles": true` in the `/start_profile` request.
However, on Ascend NPU, the merge feature does **not** fully support the `*_ascend_pt` format — merged results may be incomplete or incorrect.
It is recommended to view `trace_view.json` on each node individually instead.
For more details, see [Benchmark and Profiling](/docs/developer_guide/benchmark_and_profiling#profiler-trace-merger-for-distributed-traces).
</Note>
### 6. Re-parsing Raw Data (Optional)
@@ -487,6 +493,8 @@ analyse("./sglang_profile/<hostname>_*_ascend_pt/")
<Note>
Normally **no need** to manually run `analyse()` — the server already parses
data automatically. Only use this for re-parsing or handling interrupted data.
Running `analyse()` when ASCEND_PROFILER_OUTPUT already exists will overwrite the original directory.
If the original data is still needed, back it up before running `analyse()`.
</Note>
## Best Practices