Fix inaccuracies and add NPU constraints in ascend_npu_profiling.mdx. (#28283)
This commit is contained in:
@@ -44,12 +44,6 @@ sglang serve \
|
||||
--max-running-requests 128
|
||||
```
|
||||
|
||||
<Note>
|
||||
On Ascend NPU, SGLang uses `torch_npu._apply_patches()` to automatically
|
||||
redirect PyTorch Profiler's CUDA activity to NPU, so
|
||||
`activities: ["CPU", "GPU"]` actually captures NPU operator events.
|
||||
</Note>
|
||||
|
||||
**Profiling-related environment variables:**
|
||||
|
||||
<table>
|
||||
@@ -79,6 +73,10 @@ redirect PyTorch Profiler's CUDA activity to NPU, so
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<Note>
|
||||
Priority order: **API parameter > Environment variable > Default value**.
|
||||
</Note>
|
||||
|
||||
### 2. Collection Methods
|
||||
|
||||
SGLang provides four collection methods. The core differences are **whether you
|
||||
@@ -227,6 +225,12 @@ sending `/stop_profile`.
|
||||
shown in the server log as `Profiling done. Traces are saved to: <path>`.
|
||||
</Note>
|
||||
|
||||
<Note>
|
||||
On Ascend NPU, SGLang uses `torch_npu._apply_patches()` to automatically
|
||||
redirect PyTorch Profiler's CUDA activity to NPU, so
|
||||
`activities: ["CPU", "GPU"]` actually captures NPU operator events.
|
||||
</Note>
|
||||
|
||||
**`bench_serving --profile` parameters:**
|
||||
|
||||
<table>
|
||||
@@ -256,7 +260,9 @@ python3 -m sglang.profiler \
|
||||
--cpu --gpu &
|
||||
```
|
||||
```bash Command
|
||||
# Terminal 2: Immediately send inference requests to provide steps for profiling
|
||||
# Terminal 2: Wait for "Waiting for N steps" output from Terminal 1, then send requests.
|
||||
# The profiler starts recording once /start_profile is received by the server.
|
||||
# Requests sent before the server receives /start_profile will not be captured.
|
||||
curl http://127.0.0.1:30000/generate \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"text": "Hello", "sampling_params": {"max_new_tokens": 32}}'
|
||||
@@ -352,7 +358,8 @@ set of supported parameters:
|
||||
<tr>
|
||||
<td><code>activities</code></td>
|
||||
<td>
|
||||
Activity types: CPU, GPU, MEM, RPD. On Ascend NPU, primarily CPU and GPU
|
||||
Activity types: CPU, GPU, MEM, RPD. On Ascend NPU, only CPU and GPU are supported.
|
||||
MEM depends on CUDA memory APIs and will be silently ignored. RPD requires ROCm and will cause an error.
|
||||
</td>
|
||||
<td><code>["CPU", "GPU"]</code></td>
|
||||
</tr>
|
||||
@@ -426,7 +433,7 @@ Waiting for 10 steps and the trace to be flushed.... (profile_by_stage=False)
|
||||
The directory structure is
|
||||
`<output_dir>/<hostname>_<pid>_<timestamp>_ascend_pt/`. When using Method C
|
||||
(`bench_serving --profile`), a timestamp subdirectory is added:
|
||||
`<output_dir>/<timestamp>/`. Always check the server log for the exact path:
|
||||
`<output_dir>/<timestamp>/<hostname>_<pid>_<timestamp>_ascend_pt/`. Always check the server log for the exact path:
|
||||
`Profiling done. Traces are saved to: <path>`.
|
||||
|
||||
### 5. Viewing Results
|
||||
@@ -465,12 +472,11 @@ files — **no need to manually call `analyse()`**:
|
||||
</Note>
|
||||
|
||||
<Note>
|
||||
If you need to merge distributed trace files in a multi-node deployment, set
|
||||
`"merge_profiles": true` in the `/start_profile` request. Note: on Ascend NPU,
|
||||
the merger has limited support for the `*_ascend_pt` format — check
|
||||
`trace_view.json` on each node individually. See
|
||||
[Benchmark and Profiling](/docs/developer_guide/benchmark_and_profiling#profiler-trace-merger-for-distributed-traces)
|
||||
for details.
|
||||
In a multi-node deployment, each node generates its own trace files.
|
||||
To merge them into a single unified trace, set `"merge_profiles": true` in the `/start_profile` request.
|
||||
However, on Ascend NPU, the merge feature does **not** fully support the `*_ascend_pt` format — merged results may be incomplete or incorrect.
|
||||
It is recommended to view `trace_view.json` on each node individually instead.
|
||||
For more details, see [Benchmark and Profiling](/docs/developer_guide/benchmark_and_profiling#profiler-trace-merger-for-distributed-traces).
|
||||
</Note>
|
||||
|
||||
### 6. Re-parsing Raw Data (Optional)
|
||||
@@ -487,6 +493,8 @@ analyse("./sglang_profile/<hostname>_*_ascend_pt/")
|
||||
<Note>
|
||||
Normally **no need** to manually run `analyse()` — the server already parses
|
||||
data automatically. Only use this for re-parsing or handling interrupted data.
|
||||
Running `analyse()` when ASCEND_PROFILER_OUTPUT already exists will overwrite the original directory.
|
||||
If the original data is still needed, back it up before running `analyse()`.
|
||||
</Note>
|
||||
|
||||
## Best Practices
|
||||
|
||||
Reference in New Issue
Block a user