diff --git a/docs_new/docs/hardware-platforms/ascend-npus/ascend_npu_profiling.mdx b/docs_new/docs/hardware-platforms/ascend-npus/ascend_npu_profiling.mdx index d972583b3..f9df1af39 100644 --- a/docs_new/docs/hardware-platforms/ascend-npus/ascend_npu_profiling.mdx +++ b/docs_new/docs/hardware-platforms/ascend-npus/ascend_npu_profiling.mdx @@ -44,12 +44,6 @@ sglang serve \ --max-running-requests 128 ``` - -On Ascend NPU, SGLang uses `torch_npu._apply_patches()` to automatically -redirect PyTorch Profiler's CUDA activity to NPU, so -`activities: ["CPU", "GPU"]` actually captures NPU operator events. - - **Profiling-related environment variables:** @@ -79,6 +73,10 @@ redirect PyTorch Profiler's CUDA activity to NPU, so
+ +Priority order: **API parameter > Environment variable > Default value**. + + ### 2. Collection Methods SGLang provides four collection methods. The core differences are **whether you @@ -227,6 +225,12 @@ sending `/stop_profile`. shown in the server log as `Profiling done. Traces are saved to: `. + +On Ascend NPU, SGLang uses `torch_npu._apply_patches()` to automatically +redirect PyTorch Profiler's CUDA activity to NPU, so +`activities: ["CPU", "GPU"]` actually captures NPU operator events. + + **`bench_serving --profile` parameters:** @@ -256,7 +260,9 @@ python3 -m sglang.profiler \ --cpu --gpu & ``` ```bash Command -# Terminal 2: Immediately send inference requests to provide steps for profiling +# Terminal 2: Wait for "Waiting for N steps" output from Terminal 1, then send requests. +# The profiler starts recording once /start_profile is received by the server. +# Requests sent before the server receives /start_profile will not be captured. curl http://127.0.0.1:30000/generate \ -H "Content-Type: application/json" \ -d '{"text": "Hello", "sampling_params": {"max_new_tokens": 32}}' @@ -352,7 +358,8 @@ set of supported parameters: @@ -426,7 +433,7 @@ Waiting for 10 steps and the trace to be flushed.... (profile_by_stage=False) The directory structure is `/___ascend_pt/`. When using Method C (`bench_serving --profile`), a timestamp subdirectory is added: -`//`. Always check the server log for the exact path: +`//___ascend_pt/`. Always check the server log for the exact path: `Profiling done. Traces are saved to: `. ### 5. Viewing Results @@ -465,12 +472,11 @@ files — **no need to manually call `analyse()`**: -If you need to merge distributed trace files in a multi-node deployment, set -`"merge_profiles": true` in the `/start_profile` request. Note: on Ascend NPU, -the merger has limited support for the `*_ascend_pt` format — check -`trace_view.json` on each node individually. See -[Benchmark and Profiling](/docs/developer_guide/benchmark_and_profiling#profiler-trace-merger-for-distributed-traces) -for details. +In a multi-node deployment, each node generates its own trace files. +To merge them into a single unified trace, set `"merge_profiles": true` in the `/start_profile` request. +However, on Ascend NPU, the merge feature does **not** fully support the `*_ascend_pt` format — merged results may be incomplete or incorrect. +It is recommended to view `trace_view.json` on each node individually instead. +For more details, see [Benchmark and Profiling](/docs/developer_guide/benchmark_and_profiling#profiler-trace-merger-for-distributed-traces). ### 6. Re-parsing Raw Data (Optional) @@ -487,6 +493,8 @@ analyse("./sglang_profile/_*_ascend_pt/") Normally **no need** to manually run `analyse()` — the server already parses data automatically. Only use this for re-parsing or handling interrupted data. +Running `analyse()` when ASCEND_PROFILER_OUTPUT already exists will overwrite the original directory. +If the original data is still needed, back it up before running `analyse()`. ## Best Practices
activities - Activity types: CPU, GPU, MEM, RPD. On Ascend NPU, primarily CPU and GPU + Activity types: CPU, GPU, MEM, RPD. On Ascend NPU, only CPU and GPU are supported. + MEM depends on CUDA memory APIs and will be silently ignored. RPD requires ROCm and will cause an error. ["CPU", "GPU"]