minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

[tracing] sglang tracing v2: support exporting tracing data asynchronously (#30023)

Browse Source
This commit is contained in:
Feng Su
2026-08-10 15:23:05 +08:00
committed by GitHub
parent 06f32bab6b
commit fb3d1419fd
8 changed files with 1214 additions and 21 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/docs/references/environment_variables.mdx
+10
View File
@@ -1053,6 +1053,16 @@ SGLang supports various environment variables that can be used to configure its
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Config BatchSpanProcessor.max_export_batch_size if tracing is enabled</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>`64`</td>
</tr>
<tr>
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>`SGLANG_TRACE_ASYNC`</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Enable async tracing: span creation is offloaded to a dedicated exporter process via ZMQ, reducing OTel overhead on the inference hot path</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>`false`</td>
</tr>
<tr>
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>`SGLANG_TRACE_ASYNC_FLUSH_THRESHOLD`</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Number of buffered trace operations before an automatic flush to the exporter process</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>`100`</td>
</tr>
<tr>
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}><code>SGLANG_PROFILE_V2</code></td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Use the v2 profiler implementation.</td>
docs/docs/references/production_request_trace.mdx
+22
View File
@@ -70,6 +70,28 @@ This section explains how to configure the request tracing and export the trace
**Note**: You must set the parameter `--enable-trace`; otherwise, the trace capability will not be enabled regardless of any dynamic adjustments to the trace level.
## Async Tracing (Reducing Performance Overhead)
When batch sizes are large, synchronous OTel span creation can degrade inference throughput due to thread-safe locking and background export threads. Async tracing moves all span creation to a dedicated exporter process via ZMQ, keeping the scheduler and tokenizer hot paths free of OTel overhead.
**Enable async tracing:**
```bash
SGLANG_TRACE_ASYNC=1 python -m sglang.launch_server --enable-trace --otlp-traces-endpoint 0.0.0.0:4317 <other options>
```
**How it works:**
- A daemon exporter process is started per worker process (scheduler, tokenizer, etc.).
- The root span is still created in the caller process (one per request, negligible cost), preserving cross-process span linking via `traceparent`.
- Thread spans and slice spans are buffered as lightweight operation dicts and flushed to the exporter via ZMQ PUSH/PULL.
- Span IDs are pre-generated in the caller process using `TraceCustomIdGenerator.preset_next_span_id()`, so the exported span tree is identical to synchronous mode.
- Thread info (scheduler label, TP/DP/PP ranks) is registered once via a callback on `trace_set_thread_info()`.
**Tuning:**
| Environment Variable | Description | Default |
| --- | --- | --- |
| `SGLANG_TRACE_ASYNC` | Enable async tracing | `false` |
| `SGLANG_TRACE_ASYNC_FLUSH_THRESHOLD` | Max buffered ops before auto-flush | `100` |
## How to add Tracing for slices you're interested in?(API introduction)
We have already inserted instrumentation points in the tokenizer and scheduler main threads. If you wish to trace additional request execution segments or perform finer-grained tracing, please use the APIs from the tracing package as described below.