[model-gateway][tracing]: implement request tracing using OpenTelemetry with trace context propagation (HTTP) (#13897)

This commit is contained in:
Feng Su
2025-12-06 05:59:04 -08:00
committed by GitHub
parent 6d41791823
commit e12c6b320f
19 changed files with 814 additions and 35 deletions
+2 -1
View File
@@ -9,7 +9,7 @@ High-performance model routing control and data plane for large-scale LLM deploy
- Multi-model inference gateway mode (`--enable-igw`) that runs several routers at once and applies per-model policies.
- Conversation, response, and chat-history connectors that centralize state at the router, enabling compliant sharing across models/MCP loops with in-memory, no-op, or Oracle ATP storage options.
- Built-in reliability primitives: retries with exponential backoff, circuit breakers, token-bucket rate limiting, and queuing.
- First-class observability with structured logging and Prometheus metrics.
- First-class observability with structured logging, OpenTelemetry trace and Prometheus metrics.
### Architecture at a Glance
**Control Plane**
@@ -562,6 +562,7 @@ Only one of `--oracle-dsn` or `--oracle-tns-alias` should be supplied.
- **Prometheus Metrics**: Enable with `--prometheus-host`/`--prometheus-port` (defaults to `0.0.0.0:29000`). Metrics cover request latency, retry behavior, circuit breaker states, worker health/load, queue depth, PD pipeline stats, tokenizer timings, and MCP activity.
- **Request IDs**: Configurable headers via `--request-id-headers`; responses include `x-request-id`.
- **CORS**: Set `--cors-allowed-origins` for browser access.
- **Request Tracing via OpenTelemetry**: Enable with `--enable-trace` and set opentelemetry collector endpoint with `--otlp-traces-endpoint <ip>:<port>`.
## Security