[model-gateway][tracing]: implement request tracing using OpenTelemetry with trace context propagation (HTTP) (#13897)
This commit is contained in:
@@ -9,7 +9,7 @@ High-performance model routing control and data plane for large-scale LLM deploy
|
||||
- Multi-model inference gateway mode (`--enable-igw`) that runs several routers at once and applies per-model policies.
|
||||
- Conversation, response, and chat-history connectors that centralize state at the router, enabling compliant sharing across models/MCP loops with in-memory, no-op, or Oracle ATP storage options.
|
||||
- Built-in reliability primitives: retries with exponential backoff, circuit breakers, token-bucket rate limiting, and queuing.
|
||||
- First-class observability with structured logging and Prometheus metrics.
|
||||
- First-class observability with structured logging, OpenTelemetry trace and Prometheus metrics.
|
||||
|
||||
### Architecture at a Glance
|
||||
**Control Plane**
|
||||
@@ -562,6 +562,7 @@ Only one of `--oracle-dsn` or `--oracle-tns-alias` should be supplied.
|
||||
- **Prometheus Metrics**: Enable with `--prometheus-host`/`--prometheus-port` (defaults to `0.0.0.0:29000`). Metrics cover request latency, retry behavior, circuit breaker states, worker health/load, queue depth, PD pipeline stats, tokenizer timings, and MCP activity.
|
||||
- **Request IDs**: Configurable headers via `--request-id-headers`; responses include `x-request-id`.
|
||||
- **CORS**: Set `--cors-allowed-origins` for browser access.
|
||||
- **Request Tracing via OpenTelemetry**: Enable with `--enable-trace` and set opentelemetry collector endpoint with `--otlp-traces-endpoint <ip>:<port>`.
|
||||
|
||||
## Security
|
||||
|
||||
|
||||
Reference in New Issue
Block a user