[Router] Add bucket-aware policy domains and native cache indexing (#38108)
Signed-off-by: Vincent Gao <vincentbo@linux.alibaba.com> Co-authored-by: inkcherry <mingzhi.liu@amd.com> Co-authored-by: yangbodong22011 <13137470+yangbodong22011@users.noreply.github.com>
This commit is contained in:
co-authored by
inkcherry
yangbodong22011
parent
a176ba2f7b
commit
5bebe7a033
@@ -25,7 +25,6 @@ The dashboard graphs every family the router emits:
|
||||
| `sgl_router_worker_requests_total` | Counter | Per-worker **dispatches** by `worker_url`, `model_id`, `mode`, `outcome` (recorded after dispatch; blind to pre-dispatch drops) |
|
||||
| `sgl_router_request_duration_seconds` | Histogram | End-to-end request latency by `model_id` |
|
||||
| `sgl_router_ttft_seconds` | Histogram | Time to first token (streaming) by `model_id` |
|
||||
| `sgl_router_overlap_blocks` | Histogram | Cache-aware-zmq overlap blocks by `model_id` |
|
||||
| `sgl_router_active_load` | Gauge | Per-worker prefill-token / decode-block load |
|
||||
| `sgl_router_workers` | Gauge | Registered worker count by `mode` |
|
||||
| `sgl_router_worker_health` | Gauge | Per-worker health (1=breaker admits, 0=open) |
|
||||
@@ -35,6 +34,10 @@ The dashboard graphs every family the router emits:
|
||||
| `sgl_router_decode_affinity_total` | Counter | PD decode-affinity outcomes |
|
||||
| `sgl_router_sticky_total` | Counter | Sticky-session selection outcomes |
|
||||
|
||||
The legacy `sgl_router_overlap_blocks` metric was removed with the
|
||||
`cache_aware_zmq` policy and has no direct replacement. Remove queries, alerts,
|
||||
and dashboard panels that depend on this metric before upgrading.
|
||||
|
||||
The `sgl_router_workers` / `sgl_router_worker_*` gauges are sampled from the
|
||||
live worker registry on every scrape, so a removed worker stops emitting
|
||||
series immediately rather than leaving a stale value.
|
||||
|
||||
Reference in New Issue
Block a user