[Router] Add bucket-aware policy domains and native cache indexing (#38108)

Signed-off-by: Vincent Gao <vincentbo@linux.alibaba.com>
Co-authored-by: inkcherry <mingzhi.liu@amd.com>
Co-authored-by: yangbodong22011 <13137470+yangbodong22011@users.noreply.github.com>
This commit is contained in:
Vincent Gao
2026-09-06 19:47:51 +08:00
committed by GitHub
co-authored by inkcherry yangbodong22011
parent a176ba2f7b
commit 5bebe7a033
76 changed files with 5842 additions and 3639 deletions
@@ -5,10 +5,7 @@
//!
//! Mirrors `sgl-model-gateway/benches/manual_policy_benchmark.rs` —
//! measures how fast the routing layer returns a worker for a given
//! request context, across the policies sgl-router actually ships
//! (round-robin, random, power-of-two-choices). The cache-aware-zmq
//! policy lives in `tree_lookup.rs`; this file targets the non-tree
//! policies' steady-state hot path.
//! request context, across round-robin, random, and power-of-two choices.
use criterion::{black_box, criterion_group, criterion_main, BenchmarkId, Criterion, Throughput};
use sgl_router::discovery::{ModelId, WorkerId, WorkerMode, WorkerSpec};