[Router] Add bucket-aware policy domains and native cache indexing (#38108)
Signed-off-by: Vincent Gao <vincentbo@linux.alibaba.com> Co-authored-by: inkcherry <mingzhi.liu@amd.com> Co-authored-by: yangbodong22011 <13137470+yangbodong22011@users.noreply.github.com>
This commit is contained in:
co-authored by
inkcherry
yangbodong22011
parent
a176ba2f7b
commit
5bebe7a033
@@ -5,10 +5,7 @@
|
||||
//!
|
||||
//! Mirrors `sgl-model-gateway/benches/manual_policy_benchmark.rs` —
|
||||
//! measures how fast the routing layer returns a worker for a given
|
||||
//! request context, across the policies sgl-router actually ships
|
||||
//! (round-robin, random, power-of-two-choices). The cache-aware-zmq
|
||||
//! policy lives in `tree_lookup.rs`; this file targets the non-tree
|
||||
//! policies' steady-state hot path.
|
||||
//! request context, across round-robin, random, and power-of-two choices.
|
||||
|
||||
use criterion::{black_box, criterion_group, criterion_main, BenchmarkId, Criterion, Throughput};
|
||||
use sgl_router::discovery::{ModelId, WorkerId, WorkerMode, WorkerSpec};
|
||||
|
||||
Reference in New Issue
Block a user