From 630b1ef322c2f995adf7ecafa067b67289d1b9e0 Mon Sep 17 00:00:00 2001 From: Shangming Cai Date: Mon, 21 Sep 2026 16:42:29 +0800 Subject: [PATCH] [sgl-router] Fix reorg admission proxy test build after BucketResolver::new (#40537) Co-authored-by: Kan Wu Co-authored-by: Claude Fable 5.1 --- experimental/sgl-router/POLICY_DESIGN.md | 24 ++++++++++--------- .../src/policies_reorg/session_aware.rs | 5 ++-- .../tests/proxy/chat_routing/reorg.rs | 3 ++- 3 files changed, 18 insertions(+), 14 deletions(-) diff --git a/experimental/sgl-router/POLICY_DESIGN.md b/experimental/sgl-router/POLICY_DESIGN.md index 58cf904fb..f49d383b2 100644 --- a/experimental/sgl-router/POLICY_DESIGN.md +++ b/experimental/sgl-router/POLICY_DESIGN.md @@ -303,21 +303,23 @@ Session assignments are scoped by model, bucket ID, stage, and session key. `SessionAwarePolicy::new(store, engine_load)` receives shared state; the caller owns the store's idle timeout and eviction task. Missing or empty session keys use power-of-two without creating assignments. A new or out-of-group binding -uses power-of-two with `AllowAll`, then the session policy checks its selected -engine before binding. A concurrent live assignment wins, but is checked before -returning it; rejection ends that attempt without rewriting the binding or -retrying another engine. Existing bindings are reused regardless of pressure -when admitted. Session policies can be attached independently to each role. -Programmatic reorg callers configure `model.affinity.session_id_header` for HTTP -header extraction; this does not enable legacy global modes or backup escape. +uses power-of-two with `AdmissionLimits::default()`, then the session policy +checks its selected engine before binding. A concurrent live assignment wins, +but is checked before returning it; rejection ends that attempt without +rewriting the binding or retrying another engine. Existing bindings are reused +regardless of pressure when admitted. Session policies can be attached +independently to each role. Programmatic reorg callers configure +`model.affinity.session_id_header` for HTTP header extraction; this does not +enable legacy global modes or backup escape. Session and sticky policies do not create assignments for missing keys. A binding outside the candidates cannot win. A missing binding may invoke policy fallback within the group; hard admission rejection remains an error. Sticky fallback supports `round_robin`, `random`, `power_of_two`, and `load_based`, -with round-robin as the default. Nested fallbacks use `AllowAll`; the owning -policy explicitly checks the engine returned by its fallback. +with round-robin as the default. Nested fallbacks use +`AdmissionLimits::default()`; the owning policy explicitly checks the engine +returned by its fallback. ### Cache-aware behavior @@ -502,8 +504,8 @@ do not accept and ignore them. - Preserve configured capacity, pending-prefill, and in-flight checks, including their missing-report behavior. Power-of-two applies admission to its selected engine; other policies explicitly place checks in their selection logic. - Other paths use `AllowAll` unless a check is configured. Never silently discard - a configured budget. + Other paths use `AdmissionLimits::default()` unless a check is configured. + Never silently discard a configured budget. Listener and shutdown configuration, discovery, worker health and circuit breakers, tokenizer loading, request timeouts, sampling overrides, and logging diff --git a/experimental/sgl-router/src/policies_reorg/session_aware.rs b/experimental/sgl-router/src/policies_reorg/session_aware.rs index bb0b1e4b3..9de9cc958 100644 --- a/experimental/sgl-router/src/policies_reorg/session_aware.rs +++ b/experimental/sgl-router/src/policies_reorg/session_aware.rs @@ -87,8 +87,9 @@ impl Policy for SessionAwarePolicy { }); } - // The nested power-of-two policy uses AllowAll. The session owner - // checks its chosen engine before creating or replacing a binding. + // The nested power-of-two policy uses AdmissionLimits::default(). + // The session owner checks its chosen engine before creating or + // replacing a binding. let mut pick = self.pick_fallback(engines, request).await?; let load = self.engine_load.capture_snapshot(Instant::now()); self.check(&pick.engine, &load)?; diff --git a/experimental/sgl-router/tests/proxy/chat_routing/reorg.rs b/experimental/sgl-router/tests/proxy/chat_routing/reorg.rs index e71d66e38..6ffdb65d7 100644 --- a/experimental/sgl-router/tests/proxy/chat_routing/reorg.rs +++ b/experimental/sgl-router/tests/proxy/chat_routing/reorg.rs @@ -169,7 +169,8 @@ async fn configured_limits_reject_before_dispatch_and_admit_after_load_drops() { worker_ids: Some([WorkerId("w".into())].into()), policy: Arc::new(policy), }), - )]), + )]) + .unwrap(), )] .into(), );