[sgl-router] Fix reorg admission proxy test build after BucketResolver::new (#40537)

Co-authored-by: Kan Wu <wukanustc@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Shangming Cai
2026-09-21 16:42:29 +08:00
committed by GitHub
co-authored by Kan Wu Claude Fable 5.1
parent 8d08dfdab7
commit 630b1ef322
3 changed files with 18 additions and 14 deletions
+13 -11
View File
@@ -303,21 +303,23 @@ Session assignments are scoped by model, bucket ID, stage, and session key.
`SessionAwarePolicy::new(store, engine_load)` receives shared state; the caller `SessionAwarePolicy::new(store, engine_load)` receives shared state; the caller
owns the store's idle timeout and eviction task. Missing or empty session keys owns the store's idle timeout and eviction task. Missing or empty session keys
use power-of-two without creating assignments. A new or out-of-group binding use power-of-two without creating assignments. A new or out-of-group binding
uses power-of-two with `AllowAll`, then the session policy checks its selected uses power-of-two with `AdmissionLimits::default()`, then the session policy
engine before binding. A concurrent live assignment wins, but is checked before checks its selected engine before binding. A concurrent live assignment wins,
returning it; rejection ends that attempt without rewriting the binding or but is checked before returning it; rejection ends that attempt without
retrying another engine. Existing bindings are reused regardless of pressure rewriting the binding or retrying another engine. Existing bindings are reused
when admitted. Session policies can be attached independently to each role. regardless of pressure when admitted. Session policies can be attached
Programmatic reorg callers configure `model.affinity.session_id_header` for HTTP independently to each role. Programmatic reorg callers configure
header extraction; this does not enable legacy global modes or backup escape. `model.affinity.session_id_header` for HTTP header extraction; this does not
enable legacy global modes or backup escape.
Session and sticky policies do not create assignments for missing keys. A Session and sticky policies do not create assignments for missing keys. A
binding outside the candidates cannot win. A missing binding may invoke policy binding outside the candidates cannot win. A missing binding may invoke policy
fallback within the group; hard admission rejection remains an error. fallback within the group; hard admission rejection remains an error.
Sticky fallback supports `round_robin`, `random`, `power_of_two`, and `load_based`, Sticky fallback supports `round_robin`, `random`, `power_of_two`, and `load_based`,
with round-robin as the default. Nested fallbacks use `AllowAll`; the owning with round-robin as the default. Nested fallbacks use
policy explicitly checks the engine returned by its fallback. `AdmissionLimits::default()`; the owning policy explicitly checks the engine
returned by its fallback.
### Cache-aware behavior ### Cache-aware behavior
@@ -502,8 +504,8 @@ do not accept and ignore them.
- Preserve configured capacity, pending-prefill, and in-flight checks, including - Preserve configured capacity, pending-prefill, and in-flight checks, including
their missing-report behavior. Power-of-two applies admission to its selected their missing-report behavior. Power-of-two applies admission to its selected
engine; other policies explicitly place checks in their selection logic. engine; other policies explicitly place checks in their selection logic.
Other paths use `AllowAll` unless a check is configured. Never silently discard Other paths use `AdmissionLimits::default()` unless a check is configured.
a configured budget. Never silently discard a configured budget.
Listener and shutdown configuration, discovery, worker health and circuit Listener and shutdown configuration, discovery, worker health and circuit
breakers, tokenizer loading, request timeouts, sampling overrides, and logging breakers, tokenizer loading, request timeouts, sampling overrides, and logging
@@ -87,8 +87,9 @@ impl Policy for SessionAwarePolicy {
}); });
} }
// The nested power-of-two policy uses AllowAll. The session owner // The nested power-of-two policy uses AdmissionLimits::default().
// checks its chosen engine before creating or replacing a binding. // The session owner checks its chosen engine before creating or
// replacing a binding.
let mut pick = self.pick_fallback(engines, request).await?; let mut pick = self.pick_fallback(engines, request).await?;
let load = self.engine_load.capture_snapshot(Instant::now()); let load = self.engine_load.capture_snapshot(Instant::now());
self.check(&pick.engine, &load)?; self.check(&pick.engine, &load)?;
@@ -169,7 +169,8 @@ async fn configured_limits_reject_before_dispatch_and_admit_after_load_drops() {
worker_ids: Some([WorkerId("w".into())].into()), worker_ids: Some([WorkerId("w".into())].into()),
policy: Arc::new(policy), policy: Arc::new(policy),
}), }),
)]), )])
.unwrap(),
)] )]
.into(), .into(),
); );