[sgl-router] Fix reorg admission proxy test build after BucketResolver::new (#40537)

Co-authored-by: Kan Wu <wukanustc@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Shangming Cai
2026-09-21 16:42:29 +08:00
committed by GitHub
co-authored by Kan Wu Claude Fable 5.1
parent 8d08dfdab7
commit 630b1ef322
3 changed files with 18 additions and 14 deletions
+13 -11
View File
@@ -303,21 +303,23 @@ Session assignments are scoped by model, bucket ID, stage, and session key.
`SessionAwarePolicy::new(store, engine_load)` receives shared state; the caller
owns the store's idle timeout and eviction task. Missing or empty session keys
use power-of-two without creating assignments. A new or out-of-group binding
uses power-of-two with `AllowAll`, then the session policy checks its selected
engine before binding. A concurrent live assignment wins, but is checked before
returning it; rejection ends that attempt without rewriting the binding or
retrying another engine. Existing bindings are reused regardless of pressure
when admitted. Session policies can be attached independently to each role.
Programmatic reorg callers configure `model.affinity.session_id_header` for HTTP
header extraction; this does not enable legacy global modes or backup escape.
uses power-of-two with `AdmissionLimits::default()`, then the session policy
checks its selected engine before binding. A concurrent live assignment wins,
but is checked before returning it; rejection ends that attempt without
rewriting the binding or retrying another engine. Existing bindings are reused
regardless of pressure when admitted. Session policies can be attached
independently to each role. Programmatic reorg callers configure
`model.affinity.session_id_header` for HTTP header extraction; this does not
enable legacy global modes or backup escape.
Session and sticky policies do not create assignments for missing keys. A
binding outside the candidates cannot win. A missing binding may invoke policy
fallback within the group; hard admission rejection remains an error.
Sticky fallback supports `round_robin`, `random`, `power_of_two`, and `load_based`,
with round-robin as the default. Nested fallbacks use `AllowAll`; the owning
policy explicitly checks the engine returned by its fallback.
with round-robin as the default. Nested fallbacks use
`AdmissionLimits::default()`; the owning policy explicitly checks the engine
returned by its fallback.
### Cache-aware behavior
@@ -502,8 +504,8 @@ do not accept and ignore them.
- Preserve configured capacity, pending-prefill, and in-flight checks, including
their missing-report behavior. Power-of-two applies admission to its selected
engine; other policies explicitly place checks in their selection logic.
Other paths use `AllowAll` unless a check is configured. Never silently discard
a configured budget.
Other paths use `AdmissionLimits::default()` unless a check is configured.
Never silently discard a configured budget.
Listener and shutdown configuration, discovery, worker health and circuit
breakers, tokenizer loading, request timeouts, sampling overrides, and logging
@@ -87,8 +87,9 @@ impl Policy for SessionAwarePolicy {
});
}
// The nested power-of-two policy uses AllowAll. The session owner
// checks its chosen engine before creating or replacing a binding.
// The nested power-of-two policy uses AdmissionLimits::default().
// The session owner checks its chosen engine before creating or
// replacing a binding.
let mut pick = self.pick_fallback(engines, request).await?;
let load = self.engine_load.capture_snapshot(Instant::now());
self.check(&pick.engine, &load)?;
@@ -169,7 +169,8 @@ async fn configured_limits_reject_before_dispatch_and_admit_after_load_drops() {
worker_ids: Some([WorkerId("w".into())].into()),
policy: Arc::new(policy),
}),
)]),
)])
.unwrap(),
)]
.into(),
);