gateway g9i: keep large prefills off fast-P even without least-wait

regular_plan returned Default when regular_pool_least_wait=false, letting
the router policy pick across ALL prefill workers including the fast-P
node — ~1/3 of large prefills landed on fast-P under random policy
(observed 2026-10-12), polluting its small-prefill queue and L2. With a
fast-P configured, 'regular' now explicitly restricts to the regular
pool via the least-wait aggregate pick. Default is kept only when the
fast pool is empty (then it equals the regular pool anyway).

Co-authored-by: Kimi
This commit is contained in:
2026-10-12 08:58:03 +08:00
co-authored by Kimi
parent 6c9518fcd8
commit 71a72b8fea
+8 -1
View File
@@ -744,7 +744,14 @@ impl G9iState {
});
}
if !self.config.regular_pool_least_wait {
return PrefillPlan::Default;
// Default lets the router policy pick across ALL prefill workers,
// including fast-P — large prefills would land on the fast-P node
// and defeat its isolation (~1/3 of big prefills under random
// policy, observed 2026-10-12). With a fast-P configured, restrict
// "regular" explicitly to the regular pool (least-wait aggregate).
if self.fast_pool(prefill_workers).is_empty() {
return PrefillPlan::Default;
}
}
match picked {
Some(w) => {