1 Commits
Author SHA1 Message Date
QNan0513 fef023b64f [Fix] scheduler: Count all pending work in DP load snapshots
total_tokens DP balancing replaces its per-rank estimate with the rank's
latest load snapshot, so a snapshot that misses work already sent to the
rank lets the controller keep sending to it. Close three gaps:

- count grammar-queue requests in num_waiting_reqs and num_total_tokens
- count the in-progress chunked request's unprefilled tokens
- publish a snapshot once newly received requests are queued, since the
  one published after a forward predates the requests sent during it
2026-11-10 23:58:23 +08:00