Files
sglang/python
minke.yu 1a274df27e observability: fix prefetch ready latency to cover fast lookups
The first version stamped the request on its first in-flight
check_prefetch_progress() call. A fast L3 query can be drained by
_process_hicache_events before that check runs (the op is already gone
from ongoing_prefetch), so those requests produced no sample -- exactly
the common case.

Stamp at issue time in prefetch_from_storage() and observe once at
_retire_ongoing_prefetch(), classifying by operation.storage_hit_count.
Retire is the single exit for hit completion, below-threshold revoke,
anchor loss and host-capacity fallback, so hit and miss are both covered.

No behavior change; the hook is still behind SGLANG_BATCH_STAGE_TIMING.
2026-11-04 18:04:34 +08:00
..