Files
sglang/python
minke.yu 309af6e6e1 observability: record prefetch ready latency on the L3 hit success path
_handle_prefetch_result() retires the operation with a direct
del ongoing_prefetch[request] instead of _retire_ongoing_prefetch(), so
the ready-latency hook only ever saw miss/revoke paths: with a cache-hit
workload storage_prefetch_ready_seconds{result="hit"} stayed at 0 even
though storage_prefetch_hit_tokens_total was moving ~312k tok/s.

Observe right after the del, classifying by loaded_from_storage.
2026-11-04 19:14:02 +08:00
..