Files
sglang/python
ymk fe9c6574fe observability: trace KV transfer chunks end-to-end (M2)
Implements M2 of handoff/prefill-cpu-kv-overlap-design-20261106.md, behind
SGLANG_TRACE_MOONCAKE_DETAIL=1 (default off; reuses the existing trace stack).

- TransferKVChunk gains optional observation fields: chunk_seq, loop_seq,
  send_kind, enqueue/last_enqueue monotonic+wall stamps, queue_wait_accum_ns,
  queue_id, requeue_count, num_pages, approx_bytes, observe
- chunk_seq and enqueue timestamps are assigned at the real queue.put()
  point (after every early return), so skipped paths cannot leave holes
- Mooncake transfer worker measures queue wait, source-event wait, indices
  wait, KV/state/aux transfer, worker busy and residual; emits one
  mooncake.kv_chunk span per attempt (+ queue/source/indices wait children)
  and updates a per-room join summary
- request completion emits kv.request_join with chunk_count,
  last_chunk_queue_wait/worker, final_exposed_ms, completion_poll_ms,
  request_join_ms and first-enqueue-to-completion. First enqueue is stamped
  by the Mooncake manager at the real queue.put(); no ReqTimeStats field is
  added and the legacy queue_entry semantics are unchanged.
- nixl/mori add_transfer_request accept the observation kwarg (no-op there)

No scheduling/transfer semantics change; queue wait is excluded from worker
busy, and worker_busy excludes source/indices event waits. Unit tests: test_mooncake_chunk_detail.py (15 cases, including the
abort/worker interleaving and attempt-vs-total queue wait windows).
2026-11-06 20:01:56 +08:00
..