Implements M2 of handoff/prefill-cpu-kv-overlap-design-20261106.md, behind
SGLANG_TRACE_MOONCAKE_DETAIL=1 (default off; reuses the existing trace stack).
- TransferKVChunk gains optional observation fields: chunk_seq, loop_seq,
send_kind, enqueue/last_enqueue monotonic+wall stamps, queue_wait_accum_ns,
queue_id, requeue_count, num_pages, approx_bytes, observe
- chunk_seq and enqueue timestamps are assigned at the real queue.put()
point (after every early return), so skipped paths cannot leave holes
- Mooncake transfer worker measures queue wait, source-event wait, indices
wait, KV/state/aux transfer, worker busy and residual; emits one
mooncake.kv_chunk span per attempt (+ queue/source/indices wait children)
and updates a per-room join summary
- request completion emits kv.request_join with chunk_count,
last_chunk_queue_wait/worker, final_exposed_ms, completion_poll_ms,
request_join_ms and first-enqueue-to-completion. First enqueue is stamped
by the Mooncake manager at the real queue.put(); no ReqTimeStats field is
added and the legacy queue_entry semantics are unchanged.
- nixl/mori add_transfer_request accept the observation kwarg (no-op there)
No scheduling/transfer semantics change; queue wait is excluded from worker
busy, and worker_busy excludes source/indices event waits. Unit tests: test_mooncake_chunk_detail.py (15 cases, including the
abort/worker interleaving and attempt-vs-total queue wait windows).