Add scripted-runtime harness core and wire scheduler/IPC hooks (#27411)

2026-06-06 09:07:01 +08:00
parent 21201ef718
commit dd176387c6
24 changed files with 1313 additions and 4 deletions
@@ -0,0 +1,29 @@
+---
+name: scripted-runtime-notes
+description: Requirements for the SGLang scripted runtime, chiefly when to add (vs not add) a harness API. Use for anything related to the scripted runtime.
+---
+
+# Scripted Runtime — Notes
+
+Notes for anything related to the SGLang scripted runtime.
+
+## When to Add an API
+
+Tests read `r.req.*` and `t._scheduler.*` directly — there is no encapsulation boundary. A thin wrapper buys zero isolation; it only grows the surface.
+
+Add an API only if it does real work:
+
+1. **Control primitive** — drives the engine through a real path (`start_req`, `pause_generation`, `abort`, `evict_radix`, `exhaust_kv`). Reuse the real path; never hand-mutate state.
+2. **Hook-backed** — value cannot be read from a snapshot; accumulate via `scheduler_hook.on_run_batch` or the recv proxy (`chunks_done`). Read-only; never monkey-patch; never add `*_count` to `srt/`.
+3. **Multi-structure derivation, widely reused** — scans `chunked_req` + `waiting_queue` + `running_batch` + `last_batch` (`is_idle`, `status`, `batch_composition`).
+
+Else: don't. Read `r.req.X` / `t._scheduler.X` in the test; inline single-use accessors.
+
+Never:
+
+- Weaken an assertion to fit a missing probe.
+- Probe implementation details ("field non-None", "which branch ran") — assert the consequence.
+
+## Other Tips
+
+- **Engine-self-driven behavior: drive the real loop, don't call the private.** Never synchronously call a scheduler private (e.g. `scheduler._abort_on_waiting_timeout()`) from the harness/test — it runs at the wrong loop phase, bypasses the ordered `recv_requests` → `process_input_requests` injection, and can fire in states the real loop never reaches (e.g. while paused). For sweeps the engine runs itself (timeout/idle), enable the config/env and advance the loop with `yield`.