config: resolution declares, and nothing writes a field (#36618)

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Cheng Wan
2026-08-27 12:53:17 -07:00
committed by GitHub
co-authored by Claude Opus 5
parent d1f14431fd
commit bd4bb1781a
5 changed files with 296 additions and 37 deletions
+9 -10
View File
@@ -11,7 +11,7 @@ One container owns process-static runtime state: `sglang.srt.runtime_context.Run
| Tier | Accessor | Holds | Lifecycle |
|------|----------|-------|-----------|
| raw config seed | `get_server_args()` | the published `ServerArgs` — the startup record, for debugging, dumps and provenance. **Business code does not read fields off it**: the read ratchet pins that at zero, and "Reading config: the seed is off limits" below says what to read instead, which forms the ratchet sees, and what is outside it by construction (a runtime-computed name; a whole-object hand-off) | published at process entry; re-publish is **last-publish-wins** (the tokenizer publish in the launcher process; sequential engine rebuild in one process, e.g. unit tests) and re-projects the bags; read-only |
| resolved config | `get_exec()` `get_memory()` `get_schedule()` `get_model()` `get_spec()` `get_serving()` `get_observability()` `get_disagg()` `get_lora()` `get_mm()` `get_device()` | namespace **config bags** — the single source of truth for resolved config; leaves are real attributes (dynamo-traceable). Each is a **module function of no arguments**, and a module binds the name once: `manager.get_disagg()`, `self.get_disagg = get_disagg`, or a same-named import next to the bag one (`from model_loader import get_model`) all import fine and fail only when that path runs. `ruff --select F811` catches the import collision; `RuntimeContext` has no bag-named member and no `__getattr__`, so the member-call shapes are an `AttributeError` at call time — give it a delegating `__getattr__` and they go silent instead | projected from `server_args` at `publish`; mutated only via `get_context().override` |
| resolved config | `get_exec()` `get_memory()` `get_schedule()` `get_model()` `get_spec()` `get_serving()` `get_observability()` `get_disagg()` `get_lora()` `get_mm()` `get_device()` | namespace **config bags** — the single source of truth for resolved config; leaves are real attributes (dynamo-traceable). Each is a **module function of no arguments**, and a module binds the name once: `manager.get_disagg()`, `self.get_disagg = get_disagg`, or a same-named import next to the bag one (`from model_loader import get_model`) all import fine and fail only when that path runs. `ruff --select F811` catches the import collision; `RuntimeContext` has no bag-named member and no `__getattr__`, so the member-call shapes are an `AttributeError` at call time — give it a delegating `__getattr__` and they go silent instead | projected at `publish` from the declarations over `server_args`' raw fields; mutated only via `get_context().override` |
| runtime flags | `get_flags()` | state that is *not* a pure function of config: `capture` (cuda-graph lifecycle), `moe` (ACTIVE backends, swappable), `dp` (DP-attention runtime flags) | materialized at subsystem init; groups offer `override()` for tests |
| resources | `get_resources()`, `get_stream(name)`, `get_buffer(name, factory)` | process-level handles: graph pools, EPLB state, EP dispatcher state, named side streams, workspace buffers | lazy; cleared by `reset_context()` |
| per-forward | `get_forward()` | forward-scoped flags (multi-stream switch, MoE output buffer, attn-TP inputs, extend-in-batch) | contextvar-backed; `scoped(**kw)` restores on exit; new threads see defaults |
@@ -24,9 +24,8 @@ flags/resources/forward tiers.
**`ServerArgs` holds the raw input and nothing else. Resolution writes no field:
it declares, and the declarations are what the namespace bags are projected from.
Business code never reads the record for a decision — and after this cut, a field
read there answers with what the operator typed, not with what resolution
decided.**
Business code never reads the record for a decision: a field read there answers
with what the operator typed, not with what resolution decided.**
- Every publishing process entry calls `publish(server_args, role=...)`
(`run_scheduler_process`, the Ray `SchedulerActor`, the DP controller, tokenizer,
@@ -39,8 +38,8 @@ decided.**
`run_multi_detokenizer_router_process`: it *is* handed a `ServerArgs`, and uses
it only for `configure_logger(server_args)` today, so it has nothing to publish
for — a bag read added under that entry needs a `publish` at the entry first.
`publish` snapshots the resolved field
values into the config bags; the accessors (`get_exec()` etc.) fail closed before it
`publish` projects the config bags from the declarations over the record's raw
fields; the accessors (`get_exec()` etc.) fail closed before it
runs. `role` records which process type published, and keys per-role namespace
enforcement: `SGLANG_ROLE_NAMESPACES=record` audits which namespaces each role's
process actually reads (per-pair persisted via `SGLANG_ROLE_NAMESPACES_OUT`;
@@ -161,10 +160,10 @@ bag to override at all.
and the callee runs in a process that has published.** That second case is a
decision, not a style question: the record carries the user's raw input, so a
resolution-filled field read off it inside a runner-owned constructor answers
with the pre-resolution value instead of the effective one. Debt means a decision, not automatically a bag read: pick where the
value should come from — usually the `get_*()` bag, sometimes a runner stamp
or a constructor argument (the per-mode attention pair and the encode-server
`gpu_id` above are dispositions of exactly this debt). The per-instance
with the pre-resolution value instead of the effective one. The answer is not
automatically a bag read: pick where the value should come from — usually the
`get_*()` bag, sometimes a runner stamp or a constructor argument (the per-mode
attention pair and the encode-server `gpu_id` above are both this). The per-instance
boundaries above are **not** exempt from this unless-clause (the multi-Engine
exemption is retracted); each one gets its own disposition.
`test_supplied_instance_exposure_ratchet.py`