config: resolution declares, and nothing writes a field (#36618)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5
parent
d1f14431fd
commit
bd4bb1781a
@@ -11,7 +11,7 @@ One container owns process-static runtime state: `sglang.srt.runtime_context.Run
|
||||
| Tier | Accessor | Holds | Lifecycle |
|
||||
|------|----------|-------|-----------|
|
||||
| raw config seed | `get_server_args()` | the published `ServerArgs` — the startup record, for debugging, dumps and provenance. **Business code does not read fields off it**: the read ratchet pins that at zero, and "Reading config: the seed is off limits" below says what to read instead, which forms the ratchet sees, and what is outside it by construction (a runtime-computed name; a whole-object hand-off) | published at process entry; re-publish is **last-publish-wins** (the tokenizer publish in the launcher process; sequential engine rebuild in one process, e.g. unit tests) and re-projects the bags; read-only |
|
||||
| resolved config | `get_exec()` `get_memory()` `get_schedule()` `get_model()` `get_spec()` `get_serving()` `get_observability()` `get_disagg()` `get_lora()` `get_mm()` `get_device()` | namespace **config bags** — the single source of truth for resolved config; leaves are real attributes (dynamo-traceable). Each is a **module function of no arguments**, and a module binds the name once: `manager.get_disagg()`, `self.get_disagg = get_disagg`, or a same-named import next to the bag one (`from model_loader import get_model`) all import fine and fail only when that path runs. `ruff --select F811` catches the import collision; `RuntimeContext` has no bag-named member and no `__getattr__`, so the member-call shapes are an `AttributeError` at call time — give it a delegating `__getattr__` and they go silent instead | projected from `server_args` at `publish`; mutated only via `get_context().override` |
|
||||
| resolved config | `get_exec()` `get_memory()` `get_schedule()` `get_model()` `get_spec()` `get_serving()` `get_observability()` `get_disagg()` `get_lora()` `get_mm()` `get_device()` | namespace **config bags** — the single source of truth for resolved config; leaves are real attributes (dynamo-traceable). Each is a **module function of no arguments**, and a module binds the name once: `manager.get_disagg()`, `self.get_disagg = get_disagg`, or a same-named import next to the bag one (`from model_loader import get_model`) all import fine and fail only when that path runs. `ruff --select F811` catches the import collision; `RuntimeContext` has no bag-named member and no `__getattr__`, so the member-call shapes are an `AttributeError` at call time — give it a delegating `__getattr__` and they go silent instead | projected at `publish` from the declarations over `server_args`' raw fields; mutated only via `get_context().override` |
|
||||
| runtime flags | `get_flags()` | state that is *not* a pure function of config: `capture` (cuda-graph lifecycle), `moe` (ACTIVE backends, swappable), `dp` (DP-attention runtime flags) | materialized at subsystem init; groups offer `override()` for tests |
|
||||
| resources | `get_resources()`, `get_stream(name)`, `get_buffer(name, factory)` | process-level handles: graph pools, EPLB state, EP dispatcher state, named side streams, workspace buffers | lazy; cleared by `reset_context()` |
|
||||
| per-forward | `get_forward()` | forward-scoped flags (multi-stream switch, MoE output buffer, attn-TP inputs, extend-in-batch) | contextvar-backed; `scoped(**kw)` restores on exit; new threads see defaults |
|
||||
@@ -24,9 +24,8 @@ flags/resources/forward tiers.
|
||||
|
||||
**`ServerArgs` holds the raw input and nothing else. Resolution writes no field:
|
||||
it declares, and the declarations are what the namespace bags are projected from.
|
||||
Business code never reads the record for a decision — and after this cut, a field
|
||||
read there answers with what the operator typed, not with what resolution
|
||||
decided.**
|
||||
Business code never reads the record for a decision: a field read there answers
|
||||
with what the operator typed, not with what resolution decided.**
|
||||
|
||||
- Every publishing process entry calls `publish(server_args, role=...)`
|
||||
(`run_scheduler_process`, the Ray `SchedulerActor`, the DP controller, tokenizer,
|
||||
@@ -39,8 +38,8 @@ decided.**
|
||||
`run_multi_detokenizer_router_process`: it *is* handed a `ServerArgs`, and uses
|
||||
it only for `configure_logger(server_args)` today, so it has nothing to publish
|
||||
for — a bag read added under that entry needs a `publish` at the entry first.
|
||||
`publish` snapshots the resolved field
|
||||
values into the config bags; the accessors (`get_exec()` etc.) fail closed before it
|
||||
`publish` projects the config bags from the declarations over the record's raw
|
||||
fields; the accessors (`get_exec()` etc.) fail closed before it
|
||||
runs. `role` records which process type published, and keys per-role namespace
|
||||
enforcement: `SGLANG_ROLE_NAMESPACES=record` audits which namespaces each role's
|
||||
process actually reads (per-pair persisted via `SGLANG_ROLE_NAMESPACES_OUT`;
|
||||
@@ -161,10 +160,10 @@ bag to override at all.
|
||||
and the callee runs in a process that has published.** That second case is a
|
||||
decision, not a style question: the record carries the user's raw input, so a
|
||||
resolution-filled field read off it inside a runner-owned constructor answers
|
||||
with the pre-resolution value instead of the effective one. Debt means a decision, not automatically a bag read: pick where the
|
||||
value should come from — usually the `get_*()` bag, sometimes a runner stamp
|
||||
or a constructor argument (the per-mode attention pair and the encode-server
|
||||
`gpu_id` above are dispositions of exactly this debt). The per-instance
|
||||
with the pre-resolution value instead of the effective one. The answer is not
|
||||
automatically a bag read: pick where the value should come from — usually the
|
||||
`get_*()` bag, sometimes a runner stamp or a constructor argument (the per-mode
|
||||
attention pair and the encode-server `gpu_id` above are both this). The per-instance
|
||||
boundaries above are **not** exempt from this unless-clause (the multi-Engine
|
||||
exemption is retracted); each one gets its own disposition.
|
||||
`test_supplied_instance_exposure_ratchet.py`
|
||||
|
||||
Reference in New Issue
Block a user