[Config] One writer for the declaration stash; no exception to the write seal (#38752)
This commit is contained in:
@@ -91,8 +91,11 @@ with what the operator typed, not with what resolution decided.**
|
||||
- **Late launcher-stage resolution (pre-publish)**: a few rules cannot run inside
|
||||
`__post_init__` — LoRA normalization, and the auto-parser detection that needs a
|
||||
tokenizer/chat-template load. They are resolution, not mutation, and they
|
||||
**declare** via `arg_groups.overrides.declare_late_resolution(server_args,
|
||||
source, **fields)`, which refuses the published instance. The declaration lands
|
||||
**declare** via `arg_groups.overrides.declare_resolution(server_args, source,
|
||||
**fields)`, the same call the rest of the pipeline makes; there is no
|
||||
`declare_late_resolution` any more. *When* a declaration is made is not
|
||||
something the code marks — the guardrails that used to read that marker
|
||||
cover these sites through the ordinary keyword scan instead. The declaration lands
|
||||
in the stash on that very object, so every holder of it carries the decision —
|
||||
the HTTP server, the multi-tokenizer workers it is serialized for, the
|
||||
schedulers it forks — and each of them publishes bags projected from it. The
|
||||
@@ -408,9 +411,38 @@ derivation cannot enumerate.
|
||||
|
||||
One consequence worth knowing: because the fields are the raw input, resolving a
|
||||
bare `dataclasses.replace` copy lands in the same place as the parent — the
|
||||
pipeline reads only its own input. `replace_resolved` is the way to copy a
|
||||
resolved record (it carries the declarations and the `model_config` memo, so the
|
||||
copy does not re-resolve at all).
|
||||
pipeline reads only its own input. **So a resolved record is not copied at
|
||||
all.** A caller that needs one field different for the process it is about to
|
||||
hand the record to — the Ray paths and their `dist_init_addr` — declares it on
|
||||
the record it holds (`declare_resolution`) and hands that over: the declaration
|
||||
travels inside the object, the receiving process projects its bags from it, and
|
||||
nothing re-resolves. There is no `ServerArgs.replace_resolved` any more, and the
|
||||
`model_config`-memo bug that copying used to cause (a copy marked resolved but
|
||||
arriving without the memo cannot refill it, because the guard refuses the write)
|
||||
is gone by construction rather than guarded.
|
||||
|
||||
A bag `override` cannot stand in for this. It is *not* because overriding needs
|
||||
a publish — `set_server_args` is what projects the bags and `override` works as
|
||||
soon as the context holds a record — but because `override` writes bag leaves
|
||||
and by contract never touches the record, so its effect cannot travel inside an
|
||||
object to another process.
|
||||
|
||||
### The declaration stash has one writer
|
||||
|
||||
Everything that decides configuration goes through
|
||||
`declare_resolution(server_args, source, **fields)`. It validates the names,
|
||||
refuses the published config (the stash is projected at publish and never
|
||||
again, so a later declaration is a silent no-op), and appends. The other names
|
||||
around it are spellings, not mechanisms:
|
||||
|
||||
| name | what it adds |
|
||||
|---|---|
|
||||
| `run_post_process_pass` | runs a pass at its slot and validates its return; declares through `declare_resolution`. A pass returning an **empty** dict is a validation, not a declaration, and stays legal on the published instance — `Engine(server_args=sa)` after `Engine.shutdown()` re-runs `check_server_args` on the very instance the context holds |
|
||||
| `record_foreign_defaults` | for a resolver this tree does not own (an out-of-tree platform plugin, a registered speculative algorithm), whose interface is to *assign* fields. It gets a stand-in whose reads fall through to `resolving_view`; what it assigned is declared. The record is never written, so the write seal has no exception. In-tree code does not go through it — `handle_platform_defaults` wraps the platform hook, and the in-tree speculative dispatcher is called directly, because handed the stand-in its own `declare_resolution` calls would stash on that instead |
|
||||
|
||||
`resolution_projection` is gone; the whole-object readback is
|
||||
`ServerArgs.resolved_dict()`, which is what `/server_info` and its gRPC and
|
||||
in-process twins report.
|
||||
|
||||
### Adding a model-specific config adjustment
|
||||
|
||||
@@ -531,8 +563,10 @@ ONE thread — do not design for TBO threads that don't exist.
|
||||
|
||||
## Guardrails (these fail CI; what to do when they fire)
|
||||
|
||||
1. **Strict mutation guard** (always on): bare `server_args.x = ...` after resolution
|
||||
raises unconditionally in `ServerArgs.__setattr__` — this *is* the guarantee that
|
||||
1. **Strict mutation guard** (always on, and with no exception): bare
|
||||
`server_args.x = ...` after resolution raises unconditionally in
|
||||
`ServerArgs.__setattr__` — the named lift that out-of-tree plugins used to
|
||||
ask for is gone, they assign onto a stand-in instead — this *is* the guarantee that
|
||||
no writer can desync the bags, so there is no writer ratchet any more. Change
|
||||
resolved config with `get_context().override`; hand a per-runner value to its
|
||||
runner as a constructor argument. Projected bags are sealed the same way (leaf
|
||||
@@ -621,7 +655,7 @@ Never module-skip a test "until the migration settles" — seed the context inst
|
||||
Key source files: `python/sglang/srt/runtime_context.py` (the container, every tier,
|
||||
`publish`, `_ConfigBag`, `override_server_args`),
|
||||
`python/sglang/srt/arg_groups/overrides.py` (override registry, passes,
|
||||
`declare_late_resolution`), `python/sglang/srt/server_args.py` (`NS` metadata,
|
||||
`declare_resolution` and the spellings around it), `python/sglang/srt/server_args.py` (`NS` metadata,
|
||||
`Arg(..., resolvable=True)`, `__setattr__` strict guard), and the guardrail tests under
|
||||
`test/registered/unit/` (`test_server_args_mutation_ratchet.py`,
|
||||
`test_global_config_read_ratchet.py`,
|
||||
|
||||
Reference in New Issue
Block a user