config: one control-plane log for the process (#35028)
This commit is contained in:
@@ -54,6 +54,32 @@ resolved configuration lives in the namespace bags.**
|
||||
There is **no write-through** to the `ServerArgs` instance — it stays pristine.
|
||||
There is no in-place mutation entry on the instance at all: it is read-only after
|
||||
resolution.
|
||||
- **Reading a leaf when the caller holds the field *name*** (a readback endpoint,
|
||||
a control-plane handler): `get_context().config_leaf(name)` — the read side of
|
||||
`override`. It resolves the flat name through the same `NS` map the write side
|
||||
uses and raises on a name that is not a config leaf. Code that knows its field
|
||||
when it is written reads the bag leaf directly; `config_leaf` is for
|
||||
name-driven code, not a way around the seed ratchet.
|
||||
- **Post-startup control-plane changes** — a weight update, a HiCache mirror
|
||||
attach, a parser resolved from the chat template — go through
|
||||
`TokenizerManager.record_config_updates(source, **fields)`, a named wrapper
|
||||
over `get_context().override`. One process keeps one log: the request dumps
|
||||
ship `get_context().overrides_log()`, and `config_value(name)` /
|
||||
`resolved_config_dict(base)` answer from the bags. The exposure ratchet
|
||||
resolves the wrapper, so a field recorded through it joins the post-publish
|
||||
override surface exactly like a direct `override` and needs the same ordering
|
||||
judgment against any supplied-instance read of it
|
||||
(`test_supplied_instance_exposure_ratchet.py`).
|
||||
- **`model_path` and `served_model_name` are answered off the manager.** Both are
|
||||
`NS` leaves and `override` accepts them, but the tokenizer-side weight reload
|
||||
records only `load_format` and writes the two path fields as `TokenizerManager`
|
||||
attributes (`_MANAGER_OWNED_FIELDS`); `config_value` and `resolved_config_dict`
|
||||
overlay them on top of the bags. Bags do not cross a process boundary (above),
|
||||
so recording those two in the tokenizer process would leave every other
|
||||
process's bag on the old path while the log claimed a process-wide change. The
|
||||
scheduler rewrites its own copy where the reload happens —
|
||||
`ModelRunner.update_model_fields` overrides `model_path` / `load_format` for
|
||||
the target runner.
|
||||
- **Late launcher-stage resolution (pre-publish)**: a few rules cannot run inside
|
||||
`__post_init__` — LoRA normalization, and the auto-parser detection that needs a
|
||||
tokenizer/chat-template load. They are resolution, not mutation, and they write
|
||||
@@ -143,6 +169,51 @@ bag to override at all.
|
||||
named `server_args`), and a factory whose contract is "build X from the record
|
||||
you are handed" (`create_kt_config_from_server_args`, `DllmConfig.from_server_args`).
|
||||
|
||||
### Four ways a config sweep breaks something no test runs
|
||||
|
||||
Each of these shipped in a review round and cost a real defect; each now has a
|
||||
guard, named here so the next sweep checks the same four things by hand first.
|
||||
|
||||
1. **The other implementations of an interface.** Dropping a parameter means
|
||||
auditing implementers, not just callers: `CustomSpecAlgo` is the plugin
|
||||
base for speculative algorithms, and the dispatch calls it with the
|
||||
built-in's argument list. Nothing in the tree implements it, so only a
|
||||
plugin user hits the `TypeError`.
|
||||
Guard: `test_plugin_hook_signatures.py`.
|
||||
2. **Publish order inside a process entry, not per file.** A file containing a
|
||||
`publish` says nothing about whether a given read runs before it. Spawned
|
||||
workers (`MMEncoder` for encoder DP/TP, the Ray scheduler actor) start with
|
||||
an empty context, so a bag read above the publish raises only there.
|
||||
Guard: `test_publish_precedes_bag_reads.py`.
|
||||
3. **The role namespace a process publishes under.** `ROLE_NAMESPACE_SETS`
|
||||
narrows what each role may read; the DP controller is audited for `exec`
|
||||
alone. A helper that reaches for another namespace passes every default-mode
|
||||
test and aborts startup under `SGLANG_ROLE_NAMESPACES=enforce`. Prefer
|
||||
answering from the caller's own namespaces over widening the set.
|
||||
4. **Sibling surfaces of a readback.** Changing what one entry point reports
|
||||
means enumerating the others: HTTP, gRPC and in-process `Engine` each have
|
||||
their own server-info and model-info, and each passes its own tests while
|
||||
its users lose the field.
|
||||
Guard: `test_effective_state_surfaces.py`.
|
||||
|
||||
A fifth, from the same rounds: the accessor **name itself**. Called as an
|
||||
object member (`manager.get_disagg()`), or shadowed by a same-named import
|
||||
(`from model_loader import get_model` next to the model bag, where the later
|
||||
import silently wins and the loader call gets a zero-argument bag), it imports
|
||||
fine and fails only when that path runs. The invariant is one line: the name
|
||||
means the process-wide bag, takes no arguments, and is bound once per module.
|
||||
`ruff --select F811` catches the import collision; the member-call shapes are an
|
||||
`AttributeError` at call time only because `RuntimeContext` has no bag-named
|
||||
member and no `__getattr__` -- a delegating `__getattr__` would make them
|
||||
silent, and that is when this needs a guard again rather than a rule.
|
||||
|
||||
Write these guards over a **derived** set, never a hand-kept list: an entry
|
||||
naming a function that no longer exists, or a field list missing the one field
|
||||
nobody migrated, passes green forever. Both happened here -- a `_ENTRY_POINTS`
|
||||
row for a method the Ray actor does not have, and an effective-field set
|
||||
without `load_format` -- and both were invisible because the assertion had
|
||||
slack (`>= len(...) - 1`) or compared key names instead of value sources.
|
||||
|
||||
### `get_parallel()`: config leaves vs live topology
|
||||
|
||||
Config leaves (`nccl_port`, `enable_dp_attention`, `dp_size`, `ep_size`,
|
||||
@@ -182,6 +253,9 @@ this).
|
||||
parallel bag's own leaf — are what see post-publish overrides. Only the
|
||||
instance-derived accessors (the ones with no leaf to read) answer from the
|
||||
startup record and therefore do not.
|
||||
- **a leaf the caller names at runtime** (a readback reporting a list of fields)
|
||||
→ `get_context().config_leaf(name)`; it resolves the name through `NS` and
|
||||
raises on a non-leaf. A call site that knows its field reads the bag leaf.
|
||||
- **the live topology** → `get_parallel()`.
|
||||
- **a value derived from published leaves** → an accessor in `runtime_context` that
|
||||
derives it *from the bags*: `mamba_extra_buffer_enabled()` /
|
||||
|
||||
Reference in New Issue
Block a user