config: one control-plane log for the process (#35028)

This commit is contained in:
Cheng Wan
2026-08-17 16:19:00 -07:00
committed by GitHub
parent c70c7d72a8
commit b3c8f0d923
8 changed files with 245 additions and 95 deletions
@@ -54,6 +54,32 @@ resolved configuration lives in the namespace bags.**
There is **no write-through** to the `ServerArgs` instance — it stays pristine.
There is no in-place mutation entry on the instance at all: it is read-only after
resolution.
- **Reading a leaf when the caller holds the field *name*** (a readback endpoint,
a control-plane handler): `get_context().config_leaf(name)` — the read side of
`override`. It resolves the flat name through the same `NS` map the write side
uses and raises on a name that is not a config leaf. Code that knows its field
when it is written reads the bag leaf directly; `config_leaf` is for
name-driven code, not a way around the seed ratchet.
- **Post-startup control-plane changes** — a weight update, a HiCache mirror
attach, a parser resolved from the chat template — go through
`TokenizerManager.record_config_updates(source, **fields)`, a named wrapper
over `get_context().override`. One process keeps one log: the request dumps
ship `get_context().overrides_log()`, and `config_value(name)` /
`resolved_config_dict(base)` answer from the bags. The exposure ratchet
resolves the wrapper, so a field recorded through it joins the post-publish
override surface exactly like a direct `override` and needs the same ordering
judgment against any supplied-instance read of it
(`test_supplied_instance_exposure_ratchet.py`).
- **`model_path` and `served_model_name` are answered off the manager.** Both are
`NS` leaves and `override` accepts them, but the tokenizer-side weight reload
records only `load_format` and writes the two path fields as `TokenizerManager`
attributes (`_MANAGER_OWNED_FIELDS`); `config_value` and `resolved_config_dict`
overlay them on top of the bags. Bags do not cross a process boundary (above),
so recording those two in the tokenizer process would leave every other
process's bag on the old path while the log claimed a process-wide change. The
scheduler rewrites its own copy where the reload happens —
`ModelRunner.update_model_fields` overrides `model_path` / `load_format` for
the target runner.
- **Late launcher-stage resolution (pre-publish)**: a few rules cannot run inside
`__post_init__` — LoRA normalization, and the auto-parser detection that needs a
tokenizer/chat-template load. They are resolution, not mutation, and they write
@@ -143,6 +169,51 @@ bag to override at all.
named `server_args`), and a factory whose contract is "build X from the record
you are handed" (`create_kt_config_from_server_args`, `DllmConfig.from_server_args`).
### Four ways a config sweep breaks something no test runs
Each of these shipped in a review round and cost a real defect; each now has a
guard, named here so the next sweep checks the same four things by hand first.
1. **The other implementations of an interface.** Dropping a parameter means
auditing implementers, not just callers: `CustomSpecAlgo` is the plugin
base for speculative algorithms, and the dispatch calls it with the
built-in's argument list. Nothing in the tree implements it, so only a
plugin user hits the `TypeError`.
Guard: `test_plugin_hook_signatures.py`.
2. **Publish order inside a process entry, not per file.** A file containing a
`publish` says nothing about whether a given read runs before it. Spawned
workers (`MMEncoder` for encoder DP/TP, the Ray scheduler actor) start with
an empty context, so a bag read above the publish raises only there.
Guard: `test_publish_precedes_bag_reads.py`.
3. **The role namespace a process publishes under.** `ROLE_NAMESPACE_SETS`
narrows what each role may read; the DP controller is audited for `exec`
alone. A helper that reaches for another namespace passes every default-mode
test and aborts startup under `SGLANG_ROLE_NAMESPACES=enforce`. Prefer
answering from the caller's own namespaces over widening the set.
4. **Sibling surfaces of a readback.** Changing what one entry point reports
means enumerating the others: HTTP, gRPC and in-process `Engine` each have
their own server-info and model-info, and each passes its own tests while
its users lose the field.
Guard: `test_effective_state_surfaces.py`.
A fifth, from the same rounds: the accessor **name itself**. Called as an
object member (`manager.get_disagg()`), or shadowed by a same-named import
(`from model_loader import get_model` next to the model bag, where the later
import silently wins and the loader call gets a zero-argument bag), it imports
fine and fails only when that path runs. The invariant is one line: the name
means the process-wide bag, takes no arguments, and is bound once per module.
`ruff --select F811` catches the import collision; the member-call shapes are an
`AttributeError` at call time only because `RuntimeContext` has no bag-named
member and no `__getattr__` -- a delegating `__getattr__` would make them
silent, and that is when this needs a guard again rather than a rule.
Write these guards over a **derived** set, never a hand-kept list: an entry
naming a function that no longer exists, or a field list missing the one field
nobody migrated, passes green forever. Both happened here -- a `_ENTRY_POINTS`
row for a method the Ray actor does not have, and an effective-field set
without `load_format` -- and both were invisible because the assertion had
slack (`>= len(...) - 1`) or compared key names instead of value sources.
### `get_parallel()`: config leaves vs live topology
Config leaves (`nccl_port`, `enable_dp_attention`, `dp_size`, `ep_size`,
@@ -182,6 +253,9 @@ this).
parallel bag's own leaf — are what see post-publish overrides. Only the
instance-derived accessors (the ones with no leaf to read) answer from the
startup record and therefore do not.
- **a leaf the caller names at runtime** (a readback reporting a list of fields)
`get_context().config_leaf(name)`; it resolves the name through `NS` and
raises on a non-leaf. A call site that knows its field reads the bag leaf.
- **the live topology** → `get_parallel()`.
- **a value derived from published leaves** → an accessor in `runtime_context` that
derives it *from the bags*: `mamba_extra_buffer_enabled()` /