config: retire the multi-engine accommodation in the runtime context (#35022)

This commit is contained in:
Cheng Wan
2026-08-17 16:15:33 -07:00
committed by GitHub
parent bc312d185d
commit 2b278b4ac4
8 changed files with 172 additions and 283 deletions
+25 -25
View File
@@ -10,7 +10,7 @@ One container owns process-static runtime state: `sglang.srt.runtime_context.Run
| Tier | Accessor | Holds | Lifecycle |
|------|----------|-------|-----------|
| raw config seed | `get_server_args()` | the published `ServerArgs` — the startup record, for debugging, dumps and provenance. **Business code does not read fields off it**: the read ratchet pins that at zero, and "Reading config: the seed is off limits" below says what to read instead, which forms the ratchet sees, and what is outside it by construction (a runtime-computed name; a whole-object hand-off) | published at process entry; re-publish is **last-publish-wins** (in-process tokenizer build, multi-Engine) and re-projects the bags; read-only |
| raw config seed | `get_server_args()` | the published `ServerArgs` — the startup record, for debugging, dumps and provenance. **Business code does not read fields off it**: the read ratchet pins that at zero, and "Reading config: the seed is off limits" below says what to read instead, which forms the ratchet sees, and what is outside it by construction (a runtime-computed name; a whole-object hand-off) | published at process entry; re-publish is **last-publish-wins** (the tokenizer publish in the launcher process; sequential engine rebuild in one process, e.g. unit tests) and re-projects the bags; read-only |
| resolved config | `get_exec()` `get_memory()` `get_schedule()` `get_model()` `get_spec()` `get_serving()` `get_observability()` `get_disagg()` `get_lora()` `get_mm()` `get_device()` | namespace **config bags** — the single source of truth for resolved config; leaves are real attributes (dynamo-traceable) | projected from `server_args` at `publish`; mutated only via `get_context().override` |
| runtime flags | `get_flags()` | state that is *not* a pure function of config: `capture` (cuda-graph lifecycle), `moe` (ACTIVE backends, swappable), `dp` (DP-attention runtime flags) | materialized at subsystem init; groups offer `override()` for tests |
| resources | `get_resources()`, `get_stream(name)`, `get_buffer(name, factory)` | process-level handles: graph pools, EPLB state, EP dispatcher state, named side streams, workspace buffers | lazy; cleared by `reset_context()` |
@@ -79,11 +79,6 @@ re-projects its own bags, so a parent-side override is lost. Values that feed
construction before any bag exists (group init reads `server_args.tp_size`) have no
bag to override at all.
- **Nested publishes**: `get_context().preserve_config()` snapshots the enclosing
lifecycle (including its post-publish overrides) and reinstates it on exit. No
production caller is left — the draft build was the last one, and per-runner
values are constructor arguments now — so it survives for tests and for a future
construction step that genuinely has to publish a private copy.
### Reads that legitimately stay on a `ServerArgs` instance
@@ -110,28 +105,31 @@ bag to override at all.
the scope. When there is a runner in hand, read its
stamp; that is a different rule from "read the instance".
- **Per-instance boundaries** — the tokenizer-manager family, everything under
`entrypoints/`, and the tokenizer-process multimodal processors read
`self.server_args`: several `Engine`s can share one process, and the process-global
bags are last-publish-wins across engines. `base_gpu_id` also differs per engine,
so no process-global value can stand in for it —
`BaseMultimodalProcessor._fast_image_processor_device` is the shape to copy. (The
encode-server DP workers used to specialize a config copy for the same reason;
their device now travels as `MMEncoder(gpu_id=...)`.)
`entrypoints/`, and the tokenizer-process multimodal processors still read
`self.server_args` today. The old justification ("several `Engine`s can share
one process, bags are last-publish-wins across them") is **retracted** — owner
ruling (2026-08-15): a process holds at most one live config at a time
(concurrent multi-Engine is unsupported; sequential rebuild stays legal, unit
tests rely on it). These reads are scheduled to become bag reads in the
bag-read series; treat them as pinned debt, not as a boundary to imitate. What
genuinely stays per-instance is what differs per *worker* within one engine:
`base_gpu_id` travels as a constructor argument (`MMEncoder(gpu_id=...)`;
`BaseMultimodalProcessor._fast_image_processor_device` is the shape to copy).
- **Whole-object passes** (`f(server_args)` handing the instance along) keep the
supplied-instance contract; don't rewrite the parameter reads unless the
field is runtime-mutated (see the elastic-EP `ep_size` case in
`eplb/expert_location.py`) — **or the field is one that resolution fills in
and the callee runs in a process that has published.** That second case is
step-12 debt, not a style question: the record is destined to carry the
pinned debt, not a style question: the record is destined to carry the
user's raw input, so `server_args.page_size` inside a runner-owned
constructor will read the raw pre-resolution value instead of the effective
one. Debt means a decision, not automatically a bag read: pick where the
value should come from — usually the `get_*()` bag, sometimes a runner stamp
or a constructor argument (the per-mode attention pair and the encode-server
`gpu_id` above are dispositions of exactly this debt). And the per-instance
boundaries above stay exempt from this unless-clause: a multi-Engine site
must not become a process-global bag read even for a resolution-filled
field. `test_supplied_instance_exposure_ratchet.py`
`gpu_id` above are dispositions of exactly this debt). The per-instance
boundaries above are **not** exempt from this unless-clause (the multi-Engine
exemption is retracted); each one gets its own disposition.
`test_supplied_instance_exposure_ratchet.py`
pins the remaining set — three spellings of the read: `server_args.field`,
literal-name `getattr(server_args, "field", default)`, and the parked form
(`self.x = server_args` in a method that takes the parameter, read as
@@ -224,13 +222,15 @@ this).
`self.server_args.field` is still right for handed per-instance config (see
"Reads that legitimately stay on a ServerArgs instance" above for the full set —
per-instance boundaries and whole-object passes; there are no per-runner config
copies to read any more). The allow-list is the
tokenizer-manager family, `entrypoints/`, the tokenizer-process multimodal
processors, `GrammarManager`, `MMEncoder` — but not for one single reason:
copies to read any more). The allow-list is `GrammarManager` and `MMEncoder`;
the tokenizer-manager family, `entrypoints/`, and the tokenizer-process
multimodal processors sit beside it only as pinned debt — not for one single
reason:
- the tokenizer-manager family and `entrypoints/` are the multi-Engine case
proper: several of them can live in one process, so a bag read would answer
from whichever Engine published last;
- the tokenizer-manager family and `entrypoints/` are **pinned debt awaiting
conversion to bag reads** (the old multi-Engine justification is retracted —
one process, one live config); the reads still work today because the
instance carries resolved values;
- `GrammarManager` is a handed instance — it is constructed with the config its
owner hands it and never assumes a published namespace;
- `MMEncoder` publishes the very instance it is handed (`publish(server_args,
@@ -474,7 +474,7 @@ Never module-skip a test "until the migration settles" — seed the context inst
## Where to read the code
Key source files: `python/sglang/srt/runtime_context.py` (the container, every tier,
`publish`, `_ConfigBag`, `preserve_config`, `override_server_args`),
`publish`, `_ConfigBag`, `override_server_args`),
`python/sglang/srt/arg_groups/overrides.py` (override registry, passes,
`declare_late_resolution`), `python/sglang/srt/server_args.py` (`NS` metadata,
`Arg(..., resolvable=True)`, `__setattr__` strict guard), and the guardrail tests under