[Config] Round 6.3: the record remembers how it was asked for, and is sealed while resolution runs (#38048)

Third of four; stacked on #38047. Two small changes, both about the same thing:
the record holds the operator's input, and nothing else should be true of it.

## `/server_info` can answer what was actually typed

It reports `resolved_dict()` -- what resolution decided. There was no way to ask
the other question, and the two are not derivable from each other: a field
nobody set reads the same as one set to the value resolution would have picked
anyway.

The launcher stores the arguments it parsed and the in-process `Engine` stores
the call that built the record. All three readbacks report it beside the
resolved values, so both surfaces come back in one request: HTTP `/server_info`,
`Engine.get_server_info`, and the gRPC bridge's -- the last one builds from
`resolved_dict()` and would otherwise have been the one surface of the three
that answers only "what resolution decided".
It rides on the record rather than in a field -- it describes how the
configuration was asked for, so it is not part of the configuration: no CLI
flag, no namespace, not in the bags. Being on the record is what lets a
subprocess copy answer the same question the launcher can, and
`replace_resolved` carries it because a copy was launched by whatever launched
its parent.

The crash dump already collected all four surfaces (`server_args`,
`config_updates`, `resolved_config`, `launch_command`); this is the one that
`/server_info` was missing.

## The record is sealed for the length of resolution

The read-only guard armed on `_resolution_finished`, so for the whole run of the
pipeline nothing stopped a resolver from assigning a field. Nothing in `srt/`
does -- 0 assignments statically, and 0 writes observed across the launch-shape
matrix with a watching `__setattr__` -- but that was a convention, and the
defect it permits is invisible: a value a resolver wrote onto the record is
indistinguishable from a value the operator typed, which is the one distinction
the record exists to preserve.

It now arms when resolution starts. A resolver that assigns a field fails at
boot with a message naming `declare_resolution`, which is where the decision
belongs: the stash carries a source and leaves the input intact.

`declare_direct_writes` asks for the seal by name through `record_writable`. It
hands the record to an out-of-tree platform plugin that sets fields on it; those
implementations cannot be converted by editing a resolver here, so the write
stays and the diff is captured into the stash afterwards. Naming the exception
is the point -- an in-tree resolver reaching for it is doing something it should
be declaring.

## Verification

Costs nothing: the 211 test-side assignments all happen before `resolve_once`,
which a post-resolution write already refused. A full registered-unit sweep
(648 files) against the stack's merge-base: 19 failures on both sides, the same
19, none of them config. Driving a
deliberate write into a real handler produces the new error, so the seal is
tested by more than its own unit test.
This commit is contained in:
Cheng Wan
2026-09-06 21:41:14 -07:00
committed by GitHub
parent ed82def55f
commit 98f69ccbf3
6 changed files with 219 additions and 17 deletions
+7 -1
View File
@@ -250,7 +250,13 @@ def declare_direct_writes(
for field in dataclasses.fields(server_args)
}
already = len(getattr(server_args, "_resolved_overrides", None) or ())
result = resolve(server_args)
# The one place the input seal comes off. The plugin writes the record;
# the diff below captures what it moved into the stash so the projection
# and the bags carry it.
from sglang.srt.server_args import record_writable
with record_writable(server_args):
result = resolve(server_args)
stash = getattr(server_args, "_resolved_overrides", None)
if stash is None:
stash = []
+9
View File
@@ -261,6 +261,14 @@ class Engine(EngineScoreMixin, EngineBase):
# Do not print logs by default
kwargs["log_level"] = "error"
server_args = self.server_args_class(**kwargs)
# There was no command line, so the call is what the operator
# asked for. `log_level` is filled in above when absent, so it
# shows here even when the caller did not pass it.
object.__setattr__(
server_args,
"_launch_command",
"Engine(" + ", ".join(f"{k}={v!r}" for k, v in kwargs.items()) + ")",
)
self.server_args = server_args
logger.info(f"server_args={server_args.resolved_dict()}")
@@ -1362,6 +1370,7 @@ class Engine(EngineScoreMixin, EngineBase):
return msgspec_to_builtins(
{
**self.tokenizer_manager.server_args.resolved_dict(),
"launch_command": self.tokenizer_manager.server_args.launch_command,
**self._scheduler_init_result.scheduler_infos[0],
"startup_time": self.tokenizer_manager.startup_time,
"internal_states": internal_states,
@@ -425,6 +425,11 @@ class RuntimeHandle:
def get_server_info(self) -> str:
result: Dict[str, Any] = self.tokenizer_manager.server_args.resolved_dict()
# `resolved_dict` answers with what resolution decided; the launch
# command answers with what was asked for, and the two are not
# derivable from each other. The HTTP and in-process readbacks both
# carry it, so this one does too.
result["launch_command"] = self.tokenizer_manager.server_args.launch_command
result.update(self.scheduler_info)
result["kv_events"] = describe_kv_events_publisher(
self.tokenizer_manager.server_args
+5 -2
View File
@@ -812,8 +812,10 @@ async def get_server_info():
async def server_info():
"""The startup configuration, plus live scheduler state.
The values here are the resolution result: what the launcher was given,
with every decision resolution made applied over it. Fields the control plane changes
Two surfaces, deliberately both: the field values are the resolution
result -- what the launcher was given with every decision resolution made
applied over it -- and `launch_command` is what was actually asked for,
which no amount of reading the resolved values recovers. Fields the control plane changes
after publication -- the model a weight update swapped in, its load format,
an operator-set weight version -- are reported by `/model_info`, and the
HiCache mirror by `GET /hicache/storage-backend`.
@@ -828,6 +830,7 @@ async def server_info():
return msgspec_to_builtins(
{
**server_args.resolved_dict(),
"launch_command": server_args.launch_command,
**_global_state.scheduler_info,
"startup_time": _global_state.tokenizer_manager.startup_time,
"internal_states": internal_states,
+77 -14
View File
@@ -40,6 +40,7 @@ import functools
import logging
import tempfile
import uuid
from contextlib import contextmanager
from typing import Any, Dict, List, Optional
from sglang.kernels.ops.kv_canary.consts import RealKvHashMode
@@ -269,6 +270,11 @@ class ServerArgs:
)
from sglang.srt.arg_groups.pipeline import run_resolution_pipeline
# Sealed for the duration, not just afterwards: everything below this
# line reads the input and declares against it, and the one channel
# that still writes the record (`declare_direct_writes`, for
# out-of-tree platform plugins) asks for the seal to be lifted by name.
self._input_frozen = True
try:
run_resolution_pipeline(self)
except BaseException:
@@ -276,11 +282,30 @@ class ServerArgs:
# idempotent over their own output.
self._resolution_failed = True
raise
finally:
self._input_frozen = False
# Set here too, because the dummy/absent-model path returns before the
# end of the pipeline that normally sets it: the gate is about whether
# the handlers ran, not how far they got.
self._resolution_finished = True
@property
def launch_command(self) -> Optional[str]:
"""How this record was created, verbatim.
`resolved_dict` answers with what resolution decided; this answers with
what the operator asked for, which is a different question and the one
"why is this server configured like this?" usually means. The two are
not derivable from each other: a field the operator never set reads the
same as one they set to the value resolution would have picked anyway.
The launcher stores the arguments it parsed; the in-process `Engine`
stores the call that built the record, since there was no command line.
`None` on a record built directly (a fixture, a subprocess copy that
predates this, a config being inspected).
"""
return getattr(self, "_launch_command", None)
def resolved_dict(self) -> Dict[str, Any]:
"""This configuration as a plain dict of resolved field values.
@@ -318,6 +343,9 @@ class ServerArgs:
copy's deep structure in-process mutates the parent's too.
"""
replacement = dataclasses.replace(self, **changes)
# Provenance, not resolution state: a copy was still launched by
# whatever launched its parent, resolved or not.
object.__setattr__(replacement, "_launch_command", self.launch_command)
if not getattr(self, "_resolution_finished", False):
# Not resolved yet: the copy goes through the gate itself.
return replacement
@@ -669,19 +697,28 @@ class ServerArgs:
return cfg.startup_weight_load_mode == "overlap"
def __setattr__(self, name, value):
# Once resolution has finished the record is the READ-ONLY raw input
# the config bags were projected from. Resolved config changes go to the bags via
# get_context().override(source, ...); a value one runner or worker
# owns travels as a constructor argument to it.
if getattr(self, "_resolution_finished", False) and (
not name.startswith("_") or name in _underscore_field_names()
):
raise AttributeError(
f"server_args.{name} assigned after resolution; server_args is "
"read-only -- use get_context().override(source, ...) to change "
"resolved config; a value one runner owns travels as a "
"constructor argument."
)
# The record holds the operator's input. It is writable while the
# caller is still assembling it and sealed from the moment resolution
# starts: a resolver that writes a field would overwrite the very thing
# the record exists to remember, and the decision it meant to record
# belongs in the stash, where it carries a source and does not destroy
# the input it was derived from.
if not name.startswith("_") or name in _underscore_field_names():
if getattr(self, "_input_frozen", False):
raise AttributeError(
f"server_args.{name} assigned during resolution; the record "
"is the operator's input and resolution does not write it -- "
"declare the decision with declare_resolution(server_args, "
"source, **fields) so it carries a source and leaves the "
"input intact."
)
if getattr(self, "_resolution_finished", False):
raise AttributeError(
f"server_args.{name} assigned after resolution; server_args is "
"read-only -- use get_context().override(source, ...) to change "
"resolved config; a value one runner owns travels as a "
"constructor argument."
)
object.__setattr__(self, name, value)
def enable_mamba_extra_buffer(self) -> bool:
@@ -834,6 +871,27 @@ def get_global_server_args() -> ServerArgs:
return get_context().server_args
@contextmanager
def record_writable(server_args: Any):
"""Lift the input seal for a resolver that genuinely writes the record.
There is exactly one: `declare_direct_writes`, which hands the record to an
out-of-tree platform plugin that sets fields on it. Those implementations
live outside this tree and cannot be converted by editing a resolver here,
so the write stays and is captured into the stash afterwards. Naming the
exception is the point -- an in-tree resolver that reaches for this is
doing something it should be declaring instead.
"""
frozen = getattr(server_args, "_input_frozen", False)
if frozen:
object.__setattr__(server_args, "_input_frozen", False)
try:
yield
finally:
if frozen:
object.__setattr__(server_args, "_input_frozen", True)
def prepare_server_args(argv: List[str]) -> ServerArgs:
"""
Prepare the server arguments from the command line arguments.
@@ -868,7 +926,12 @@ def prepare_server_args(argv: List[str]) -> ServerArgs:
force=True,
)
return ServerArgs.from_cli_args(raw_args)
server_args = ServerArgs.from_cli_args(raw_args)
# Not a field: the record's fields are the configuration, and this is how
# the configuration was asked for. It rides along on the record so a
# subprocess copy can answer the same question the launcher can.
object.__setattr__(server_args, "_launch_command", " ".join(argv))
return server_args
# --------------------------------------------------------------------------