Last of four; stacked on #38048.
The record is the operator's input; the bags are what is in effect. A reader
that takes the record and reads a field off it gets the input, which is the
wrong one of the two whenever resolution decided something -- and the mistake is
silent, because for most fields and most launches the two agree. Several of
these files already read both ways, sometimes in the same expression:
```python
get_tokenizer(
get_serving().tokenizer_path,
tokenizer_mode=server_args.tokenizer_mode, # the input, not the decision
...
)
```
Sixty-odd files convert. Record field reads in runtime code go from 199 to 11.
Nine parameters that the conversion emptied are dropped along with the argument
at every call site -- the dead-parameter ratchet is what names them.
### "Runs after its process publishes" is a per-entry-point claim
Most converted reads sit in the serving and model-executor layers, which only
exist after publication, or in the two subprocess entry points, which publish
first thing. Three places are not like that, and they keep reading the record
they were handed:
- **`HttpServerEngineAdapter`** launches the server as a *child*. The parent
resolves the record and never publishes, so the adapter's own reads -- the
launch banner, the API key in its readiness loop, the TP width in
`update_weights_from_tensor` -- are of `self.server_args`. A bag read here
fails closed in a bare process, or answers for an unrelated engine in one that
happens to have published.
- **`serve_grpc`** reads its sidecar port before the integrated servicer builds
the `Engine` that publishes. The comment above that line already said so and
already bound `cfg = resolving_view(server_args)` for it; the sidecar port and
the port it derives from read `cfg`.
- **`initialize_dp_attention`** runs from callers whose publish is not
guaranteed, so its one predicate stays on the resolution view.
`ROLE_NAMESPACE_SETS["dp_controller"]` gains `observability` and `serving`,
because the controller's metrics gate, tracing setup and worker-port broadcast
now read those namespaces. Under `SGLANG_ROLE_NAMESPACES=enforce` that set is
what the process may read, so a conversion that reaches a new namespace has to
widen it in the same change.
## Three things worth a reviewer's attention
**Eleven reads were `getattr(record, "field", default)`.** An AST scan for
attribute access does not see those, so the census that said "43 readers" was
counting the shape it could match rather than the thing it was after.
`incremental_streaming_output` was read that way twice, and the transcription
tests were the only reason it surfaced.
**Not every record read is a bag read waiting to happen.** A multimodal
processor's `base_gpu_id` is the instance's, not the process's: two engines in
one process keep different ones, and
`test_publishing_another_config_does_not_move_the_device` exists to say so. It
stays on the record while `rl_on_policy_target` beside it moves.
`RequestMetricsExporter` is the same shape -- it is handed the directory it
writes to, and a test builds several with different ones. `configure_logger` is
a third: 17 call sites, one of which passes an `argparse.Namespace`, so it is
not a global-context reader at all. Those eleven remaining reads are the ones
with a reason.
**The fixtures move with the code.** Tests that hung config off a mock manager
now publish a record, which is what the serving layer reads; where a test states
a value it says so with `override_server_args` instead of assigning through the
mock. `test_hisparse_unit` is the last of them: it stubbed a `server_args` onto
a fake scheduler to say the decode radix cache was off, and the value it was
standing in for is the published default, so the stub goes and the class
publishes.
## Two things CI caught that a local sweep could not
**`unittest.TestCase.enterContext` is Python 3.11+.** The converted fixtures used
it at 18 sites; `requires-python` is `>=3.10` and CI runs 3.10, so every one of
them raised `AttributeError` there while passing on a newer local interpreter.
They call `enter_override(self, ...)` now -- a four-line helper in
`sglang/test/test_utils.py` over the override's own `install()` / `restore()`.
**A batched sweep cannot see a missing publish.** Three fixtures needed a
published config and did not have one; each *passed* inside a shard where some
other file had published, and failed when run alone. The affected cases are
`test_serving_completions` (which set `incremental_streaming_output` on the mock
manager's record, where nothing reads it now), `test_qwen3_vl_feature_materialization`
(same shape for `mm_enable_dp_encoder`), and the two Qwen Rust tests -- whose
fixture already carried the comment `# Non-auto: get_resolved_model_impl would
choke on a SimpleNamespace` next to the `model_impl` it sets, which is exactly
what happened once `get_mm_processor_cls` started reading that value from the
bag. Its `publish` mirrors `model_impl` now, like the four fields it already
mirrored.
## Verification
A full registered-unit sweep (648 files) against this stack's merge-base:
19 failures on both sides, the same 19, none of them config. That sweep is what
caught 23 failures the file-scoped runs missed -- and, later, that the narrower
139-file list did not even contain the files this change reaches. It is also
what caught the `test_hisparse_unit` fixture above: the file passes inside a
shard where something else published, and fails when it is run on its own,
which is why every failing file is re-run alone before it is counted.
Third of four; stacked on #38047. Two small changes, both about the same thing:
the record holds the operator's input, and nothing else should be true of it.
## `/server_info` can answer what was actually typed
It reports `resolved_dict()` -- what resolution decided. There was no way to ask
the other question, and the two are not derivable from each other: a field
nobody set reads the same as one set to the value resolution would have picked
anyway.
The launcher stores the arguments it parsed and the in-process `Engine` stores
the call that built the record. All three readbacks report it beside the
resolved values, so both surfaces come back in one request: HTTP `/server_info`,
`Engine.get_server_info`, and the gRPC bridge's -- the last one builds from
`resolved_dict()` and would otherwise have been the one surface of the three
that answers only "what resolution decided".
It rides on the record rather than in a field -- it describes how the
configuration was asked for, so it is not part of the configuration: no CLI
flag, no namespace, not in the bags. Being on the record is what lets a
subprocess copy answer the same question the launcher can, and
`replace_resolved` carries it because a copy was launched by whatever launched
its parent.
The crash dump already collected all four surfaces (`server_args`,
`config_updates`, `resolved_config`, `launch_command`); this is the one that
`/server_info` was missing.
## The record is sealed for the length of resolution
The read-only guard armed on `_resolution_finished`, so for the whole run of the
pipeline nothing stopped a resolver from assigning a field. Nothing in `srt/`
does -- 0 assignments statically, and 0 writes observed across the launch-shape
matrix with a watching `__setattr__` -- but that was a convention, and the
defect it permits is invisible: a value a resolver wrote onto the record is
indistinguishable from a value the operator typed, which is the one distinction
the record exists to preserve.
It now arms when resolution starts. A resolver that assigns a field fails at
boot with a message naming `declare_resolution`, which is where the decision
belongs: the stash carries a source and leaves the input intact.
`declare_direct_writes` asks for the seal by name through `record_writable`. It
hands the record to an out-of-tree platform plugin that sets fields on it; those
implementations cannot be converted by editing a resolver here, so the write
stays and the diff is captured into the stash afterwards. Naming the exception
is the point -- an in-tree resolver reaching for it is doing something it should
be declaring.
## Verification
Costs nothing: the 211 test-side assignments all happen before `resolve_once`,
which a post-resolution write already refused. A full registered-unit sweep
(648 files) against the stack's merge-base: 19 failures on both sides, the same
19, none of them config. Driving a
deliberate write into a real handler produces the new error, so the seal is
tested by more than its own unit test.
Second of four; stacked on #38046. Mechanical relocation plus one design change
that the relocation makes possible. **Review by checking the identity proofs at
the bottom** -- nothing here is meant to change behaviour.
## The declarations move
`ServerArgs` carried all 487 declarations in one 4,462-line file, each tagged
with an `NS("...")` marker naming the namespace it belongs to -- structure
supplied by annotation, in a file a namespace away from the
`arg_groups/*_hook.py` that resolves it.
They move to `arg_groups/fields/`: one module per top-level namespace, one class
per leaf namespace (21 of them, `exec.moe` becomes `exec_.py::ExecMoe`). The
class carries the `_NS_PATH` it stands for, so the module a field is declared in
*is* its namespace and the marker is redundant -- `namespace_of` reads the
declaring class instead. `NS` stays for the one case a class cannot express: a
single ad-hoc dataclass whose fields span namespaces, which is what the
config-bag tests build.
Two things travel with the declarations. The `*_CHOICES` lists and the
`add_*_choices` adders that extend them move to `arg_groups/choices.py`, since
the fields naming them can no longer import from `server_args` without a cycle;
`server_args` re-exports all of them, because out-of-tree plugins have always
reached them there. And five fields whose only annotation element was the
namespace marker become plain annotations -- `A` is `Annotated`, which needs two
arguments, so stripping the marker would have left them invalid.
`server_args.py` goes from 4,458 lines to about 1,000.
## The record is assembled, not inherited
Inheriting the namespace classes would make the record's contents a property of
which classes happen to appear in a base list. That is correct only while every
namespace declares nothing but operator input, and it stops being correct the
moment a derived field is declared: `attn_tp_size` belongs in `parallel.py`
next to the leaves it is derived from, and inheriting `Parallel` would put it on
the record -- where it is neither input nor safe, since the record is what
crosses a process boundary and a derived width pickled to a subprocess is a
stamp that elastic scale-up will not refresh.
`collect_input_fields` takes the classes that declare input and returns their
annotations, defaults and namespaces. Each source's annotations are resolved in
its own module and handed on as type objects; carried across as text they would
be re-evaluated where they land, and the composing module deliberately imports
none of the names the declarations use. A namespace can now declare both halves
side by side, and which half reaches the record is one readable call rather than
an invariant spread across a base-class list. Nothing is registered on the
derived side yet -- this is what makes it possible.
`ServerArgs` is still one flat dataclass with 494 attributes, so
`server_args.tp_size`, `ServerArgs(model_path=..., tp_size=8)`, pickling to a
subprocess and every existing call site are untouched.
### Field order is a contract, so it is written down
A dataclass turns field order into a positional constructor signature, and
collecting whole namespaces groups fields that used to be interleaved. Keeping
`model_path` first is not enough: `ServerArgs("dummy", "/tmp/tokenizer")` would
set `load_format="/tmp/tokenizer"` and leave `tokenizer_path=None`, which then
selects an invalid model loader -- silently, at a call site that did not change.
So `arg_groups/field_order.py` records the order the record had before the
split, and `collect_input_fields` orders what it collects by it. A field the
record declares that the frozen order does not name goes after it, in
declaration order -- the only backward-compatible place for a new field anyway,
so a new declaration needs no edit there. The list is a compatibility record and
nothing else reads it; the namespace a field belongs to is still the module it
is declared in.
## Verification
Four ways, all against the base commit:
| check | result |
|---|---|
| `namespace_of` map, field by field | 494 / 494, **0 differences** |
| CLI surface (options, defaults, choices, actions) | 507 / 507, **0 differences** |
| field order, name by name | 494 / 494, **identical to the base** |
| resolution result, 24 launch shapes x 489 fields | **0 differences** |
| names importable from `sglang.srt.server_args` | nothing lost |
Plus a full registered-unit sweep (648 files) against the stack's merge-base:
19 failures on both sides, the same 19, none of them config.
First of five. The stack continues a series that moved configuration out of
`ServerArgs` and into the runtime context's namespace bags. This one fixes
something that was actually broken, and gives the fix its other half.
## "Unset" gets its own spelling on two ratio fields
`swa_full_tokens_ratio` and `mamba_full_memory_ratio` carried real values as
their class defaults (0.8, 0.9), so a model family with an opinion had to ask
"is this field still equal to the class default?" to find out whether the
operator had set it. That question has two wrong answers: it says "the operator
set it" as soon as any earlier pass declares the field, and it says "the
operator did not set it" when the operator types the default value.
Both become `Optional[float] = None`. The record carries what the operator typed
and nothing else, and the family test becomes `is None`.
`mamba_radix_cache_strategy` keeps `"auto"`: unlike the ratios it already has a
spelling for "unset" that an operator can type and that means exactly that --
only its comparison changes, from the class default to the token itself, which
is the fix the comment at that site already prescribed. With that, neither
family module imports `ServerArgs` any more.
## And the declaration says what the field means when nobody answers
Making the default `None` leaves a hole: something has to supply the generic
value. `Arg(fallback=...)` supplies it from the declaration.
```python
swa_full_tokens_ratio: A[
Optional[float],
Arg(help="...", resolvable=True, fallback=0.8),
NS("schedule"),
] = None
```
The dataclass default stays `None`. A fallback is not a default: the record is
the wire format, and a child process has to keep being able to tell "unset" from
"set to the value resolution would have picked anyway".
### Which surface it lives on is the whole design
Precedence becomes **override -> decision -> input -> fallback**, applied in
`resolution_result` -- which the projection, `/server_info` and every config bag
read through.
Deliberately **not** in `resolving_view` / `resolved_view`. Those are the
decision-over-input surface a pass reads *while it is deciding*, and two model
families branch on exactly this:
```python
# model_overrides/inkling.py, and the same shape in deepseek_v4.py
if cfg.swa_full_tokens_ratio is None:
overrides["swa_full_tokens_ratio"] = 0.1
```
A fallback answering there is not "the generic value, later" -- a `__getattr__`
layer is read-time, so there is no later. Every read during resolution would
already get 0.8 and the branch would never fire. Running `_inkling_overrides`
against both versions:
```
--- fallback on the effective surface only (this PR) ---
cfg.swa_full_tokens_ratio during resolution = None
family declared swa = 0.1 mamba = 0.1
--- fallback also on the view a pass reads ---
cfg.swa_full_tokens_ratio during resolution = 0.8
family declared swa = None mamba = None <- the key never lands
```
So "resolution first, then the fallback" holds -- not because a step is appended
to the pipeline, but because of which surface the value lives on. Exactly one
reader consults the effective surface during resolution: the range check on the
ratio, which wants the value the pools will be sized against. It asks
`resolution_result` directly -- what its comment already claimed it was doing --
and it runs after the model families.
### The alternative, and why not
A pass that fills the field in when nothing claimed it needs a slot (after the
families, or it beats them), a second call site (the dummy-model short circuit
returns long before that slot), an idempotence requirement so the second call is
harmless, and the value written twice -- once as a literal, once as prose in the
help (`"Unset means 0.8"`). An earlier revision of this series did exactly that
and deleted it four PRs later. A declaration needs none of it, and `pipeline.py`
is untouched by the whole series as a result.
### What may be declared this way, and what may not
Across every hook, `if x is None: x = ...` appears at **55 sites over 29
fields**. They are not one thing:
| | count | examples | declarable |
|---|---|---|---|
| unconditional constant | 5 | the two ratios, `grammar_backend="xgrammar"`, `mm_process_config={}`, `custom_weight_loader=[]` | **yes** |
| unconditional, computed from another field | 4 | `tokenizer_path=model_path`, `device=get_device()`, `served_model_name`, `speculative_draft_model_quantization` | needs a `fallback="dotted.path"` form; not here |
| **conditional decision** | ~20 | `chunked_prefill_size` across seven memory tiers, `max_bs` across eight, `max_running_requests` at 48 or 256 by model family | **no, and it should not be** |
Only a value fixed for the life of the configuration belongs in a declaration.
One that depends on the machine, on another field, or on anything impure
(`random_seed = random.randint(...)`) is a decision, and decisions stay in a hook
where their order is visible. This PR converts the two ratios only.
## Verification
- `resolve_once` ends with the same effective values: the resolution result is
identical across 24 launch shapes x 489 fields except for the two intended
ratio changes. Separately, 16 launch shapes resolved on both sides, real model
and dummy: 7,904 field readings, and the only difference is `random_seed`, a
fresh `random.randint` per process.
- The CLI registers the same 507 options with the same choices and actions; only
the two defaults move.
- `test_declared_fallbacks.py`, 17 cases. One pins the inverse of the dead branch
above: what a pass sees while deciding is still `None`.
- The whole series was swept over all 648 registered unit-test files against its
merge-base: 19 failures on both sides, the same 19, none of them config.
---
### CI States
Latest PR Test (Base): <!-- slot:pr-test:start -->❌ [Run #34083705463](https://github.com/sgl-project/sglang/actions/runs/34083705463)<!-- slot:pr-test:end -->
Latest PR Test (Extra): <!-- slot:pr-test-extra:start -->❌ [Run #34083705284](https://github.com/sgl-project/sglang/actions/runs/34083705284)<!-- slot:pr-test-extra:end -->
Latest PR Test (AMD ROCm 7.2): <!-- slot:pr-test-amd-rocm720:start -->❌ [Run #34083705383](https://github.com/sgl-project/sglang/actions/runs/34083705383)<!-- slot:pr-test-amd-rocm720:end -->
<!-- pr-states:end -->
Cheng Wan