Fifth of five; stacked on #38049. The split gave every namespace a file, but
only for the half an operator types. This is the other half.
## The parallel quotients are declared, not written out
`attn_tp_size` and its five siblings were sixty lines of near-identical
properties in the runtime context, a file away from the leaves they are
quotients of, so reading `parallel.py` told you what you could set and nothing
about what that decides.
They are declared in `Parallel` now, in the same class as those leaves. They
carry no annotation, so they are not dataclass fields and
`collect_input_fields` never puts them on the record -- the same mechanism that
already keeps `_NS_PATH` off it. That is the right exclusion: a quotient has no
operator input to preserve, and the record is what crosses a process boundary,
where a stamped width is one an elastic scale-up will not refresh.
## A quotient is a value in the bag, like every other derived one
`_derived_width` answered from a stamp or, failing that, a live process group.
The group read could never disagree with the stamp:
- `initialize_model_parallel` stamps all six as its last statement,
unconditionally;
- an elastic scale-up restamps `attn_dp_size` through
`update_dp_attention_post_scale` -- the comment claiming it does *not* was
wrong;
- no hardware backend builds groups of its own;
- `multimodal_gen`, which has its own `initialize_model_parallel` and does not
stamp, never reads a quotient.
So a built group was always already stamped, and the group read goes -- and with
it the last reason for a quotient to be resolved on every read.
Every input to `derive_parallel_widths` is a record field. `dcp_enabled` is
`decode_context_parallel_size > 1`, not a fact about a built group; it was
spelled `_DCP is not None`, which is a longer way to say the same thing. So the
six are fixed once the configuration is fixed -- the same test every other
`Derived(fn=...)` in this PR passes. They are declared the same way and computed
the same way: once, at publish, into ordinary bag leaves.
What remains is override -> stamp -> published leaf. The stamp stays above the
leaf because an elastic scale-up restamps `attn_dp_size`; the override stays on
top because that is how a test names a width.
## One answer for the config-derived predicates
`enable_mamba_extra_buffer` and its lazy variant, `is_ep_joiner`,
`is_ep_scale_joiner`, `is_startup_weight_load_overlap`: each existed as a
`ServerArgs` member for the resolution pipeline and, for most of them, again as
a `runtime_context` function for readers after publish. Three places to keep
saying the same thing.
A `Derived(fn=...)` is a pure function of the published configuration, so
`publish` computes it once and stores it as an ordinary bag leaf -- a plain
attribute load, which is what a read inside compiled model code needs. The
function is handed the whole resolved config rather than the bag it lands in,
because a derivation is free to span namespaces and the mamba one does: it
reads `memory.disable_radix_cache` alongside its own `exec.mamba` strategy,
which is why it could never have been a method on either bag.
The pre-publish helpers stay -- resolution needs the predicate before there is
a bag to read -- and three readers keep them, because they run before their own
process publishes: `initialize_dp_attention`, which the weight-cache daemon
calls while building its groups thirty lines before its `publish`, and
`PortArgs.init_new`, a factory handed the record that already reads eighteen
other fields off it.
## Notes for a reviewer
**Overriding a leaf does not move its quotient.** `override(tp_size=2)` leaves
`attn_tp_size` where the published config put it, because nothing is recomputed
on read. A test states a topology by publishing a config -- which is what a
real process does -- or by naming the width it wants, `override(attn_tp_size=2)`.
Six tests say it that way now. This is the price of having one answer computed
once, and it is the same price every other derived value in the config already
carries.
A caller that reads a quotient without publishing or overriding now gets an
explicit error naming the field, instead of a default that an uninitialised
group happened to supply. One fixture was in that state --
`TestMlaWriteDoorsUnderDcp` built a bare pool and asked whether DCP was on --
and it publishes a config now, which is what the process it stands in for
does.
Eighteen sites read these predicates without calling them. That is correct --
they are properties -- but it is worth saying they were checked, because a
census that assumes otherwise reports eighteen always-true conditions.
## The skill that documents this subsystem is updated with it
`.claude/rules/modify-component-must-read.md` points at
`.claude/skills/sglang-runtime-context/SKILL.md` before anyone touches these
files, so a stale sentence there is a wrong instruction rather than a stale
note. Four of its load-bearing statements stopped being true across this series
and are corrected here: `NS(...)` is no longer how a field states its namespace
(the declaring class is); the DCP degrade rule is gone, because the quotients
are not live reads; `mamba_extra_buffer_enabled()` and the other predicate
functions it named as the shape to copy no longer exist; and the
namespace-coverage ratchet is described in terms of the marker. The docstring of
`test_server_args_namespaces.py` said the same thing and is fixed too.
The consequence a test author actually trips over is stated there as well:
overriding a leaf no longer moves its quotient, so a topology is stated by
publishing a config or by naming the width.
## Verification
A full registered-unit sweep (648 files) against this stack's merge-base:
19 failures on both sides, the same 19 -- AMD `gfx950`, `modelopt`,
`cuda_vmm`, `weight_checker` and friends, none of them config. The narrower 139-file config sweep used earlier in this series
does not contain the files this change reaches -- `test_kv_index_translator`
never names `get_parallel()`, it constructs an object that does -- which is why
the baseline differential over everything is what is quoted here.
Last of four; stacked on #38048.
The record is the operator's input; the bags are what is in effect. A reader
that takes the record and reads a field off it gets the input, which is the
wrong one of the two whenever resolution decided something -- and the mistake is
silent, because for most fields and most launches the two agree. Several of
these files already read both ways, sometimes in the same expression:
```python
get_tokenizer(
get_serving().tokenizer_path,
tokenizer_mode=server_args.tokenizer_mode, # the input, not the decision
...
)
```
Sixty-odd files convert. Record field reads in runtime code go from 199 to 11.
Nine parameters that the conversion emptied are dropped along with the argument
at every call site -- the dead-parameter ratchet is what names them.
### "Runs after its process publishes" is a per-entry-point claim
Most converted reads sit in the serving and model-executor layers, which only
exist after publication, or in the two subprocess entry points, which publish
first thing. Three places are not like that, and they keep reading the record
they were handed:
- **`HttpServerEngineAdapter`** launches the server as a *child*. The parent
resolves the record and never publishes, so the adapter's own reads -- the
launch banner, the API key in its readiness loop, the TP width in
`update_weights_from_tensor` -- are of `self.server_args`. A bag read here
fails closed in a bare process, or answers for an unrelated engine in one that
happens to have published.
- **`serve_grpc`** reads its sidecar port before the integrated servicer builds
the `Engine` that publishes. The comment above that line already said so and
already bound `cfg = resolving_view(server_args)` for it; the sidecar port and
the port it derives from read `cfg`.
- **`initialize_dp_attention`** runs from callers whose publish is not
guaranteed, so its one predicate stays on the resolution view.
`ROLE_NAMESPACE_SETS["dp_controller"]` gains `observability` and `serving`,
because the controller's metrics gate, tracing setup and worker-port broadcast
now read those namespaces. Under `SGLANG_ROLE_NAMESPACES=enforce` that set is
what the process may read, so a conversion that reaches a new namespace has to
widen it in the same change.
## Three things worth a reviewer's attention
**Eleven reads were `getattr(record, "field", default)`.** An AST scan for
attribute access does not see those, so the census that said "43 readers" was
counting the shape it could match rather than the thing it was after.
`incremental_streaming_output` was read that way twice, and the transcription
tests were the only reason it surfaced.
**Not every record read is a bag read waiting to happen.** A multimodal
processor's `base_gpu_id` is the instance's, not the process's: two engines in
one process keep different ones, and
`test_publishing_another_config_does_not_move_the_device` exists to say so. It
stays on the record while `rl_on_policy_target` beside it moves.
`RequestMetricsExporter` is the same shape -- it is handed the directory it
writes to, and a test builds several with different ones. `configure_logger` is
a third: 17 call sites, one of which passes an `argparse.Namespace`, so it is
not a global-context reader at all. Those eleven remaining reads are the ones
with a reason.
**The fixtures move with the code.** Tests that hung config off a mock manager
now publish a record, which is what the serving layer reads; where a test states
a value it says so with `override_server_args` instead of assigning through the
mock. `test_hisparse_unit` is the last of them: it stubbed a `server_args` onto
a fake scheduler to say the decode radix cache was off, and the value it was
standing in for is the published default, so the stub goes and the class
publishes.
## Two things CI caught that a local sweep could not
**`unittest.TestCase.enterContext` is Python 3.11+.** The converted fixtures used
it at 18 sites; `requires-python` is `>=3.10` and CI runs 3.10, so every one of
them raised `AttributeError` there while passing on a newer local interpreter.
They call `enter_override(self, ...)` now -- a four-line helper in
`sglang/test/test_utils.py` over the override's own `install()` / `restore()`.
**A batched sweep cannot see a missing publish.** Three fixtures needed a
published config and did not have one; each *passed* inside a shard where some
other file had published, and failed when run alone. The affected cases are
`test_serving_completions` (which set `incremental_streaming_output` on the mock
manager's record, where nothing reads it now), `test_qwen3_vl_feature_materialization`
(same shape for `mm_enable_dp_encoder`), and the two Qwen Rust tests -- whose
fixture already carried the comment `# Non-auto: get_resolved_model_impl would
choke on a SimpleNamespace` next to the `model_impl` it sets, which is exactly
what happened once `get_mm_processor_cls` started reading that value from the
bag. Its `publish` mirrors `model_impl` now, like the four fields it already
mirrored.
## Verification
A full registered-unit sweep (648 files) against this stack's merge-base:
19 failures on both sides, the same 19, none of them config. That sweep is what
caught 23 failures the file-scoped runs missed -- and, later, that the narrower
139-file list did not even contain the files this change reaches. It is also
what caught the `test_hisparse_unit` fixture above: the file passes inside a
shard where something else published, and fails when it is run on its own,
which is why every failing file is re-run alone before it is counted.
Third of four; stacked on #38047. Two small changes, both about the same thing:
the record holds the operator's input, and nothing else should be true of it.
## `/server_info` can answer what was actually typed
It reports `resolved_dict()` -- what resolution decided. There was no way to ask
the other question, and the two are not derivable from each other: a field
nobody set reads the same as one set to the value resolution would have picked
anyway.
The launcher stores the arguments it parsed and the in-process `Engine` stores
the call that built the record. All three readbacks report it beside the
resolved values, so both surfaces come back in one request: HTTP `/server_info`,
`Engine.get_server_info`, and the gRPC bridge's -- the last one builds from
`resolved_dict()` and would otherwise have been the one surface of the three
that answers only "what resolution decided".
It rides on the record rather than in a field -- it describes how the
configuration was asked for, so it is not part of the configuration: no CLI
flag, no namespace, not in the bags. Being on the record is what lets a
subprocess copy answer the same question the launcher can, and
`replace_resolved` carries it because a copy was launched by whatever launched
its parent.
The crash dump already collected all four surfaces (`server_args`,
`config_updates`, `resolved_config`, `launch_command`); this is the one that
`/server_info` was missing.
## The record is sealed for the length of resolution
The read-only guard armed on `_resolution_finished`, so for the whole run of the
pipeline nothing stopped a resolver from assigning a field. Nothing in `srt/`
does -- 0 assignments statically, and 0 writes observed across the launch-shape
matrix with a watching `__setattr__` -- but that was a convention, and the
defect it permits is invisible: a value a resolver wrote onto the record is
indistinguishable from a value the operator typed, which is the one distinction
the record exists to preserve.
It now arms when resolution starts. A resolver that assigns a field fails at
boot with a message naming `declare_resolution`, which is where the decision
belongs: the stash carries a source and leaves the input intact.
`declare_direct_writes` asks for the seal by name through `record_writable`. It
hands the record to an out-of-tree platform plugin that sets fields on it; those
implementations cannot be converted by editing a resolver here, so the write
stays and the diff is captured into the stash afterwards. Naming the exception
is the point -- an in-tree resolver reaching for it is doing something it should
be declaring.
## Verification
Costs nothing: the 211 test-side assignments all happen before `resolve_once`,
which a post-resolution write already refused. A full registered-unit sweep
(648 files) against the stack's merge-base: 19 failures on both sides, the same
19, none of them config. Driving a
deliberate write into a real handler produces the new error, so the seal is
tested by more than its own unit test.
Second of four; stacked on #38046. Mechanical relocation plus one design change
that the relocation makes possible. **Review by checking the identity proofs at
the bottom** -- nothing here is meant to change behaviour.
## The declarations move
`ServerArgs` carried all 487 declarations in one 4,462-line file, each tagged
with an `NS("...")` marker naming the namespace it belongs to -- structure
supplied by annotation, in a file a namespace away from the
`arg_groups/*_hook.py` that resolves it.
They move to `arg_groups/fields/`: one module per top-level namespace, one class
per leaf namespace (21 of them, `exec.moe` becomes `exec_.py::ExecMoe`). The
class carries the `_NS_PATH` it stands for, so the module a field is declared in
*is* its namespace and the marker is redundant -- `namespace_of` reads the
declaring class instead. `NS` stays for the one case a class cannot express: a
single ad-hoc dataclass whose fields span namespaces, which is what the
config-bag tests build.
Two things travel with the declarations. The `*_CHOICES` lists and the
`add_*_choices` adders that extend them move to `arg_groups/choices.py`, since
the fields naming them can no longer import from `server_args` without a cycle;
`server_args` re-exports all of them, because out-of-tree plugins have always
reached them there. And five fields whose only annotation element was the
namespace marker become plain annotations -- `A` is `Annotated`, which needs two
arguments, so stripping the marker would have left them invalid.
`server_args.py` goes from 4,458 lines to about 1,000.
## The record is assembled, not inherited
Inheriting the namespace classes would make the record's contents a property of
which classes happen to appear in a base list. That is correct only while every
namespace declares nothing but operator input, and it stops being correct the
moment a derived field is declared: `attn_tp_size` belongs in `parallel.py`
next to the leaves it is derived from, and inheriting `Parallel` would put it on
the record -- where it is neither input nor safe, since the record is what
crosses a process boundary and a derived width pickled to a subprocess is a
stamp that elastic scale-up will not refresh.
`collect_input_fields` takes the classes that declare input and returns their
annotations, defaults and namespaces. Each source's annotations are resolved in
its own module and handed on as type objects; carried across as text they would
be re-evaluated where they land, and the composing module deliberately imports
none of the names the declarations use. A namespace can now declare both halves
side by side, and which half reaches the record is one readable call rather than
an invariant spread across a base-class list. Nothing is registered on the
derived side yet -- this is what makes it possible.
`ServerArgs` is still one flat dataclass with 494 attributes, so
`server_args.tp_size`, `ServerArgs(model_path=..., tp_size=8)`, pickling to a
subprocess and every existing call site are untouched.
### Field order is a contract, so it is written down
A dataclass turns field order into a positional constructor signature, and
collecting whole namespaces groups fields that used to be interleaved. Keeping
`model_path` first is not enough: `ServerArgs("dummy", "/tmp/tokenizer")` would
set `load_format="/tmp/tokenizer"` and leave `tokenizer_path=None`, which then
selects an invalid model loader -- silently, at a call site that did not change.
So `arg_groups/field_order.py` records the order the record had before the
split, and `collect_input_fields` orders what it collects by it. A field the
record declares that the frozen order does not name goes after it, in
declaration order -- the only backward-compatible place for a new field anyway,
so a new declaration needs no edit there. The list is a compatibility record and
nothing else reads it; the namespace a field belongs to is still the module it
is declared in.
## Verification
Four ways, all against the base commit:
| check | result |
|---|---|
| `namespace_of` map, field by field | 494 / 494, **0 differences** |
| CLI surface (options, defaults, choices, actions) | 507 / 507, **0 differences** |
| field order, name by name | 494 / 494, **identical to the base** |
| resolution result, 24 launch shapes x 489 fields | **0 differences** |
| names importable from `sglang.srt.server_args` | nothing lost |
Plus a full registered-unit sweep (648 files) against the stack's merge-base:
19 failures on both sides, the same 19, none of them config.
First of five. The stack continues a series that moved configuration out of
`ServerArgs` and into the runtime context's namespace bags. This one fixes
something that was actually broken, and gives the fix its other half.
## "Unset" gets its own spelling on two ratio fields
`swa_full_tokens_ratio` and `mamba_full_memory_ratio` carried real values as
their class defaults (0.8, 0.9), so a model family with an opinion had to ask
"is this field still equal to the class default?" to find out whether the
operator had set it. That question has two wrong answers: it says "the operator
set it" as soon as any earlier pass declares the field, and it says "the
operator did not set it" when the operator types the default value.
Both become `Optional[float] = None`. The record carries what the operator typed
and nothing else, and the family test becomes `is None`.
`mamba_radix_cache_strategy` keeps `"auto"`: unlike the ratios it already has a
spelling for "unset" that an operator can type and that means exactly that --
only its comparison changes, from the class default to the token itself, which
is the fix the comment at that site already prescribed. With that, neither
family module imports `ServerArgs` any more.
## And the declaration says what the field means when nobody answers
Making the default `None` leaves a hole: something has to supply the generic
value. `Arg(fallback=...)` supplies it from the declaration.
```python
swa_full_tokens_ratio: A[
Optional[float],
Arg(help="...", resolvable=True, fallback=0.8),
NS("schedule"),
] = None
```
The dataclass default stays `None`. A fallback is not a default: the record is
the wire format, and a child process has to keep being able to tell "unset" from
"set to the value resolution would have picked anyway".
### Which surface it lives on is the whole design
Precedence becomes **override -> decision -> input -> fallback**, applied in
`resolution_result` -- which the projection, `/server_info` and every config bag
read through.
Deliberately **not** in `resolving_view` / `resolved_view`. Those are the
decision-over-input surface a pass reads *while it is deciding*, and two model
families branch on exactly this:
```python
# model_overrides/inkling.py, and the same shape in deepseek_v4.py
if cfg.swa_full_tokens_ratio is None:
overrides["swa_full_tokens_ratio"] = 0.1
```
A fallback answering there is not "the generic value, later" -- a `__getattr__`
layer is read-time, so there is no later. Every read during resolution would
already get 0.8 and the branch would never fire. Running `_inkling_overrides`
against both versions:
```
--- fallback on the effective surface only (this PR) ---
cfg.swa_full_tokens_ratio during resolution = None
family declared swa = 0.1 mamba = 0.1
--- fallback also on the view a pass reads ---
cfg.swa_full_tokens_ratio during resolution = 0.8
family declared swa = None mamba = None <- the key never lands
```
So "resolution first, then the fallback" holds -- not because a step is appended
to the pipeline, but because of which surface the value lives on. Exactly one
reader consults the effective surface during resolution: the range check on the
ratio, which wants the value the pools will be sized against. It asks
`resolution_result` directly -- what its comment already claimed it was doing --
and it runs after the model families.
### The alternative, and why not
A pass that fills the field in when nothing claimed it needs a slot (after the
families, or it beats them), a second call site (the dummy-model short circuit
returns long before that slot), an idempotence requirement so the second call is
harmless, and the value written twice -- once as a literal, once as prose in the
help (`"Unset means 0.8"`). An earlier revision of this series did exactly that
and deleted it four PRs later. A declaration needs none of it, and `pipeline.py`
is untouched by the whole series as a result.
### What may be declared this way, and what may not
Across every hook, `if x is None: x = ...` appears at **55 sites over 29
fields**. They are not one thing:
| | count | examples | declarable |
|---|---|---|---|
| unconditional constant | 5 | the two ratios, `grammar_backend="xgrammar"`, `mm_process_config={}`, `custom_weight_loader=[]` | **yes** |
| unconditional, computed from another field | 4 | `tokenizer_path=model_path`, `device=get_device()`, `served_model_name`, `speculative_draft_model_quantization` | needs a `fallback="dotted.path"` form; not here |
| **conditional decision** | ~20 | `chunked_prefill_size` across seven memory tiers, `max_bs` across eight, `max_running_requests` at 48 or 256 by model family | **no, and it should not be** |
Only a value fixed for the life of the configuration belongs in a declaration.
One that depends on the machine, on another field, or on anything impure
(`random_seed = random.randint(...)`) is a decision, and decisions stay in a hook
where their order is visible. This PR converts the two ratios only.
## Verification
- `resolve_once` ends with the same effective values: the resolution result is
identical across 24 launch shapes x 489 fields except for the two intended
ratio changes. Separately, 16 launch shapes resolved on both sides, real model
and dummy: 7,904 field readings, and the only difference is `random_seed`, a
fresh `random.randint` per process.
- The CLI registers the same 507 options with the same choices and actions; only
the two defaults move.
- `test_declared_fallbacks.py`, 17 cases. One pins the inverse of the dead branch
above: what a pass sees while deciding is still `None`.
- The whole series was swept over all 648 registered unit-test files against its
merge-base: 19 failures on both sides, the same 19, none of them config.
---
### CI States
Latest PR Test (Base): <!-- slot:pr-test:start -->❌ [Run #34083705463](https://github.com/sgl-project/sglang/actions/runs/34083705463)<!-- slot:pr-test:end -->
Latest PR Test (Extra): <!-- slot:pr-test-extra:start -->❌ [Run #34083705284](https://github.com/sgl-project/sglang/actions/runs/34083705284)<!-- slot:pr-test-extra:end -->
Latest PR Test (AMD ROCm 7.2): <!-- slot:pr-test-amd-rocm720:start -->❌ [Run #34083705383](https://github.com/sgl-project/sglang/actions/runs/34083705383)<!-- slot:pr-test-amd-rocm720:end -->
<!-- pr-states:end -->
Mick