Fifth of five; stacked on #38049. The split gave every namespace a file, but
only for the half an operator types. This is the other half.
## The parallel quotients are declared, not written out
`attn_tp_size` and its five siblings were sixty lines of near-identical
properties in the runtime context, a file away from the leaves they are
quotients of, so reading `parallel.py` told you what you could set and nothing
about what that decides.
They are declared in `Parallel` now, in the same class as those leaves. They
carry no annotation, so they are not dataclass fields and
`collect_input_fields` never puts them on the record -- the same mechanism that
already keeps `_NS_PATH` off it. That is the right exclusion: a quotient has no
operator input to preserve, and the record is what crosses a process boundary,
where a stamped width is one an elastic scale-up will not refresh.
## A quotient is a value in the bag, like every other derived one
`_derived_width` answered from a stamp or, failing that, a live process group.
The group read could never disagree with the stamp:
- `initialize_model_parallel` stamps all six as its last statement,
unconditionally;
- an elastic scale-up restamps `attn_dp_size` through
`update_dp_attention_post_scale` -- the comment claiming it does *not* was
wrong;
- no hardware backend builds groups of its own;
- `multimodal_gen`, which has its own `initialize_model_parallel` and does not
stamp, never reads a quotient.
So a built group was always already stamped, and the group read goes -- and with
it the last reason for a quotient to be resolved on every read.
Every input to `derive_parallel_widths` is a record field. `dcp_enabled` is
`decode_context_parallel_size > 1`, not a fact about a built group; it was
spelled `_DCP is not None`, which is a longer way to say the same thing. So the
six are fixed once the configuration is fixed -- the same test every other
`Derived(fn=...)` in this PR passes. They are declared the same way and computed
the same way: once, at publish, into ordinary bag leaves.
What remains is override -> stamp -> published leaf. The stamp stays above the
leaf because an elastic scale-up restamps `attn_dp_size`; the override stays on
top because that is how a test names a width.
## One answer for the config-derived predicates
`enable_mamba_extra_buffer` and its lazy variant, `is_ep_joiner`,
`is_ep_scale_joiner`, `is_startup_weight_load_overlap`: each existed as a
`ServerArgs` member for the resolution pipeline and, for most of them, again as
a `runtime_context` function for readers after publish. Three places to keep
saying the same thing.
A `Derived(fn=...)` is a pure function of the published configuration, so
`publish` computes it once and stores it as an ordinary bag leaf -- a plain
attribute load, which is what a read inside compiled model code needs. The
function is handed the whole resolved config rather than the bag it lands in,
because a derivation is free to span namespaces and the mamba one does: it
reads `memory.disable_radix_cache` alongside its own `exec.mamba` strategy,
which is why it could never have been a method on either bag.
The pre-publish helpers stay -- resolution needs the predicate before there is
a bag to read -- and three readers keep them, because they run before their own
process publishes: `initialize_dp_attention`, which the weight-cache daemon
calls while building its groups thirty lines before its `publish`, and
`PortArgs.init_new`, a factory handed the record that already reads eighteen
other fields off it.
## Notes for a reviewer
**Overriding a leaf does not move its quotient.** `override(tp_size=2)` leaves
`attn_tp_size` where the published config put it, because nothing is recomputed
on read. A test states a topology by publishing a config -- which is what a
real process does -- or by naming the width it wants, `override(attn_tp_size=2)`.
Six tests say it that way now. This is the price of having one answer computed
once, and it is the same price every other derived value in the config already
carries.
A caller that reads a quotient without publishing or overriding now gets an
explicit error naming the field, instead of a default that an uninitialised
group happened to supply. One fixture was in that state --
`TestMlaWriteDoorsUnderDcp` built a bare pool and asked whether DCP was on --
and it publishes a config now, which is what the process it stands in for
does.
Eighteen sites read these predicates without calling them. That is correct --
they are properties -- but it is worth saying they were checked, because a
census that assumes otherwise reports eighteen always-true conditions.
## The skill that documents this subsystem is updated with it
`.claude/rules/modify-component-must-read.md` points at
`.claude/skills/sglang-runtime-context/SKILL.md` before anyone touches these
files, so a stale sentence there is a wrong instruction rather than a stale
note. Four of its load-bearing statements stopped being true across this series
and are corrected here: `NS(...)` is no longer how a field states its namespace
(the declaring class is); the DCP degrade rule is gone, because the quotients
are not live reads; `mamba_extra_buffer_enabled()` and the other predicate
functions it named as the shape to copy no longer exist; and the
namespace-coverage ratchet is described in terms of the marker. The docstring of
`test_server_args_namespaces.py` said the same thing and is fixed too.
The consequence a test author actually trips over is stated there as well:
overriding a leaf no longer moves its quotient, so a topology is stated by
publishing a config or by naming the width.
## Verification
A full registered-unit sweep (648 files) against this stack's merge-base:
19 failures on both sides, the same 19 -- AMD `gfx950`, `modelopt`,
`cuda_vmm`, `weight_checker` and friends, none of them config. The narrower 139-file config sweep used earlier in this series
does not contain the files this change reaches -- `test_kv_index_translator`
never names `get_parallel()`, it constructs an object that does -- which is why
the baseline differential over everything is what is quoted here.
Last of four; stacked on #38048.
The record is the operator's input; the bags are what is in effect. A reader
that takes the record and reads a field off it gets the input, which is the
wrong one of the two whenever resolution decided something -- and the mistake is
silent, because for most fields and most launches the two agree. Several of
these files already read both ways, sometimes in the same expression:
```python
get_tokenizer(
get_serving().tokenizer_path,
tokenizer_mode=server_args.tokenizer_mode, # the input, not the decision
...
)
```
Sixty-odd files convert. Record field reads in runtime code go from 199 to 11.
Nine parameters that the conversion emptied are dropped along with the argument
at every call site -- the dead-parameter ratchet is what names them.
### "Runs after its process publishes" is a per-entry-point claim
Most converted reads sit in the serving and model-executor layers, which only
exist after publication, or in the two subprocess entry points, which publish
first thing. Three places are not like that, and they keep reading the record
they were handed:
- **`HttpServerEngineAdapter`** launches the server as a *child*. The parent
resolves the record and never publishes, so the adapter's own reads -- the
launch banner, the API key in its readiness loop, the TP width in
`update_weights_from_tensor` -- are of `self.server_args`. A bag read here
fails closed in a bare process, or answers for an unrelated engine in one that
happens to have published.
- **`serve_grpc`** reads its sidecar port before the integrated servicer builds
the `Engine` that publishes. The comment above that line already said so and
already bound `cfg = resolving_view(server_args)` for it; the sidecar port and
the port it derives from read `cfg`.
- **`initialize_dp_attention`** runs from callers whose publish is not
guaranteed, so its one predicate stays on the resolution view.
`ROLE_NAMESPACE_SETS["dp_controller"]` gains `observability` and `serving`,
because the controller's metrics gate, tracing setup and worker-port broadcast
now read those namespaces. Under `SGLANG_ROLE_NAMESPACES=enforce` that set is
what the process may read, so a conversion that reaches a new namespace has to
widen it in the same change.
## Three things worth a reviewer's attention
**Eleven reads were `getattr(record, "field", default)`.** An AST scan for
attribute access does not see those, so the census that said "43 readers" was
counting the shape it could match rather than the thing it was after.
`incremental_streaming_output` was read that way twice, and the transcription
tests were the only reason it surfaced.
**Not every record read is a bag read waiting to happen.** A multimodal
processor's `base_gpu_id` is the instance's, not the process's: two engines in
one process keep different ones, and
`test_publishing_another_config_does_not_move_the_device` exists to say so. It
stays on the record while `rl_on_policy_target` beside it moves.
`RequestMetricsExporter` is the same shape -- it is handed the directory it
writes to, and a test builds several with different ones. `configure_logger` is
a third: 17 call sites, one of which passes an `argparse.Namespace`, so it is
not a global-context reader at all. Those eleven remaining reads are the ones
with a reason.
**The fixtures move with the code.** Tests that hung config off a mock manager
now publish a record, which is what the serving layer reads; where a test states
a value it says so with `override_server_args` instead of assigning through the
mock. `test_hisparse_unit` is the last of them: it stubbed a `server_args` onto
a fake scheduler to say the decode radix cache was off, and the value it was
standing in for is the published default, so the stub goes and the class
publishes.
## Two things CI caught that a local sweep could not
**`unittest.TestCase.enterContext` is Python 3.11+.** The converted fixtures used
it at 18 sites; `requires-python` is `>=3.10` and CI runs 3.10, so every one of
them raised `AttributeError` there while passing on a newer local interpreter.
They call `enter_override(self, ...)` now -- a four-line helper in
`sglang/test/test_utils.py` over the override's own `install()` / `restore()`.
**A batched sweep cannot see a missing publish.** Three fixtures needed a
published config and did not have one; each *passed* inside a shard where some
other file had published, and failed when run alone. The affected cases are
`test_serving_completions` (which set `incremental_streaming_output` on the mock
manager's record, where nothing reads it now), `test_qwen3_vl_feature_materialization`
(same shape for `mm_enable_dp_encoder`), and the two Qwen Rust tests -- whose
fixture already carried the comment `# Non-auto: get_resolved_model_impl would
choke on a SimpleNamespace` next to the `model_impl` it sets, which is exactly
what happened once `get_mm_processor_cls` started reading that value from the
bag. Its `publish` mirrors `model_impl` now, like the four fields it already
mirrored.
## Verification
A full registered-unit sweep (648 files) against this stack's merge-base:
19 failures on both sides, the same 19, none of them config. That sweep is what
caught 23 failures the file-scoped runs missed -- and, later, that the narrower
139-file list did not even contain the files this change reaches. It is also
what caught the `test_hisparse_unit` fixture above: the file passes inside a
shard where something else published, and fails when it is run on its own,
which is why every failing file is re-run alone before it is counted.
Third of four; stacked on #38047. Two small changes, both about the same thing:
the record holds the operator's input, and nothing else should be true of it.
## `/server_info` can answer what was actually typed
It reports `resolved_dict()` -- what resolution decided. There was no way to ask
the other question, and the two are not derivable from each other: a field
nobody set reads the same as one set to the value resolution would have picked
anyway.
The launcher stores the arguments it parsed and the in-process `Engine` stores
the call that built the record. All three readbacks report it beside the
resolved values, so both surfaces come back in one request: HTTP `/server_info`,
`Engine.get_server_info`, and the gRPC bridge's -- the last one builds from
`resolved_dict()` and would otherwise have been the one surface of the three
that answers only "what resolution decided".
It rides on the record rather than in a field -- it describes how the
configuration was asked for, so it is not part of the configuration: no CLI
flag, no namespace, not in the bags. Being on the record is what lets a
subprocess copy answer the same question the launcher can, and
`replace_resolved` carries it because a copy was launched by whatever launched
its parent.
The crash dump already collected all four surfaces (`server_args`,
`config_updates`, `resolved_config`, `launch_command`); this is the one that
`/server_info` was missing.
## The record is sealed for the length of resolution
The read-only guard armed on `_resolution_finished`, so for the whole run of the
pipeline nothing stopped a resolver from assigning a field. Nothing in `srt/`
does -- 0 assignments statically, and 0 writes observed across the launch-shape
matrix with a watching `__setattr__` -- but that was a convention, and the
defect it permits is invisible: a value a resolver wrote onto the record is
indistinguishable from a value the operator typed, which is the one distinction
the record exists to preserve.
It now arms when resolution starts. A resolver that assigns a field fails at
boot with a message naming `declare_resolution`, which is where the decision
belongs: the stash carries a source and leaves the input intact.
`declare_direct_writes` asks for the seal by name through `record_writable`. It
hands the record to an out-of-tree platform plugin that sets fields on it; those
implementations cannot be converted by editing a resolver here, so the write
stays and the diff is captured into the stash afterwards. Naming the exception
is the point -- an in-tree resolver reaching for it is doing something it should
be declaring.
## Verification
Costs nothing: the 211 test-side assignments all happen before `resolve_once`,
which a post-resolution write already refused. A full registered-unit sweep
(648 files) against the stack's merge-base: 19 failures on both sides, the same
19, none of them config. Driving a
deliberate write into a real handler produces the new error, so the seal is
tested by more than its own unit test.
Second of four; stacked on #38046. Mechanical relocation plus one design change
that the relocation makes possible. **Review by checking the identity proofs at
the bottom** -- nothing here is meant to change behaviour.
## The declarations move
`ServerArgs` carried all 487 declarations in one 4,462-line file, each tagged
with an `NS("...")` marker naming the namespace it belongs to -- structure
supplied by annotation, in a file a namespace away from the
`arg_groups/*_hook.py` that resolves it.
They move to `arg_groups/fields/`: one module per top-level namespace, one class
per leaf namespace (21 of them, `exec.moe` becomes `exec_.py::ExecMoe`). The
class carries the `_NS_PATH` it stands for, so the module a field is declared in
*is* its namespace and the marker is redundant -- `namespace_of` reads the
declaring class instead. `NS` stays for the one case a class cannot express: a
single ad-hoc dataclass whose fields span namespaces, which is what the
config-bag tests build.
Two things travel with the declarations. The `*_CHOICES` lists and the
`add_*_choices` adders that extend them move to `arg_groups/choices.py`, since
the fields naming them can no longer import from `server_args` without a cycle;
`server_args` re-exports all of them, because out-of-tree plugins have always
reached them there. And five fields whose only annotation element was the
namespace marker become plain annotations -- `A` is `Annotated`, which needs two
arguments, so stripping the marker would have left them invalid.
`server_args.py` goes from 4,458 lines to about 1,000.
## The record is assembled, not inherited
Inheriting the namespace classes would make the record's contents a property of
which classes happen to appear in a base list. That is correct only while every
namespace declares nothing but operator input, and it stops being correct the
moment a derived field is declared: `attn_tp_size` belongs in `parallel.py`
next to the leaves it is derived from, and inheriting `Parallel` would put it on
the record -- where it is neither input nor safe, since the record is what
crosses a process boundary and a derived width pickled to a subprocess is a
stamp that elastic scale-up will not refresh.
`collect_input_fields` takes the classes that declare input and returns their
annotations, defaults and namespaces. Each source's annotations are resolved in
its own module and handed on as type objects; carried across as text they would
be re-evaluated where they land, and the composing module deliberately imports
none of the names the declarations use. A namespace can now declare both halves
side by side, and which half reaches the record is one readable call rather than
an invariant spread across a base-class list. Nothing is registered on the
derived side yet -- this is what makes it possible.
`ServerArgs` is still one flat dataclass with 494 attributes, so
`server_args.tp_size`, `ServerArgs(model_path=..., tp_size=8)`, pickling to a
subprocess and every existing call site are untouched.
### Field order is a contract, so it is written down
A dataclass turns field order into a positional constructor signature, and
collecting whole namespaces groups fields that used to be interleaved. Keeping
`model_path` first is not enough: `ServerArgs("dummy", "/tmp/tokenizer")` would
set `load_format="/tmp/tokenizer"` and leave `tokenizer_path=None`, which then
selects an invalid model loader -- silently, at a call site that did not change.
So `arg_groups/field_order.py` records the order the record had before the
split, and `collect_input_fields` orders what it collects by it. A field the
record declares that the frozen order does not name goes after it, in
declaration order -- the only backward-compatible place for a new field anyway,
so a new declaration needs no edit there. The list is a compatibility record and
nothing else reads it; the namespace a field belongs to is still the module it
is declared in.
## Verification
Four ways, all against the base commit:
| check | result |
|---|---|
| `namespace_of` map, field by field | 494 / 494, **0 differences** |
| CLI surface (options, defaults, choices, actions) | 507 / 507, **0 differences** |
| field order, name by name | 494 / 494, **identical to the base** |
| resolution result, 24 launch shapes x 489 fields | **0 differences** |
| names importable from `sglang.srt.server_args` | nothing lost |
Plus a full registered-unit sweep (648 files) against the stack's merge-base:
19 failures on both sides, the same 19, none of them config.
First of five. The stack continues a series that moved configuration out of
`ServerArgs` and into the runtime context's namespace bags. This one fixes
something that was actually broken, and gives the fix its other half.
## "Unset" gets its own spelling on two ratio fields
`swa_full_tokens_ratio` and `mamba_full_memory_ratio` carried real values as
their class defaults (0.8, 0.9), so a model family with an opinion had to ask
"is this field still equal to the class default?" to find out whether the
operator had set it. That question has two wrong answers: it says "the operator
set it" as soon as any earlier pass declares the field, and it says "the
operator did not set it" when the operator types the default value.
Both become `Optional[float] = None`. The record carries what the operator typed
and nothing else, and the family test becomes `is None`.
`mamba_radix_cache_strategy` keeps `"auto"`: unlike the ratios it already has a
spelling for "unset" that an operator can type and that means exactly that --
only its comparison changes, from the class default to the token itself, which
is the fix the comment at that site already prescribed. With that, neither
family module imports `ServerArgs` any more.
## And the declaration says what the field means when nobody answers
Making the default `None` leaves a hole: something has to supply the generic
value. `Arg(fallback=...)` supplies it from the declaration.
```python
swa_full_tokens_ratio: A[
Optional[float],
Arg(help="...", resolvable=True, fallback=0.8),
NS("schedule"),
] = None
```
The dataclass default stays `None`. A fallback is not a default: the record is
the wire format, and a child process has to keep being able to tell "unset" from
"set to the value resolution would have picked anyway".
### Which surface it lives on is the whole design
Precedence becomes **override -> decision -> input -> fallback**, applied in
`resolution_result` -- which the projection, `/server_info` and every config bag
read through.
Deliberately **not** in `resolving_view` / `resolved_view`. Those are the
decision-over-input surface a pass reads *while it is deciding*, and two model
families branch on exactly this:
```python
# model_overrides/inkling.py, and the same shape in deepseek_v4.py
if cfg.swa_full_tokens_ratio is None:
overrides["swa_full_tokens_ratio"] = 0.1
```
A fallback answering there is not "the generic value, later" -- a `__getattr__`
layer is read-time, so there is no later. Every read during resolution would
already get 0.8 and the branch would never fire. Running `_inkling_overrides`
against both versions:
```
--- fallback on the effective surface only (this PR) ---
cfg.swa_full_tokens_ratio during resolution = None
family declared swa = 0.1 mamba = 0.1
--- fallback also on the view a pass reads ---
cfg.swa_full_tokens_ratio during resolution = 0.8
family declared swa = None mamba = None <- the key never lands
```
So "resolution first, then the fallback" holds -- not because a step is appended
to the pipeline, but because of which surface the value lives on. Exactly one
reader consults the effective surface during resolution: the range check on the
ratio, which wants the value the pools will be sized against. It asks
`resolution_result` directly -- what its comment already claimed it was doing --
and it runs after the model families.
### The alternative, and why not
A pass that fills the field in when nothing claimed it needs a slot (after the
families, or it beats them), a second call site (the dummy-model short circuit
returns long before that slot), an idempotence requirement so the second call is
harmless, and the value written twice -- once as a literal, once as prose in the
help (`"Unset means 0.8"`). An earlier revision of this series did exactly that
and deleted it four PRs later. A declaration needs none of it, and `pipeline.py`
is untouched by the whole series as a result.
### What may be declared this way, and what may not
Across every hook, `if x is None: x = ...` appears at **55 sites over 29
fields**. They are not one thing:
| | count | examples | declarable |
|---|---|---|---|
| unconditional constant | 5 | the two ratios, `grammar_backend="xgrammar"`, `mm_process_config={}`, `custom_weight_loader=[]` | **yes** |
| unconditional, computed from another field | 4 | `tokenizer_path=model_path`, `device=get_device()`, `served_model_name`, `speculative_draft_model_quantization` | needs a `fallback="dotted.path"` form; not here |
| **conditional decision** | ~20 | `chunked_prefill_size` across seven memory tiers, `max_bs` across eight, `max_running_requests` at 48 or 256 by model family | **no, and it should not be** |
Only a value fixed for the life of the configuration belongs in a declaration.
One that depends on the machine, on another field, or on anything impure
(`random_seed = random.randint(...)`) is a decision, and decisions stay in a hook
where their order is visible. This PR converts the two ratios only.
## Verification
- `resolve_once` ends with the same effective values: the resolution result is
identical across 24 launch shapes x 489 fields except for the two intended
ratio changes. Separately, 16 launch shapes resolved on both sides, real model
and dummy: 7,904 field readings, and the only difference is `random_seed`, a
fresh `random.randint` per process.
- The CLI registers the same 507 options with the same choices and actions; only
the two defaults move.
- `test_declared_fallbacks.py`, 17 cases. One pins the inverse of the dead branch
above: what a pass sees while deciding is still `None`.
- The whole series was swept over all 648 registered unit-test files against its
merge-base: 19 failures on both sides, the same 19, none of them config.
---
### CI States
Latest PR Test (Base): <!-- slot:pr-test:start -->❌ [Run #34083705463](https://github.com/sgl-project/sglang/actions/runs/34083705463)<!-- slot:pr-test:end -->
Latest PR Test (Extra): <!-- slot:pr-test-extra:start -->❌ [Run #34083705284](https://github.com/sgl-project/sglang/actions/runs/34083705284)<!-- slot:pr-test-extra:end -->
Latest PR Test (AMD ROCm 7.2): <!-- slot:pr-test-amd-rocm720:start -->❌ [Run #34083705383](https://github.com/sgl-project/sglang/actions/runs/34083705383)<!-- slot:pr-test-amd-rocm720:end -->
<!-- pr-states:end -->
`test_bag_values_match_server_args` asserted `bag == field`. That holds today
only because construction resolves in place; step 12 keeps the record raw, and
the plan doc calls this test out as one that becomes **false by design** for
every field resolution fills in.
Rewritten against the resolved projection, which is the half that survives: the
bag carries what resolution produced. The `bag == field` assertion stays as one
line at the end, labelled as the tripwire -- when it starts failing for a
resolution-written leaf, the flip has landed and the bag is the only place the
effective value lives.
The reference is an independent resolution of the same raw input (a fresh,
never-published record) rather than `resolved_server_args_dict()`, which reads
`vars(server_args)` back and therefore only restates the published instance.
And the record goes through the real pipeline on a real mini config, published
through `publish()`: the dummy-model path returns at the dummy boundary with
every sampled leaf still raw, so the old comparison was raw==raw and vacuous
(both Codex catches). Reproducibility (#34094) licenses the sibling as a
stand-in for the pipeline output.
The sample admits only leaves resolution writes on this input on both CI
device shapes (attention_backend, page_size, chunked_prefill_size,
mem_fraction_static), and the raw-differs guard asserts it per leaf -- a
default-count threshold let supplied inputs like `model_path` (no dataclass
default, so any path "differs") stand in for resolution work. Passthrough
leaves (host, hicache_ratio, moe_runner_backend, model_path) move to a
separate projection smoke that claims only what it checks: publish projected
an unchanged field into its namespace. Between the two resolutions the test
restores environ and the EnvField none-flags, so the sibling resolves the
same pristine input rather than the first resolution's leftovers.
And the class runs its body exactly once, like the other dual-resolve
harnesses: a CI retry re-enters after the first attempt leaked process
state, which is the hazard the pristine snapshot exists to rule out.
docs(skill): a supplied-instance read is not automatically safe
The whole-object rule said "keep the supplied-instance contract; don't rewrite
the parameter reads unless the field is runtime-mutated". That is the right rule
for the *object* and the wrong stopping point for the *field*: after step 12 the
record carries the user's raw input, so `server_args.page_size` inside a
runner-owned constructor reads the CLI default rather than the effective value.
The rule now names that second case as step-12 debt with a guard attached
(`test_supplied_instance_exposure_ratchet.py` fails on a new pair, so the
decision is made when the read is written), and names the two shapes that stay
parameter-form on purpose: a helper the resolution pipeline calls with a
`resolved_view`, and a factory whose contract is "build X from the record you
are handed".
A callee that takes `server_args` keeps the supplied-instance contract, so no
ratchet counts its reads -- and that is right for the *object*. What it does not
cover is what the object will carry after step 12: the instance stays at the
user's raw input, so a callee reading a field resolution fills in starts seeing
the CLI default instead of the effective value.
Measured, not guessed: **297 distinct file/field pairs read one of the 125
fields resolution can write** -- what remains after the earlier members'
conversions (the census found 314 across the package; the page_size,
chunked_prefill_size and graph/limit families were converted by the members
below this one, so the list lands at the remaining debt with no churn). The
census counts three spellings of the read: `server_args.field` off the
parameter, `getattr(server_args, "field", default)` with a literal name, and
the *parked* form -- `self.x = server_args` in a method that takes the
parameter, read as `self.x.field` anywhere in the class. Parking under a
different object, a container, or a computed name stays invisible, like in
every census of this family.
The written-field set is derived in-test from resolved configs against the
dataclass defaults -- the same matrix the context repo's audit tool uses -- and
the union is only as complete as the matrix: fields a matrix entry passes in
are excluded, so each entry must let resolution make the decision the entry is
about. The DWDP shape is the loudest example: `_handle_dwdp` writes `dp_size`,
`enable_dp_attention`, `ep_size` and friends itself, and without a
`{tp_size: 2, dwdp_size: 2}` entry the whole DP/EP topology family (37 pairs
across the launcher, the controllers, the tokenizer and the spec workers)
never entered the written set at all. Multi-item scoring is the same shape in
miniature -- `_handle_multi_item_scoring` writes `disable_radix_cache` itself,
and the `{enable_mis, attention_backend=flashinfer}` entry (the backend is
passed because the handler asserts rather than switches) pins the radix-cache
builder and its friends.
The written set carries **may-write semantics**, statically collected from
every mechanism that can put a resolved value on the record, unioned with the
matrix (construct-and-diff still catches value-level writes statics cannot
name). The enumeration approach kept losing to review -- the DFLASH hook hole,
then the declarative registry (`MODEL_OVERRIDES` forces dtype for two arches
through a setattr applier no assignment scan sees), then the deprecated-alias
loop that writes through a name tuple -- because each round found one more
*mechanism*, not one more field. So all of them are collected now: hook
assignments under `arg_groups/`, the record's own method assignments (the
mooncake layout rewrite, the deepseek-EP mode defaults, the seed fill that
only fires when the caller did NOT supply one -- which construct-and-diff can
never see, since measuring requires supplying), the declarative override
registry, the alias-normalization tuple (drift-guarded), and the
late-resolution keywords. A statically-collected write site that can never
fire is a dead branch to delete upstream, not a reason to shrink the census
(maintainer's ruling). And the pin is split by host: `_EXPOSED` is asserted everywhere,
`_EXPOSED_CUDA_ONLY` (empty today) is where a capability-gated write's
readers go -- one shared exact list cannot hold such a pair at all, since
pinning it fails CPU as "gone" and omitting it fails CUDA as "new".
The resolve-once shape also undercounts on axes a construction call never
sees (each of these was a review catch, and each added pinned families):
resolution branches on the *environment*, so every matrix entry resolves under
the plain env and under the CI shape (`SGLANG_IS_IN_CI`), with the pristine
process state (environ plus the `EnvField` descriptor flags) restored between
entries -- that is where `soft_watchdog_timeout`'s four readers come from. Some
fields resolve *late*, at validation rather than construction
(`declare_late_resolution`): those writers are collected statically by keyword
-- `lora_paths`, `reasoning_parser`, `tool_call_parser` -- with the one dynamic
`**detected` site spelled out in a table guarded against drift. Fields holding
only a `default_factory` are materialized rather than skipped, and
`tokenizer_path` / `served_model_name` -- always filled from `model_path` --
leave the passed-inputs exemption and pin their twelve readers. A module the
census cannot parse fails the test instead of shrinking it, which immediately
caught a BOM-carrying file every previous census had silently skipped (the
scans read `utf-8-sig` now). And the test registers on the CUDA runner besides
the CPU suite, because capability-gated writes only open on real hardware;
AMD is intentionally not registered -- an exact pin cannot be verified from
any pinning host -- with the reasoning in the header.
**A second axis is already wrong today**, independent of step 12. Some config is
decided after publish and recorded with `get_context().override(...)` -- elastic
EP resizing `ep_size`, a weight update rewriting `model_path` / `load_format`,
HiCache attach naming a storage backend, adaptive speculative decoding moving
`speculative_num_steps`. That write reaches the bags and never the record, so a
supplied-instance read of one of those fields answers with the startup value from
the moment the override lands. **73 pairs over 13 fields** are in that position -- including the overrides
that arrive as `**kwargs`: the collector statically resolves dict-literal
expansions (the HiCache attach shape, whose write/read pairs on
`hicache_write_policy` / `hicache_storage_prefetch_policy` were invisible
before) and fails loudly on anything it cannot resolve, with
`update_server_args` exempted by name because its key set is the API's
caller's, not this file's.
Whether each is a defect depends on ordering -- a value copied at construction,
before any override, is fine -- so the axis is pinned as a measurement with the
same growth guard, not as a list of bugs.
One of them *was* a defect and is fixed at the base of this stack: the
linear-attn dispatch table rebuilt itself from the record after the SM100 GDN
prefill decision had been recorded in the bag, so a second runner's rebuild
dropped it. That choice is a per-runner stamp now and is not recorded
process-wide at all, which is why neither the read nor the field appears on this
axis.
The list is pinned both ways, on both axes. A new pair fails, because the moment
to decide where a resolved value comes from is when the read is written, not
during the flip; a disappeared pair fails too, naming the entry to delete, so the
registry stays a measurement rather than a memory of one. Both axes
reverse-verified: a new read of a written field is reported by file and field.
Per-field dispositions live in the plan doc; several of these are "should this
callee take a config at all?", which is a design call rather than a sweep.
test(step-12): tripwire on the EPD guard that a raw record would silence
`_reject_missing_dispatched_encoder_embedding` is one of the two reads the
step-12 audit calls a blocker: it keys on `encoder_transfer_backend`, a field
resolution fills in, off a handed record. Today that record carries the
resolved value; after the flip it stays at the argument default `"auto"`
(`ENCODER_TRANSFER_BACKEND_CHOICES[0]`) for every auto-resolved launch and the
503 stops firing -- a guard that goes quiet, which no existing case notices.
The tripwire resolves a real language-only Kimi-K3 TP2 launch (a mini config,
the shape whose auto pick is `"zmq_to_tokenizer"`) and asserts the guard
rejects with the record resolution produced. A fixed double cannot trip on the
flip -- it would keep handing the guard the resolved value by construction --
so the record has to come from resolution itself: when step 12 lands, this
same launch hands the guard `"auto"`, the rejection silently stops, and this
test fails, which is exactly the signal that this reader needs the resolved
value from somewhere else (the per-engine overlay or the bag).
The launch pins `mamba_radix_cache_strategy=no_buffer` (+ the overlap-off it
requires): resolution's hybrid state-cache sizing branches on the host device
and asserts a GPU stack for extra_buffer, which a CPU CI runner does not have,
while the guard under test reads a field independent of that branch. The case
restores env *and* the EnvField descriptor flags -- a real resolution leaves
state os.environ does not carry.
config: the speculative workers take page_size from the bags
Seven worker constructors stored `self.page_size = server_args.page_size` off
the handed record. They all run after publish and all keep a copy of a
process-level value, which is the first row of the plan doc's supplied-instance
disposition table -- so they read `get_schedule().page_size`, and a post-publish
override now reaches them like it reaches every other consumer.
The supplied-instance census named the seven pairs; the exposure ratchet in the
next member pins what remains after this batch of conversions.
config: the post-publish chunked_prefill_size consumers read the bags
Four of the ten supplied-instance `chunked_prefill_size` reads are plain
post-publish consumers -- the EPLB recorder's buffer sizing, the deep-gemm
compile warmup (five reads), the KV-cache builder's effective size, and the
ngram embedding manager's assert. All are reached from runner init, so they read
`get_schedule()`.
Two are deliberately left: `create_kt_config_from_server_args` builds a config
*from a supplied record* by name and contract, and `CanaryLaunchCapacities.from_args`
is the same shape. Converting those would change what the function is, not where
it reads -- the plan doc's disposition table says so per field.
config: the remaining post-publish graph/limit consumers read the bags
Three more of the census's supplied-instance debts are plain post-publish reads: the dspark worker's
cuda-graph decode sizes, the dspark planner's SPS table bound
(`max_running_requests`), and the LoRA manager's cuda-graph moe buffers. The
dspark worker is the clearest of them -- it already read
`get_exec().graph.cuda_graph_config.decode.bs` thirty lines below the instance
read, so the file disagreed with itself about where the same value comes from.
Left where the function's contract is "build a config from the record you are
handed" rather than "read this process's config":
`create_kt_config_from_server_args`, `DllmConfig.from_server_args`,
`CanaryLaunchCapacities.from_args`, `build_compilation_config`. Changing those
would change what the function is.
config: the runner, scheduler and offload manager take page_size from the bags
The same `self.page_size = server_args.page_size` shape as the speculative
workers, in the three remaining process-owned constructors: `ModelRunner`,
`Scheduler`, and the decode-side KV offload manager. The scheduler process
publishes before any of them run. The one path that did not is `ModelRunner`
constructed standalone -- `python -m sglang.benchmark.one_batch` and the manual
runner tests build it with no prior publish, and the constructor's own publish
sat below this read -- so that publish moves above the constructor's first bag
read instead of leaving a window where the runner half-exists unpublished.
Left where the read belongs to something else: `utils/common`'s predicates are
called only from the resolution pipeline with a `resolved_view`,
`allocation_sizing` takes the config its callers supply by contract, and
`CudaVmmFeatureTransport` is tokenizer-owned -- one per tokenizer worker, which
is the per-instance boundary.
The conversion left the offload manager parking a record it no longer
reads; the parked copy goes with the read (the constructor parameter stays
-- its hicache sizing still reads it directly).
The previous batch counted `self.server_args.X` and called the runner surface
done. It was not: the same read spelled through a local alias --
`server_args = model_runner.server_args` (or `sa = kvc.server_args`, `args = ...`)
followed by `server_args.leaf` -- is the same process-global read wearing a
local name, and the AST census counts **57 of them** across eleven files that
the grep never saw. Census per function, following the alias.
52 were leaves and go to their bag (`spec` 11, `schedule` 9, `memory` 7,
`exec.graph` 5, `exec.moe` 5, `parallel` 4, `disagg` 4, `model` 3,
`exec.mamba` 2, `exec.overlap` 2). Five were not leaves:
three derived members on the eager runner --
`max_speculative_num_draft_tokens` and `enable_mamba_extra_buffer` already had
accessors, and `max_prefill_buffer_tokens` gets one (all its inputs are `schedule`
leaves plus the configured PP size, so it derives from the bags and follows a
post-publish override; `TestDerivedPredicatesAgreeAcrossTiers` pins it against
the member over a 48-case matrix) -- plus `get_attention_backends()`, which the
same commit routes through `attention_backends()`, and a dict that merely shares
the name (`server_args_dict.items`). That dict is the one read left behind.
`build_attention_backends` also stops resolving the pair from the record: it
runs after publish, so it asks `attention_backends()` like every other consumer.
The draft override on the runner still wins first.
`dispatch_event_loop`'s three PP checks read the *configured* PP size, not the
live topology: the MLX runner stub never initializes torch.distributed, so the
live property asserts before the MLX event loop can start (a Codex catch). The
configured leaf answers the same value wherever the live groups exist.
`flashinfer_gdn_prefill_default`'s guard is the one read here that asks what the
*operator* named rather than what the config resolved to, and the bag leaf now
answers exactly that: the per-runner auto-default is stamped on the runner and
deliberately never recorded process-wide, so nothing writes that leaf after
launch and reading it back cannot mistake another runner's default for a flag.
Three test doubles injected a `SimpleNamespace`/`MagicMock` record for exactly
these reads and now publish instead (pool configurator, cache registry, GDN
prefill policy) -- the fixture publishes what the case configures and hands the
published instance to the whole-object contracts that still take one.
The functions this sweep partially converted stop mixing sources (review
catches): the flash-attention constructor's remaining seed reads
(`speculative_eagle_topk`, `speculative_algorithm`, both deterministic gates)
read their bags next to the leaves already converted;
`_should_disable_scheduler_metadata_precompute` reads the parallel config
leaves itself instead of taking the record (its alias binding was the last
use); and the autotune gates (`disable_flashinfer_autotune`, deterministic,
`flashinfer_autotune_skip_ops`) join the moe leaves the same function already
reads from the bags. The pool-configurator fixture drops a parameter nothing
published or read.
`_is_dsa_active` asked `getattr(server_args, "_is_dsa_model_arch", False)`, and
that name has never existed on `ServerArgs` -- it arrived as a placeholder with
the CP strategy abstractions (#27313), so the getattr default has always decided
the predicate. A dynamic read of a name nothing sets is the one shape the config
census cannot follow, and it looked like a live decision while being dead.
Spelled as the constant it evaluates to, with the placeholder written down: what
it should ask (whether this process runs a DSA model arch) is the CP path's
call, and its only consumer, `ContextParallelStrategy.per_layer_attn_cp_comm`,
has no readers yet.
That was the sole entry in the read ratchet's `_INERT_DYNAMIC_READS`, so the
exemption list is gone with it -- there is no way to exempt a read from the
baselines any more, which is the invariant worth having. The `counted()`
indirection it existed for goes too (verified the three shapes it guarded still
report: direct, `getattr`, and an attribute-parked alias).
`--attention-backend` is one field of three: a launch that sets only
`--prefill-attention-backend` or `--decode-attention-backend` leaves the base
field at `None`. Seven decisions read that base field alone and therefore
answered from a field the operator never set. `attention_backends()` is the
pair with the base-field fallback already applied, so each site now asks it for
the half it actually needs:
- `inkling_common/attn` assembles backend-specific kwargs (rel_bias / score
mods) and gates its fused prologue; the backend those describe is the one
`self.attn` dispatches to, so `serving_attention_backend()` selects the pair
member by `forward_batch.forward_mode`, mirroring
`HybridAttnBackend._select_backend` exactly -- draft-extend routes through
the prefill branch like the dispatcher does -- and preferring the
runner-stamped pair, so a draft runner answers with its own backend. That
preference only works if every backend that can enter a ForwardContext
carries the stamp, so `DraftBackendFactory._create_backend` now stamps its
products with the backend it resolved (draft override first), and the
draft-extend conv-sidecar wrapper copies the wrapped backend's stamp -- the
replacement backends the spec workers install had no stamp at all and fell
back to the target's configured pair.
- The chunked-prefix-cache gate is a *prefill* feature -> prefill half. Reading
the base field switched the feature off for every prefill-only configuration.
- `init_deterministic_inference_config` maps *prefill* knobs
(SPLIT_TILE / PREFILL_TRUNCATION_ALIGN) -> prefill half; the map missed and
left truncation unset.
- `two_batch_overlap` computes extend positions -> prefill half.
- mrope's interleaved-rope kernel runs in both phases -> both halves must
support triton. This one is not conservative when it misreads:
`support_triton(None)` answers **True**, so a `--prefill-attention-backend
torch_native` launch took the triton path.
- The req-to-token writer has one caller, `alloc_for_extend` -> prefill half;
its fallback pays several `.item()` syncs per request, so gating it on the
decode half too would send every extend of a mixed launch through the slow
path. `get_last_loc` (the spec-decode allocator's helper) keeps the
both-halves reading: verify tokens are served by either half depending on
`speculative_attention_mode`.
- The flashinfer version floor is a guard; it never fired for a launch that
pinned flashinfer through a split field.
One more site the census found is not converted here: `gpt_oss` derives its
`sinks` parameter dtype from the backend, and a single parameter dtype cannot
serve a split pair (FA4 asserts bfloat16, trtllm_mha consumes float32), so
that one is a behaviour question rather than a config-source one and is fixed
in its own PR.
`test_split_attention_backend_decisions.py` pins the callable decisions by
calling them under a split-only publish, and pins the remaining ones
statically -- the file/why map fails if any of them goes back to the base field
(reverse-verified). It also asserts the `support_triton(None) is True` trap the
sweep exists for.
The stamp comes from the constructor, not the request: every factory leaf
answers ("effective_name", backend), because several map entries do not build
what their key says -- cutedsl_mla draft-extend builds the trtllm-mla backend,
"nsa" is a deprecated alias building dsa, and the hybrid-linear entries pick
fa3/intel_amx/triton by host, which no static rename table can express (a
review catch: on Blackwell the alias stamp reached Inkling's per-forward
kwargs assembly, which asserts a concrete kernel name, and crashed the first
draft-extend forward). The stamping is pinned by unit tests, not only by a
spec e2e: removing the child-stamping loop, stamping an alias from a leaf, or
dropping the wrapper copy goes red (reverse-verified), and a static guard
walks the factory source asserting no leaf answers an alias name. The child loop states its contract explicitly --
`create_decode_backend` passes `stamps_children=True` because its products
are per-step containers by construction, so a container without
`attn_backends` raises instead of being silently skipped by a defensive
probe. The `_version` invalidation names its contract (autograd's in-place
counter: private, chosen because it is the only per-tensor signal that ticks
on copy_-style updates; removal fails loudly). The version-floor guard's file
joins the pair-reader ratchet, and the one runner-seed chain read sharing the
backend's __init__ (`speculative_eagle_topk`) reads the spec bag.
Six reads were left on `self.server_args` outside the per-instance boundary the
plan reserves for the tokenizer-manager family, and each had a different reason
to be there:
- `scheduler.process_input_requests` (`mm_feature_transport`) and
`BaseSpecWorker._build_hicache_draft_plan` (`enable_hierarchical_cache`) are
plain leaves -> `get_mm()` / `get_memory()`.
- `DraftBackendFactory._create_backend` read the split backend through a
*runtime-computed name* (`getattr(self.server_args, backend_name)`) and then
fell back to the base field by hand -- the census's documented blind spot.
The two names it can be handed are exactly the pair `attention_backends()`
returns with that fallback already applied, so it reads the pair and indexes
it. The draft runner's own stamp still wins when it has one.
- `remote_instance_weight_loader_use_transfer_engine` and
`pre_capture_activation_reserve_mb` are derived members. Both are computed
from published leaves only, so both get a named accessor that derives from
the bags (and therefore follows a post-publish override).
The first of those two has all its inputs in one bag, so it follows the
established shape: one `*_of(cfg)` helper in `arg_groups/overrides.py`, the
`ServerArgs` member delegating to it, and the accessor calling it on
`get_model()`. `modelexpress_transport_of` splits out the JSON parse both
sides need. The second spans four bags plus the configured parallel sizes, so
it exists twice like the mamba pair -- and `TestDerivedPredicatesAgreeAcrossTiers`
now pins both new pairs equal over their input matrices (92 subtests).
`self.server_args.X` outside the tokenizer-manager family: 11 -> 5, and the
five that remain are the documented ones (the encode server's own record, the
nixl connector's rank arithmetic, `GrammarManager`'s handed instance).
The post-capture headroom path calls the same bag-backed
`pre_capture_activation_reserve_mb` accessor the configurator uses -- the
accessor advertises override-following, and a reserve that reads the record
while its sibling reads the bags can disagree after a post-publish override.
And the conversions' orphans go with them: `RemoteInstanceWeightTransporter`
kept a `server_args` field nothing reads, and `DraftBackendFactory` parked a
record it no longer consults -- both drop the parameter, and the four factory
call sites stop threading one.
Two problems in the same family as #33312 (a per-runner decision that one
participant answered differently), one fixed here and one guarded.
**The linear-attn kernel backends were process-wide.** `attn_backend_wrapper`
rebuilt a module-level dict once per runner, from the handed record plus a local
`prefill_default`. Two things follow, and both are wrong:
- **A draft could not hold a different choice than its target.** Only the runner
whose model is GDN gets the SM100 FlashInfer prefill default; the operator's
explicit flag belongs to the launch. The full-attention backends already model
this correctly -- the runner stamps `prefill_attention_backend_str` /
`decode_attention_backend_str` and its backend objects are built from the
stamp. Linear attn had no stamp at all.
- **The second rebuild replaced the first one's choice.** The default was also
recorded into the process-wide config, which the record does not see, so a
runner rebuilding without a default of its own resolved `prefill` back to the
base backend -- silently swapping the kernel the earlier runner selected.
Demonstrated in-process before this change: table `FLASHINFER`, then `TRITON`.
`resolve_linear_attn_backends(prefill_default=None)` returns a frozen
`LinearAttnBackends(decode, prefill, verify)` from the published `exec.mamba`
leaves; the wrapper stamps it as `runner.linear_attn_backends` before building
the backends that read it; and the three consumers (GDN, KDA, Ascend GDN) read it
off the runner they are built for. Each already took `model_runner` and cached
the result on itself, so the value now simply comes from the right place. A
backend built outside that path has no stamp and raises on the attribute, the way
the full-attention strings do -- no silent fallback to hide the wiring mistake.
The recording goes away with it. A per-runner choice in the process-wide config
has no meaning the second runner can read correctly: the leaf is how the gate
asks "did the operator name a backend", so a recorded default reads back as an
operator flag and the next runner declines its own. The leaf now keeps meaning
what was asked for at launch, and the effective choice lives in the stamp (and
in the log line the gate already emits).
Precedence is unchanged: the resolver takes the default as an argument and an
explicit `--linear-attn-prefill-backend` wins over it, with the gate declining
early so it neither probes the device nor logs.
**A draft entry class must answer the loader exactly when its target does.** The
loader asks the entry class it instantiates for the shared-experts-fusion
decision, and a draft is its own entry class. When the target family carries
auto-disable conditions and the draft's class does not expose them, the loader
installs one decision for each and the draft's weights are laid out for the wrong
one. That shipped: the DSV4 DSpark draft skipped its bundled shared-expert
tensors until #33312 gave it the gate, costing accept length 5.60 -> 2.05.
`test_fusion_gate_coverage.py` walks the same registry but asks whether an entry
class *touches* the decision -- reads the flag, names a gated class. That catches
a class once it already consumes the decision; it could not catch one that should
consume it and does not, which is what the DSpark class looked like (it built the
family's *layer* classes, so the flag reader lived in another module and its own
source named no gated class). `test_draft_entry_hook_parity.py` asks the
invariant directly: presence parity between a draft entry class and the target it
is named after. Identity is deliberately not required -- the Qwen3.5 MTP
delegates with adapted arguments (unwrapping `text_config`, using the MTP
quantization config), which is right -- and weight-name maps are out of scope,
since a draft's checkpoint has its own names.
Reverse-verified against the original defect: with the DSpark gate removed the
case names the pair and the side that is missing it; with #33312 in place it
passes.
`build_draft_tp_worker` built a `ServerArgs` variant whose only job was to make
four config reads answer with the draft's backend instead of the target's, and
published it for the duration of the build so the bags agreed. The backend is a
per-runner fact — target and draft coexist in one process — so it moves onto the
runner, and the variant and the construction-time publish both go away.
`ModelRunner` takes `draft_attention_backend` and resolves the runner's effective
value once (`resolve_draft_attention_backend`: the algorithm's resolved backend,
else `--speculative-draft-attention-backend`, else None for a target runner);
`TpModelWorker` threads it to both runner constructions.
`resolve_attention_backend_strs` reads it off the runner, and `ModelRunner`
stamps the resolved pair *before* building backends so a backend can read it
while it constructs — which is what the FlashInfer KV-access check needs now that
it no longer asks the config. `configure_kv_cache_dtype` and the draft backend
factory read the runner too.
One latent bug falls out: the non-hybrid branch of the backend build ignored the
resolved pair and re-read `server_args.attention_backend`, which is why the
variant had to set that field as well as the split pair. It now uses the value
that was resolved for the runner.
`draft_server_args_overrides` and the `preserve_config()` publish switch are
deleted; with them goes the last production `ServerArgs.derive` outside
pre-publish config building, and the last construction-time publish. The
chunked-prefix gate the target resolved simply stays in the bags, since nothing
re-projects them.
The v2 spec workers got a published `ServerArgs` copy carrying two values: the
target's context length and `--speculative-draft-load-format`. Neither is a
process-wide config change — each is consumed by exactly one constructor — so
the copy, the publish switch around the draft build, and the replay of the
target's resolved overrides onto it all go away, and the values travel to the
runner that owns them:
- **Context length.** `TpModelWorker` already takes it (`context_length=None`
keeps `server_args.context_length`); the four v2 draft workers and
`build_draft_tp_worker` pass the target's, which every one of them has in
scope as `target_worker` / `target_model_config`.
- **Load format.** `ModelRunner._draft_load_format()` resolves it for a draft
runner and `build_load_config` takes it, so the `LoadConfig` is per-runner.
Model code also reads it off the bag while it builds — Inkling replaces
per-element noise in its shared-expert scales under dummy loading — so the
load is wrapped in a scoped bag override that puts the target's value back.
- `skip_tokenizer_init` was on the copy for nobody: `TpModelWorker` already
short-circuits the tokenizer for a draft worker (`or self.is_draft_worker`).
`PrefillCudaGraphRunner._max_addressable_prefix_len` capped the prefix by
`server_args.context_length`, which the copy used to carry for the draft; it now
reads the runner's own `model_config.context_len`. That is also more accurate for
the target, whose `--context-length` may be unset while the resolved context is
shorter than the token table.
What stays a variant is the dflash/dspark path's attention backend: backend
selection reads it off the config object the draft runner holds, and the
resolved gate has to survive the variant's publish. `draft_server_args_overrides`
now carries only those fields and says why.
`ServerArgs.override(source, **fields)` was the last way to change a resolved
`ServerArgs` in place. Every remaining call-site was one of two things, and
neither wanted an in-place write:
- **A config for someone else.** A draft worker's context length, an encode
worker's device, the compile script's watchdog, the client's port pick, a test
fixture's backends. These already deepcopied first — the write was on the copy.
- **A launcher-stage resolution.** `resolve_auto_parsers` detected the chat
template's parsers and wrote them back, to be inherited by the schedulers it
spawns.
Both are "one config becomes another", so `derive(source, **fields)` returns the
variant and leaves the receiver — and any bags projected from it — untouched. It
deliberately is not `dataclasses.replace`: resolution does not re-run, because
the values being set are decided after it, from inputs it never had. Provenance
and the resolvable-field stash work as before, on the copy.
`resolve_auto_parsers` now computes the parsers and returns the config to launch
with; the detection helpers stop taking a config to mutate. `HiMambaRadixCache`
re-applied a HiCache layout normalization `__post_init__` already performs (the
same duplicate removed from `UnifiedRadixCache` in ebb1c88d23) and just goes.
With no in-place mutation left, `ServerArgs.__setattr__` raising after
resolution *is* the guarantee, so the textual writer ratchet retires and
`test_server_args_derive.py` pins the contract instead: the receiver survives
deriving, the published instance still refuses assignment, and deriving does not
publish. `SGLANG_STRICT_CONFIG_MUTATION` was already unused — the guard has been
unconditional since the mutation sweep — and goes with it.
The detection tests drop their `SimpleNamespace` stand-in for a real
`ServerArgs`; the test kit and the MLA chunk-metadata fixture publish a derived
variant instead of writing the runner's published config.
`init_tokenizer_manager` wrote the chat-template-detected `reasoning_parser` /
`tool_call_parser` onto the published `ServerArgs`, after
`TokenizerManager.__init__` had already projected the config bags — so the
namespace readers and the resolved-config readback disagreed with the instance,
and a second `Engine` in the same process would inherit the first one's
detection through the shared bags.
Detection is per-engine control-plane state, which the manager already models:
`record_config_updates` records it, the readback endpoints overlay it, and
`config_value` reports what is in effect. `OpenAIServingChat` — the only reader
of these two fields in the tokenizer process — follows the overlay. The
architecture pass (`resolve_auto_parsers`, before the schedulers fork) is
unchanged: the scheduler resolves its own bags from the instance it receives.
`sa = get_server_args()` followed by `sa.field` reads the same startup record as
the direct form; the read ratchet added in the previous slice pinned twelve of
them as the remaining surface. Eleven now read the accessor for what they
actually want:
- `is_enable_moe_cp_allgather` compares the attention-CP and MoE-DP sizes to
decide whether a forward needs an allgather, so it reads the live topology
through `get_parallel()` — the same source `get_moe_cp_size()` right above it
already uses. Both groups exist once model-parallel init has run, which is
before any forward.
- The DeepSeek MLA decode-backend gate and Inkling's attention paths read
`get_exec().kernel`; Inkling's KV-dtype checks read `get_model()`. These are
per-runner fields, and the value they get is the config published for the
runner being built — unchanged from what the alias returned.
- The int8 mamba checkpoint pool reads `get_exec().mamba`. It keeps its guard
for callers that construct the pool with no published config; that guard now
catches the namespace accessor instead of the slot.
`model_loader`'s `moe_dp_size` stays on the instance and is exempt: the dict it
belongs to already reports the live size under `"dp"`, so that entry is the
configured intent, and `get_parallel()` shadows the name with the live value.
Alias-form baseline 12 -> 0. What remains on `get_server_args()` in the package
is the derived API (properties and methods computed from several fields plus the
HF config) and four config-intent reads of live-shadowed sizes, each exempt by
name with its reason.
`RayEngine.__init__` set the caller's `PlacementGroup` on `ServerArgs` as an
undeclared attribute, and every consumer read it back off the config object:
the non-DP launch path took `pg` and `is_custom_pg` from it, the DP path had to
re-attach the handle after `dataclasses.replace` dropped it, the DP controller
asked the config whether the group was the caller's, and the Ray HTTP launcher
had to clear the field so a stale handle could not leak in.
A live cluster object is not config. It now travels as a `placement_group`
argument on the two launch hooks — `Engine._launch_subprocesses` and
`Engine._launch_scheduler_processes` — defaulting to `None`, with `RayEngine`
holding the caller's group in `_placement_group` and the DP path passing
`is_custom_pg` down to the controller. The public API is unchanged:
`RayEngine(placement_group=pg, ...)` still works, `pg` still falls back to the
ambient group, and the HTTP launcher gets `None` from the default.
Writer ratchet 18 -> 15.
`get_server_args().<field>` reads one process's startup record. Nine sites still
did that for a value that has a namespace: the attention backend (5),
`skip_tokenizer_init` (2), the draft-aware `load_format`, and a chunked-prefill
size in `sglang.kernels`. They now read `get_exec().kernel` / `get_serving()` /
`get_model()` / `get_schedule()`, so they see the resolved value including
post-publish overrides.
The multimodal processor's device selection moves to the instance it was
constructed with rather than to a namespace: `base_gpu_id` differs per worker (the
encode-server DP workers each specialise their own copy), so no process-global
value can stand in for it, and engines sharing a tokenizer process each need their
own. Branch order, the NPU preprocess patches, and the case that leaves "device"
unset are unchanged.
What stays on `get_server_args()` is the derived API — `@property` and method
members computed from several fields plus the HF config
(`mamba_cache_chunk_size`, `get_model_config()`, `enable_mamba_extra_buffer*`) —
plus three config-intent reads of live-shadowed sizes, each of which needs an
answer the live topology property cannot give (the DSA indexer's PP gate must
short-circuit before touching the PP group, `allocation`'s DCP gate asks whether
DCP was configured at all, and the CUDA-IPC recycler runs where no group exists).
A new AST ratchet pins both shapes it can see — the direct call and an alias
bound from it in the same function — at 0 and 12 respectively, exempting the
derived APIs and those three sites by name. The alias-form baseline is not zero:
those reads are mostly per-runner fields in model code, and lowering them is the
next slice.
Two fixtures stopped faking config: `test_dllm_fdfo_kv_reuse` rebound
`allocation.get_server_args` to a SimpleNamespace, which silently stops
intercepting the moment a reader migrates; it publishes a real config instead.
When --forward-pass-metrics-ipc-name is left unset the reporter generates an
endpoint and has to hand it to external consumers (the documented contract is
that they read it back from the server config). That readback is the scheduler's
get_internal_state, which already reports get_context().resolved_server_args_dict(),
so the write moves to get_context().override and the read alongside it to
get_observability() — the endpoint still shows up in /server_info's
internal_states, and the ServerArgs instance stops being a message bus.
The test's server_args stand-in (a SimpleNamespace with a hand-rolled override)
becomes a real published config, so the reporter exercises the same accessors as
production.
Writer ratchet 19 -> 18.
The scheduler's runtime HiCache attach/detach wrote its own ServerArgs so the
internal-state readback would show the change; that readback already reports
the resolved config, so the writes become get_context().override(...) and the
namespace readers see them too.
The tokenizer side is per-engine — several Engines can share one process — so
its control-plane updates (weight version, model path + load format, HiCache
attach/detach) stay with the manager instead of moving to the process-global
bags. TokenizerManager gains record_config_updates / config_value /
resolved_config_dict, and the readbacks that used to observe the instance write
(/server_info, /model_info, the HiCache status endpoint, the gRPC bridge) now
overlay those updates onto the startup config.
test_server_info's stub grew the real manager instead of a SimpleNamespace, so
the overlay it now exercises cannot drift from production.
Writer ratchet 26 -> 19.
Cheng Wan