Fifth of five; stacked on #38049. The split gave every namespace a file, but
only for the half an operator types. This is the other half.
## The parallel quotients are declared, not written out
`attn_tp_size` and its five siblings were sixty lines of near-identical
properties in the runtime context, a file away from the leaves they are
quotients of, so reading `parallel.py` told you what you could set and nothing
about what that decides.
They are declared in `Parallel` now, in the same class as those leaves. They
carry no annotation, so they are not dataclass fields and
`collect_input_fields` never puts them on the record -- the same mechanism that
already keeps `_NS_PATH` off it. That is the right exclusion: a quotient has no
operator input to preserve, and the record is what crosses a process boundary,
where a stamped width is one an elastic scale-up will not refresh.
## A quotient is a value in the bag, like every other derived one
`_derived_width` answered from a stamp or, failing that, a live process group.
The group read could never disagree with the stamp:
- `initialize_model_parallel` stamps all six as its last statement,
unconditionally;
- an elastic scale-up restamps `attn_dp_size` through
`update_dp_attention_post_scale` -- the comment claiming it does *not* was
wrong;
- no hardware backend builds groups of its own;
- `multimodal_gen`, which has its own `initialize_model_parallel` and does not
stamp, never reads a quotient.
So a built group was always already stamped, and the group read goes -- and with
it the last reason for a quotient to be resolved on every read.
Every input to `derive_parallel_widths` is a record field. `dcp_enabled` is
`decode_context_parallel_size > 1`, not a fact about a built group; it was
spelled `_DCP is not None`, which is a longer way to say the same thing. So the
six are fixed once the configuration is fixed -- the same test every other
`Derived(fn=...)` in this PR passes. They are declared the same way and computed
the same way: once, at publish, into ordinary bag leaves.
What remains is override -> stamp -> published leaf. The stamp stays above the
leaf because an elastic scale-up restamps `attn_dp_size`; the override stays on
top because that is how a test names a width.
## One answer for the config-derived predicates
`enable_mamba_extra_buffer` and its lazy variant, `is_ep_joiner`,
`is_ep_scale_joiner`, `is_startup_weight_load_overlap`: each existed as a
`ServerArgs` member for the resolution pipeline and, for most of them, again as
a `runtime_context` function for readers after publish. Three places to keep
saying the same thing.
A `Derived(fn=...)` is a pure function of the published configuration, so
`publish` computes it once and stores it as an ordinary bag leaf -- a plain
attribute load, which is what a read inside compiled model code needs. The
function is handed the whole resolved config rather than the bag it lands in,
because a derivation is free to span namespaces and the mamba one does: it
reads `memory.disable_radix_cache` alongside its own `exec.mamba` strategy,
which is why it could never have been a method on either bag.
The pre-publish helpers stay -- resolution needs the predicate before there is
a bag to read -- and three readers keep them, because they run before their own
process publishes: `initialize_dp_attention`, which the weight-cache daemon
calls while building its groups thirty lines before its `publish`, and
`PortArgs.init_new`, a factory handed the record that already reads eighteen
other fields off it.
## Notes for a reviewer
**Overriding a leaf does not move its quotient.** `override(tp_size=2)` leaves
`attn_tp_size` where the published config put it, because nothing is recomputed
on read. A test states a topology by publishing a config -- which is what a
real process does -- or by naming the width it wants, `override(attn_tp_size=2)`.
Six tests say it that way now. This is the price of having one answer computed
once, and it is the same price every other derived value in the config already
carries.
A caller that reads a quotient without publishing or overriding now gets an
explicit error naming the field, instead of a default that an uninitialised
group happened to supply. One fixture was in that state --
`TestMlaWriteDoorsUnderDcp` built a bare pool and asked whether DCP was on --
and it publishes a config now, which is what the process it stands in for
does.
Eighteen sites read these predicates without calling them. That is correct --
they are properties -- but it is worth saying they were checked, because a
census that assumes otherwise reports eighteen always-true conditions.
## The skill that documents this subsystem is updated with it
`.claude/rules/modify-component-must-read.md` points at
`.claude/skills/sglang-runtime-context/SKILL.md` before anyone touches these
files, so a stale sentence there is a wrong instruction rather than a stale
note. Four of its load-bearing statements stopped being true across this series
and are corrected here: `NS(...)` is no longer how a field states its namespace
(the declaring class is); the DCP degrade rule is gone, because the quotients
are not live reads; `mamba_extra_buffer_enabled()` and the other predicate
functions it named as the shape to copy no longer exist; and the
namespace-coverage ratchet is described in terms of the marker. The docstring of
`test_server_args_namespaces.py` said the same thing and is fixed too.
The consequence a test author actually trips over is stated there as well:
overriding a leaf no longer moves its quotient, so a topology is stated by
publishing a config or by naming the width.
## Verification
A full registered-unit sweep (648 files) against this stack's merge-base:
19 failures on both sides, the same 19 -- AMD `gfx950`, `modelopt`,
`cuda_vmm`, `weight_checker` and friends, none of them config. The narrower 139-file config sweep used earlier in this series
does not contain the files this change reaches -- `test_kv_index_translator`
never names `get_parallel()`, it constructs an object that does -- which is why
the baseline differential over everything is what is quoted here.
`build_draft_tp_worker` built a `ServerArgs` variant whose only job was to make
four config reads answer with the draft's backend instead of the target's, and
published it for the duration of the build so the bags agreed. The backend is a
per-runner fact — target and draft coexist in one process — so it moves onto the
runner, and the variant and the construction-time publish both go away.
`ModelRunner` takes `draft_attention_backend` and resolves the runner's effective
value once (`resolve_draft_attention_backend`: the algorithm's resolved backend,
else `--speculative-draft-attention-backend`, else None for a target runner);
`TpModelWorker` threads it to both runner constructions.
`resolve_attention_backend_strs` reads it off the runner, and `ModelRunner`
stamps the resolved pair *before* building backends so a backend can read it
while it constructs — which is what the FlashInfer KV-access check needs now that
it no longer asks the config. `configure_kv_cache_dtype` and the draft backend
factory read the runner too.
One latent bug falls out: the non-hybrid branch of the backend build ignored the
resolved pair and re-read `server_args.attention_backend`, which is why the
variant had to set that field as well as the split pair. It now uses the value
that was resolved for the runner.
`draft_server_args_overrides` and the `preserve_config()` publish switch are
deleted; with them goes the last production `ServerArgs.derive` outside
pre-publish config building, and the last construction-time publish. The
chunked-prefix gate the target resolved simply stays in the bags, since nothing
re-projects them.
The v2 spec workers got a published `ServerArgs` copy carrying two values: the
target's context length and `--speculative-draft-load-format`. Neither is a
process-wide config change — each is consumed by exactly one constructor — so
the copy, the publish switch around the draft build, and the replay of the
target's resolved overrides onto it all go away, and the values travel to the
runner that owns them:
- **Context length.** `TpModelWorker` already takes it (`context_length=None`
keeps `server_args.context_length`); the four v2 draft workers and
`build_draft_tp_worker` pass the target's, which every one of them has in
scope as `target_worker` / `target_model_config`.
- **Load format.** `ModelRunner._draft_load_format()` resolves it for a draft
runner and `build_load_config` takes it, so the `LoadConfig` is per-runner.
Model code also reads it off the bag while it builds — Inkling replaces
per-element noise in its shared-expert scales under dummy loading — so the
load is wrapped in a scoped bag override that puts the target's value back.
- `skip_tokenizer_init` was on the copy for nobody: `TpModelWorker` already
short-circuits the tokenizer for a draft worker (`or self.is_draft_worker`).
`PrefillCudaGraphRunner._max_addressable_prefix_len` capped the prefix by
`server_args.context_length`, which the copy used to carry for the draft; it now
reads the runner's own `model_config.context_len`. That is also more accurate for
the target, whose `--context-length` may be unset while the resolved context is
shorter than the token table.
What stays a variant is the dflash/dspark path's attention backend: backend
selection reads it off the config object the draft runner holds, and the
resolved gate has to survive the variant's publish. `draft_server_args_overrides`
now carries only those fields and says why.
EAGLEWorkerV2, StandaloneWorkerV2, MultiLayerEagleWorkerV2 and
FrozenKVMTPWorkerV2 wrote the draft's context_length onto the ServerArgs
instance they share with the target worker, and the scheduler wrote the draft's
load_format onto that same object just before creating them. The target's config
carried draft values from then on, and anything constructed later in the process
inherited them.
Scheduler.maybe_init_draft_worker now makes one draft copy through
draft_server_args_copy() and hands it to both the worker factory and the worker,
so every algorithm gets it — the four built-ins, dflash/dspark (which deepcopy
it again inside build_draft_tp_worker), and anything registered through
SpeculativeAlgorithm.register. The copy starts from the config the process
resolved, not from the pristine seed, so load-time overrides made before this
point (the chunked-prefix gate, the SM100 GDN prefill default) are part of what
the draft sees; context_length and load_format are applied on top.
The construction runs under a preserved publish of that copy, the shape
build_draft_tp_worker already used. Weight loading reads the bags rather than the
instance it was handed — Inkling's ModelOpt scale normalization keys on
load_format — so the draft has to be built with its own config published, and
the target's is back in the slot when construction returns.
The EAGLE hot-token-map write is deleted, not moved. init_token_map runs from
alloc_memory_pool, long after the draft's TpModelWorker built its ModelConfig,
and hot_vocab_size is only ever read off model_config.hf_config, which
json_model_override_args reaches at ModelConfig construction. The write could not
affect the draft model; only the shared instance saw it. hot_token_id is
unchanged, so a draft checkpoint that declares hot_vocab_size behaves as before.
Tests: draft_server_args_copy carries the target context_length, a configured
draft load_format and any load-time override while leaving the target's instance
alone; and the scheduler handoff pins that the factory and the worker both
receive the copy, that the copy is the published config during construction, and
that the target's is restored afterwards.
Writer ratchet 31 -> 26.
The module-skipped tests injected config by faking get_server_args (a
SimpleNamespace stand-in patched onto the module) or by writing fields
onto a ServerArgs instance post-publish — both invisible to the namespace
accessors the production code now reads. Re-enable them by publishing the
config they need (get_context().override_server_args seeding, scoped per
test), asserting bag state where the old assertions checked instance
write-through (declare_load_time_override is bag-only), and extending the
per-runner stubs the code genuinely reads (kv_cache_dtype_str,
max_total_tokens, context_len).
The unified-radix-cache file (which grew a large hicache/insert-walk suite
while skipped) is recovered in the same change:
- test_cache_finished_req_strips_thinking (19 parametrized classes) wrote
strip_thinking_cache onto the ServerArgs instance; the cache reads
get_serving().strip_thinking_cache — use the serving bag's scoped
override.
- test_shallower_crossing_backs_up_above_backuped_middle staged its
broken-backup-continuity setup through insert_host, which now
deliberately drops refills below an un-backed-up node under
write-through (host_insert_dropped). Build the same tree state through
an explicit backup + device eviction.
Every config-namespace-migration deferral is recovered, so the deferral
ratchet (test_migration_deferral_ratchet.py) has done its job and is
retired.
