`get_server_args().<field>` reads one process's startup record. Nine sites still
did that for a value that has a namespace: the attention backend (5),
`skip_tokenizer_init` (2), the draft-aware `load_format`, and a chunked-prefill
size in `sglang.kernels`. They now read `get_exec().kernel` / `get_serving()` /
`get_model()` / `get_schedule()`, so they see the resolved value including
post-publish overrides.
The multimodal processor's device selection moves to the instance it was
constructed with rather than to a namespace: `base_gpu_id` differs per worker (the
encode-server DP workers each specialise their own copy), so no process-global
value can stand in for it, and engines sharing a tokenizer process each need their
own. Branch order, the NPU preprocess patches, and the case that leaves "device"
unset are unchanged.
What stays on `get_server_args()` is the derived API — `@property` and method
members computed from several fields plus the HF config
(`mamba_cache_chunk_size`, `get_model_config()`, `enable_mamba_extra_buffer*`) —
plus three config-intent reads of live-shadowed sizes, each of which needs an
answer the live topology property cannot give (the DSA indexer's PP gate must
short-circuit before touching the PP group, `allocation`'s DCP gate asks whether
DCP was configured at all, and the CUDA-IPC recycler runs where no group exists).
A new AST ratchet pins both shapes it can see — the direct call and an alias
bound from it in the same function — at 0 and 12 respectively, exempting the
derived APIs and those three sites by name. The alias-form baseline is not zero:
those reads are mostly per-runner fields in model code, and lowering them is the
next slice.
Two fixtures stopped faking config: `test_dllm_fdfo_kv_reuse` rebound
`allocation.get_server_args` to a SimpleNamespace, which silently stops
intercepting the moment a reader migrates; it publishes a real config instead.
When --forward-pass-metrics-ipc-name is left unset the reporter generates an
endpoint and has to hand it to external consumers (the documented contract is
that they read it back from the server config). That readback is the scheduler's
get_internal_state, which already reports get_context().resolved_server_args_dict(),
so the write moves to get_context().override and the read alongside it to
get_observability() — the endpoint still shows up in /server_info's
internal_states, and the ServerArgs instance stops being a message bus.
The test's server_args stand-in (a SimpleNamespace with a hand-rolled override)
becomes a real published config, so the reporter exercises the same accessors as
production.
Writer ratchet 19 -> 18.
The scheduler's runtime HiCache attach/detach wrote its own ServerArgs so the
internal-state readback would show the change; that readback already reports
the resolved config, so the writes become get_context().override(...) and the
namespace readers see them too.
The tokenizer side is per-engine — several Engines can share one process — so
its control-plane updates (weight version, model path + load format, HiCache
attach/detach) stay with the manager instead of moving to the process-global
bags. TokenizerManager gains record_config_updates / config_value /
resolved_config_dict, and the readbacks that used to observe the instance write
(/server_info, /model_info, the HiCache status endpoint, the gRPC bridge) now
overlay those updates onto the startup config.
test_server_info's stub grew the real manager instead of a SimpleNamespace, so
the overlay it now exercises cannot drift from production.
Writer ratchet 26 -> 19.
EAGLEWorkerV2, StandaloneWorkerV2, MultiLayerEagleWorkerV2 and
FrozenKVMTPWorkerV2 wrote the draft's context_length onto the ServerArgs
instance they share with the target worker, and the scheduler wrote the draft's
load_format onto that same object just before creating them. The target's config
carried draft values from then on, and anything constructed later in the process
inherited them.
Scheduler.maybe_init_draft_worker now makes one draft copy through
draft_server_args_copy() and hands it to both the worker factory and the worker,
so every algorithm gets it — the four built-ins, dflash/dspark (which deepcopy
it again inside build_draft_tp_worker), and anything registered through
SpeculativeAlgorithm.register. The copy starts from the config the process
resolved, not from the pristine seed, so load-time overrides made before this
point (the chunked-prefix gate, the SM100 GDN prefill default) are part of what
the draft sees; context_length and load_format are applied on top.
The construction runs under a preserved publish of that copy, the shape
build_draft_tp_worker already used. Weight loading reads the bags rather than the
instance it was handed — Inkling's ModelOpt scale normalization keys on
load_format — so the draft has to be built with its own config published, and
the target's is back in the slot when construction returns.
The EAGLE hot-token-map write is deleted, not moved. init_token_map runs from
alloc_memory_pool, long after the draft's TpModelWorker built its ModelConfig,
and hot_vocab_size is only ever read off model_config.hf_config, which
json_model_override_args reaches at ModelConfig construction. The write could not
affect the draft model; only the shared instance saw it. hot_token_id is
unchanged, so a draft checkpoint that declares hot_vocab_size behaves as before.
Tests: draft_server_args_copy carries the target context_length, a configured
draft load_format and any load-time override while leaving the target's instance
alone; and the scheduler handoff pins that the factory and the worker both
receive the copy, that the copy is the published config during construction, and
that the target's is restored afterwards.
Writer ratchet 31 -> 26.
Each of these wrote a value after resolution so a later reader would find it on
the instance. None of them needed the instance: one write was redundant, and the
two that carry a value the resolved-config readback reports move to
get_context().override, which the readback overlays.
- The SM100 GDN prefill default was written onto ServerArgs and read back one
line later by initialize_linear_attn_config. It is now the return value of
flashinfer_gdn_prefill_default, threaded into initialize_linear_attn_config
(an explicit --linear-attn-prefill-backend still wins) and recorded with
get_context().override so /server_info reports the backend in effect.
- The XGrammar fallback recorded grammar_backend="none" on the instance. No code
reads the field after the factory reads it once, but get_internal_state
reports the whole resolved config, so the fallback now lands there instead:
the readback tells the truth and the seed keeps the requested backend.
- UnifiedRadixCache.init_hicache re-applied the direct-IO layout fixup that
__post_init__ already applies: init_hicache only runs when hierarchical cache
is on, which is exactly when _handle_hicache normalizes page_first to
page_first_direct (pinned by test_hicache_io_backend_and_mem_layout_
compatibility::direct_with_page_first). Three fixtures reached the fixup by
building ServerArgs(model_path="dummy"), whose resolution is skipped, so they
now declare the layout resolution would have produced.
Writer ratchet 34 -> 31.
The skill still described the retired resolve-at-end contract (a single
resolved ServerArgs as the source of truth, ServerArgs.override as the
mutation entry). Rewrite the config sections for the landed model:
- ServerArgs is a pristine published seed; resolved config lives in the
namespace bags (get_exec()/get_memory()/.../get_device()), projected at
publish(server_args, role=...) per process entry.
- Post-publish mutation goes through get_context().override (bag-only);
ServerArgs.override is being retired behind the writer ratchet, and
rerouting a writer co-flips all its readers in the same commit.
- Documented the reads that stay on an instance (per-runner fork fields,
per-instance tokenizer/entrypoints boundaries, whole-object passes),
get_parallel()'s config/live dual semantics with the live-shadowed
sizes rule, preserve_config for nested publishes, and the bag-only
declare_load_time_override route.
- Testing idioms: publish-seeding (override_server_args + scoped bag
overrides) instead of faking accessors or SimpleNamespace stand-ins;
per-file test runs to avoid leaked-publish masking.
- Guardrails: added the writer ratchet, namespace-coverage lint, and the
migration-deferral ratchet.
publish(role=...) has recorded provenance since the namespace split; this
wires the enforcement the role was reserved for. SGLANG_ROLE_NAMESPACES
selects the mode:
- off (default): no bookkeeping; the mode gate in config_bag stays a
dead-branch-prunable check under dynamo (bag reads run inside compiled
forwards — pinned by a fullgraph test).
- record: audit mode — collect (role, namespace) pairs per process and
persist each new pair immediately to SGLANG_ROLE_NAMESPACES_OUT (worker
teardown skips atexit), plus a per-process stderr summary at exit.
- enforce: a bag read outside the role's ROLE_NAMESPACE_SETS entry fails
closed with an actionable error; None entries mean full tree.
Sets are filled only where audits back them: dp_controller reads only
exec (record-mode plain + DP-attention smokes agree with the module's
static read set — the elastic-EP gate). tokenizer observed zero bag
reads (per-instance managers read self.server_args by design) but keeps
the full tree until the multi-tokenizer disagg shape (TokenizerWorker's
get_disagg read) is audited; encoder / expert_backup /
weight_cache_daemon likewise await their deployment shapes.
Verified end-to-end: DP-attention smoke under enforce boots and serves
with zero violations.
The module-skipped tests injected config by faking get_server_args (a
SimpleNamespace stand-in patched onto the module) or by writing fields
onto a ServerArgs instance post-publish — both invisible to the namespace
accessors the production code now reads. Re-enable them by publishing the
config they need (get_context().override_server_args seeding, scoped per
test), asserting bag state where the old assertions checked instance
write-through (declare_load_time_override is bag-only), and extending the
per-runner stubs the code genuinely reads (kv_cache_dtype_str,
max_total_tokens, context_len).
The unified-radix-cache file (which grew a large hicache/insert-walk suite
while skipped) is recovered in the same change:
- test_cache_finished_req_strips_thinking (19 parametrized classes) wrote
strip_thinking_cache onto the ServerArgs instance; the cache reads
get_serving().strip_thinking_cache — use the serving bag's scoped
override.
- test_shallower_crossing_backs_up_above_backuped_middle staged its
broken-backup-continuity setup through insert_host, which now
deliberately drops refills below an un-backed-up node under
write-through (host_insert_dropped). Build the same tree state through
an explicit backup + device eviction.
Every config-namespace-migration deferral is recovered, so the deferral
ratchet (test_migration_deferral_ratchet.py) has done its job and is
retired.
The parallel namespace joins the accessor migration: 106 config-leaf reads
(enable_dp_lm_head, enable_dp_attention, pp_async_batch_depth, dp_size,
ep_join_rank_offset, dwdp_size, ...) flip from get_server_args()/
self.server_args to get_parallel(), which serves config leaves from the
published parallel bag via __getattr__.
- ParallelContext.__getattr__ is restructured to stay dynamo-traceable
(object.__getattribute__ graph-breaks): gate helpers such as
enable_moe_dense_fully_dp() run inside compiled model forwards. A
fullgraph regression test pins the pattern.
- The five live-shadowed topology sizes (tp/pp/dcp/attn_cp/moe_dp_size)
keep their server_args reads: the live @property wins on the accessor,
and conditionally-initialized groups would fail loud at unconditional
call sites.
- Elastic-EP scale writers (ep_size/dp_size x4 in model_runner) reroute
to get_context().override together with their remaining instance
readers (expert_location gpus-per-node paths); the ServerArgs.override
ratchet drops 39 -> 35.
- The expert placement helpers (compute_logical_to_rank_dispatch_
physical_map, _compute_logical_to_all_physical_map,
_prefer_same_node_experts) now read everything from the bags and drop
their server_args parameter; their unit tests publish the config they
need instead of stubbing it.
The load-time gate (maybe_disable_chunked_prefix_cache) wrote its
ServerArgs instance while every reader has moved to the published
config: the attention backends assert / branch on
get_schedule().disable_chunked_prefix_cache when they initialize, so the
flip never reached them and a backend outside
CHUNKED_PREFIX_CACHE_SUPPORTED_ATTENTION_BACKENDS kept chunked prefix
enabled.
Reroute the writer through get_context().override (which writes the
published bags) and flip the two remaining instance reads — the gate's
own log check and the prefill cuda-graph runner's capture flag — to the
bag. A regression test pins the three contracts: the gate lands on the
bag, the pristine ServerArgs instance stays untouched, and the
draft-worker guard never writes.
The ServerArgs.override call-site ratchet drops 39 -> 38.
