config: the last runner-side instance reads read the bags

Six reads were left on `self.server_args` outside the per-instance boundary the
plan reserves for the tokenizer-manager family, and each had a different reason
to be there:

- `scheduler.process_input_requests` (`mm_feature_transport`) and
  `BaseSpecWorker._build_hicache_draft_plan` (`enable_hierarchical_cache`) are
  plain leaves -> `get_mm()` / `get_memory()`.
- `DraftBackendFactory._create_backend` read the split backend through a
  *runtime-computed name* (`getattr(self.server_args, backend_name)`) and then
  fell back to the base field by hand -- the census's documented blind spot.
  The two names it can be handed are exactly the pair `attention_backends()`
  returns with that fallback already applied, so it reads the pair and indexes
  it. The draft runner's own stamp still wins when it has one.
- `remote_instance_weight_loader_use_transfer_engine` and
  `pre_capture_activation_reserve_mb` are derived members. Both are computed
  from published leaves only, so both get a named accessor that derives from
  the bags (and therefore follows a post-publish override).

The first of those two has all its inputs in one bag, so it follows the
established shape: one `*_of(cfg)` helper in `arg_groups/overrides.py`, the
`ServerArgs` member delegating to it, and the accessor calling it on
`get_model()`. `modelexpress_transport_of` splits out the JSON parse both
sides need. The second spans four bags plus the configured parallel sizes, so
it exists twice like the mamba pair -- and `TestDerivedPredicatesAgreeAcrossTiers`
now pins both new pairs equal over their input matrices (92 subtests).

`self.server_args.X` outside the tokenizer-manager family: 11 -> 5, and the
five that remain are the documented ones (the encode server's own record, the
nixl connector's rank arithmetic, `GrammarManager`'s handed instance).

The post-capture headroom path calls the same bag-backed
`pre_capture_activation_reserve_mb` accessor the configurator uses -- the
accessor advertises override-following, and a reserve that reads the record
while its sibling reads the bags can disagree after a post-publish override.
And the conversions' orphans go with them: `RemoteInstanceWeightTransporter`
kept a `server_args` field nothing reads, and `DraftBackendFactory` parked a
record it no longer consults -- both drop the parameter, and the four factory
call sites stop threading one.
This commit is contained in:
Cheng Wan
2026-08-15 00:37:06 -07:00
committed by GitHub
parent ab810e4052
commit 97279980cf
16 changed files with 201 additions and 43 deletions
+47
View File
@@ -1420,6 +1420,53 @@ def mamba_extra_buffer_lazy_enabled() -> bool:
)
def remote_instance_transfer_engine_enabled(load_format: str | None = None) -> bool:
"""Whether remote-instance weight loading runs over the transfer engine.
Every input is a ``model`` leaf, so this derives from the bags and follows a
post-publish override; ``ServerArgs.remote_instance_weight_loader_use_transfer_engine``
is the pre-publish equivalent, and both go through the same helper.
``load_format`` is the caller's own (a draft runner loading under
``--speculative-draft-load-format`` has one the process record does not).
"""
from sglang.srt.arg_groups.overrides import remote_instance_transfer_engine_of
return remote_instance_transfer_engine_of(get_model(), load_format)
def pre_capture_activation_reserve_mb(gpu_mem: float | None) -> float:
"""The activation working-set reserve held back before cuda-graph capture.
Derived from published leaves across four bags (``disagg`` / ``schedule`` /
``exec.graph`` / ``spec``) plus the configured parallel sizes, so it follows
a post-publish override; ``ServerArgs.pre_capture_activation_reserve_mb`` is
the pre-publish equivalent and
``TestDerivedPredicatesAgreeAcrossTiers`` pins the two equal.
"""
schedule = get_schedule()
if get_disagg().disaggregation_mode == "decode":
running_requests = (
schedule.max_running_requests
or get_exec().graph.cuda_graph_config.decode.max_bs
or 1
)
activation_tokens = max(
running_requests * (get_spec().speculative_num_draft_tokens or 1), 2048
)
elif schedule.chunked_prefill_size > 0:
activation_tokens = max(schedule.chunked_prefill_size, 2048)
else:
activation_tokens = max(schedule.max_prefill_tokens, 2048)
reserved_mem = (
512
+ activation_tokens * 1.5
+ _configured_parallel("tp_size") * _configured_parallel("pp_size") / 8 * 1024
)
if gpu_mem is not None and gpu_mem > 60 * 1024:
reserved_mem = max(reserved_mem, 10 * 1024)
return reserved_mem
# --- Derived config accessors ------------------------------------------------
#
# A few values are computed from several config fields plus the HF config, so