config: the last runner-side instance reads read the bags
Six reads were left on `self.server_args` outside the per-instance boundary the plan reserves for the tokenizer-manager family, and each had a different reason to be there: - `scheduler.process_input_requests` (`mm_feature_transport`) and `BaseSpecWorker._build_hicache_draft_plan` (`enable_hierarchical_cache`) are plain leaves -> `get_mm()` / `get_memory()`. - `DraftBackendFactory._create_backend` read the split backend through a *runtime-computed name* (`getattr(self.server_args, backend_name)`) and then fell back to the base field by hand -- the census's documented blind spot. The two names it can be handed are exactly the pair `attention_backends()` returns with that fallback already applied, so it reads the pair and indexes it. The draft runner's own stamp still wins when it has one. - `remote_instance_weight_loader_use_transfer_engine` and `pre_capture_activation_reserve_mb` are derived members. Both are computed from published leaves only, so both get a named accessor that derives from the bags (and therefore follows a post-publish override). The first of those two has all its inputs in one bag, so it follows the established shape: one `*_of(cfg)` helper in `arg_groups/overrides.py`, the `ServerArgs` member delegating to it, and the accessor calling it on `get_model()`. `modelexpress_transport_of` splits out the JSON parse both sides need. The second spans four bags plus the configured parallel sizes, so it exists twice like the mamba pair -- and `TestDerivedPredicatesAgreeAcrossTiers` now pins both new pairs equal over their input matrices (92 subtests). `self.server_args.X` outside the tokenizer-manager family: 11 -> 5, and the five that remain are the documented ones (the encode server's own record, the nixl connector's rank arithmetic, `GrammarManager`'s handed instance). The post-capture headroom path calls the same bag-backed `pre_capture_activation_reserve_mb` accessor the configurator uses -- the accessor advertises override-following, and a reserve that reads the record while its sibling reads the bags can disagree after a post-publish override. And the conversions' orphans go with them: `RemoteInstanceWeightTransporter` kept a `server_args` field nothing reads, and `DraftBackendFactory` parked a record it no longer consults -- both drop the parameter, and the four factory call sites stop threading one.
This commit is contained in:
@@ -44,6 +44,7 @@ from sglang.srt.arg_groups.overrides import (
|
||||
attention_backends_of,
|
||||
mamba_extra_buffer_lazy_of,
|
||||
mamba_extra_buffer_of,
|
||||
remote_instance_transfer_engine_of,
|
||||
resolved_view,
|
||||
)
|
||||
from sglang.srt.configs.embedding_model_spec import BCGPrefillPolicy
|
||||
@@ -9470,20 +9471,7 @@ class ServerArgs:
|
||||
def remote_instance_weight_loader_use_transfer_engine(self, load_format=None):
|
||||
"""``load_format`` overrides the seed's: a draft runner loading under
|
||||
``--speculative-draft-load-format`` needs its own transfer engine."""
|
||||
# Use TransferEngine as seed backend.
|
||||
if self.remote_instance_weight_loader_start_seed_via_transfer_engine:
|
||||
return True
|
||||
# Use TransferEngine as client backend.
|
||||
if (load_format or self.load_format) == "remote_instance" and (
|
||||
self.remote_instance_weight_loader_backend == "transfer_engine"
|
||||
or (
|
||||
self.remote_instance_weight_loader_backend == "modelexpress"
|
||||
and self.modelexpress_transport == "transfer_engine"
|
||||
)
|
||||
):
|
||||
return True
|
||||
else:
|
||||
return False
|
||||
return remote_instance_transfer_engine_of(self, load_format)
|
||||
|
||||
def describe_kv_events_publisher(self) -> Optional[dict]:
|
||||
"""Return a structured description of this server's KV-event
|
||||
|
||||
Reference in New Issue
Block a user