config: the alias form of the runner-side instance read
The previous batch counted `self.server_args.X` and called the runner surface done. It was not: the same read spelled through a local alias -- `server_args = model_runner.server_args` (or `sa = kvc.server_args`, `args = ...`) followed by `server_args.leaf` -- is the same process-global read wearing a local name, and the AST census counts **57 of them** across eleven files that the grep never saw. Census per function, following the alias. 52 were leaves and go to their bag (`spec` 11, `schedule` 9, `memory` 7, `exec.graph` 5, `exec.moe` 5, `parallel` 4, `disagg` 4, `model` 3, `exec.mamba` 2, `exec.overlap` 2). Five were not leaves: three derived members on the eager runner -- `max_speculative_num_draft_tokens` and `enable_mamba_extra_buffer` already had accessors, and `max_prefill_buffer_tokens` gets one (all its inputs are `schedule` leaves plus the configured PP size, so it derives from the bags and follows a post-publish override; `TestDerivedPredicatesAgreeAcrossTiers` pins it against the member over a 48-case matrix) -- plus `get_attention_backends()`, which the same commit routes through `attention_backends()`, and a dict that merely shares the name (`server_args_dict.items`). That dict is the one read left behind. `build_attention_backends` also stops resolving the pair from the record: it runs after publish, so it asks `attention_backends()` like every other consumer. The draft override on the runner still wins first. `dispatch_event_loop`'s three PP checks read the *configured* PP size, not the live topology: the MLX runner stub never initializes torch.distributed, so the live property asserts before the MLX event loop can start (a Codex catch). The configured leaf answers the same value wherever the live groups exist. `flashinfer_gdn_prefill_default`'s guard is the one read here that asks what the *operator* named rather than what the config resolved to, and the bag leaf now answers exactly that: the per-runner auto-default is stamped on the runner and deliberately never recorded process-wide, so nothing writes that leaf after launch and reading it back cannot mistake another runner's default for a flag. Three test doubles injected a `SimpleNamespace`/`MagicMock` record for exactly these reads and now publish instead (pool configurator, cache registry, GDN prefill policy) -- the fixture publishes what the case configures and hands the published instance to the whole-object contracts that still take one. The functions this sweep partially converted stop mixing sources (review catches): the flash-attention constructor's remaining seed reads (`speculative_eagle_topk`, `speculative_algorithm`, both deterministic gates) read their bags next to the leaves already converted; `_should_disable_scheduler_metadata_precompute` reads the parallel config leaves itself instead of taking the record (its alias binding was the last use); and the autotune gates (`disable_flashinfer_autotune`, deterministic, `flashinfer_autotune_skip_ops`) join the moe leaves the same function already reads from the bags. The pool-configurator fixture drops a parameter nothing published or read.
This commit is contained in:
@@ -401,6 +401,7 @@ class _FakeResolvedArgs:
|
||||
max_running_requests: A[int | None, Arg(help="mrr"), NS("schedule")] = None
|
||||
chunked_prefill_size: A[int, Arg(help="cps"), NS("schedule")] = -1
|
||||
max_prefill_tokens: A[int, Arg(help="mpt"), NS("schedule")] = 16384
|
||||
enable_dynamic_chunking: A[bool, Arg(help="edc"), NS("schedule")] = False
|
||||
cuda_graph_config: A[object | None, Arg(help="cgc"), NS("exec.graph")] = None
|
||||
tp_size: A[int, Arg(help="tp"), NS("parallel")] = 1
|
||||
pp_size: A[int, Arg(help="pp"), NS("parallel")] = 1
|
||||
@@ -1026,6 +1027,31 @@ class TestDerivedPredicatesAgreeAcrossTiers(_IsolatedServerArgs):
|
||||
mamba_extra_buffer_lazy_enabled(),
|
||||
)
|
||||
|
||||
def test_prefill_buffer_ceiling_matches_the_member(self):
|
||||
from sglang.srt.runtime_context import max_prefill_buffer_tokens
|
||||
|
||||
for chunked in (-1, 0, 1024, 8192):
|
||||
for dynamic in (False, True):
|
||||
for pp in (1, 4):
|
||||
for max_prefill in (0, 2048, 16384):
|
||||
with self.subTest(
|
||||
chunked=chunked,
|
||||
dynamic=dynamic,
|
||||
pp=pp,
|
||||
max_prefill=max_prefill,
|
||||
):
|
||||
args = _FakeResolvedArgs(
|
||||
chunked_prefill_size=chunked,
|
||||
enable_dynamic_chunking=dynamic,
|
||||
pp_size=pp,
|
||||
max_prefill_tokens=max_prefill,
|
||||
)
|
||||
get_context().set_server_args(args)
|
||||
self.assertEqual(
|
||||
ServerArgs.max_prefill_buffer_tokens(args),
|
||||
max_prefill_buffer_tokens(),
|
||||
)
|
||||
|
||||
def test_activation_reserve_matches_the_member(self):
|
||||
from types import SimpleNamespace
|
||||
|
||||
|
||||
Reference in New Issue
Block a user