From d804b6bd9851ef94d1e7a5cf7d66dac050f68c1d Mon Sep 17 00:00:00 2001 From: Cheng Wan <54331508+ch-wan@users.noreply.github.com> Date: Sat, 15 Aug 2026 00:40:07 -0700 Subject: [PATCH] config: pin the step-12 debt on the supplied-instance surface A callee that takes `server_args` keeps the supplied-instance contract, so no ratchet counts its reads -- and that is right for the *object*. What it does not cover is what the object will carry after step 12: the instance stays at the user's raw input, so a callee reading a field resolution fills in starts seeing the CLI default instead of the effective value. Measured, not guessed: **297 distinct file/field pairs read one of the 125 fields resolution can write** -- what remains after the earlier members' conversions (the census found 314 across the package; the page_size, chunked_prefill_size and graph/limit families were converted by the members below this one, so the list lands at the remaining debt with no churn). The census counts three spellings of the read: `server_args.field` off the parameter, `getattr(server_args, "field", default)` with a literal name, and the *parked* form -- `self.x = server_args` in a method that takes the parameter, read as `self.x.field` anywhere in the class. Parking under a different object, a container, or a computed name stays invisible, like in every census of this family. The written-field set is derived in-test from resolved configs against the dataclass defaults -- the same matrix the context repo's audit tool uses -- and the union is only as complete as the matrix: fields a matrix entry passes in are excluded, so each entry must let resolution make the decision the entry is about. The DWDP shape is the loudest example: `_handle_dwdp` writes `dp_size`, `enable_dp_attention`, `ep_size` and friends itself, and without a `{tp_size: 2, dwdp_size: 2}` entry the whole DP/EP topology family (37 pairs across the launcher, the controllers, the tokenizer and the spec workers) never entered the written set at all. Multi-item scoring is the same shape in miniature -- `_handle_multi_item_scoring` writes `disable_radix_cache` itself, and the `{enable_mis, attention_backend=flashinfer}` entry (the backend is passed because the handler asserts rather than switches) pins the radix-cache builder and its friends. The written set carries **may-write semantics**, statically collected from every mechanism that can put a resolved value on the record, unioned with the matrix (construct-and-diff still catches value-level writes statics cannot name). The enumeration approach kept losing to review -- the DFLASH hook hole, then the declarative registry (`MODEL_OVERRIDES` forces dtype for two arches through a setattr applier no assignment scan sees), then the deprecated-alias loop that writes through a name tuple -- because each round found one more *mechanism*, not one more field. So all of them are collected now: hook assignments under `arg_groups/`, the record's own method assignments (the mooncake layout rewrite, the deepseek-EP mode defaults, the seed fill that only fires when the caller did NOT supply one -- which construct-and-diff can never see, since measuring requires supplying), the declarative override registry, the alias-normalization tuple (drift-guarded), and the late-resolution keywords. A statically-collected write site that can never fire is a dead branch to delete upstream, not a reason to shrink the census (maintainer's ruling). And the pin is split by host: `_EXPOSED` is asserted everywhere, `_EXPOSED_CUDA_ONLY` (empty today) is where a capability-gated write's readers go -- one shared exact list cannot hold such a pair at all, since pinning it fails CPU as "gone" and omitting it fails CUDA as "new". The resolve-once shape also undercounts on axes a construction call never sees (each of these was a review catch, and each added pinned families): resolution branches on the *environment*, so every matrix entry resolves under the plain env and under the CI shape (`SGLANG_IS_IN_CI`), with the pristine process state (environ plus the `EnvField` descriptor flags) restored between entries -- that is where `soft_watchdog_timeout`'s four readers come from. Some fields resolve *late*, at validation rather than construction (`declare_late_resolution`): those writers are collected statically by keyword -- `lora_paths`, `reasoning_parser`, `tool_call_parser` -- with the one dynamic `**detected` site spelled out in a table guarded against drift. Fields holding only a `default_factory` are materialized rather than skipped, and `tokenizer_path` / `served_model_name` -- always filled from `model_path` -- leave the passed-inputs exemption and pin their twelve readers. A module the census cannot parse fails the test instead of shrinking it, which immediately caught a BOM-carrying file every previous census had silently skipped (the scans read `utf-8-sig` now). And the test registers on the CUDA runner besides the CPU suite, because capability-gated writes only open on real hardware; AMD is intentionally not registered -- an exact pin cannot be verified from any pinning host -- with the reasoning in the header. **A second axis is already wrong today**, independent of step 12. Some config is decided after publish and recorded with `get_context().override(...)` -- elastic EP resizing `ep_size`, a weight update rewriting `model_path` / `load_format`, HiCache attach naming a storage backend, adaptive speculative decoding moving `speculative_num_steps`. That write reaches the bags and never the record, so a supplied-instance read of one of those fields answers with the startup value from the moment the override lands. **73 pairs over 13 fields** are in that position -- including the overrides that arrive as `**kwargs`: the collector statically resolves dict-literal expansions (the HiCache attach shape, whose write/read pairs on `hicache_write_policy` / `hicache_storage_prefetch_policy` were invisible before) and fails loudly on anything it cannot resolve, with `update_server_args` exempted by name because its key set is the API's caller's, not this file's. Whether each is a defect depends on ordering -- a value copied at construction, before any override, is fine -- so the axis is pinned as a measurement with the same growth guard, not as a list of bugs. One of them *was* a defect and is fixed at the base of this stack: the linear-attn dispatch table rebuilt itself from the record after the SM100 GDN prefill decision had been recorded in the bag, so a second runner's rebuild dropped it. That choice is a per-runner stamp now and is not recorded process-wide at all, which is why neither the read nor the field appears on this axis. The list is pinned both ways, on both axes. A new pair fails, because the moment to decide where a resolved value comes from is when the read is written, not during the flip; a disappeared pair fails too, naming the entry to delete, so the registry stays a measurement rather than a memory of one. Both axes reverse-verified: a new read of a written field is reported by file and field. Per-field dispositions live in the plan doc; several of these are "should this callee take a config at all?", which is a design call rather than a sweep. test(step-12): tripwire on the EPD guard that a raw record would silence `_reject_missing_dispatched_encoder_embedding` is one of the two reads the step-12 audit calls a blocker: it keys on `encoder_transfer_backend`, a field resolution fills in, off a handed record. Today that record carries the resolved value; after the flip it stays at the argument default `"auto"` (`ENCODER_TRANSFER_BACKEND_CHOICES[0]`) for every auto-resolved launch and the 503 stops firing -- a guard that goes quiet, which no existing case notices. The tripwire resolves a real language-only Kimi-K3 TP2 launch (a mini config, the shape whose auto pick is `"zmq_to_tokenizer"`) and asserts the guard rejects with the record resolution produced. A fixed double cannot trip on the flip -- it would keep handing the guard the resolved value by construction -- so the record has to come from resolution itself: when step 12 lands, this same launch hands the guard `"auto"`, the rejection silently stops, and this test fails, which is exactly the signal that this reader needs the resolved value from somewhere else (the per-engine overlay or the bag). The launch pins `mamba_radix_cache_strategy=no_buffer` (+ the overlap-off it requires): resolution's hybrid state-cache sizing branches on the host device and asserts a GPU stack for extra_buffer, which a CPU CI runner does not have, while the guard under test reads a field independent of that branch. The case restores env *and* the EnvField descriptor flags -- a real resolution leaves state os.environ does not carry. --- .../test_kimi_k3_encoder_mode.py | 104 ++ ...test_supplied_instance_exposure_ratchet.py | 1165 +++++++++++++++++ 2 files changed, 1269 insertions(+) create mode 100644 test/registered/unit/test_supplied_instance_exposure_ratchet.py diff --git a/test/registered/unit/disaggregation/test_kimi_k3_encoder_mode.py b/test/registered/unit/disaggregation/test_kimi_k3_encoder_mode.py index cba64f49b..8c704be57 100644 --- a/test/registered/unit/disaggregation/test_kimi_k3_encoder_mode.py +++ b/test/registered/unit/disaggregation/test_kimi_k3_encoder_mode.py @@ -86,6 +86,110 @@ def test_epd_language_only_rejects_missing_dispatched_embedding(): assert getattr(exc_info.value, "status_code", None) == 503 +def test_epd_rejection_reads_the_resolved_transfer_backend(): + """Tripwire for step 12: this guard fires on the *resolved* backend. + + The record is produced by actual resolution -- a language-only Kimi-K3 + launch at TP2, whose `encoder_transfer_backend` starts at the argument + default `"auto"` (`ENCODER_TRANSFER_BACKEND_CHOICES[0]`) and is filled in + by `resolve_encoder_transfer_backend` to `"zmq_to_tokenizer"`. Today the + guard therefore rejects. When step 12 makes the instance raw, this same + launch hands the guard a record still at `"auto"`, the rejection silently + stops, and *this test fails* -- which is the signal to give this reader + the resolved value (per-engine overlay or bag) rather than the record. + Fixed doubles cannot trip on that change, so the record here must come + from resolution, not a SimpleNamespace. + """ + import json + import os + import shutil + import tempfile + + from sglang.srt.server_args import ServerArgs + + def env_field_flags(): + from sglang.srt.environ import EnvField, envs + + return { + name: field._set_to_none + for klass in reversed(type(envs).__mro__) + for name, field in vars(klass).items() + if isinstance(field, EnvField) + } + + config_dir = tempfile.mkdtemp(prefix="epd_tripwire_") + try: + payload = { + "architectures": ["KimiK3ForConditionalGeneration"], + "model_type": "kimi_k3", + "text_config": { + "architectures": ["DeepseekV3ForCausalLM"], + "model_type": "deepseek_v3", + "hidden_size": 16, + "intermediate_size": 32, + "moe_intermediate_size": 32, + "num_attention_heads": 2, + "num_key_value_heads": 2, + "num_hidden_layers": 2, + "n_routed_experts": 8, + "n_shared_experts": 1, + "num_experts_per_tok": 2, + "first_k_dense_replace": 1, + "vocab_size": 128, + "max_position_embeddings": 2048, + "kv_lora_rank": 8, + "q_lora_rank": 8, + "qk_nope_head_dim": 8, + "qk_rope_head_dim": 8, + "v_head_dim": 8, + "topk_method": "greedy", + "scoring_func": "softmax", + }, + "vision_config": { + "model_type": "kimi_k3_vision", + "hidden_size": 16, + "num_heads": 2, + "depth": 2, + "patch_size": 14, + "merge_kernel_size": [2, 2], + }, + } + with open(os.path.join(config_dir, "config.json"), "w") as handle: + json.dump(payload, handle) + environ_before = dict(os.environ) + flags_before = env_field_flags() + try: + resolved = ServerArgs( + model_path=config_dir, + device="cuda", + random_seed=42, + language_only=True, + tp_size=2, + # Resolution branches on the host device for the hybrid + # state-cache sizing (extra_buffer asserts a GPU stack, which + # the CPU CI runner does not have); the guard under test reads + # `encoder_transfer_backend`, independent of that branch, so + # pin the strategy every host can resolve. + mamba_radix_cache_strategy="no_buffer", + disable_overlap_schedule=True, + ) + finally: + os.environ.clear() + os.environ.update(environ_before) + from sglang.srt.environ import envs + + for name, was_none in flags_before.items(): + getattr(type(envs), name)._set_to_none = was_none + finally: + shutil.rmtree(config_dir, ignore_errors=True) + + assert resolved.encoder_transfer_backend == "zmq_to_tokenizer" + request = SimpleNamespace(need_wait_for_mm_inputs=True) + with pytest.raises(HTTPException) as exc_info: + _reject_missing_dispatched_encoder_embedding(resolved, request, None) + assert getattr(exc_info.value, "status_code", None) == 503 + + def test_epd_allows_local_processing_when_request_was_not_dispatched(): server_args = SimpleNamespace( language_only=True, diff --git a/test/registered/unit/test_supplied_instance_exposure_ratchet.py b/test/registered/unit/test_supplied_instance_exposure_ratchet.py new file mode 100644 index 000000000..fb40b117f --- /dev/null +++ b/test/registered/unit/test_supplied_instance_exposure_ratchet.py @@ -0,0 +1,1165 @@ +"""The supplied-instance surface is measured on two axes, and may only shrink. + +A callee that takes ``server_args`` keeps the supplied-instance contract: the +caller chose the object, so no global-read ratchet counts it. Step 12 changes +what that object *carries* — the instance stays at the user's raw input — so a +callee reading a field **resolution fills in** would start seeing the CLI default +instead of the effective value. + +This pins that intersection. Each entry is one (file, field) pair where a +parameter named ``server_args`` is read for a field resolution writes; the plan +doc carries the proposed disposition per field +(``global_context/12-raw-input-config.md``, "the supplied-instance conversion +list"). New pairs fail: a new one is new step-12 work, and the moment to decide +where the value should come from is when the read is written, not during the +flip. Pairs that disappear also fail, with the entry to delete — the list is the +measurement, not a memory of one. + +The written-field set is derived here rather than hardcoded: the +representative configs in ``_MATRIX`` (one per resolution family it exercises) +are resolved and compared against the dataclass defaults, the same matrix the +context repo's audit tool uses. Ambient environment is normalized per entry -- +resolution branches on CI detection and leaves sticky process state, so each +entry resolves from the pristine snapshot, and the CI shape is an explicit +entry rather than an accident of the runner. The read scan mirrors that +tool's three shapes — a parameter attribute, ``getattr(server_args, "literal")``, +and the parameter parked on ``self`` — because two implementations of one census +that disagree are worse than either alone. + +The second axis is **already wrong today**, not after a flip. Some config is +decided *after* publish and recorded with ``get_context().override(...)`` — +elastic-EP resizing `ep_size`, a weight update rewriting `model_path` / +`load_format`, HiCache attach naming a storage backend, adaptive speculative +decoding moving `speculative_num_steps`. That write reaches the bags and never +the record, so a supplied-instance read of one of those fields answers with the +startup value from the moment the override lands. Whether that is a defect +depends on ordering — a value copied at construction, before any override, is +fine — so this axis is pinned as a measurement with the same growth guard rather +than as a list of bugs. One of them *was* a defect and is fixed at the base of +this stack: the linear-attn dispatch table rebuilt itself from the record after +the SM100 GDN prefill decision had been recorded in the bag, so a second runner's +rebuild dropped it. That choice is a per-runner stamp now and is not recorded +process-wide at all, so neither the read nor the field is on this axis. +""" + +import ast +import dataclasses +import json +import os +import shutil +import tempfile +import unittest +from pathlib import Path + +import sglang +from sglang.srt.server_args import ServerArgs +from sglang.test.ci.ci_register import register_cpu_ci, register_cuda_ci +from sglang.test.test_utils import CustomTestCase + +register_cpu_ci(est_time=20, suite="base-a-test-cpu") +# Also on a CUDA runner: the written set is derived by resolving on the running +# host, and `is_cuda()` / capability gates only open on real hardware. The pin +# is split by host so both registrations stay exact: `_EXPOSED` is asserted +# everywhere, and a pair whose write only happens on CUDA belongs in +# `_EXPOSED_CUDA_ONLY` -- pinned on the CUDA runner, invisible to the CPU +# assertion. Without the split, one shared exact list could not hold such a +# pair at all: pinning it fails the CPU run as "gone", omitting it fails the +# CUDA run as "new". (No AMD registration: an `is_hip()`-gated write would +# shift the exact sets in ways none of the pinning hosts can verify; the ROCm +# resolution surface is covered by `test_resolution_is_reproducible.py` +# instead, whose assertion is device-agnostic.) +register_cuda_ci(est_time=20, stage="base-b", runner_config="1-gpu-small") + +_PACKAGE_ROOT = Path(next(iter(sglang.__path__))) / "srt" + +# The config the resolution pipeline owns; reading the in-flight record is their +# job, not a supplied-instance read. +_OWNERS = ("server_args.py", "runtime_context.py", "arg_groups/") + +_MINI_CONFIG = { + "architectures": ["LlamaForCausalLM"], + "model_type": "llama", + "hidden_size": 16, + "intermediate_size": 32, + "num_attention_heads": 2, + "num_key_value_heads": 2, + "num_hidden_layers": 2, + "vocab_size": 128, + "max_position_embeddings": 2048, +} + +# One config resolves only its own decisions, so the written set is a union. +_MATRIX = ( + {}, + { + "speculative_algorithm": "EAGLE", + "speculative_num_steps": 3, + "speculative_eagle_topk": 1, + "speculative_num_draft_tokens": 4, + }, + {"dp_size": 2, "tp_size": 2, "enable_dp_attention": True}, + # DWDP resolves dp_size and enable_dp_attention *itself* -- the plain DP + # entry above passes them in, and passed-in fields are excluded from the + # written set, so without this entry the dp_size readers would never pin. + {"tp_size": 2, "dwdp_size": 2}, + {"enable_hierarchical_cache": True, "hicache_ratio": 2.0}, + {"disaggregation_mode": "prefill"}, + {"tp_size": 2, "attn_cp_size": 2}, + {"enable_lora": True, "max_lora_rank": 16}, + {"kv_cache_dtype": "fp8_e4m3", "page_size": 64}, + # MIS resolves disable_radix_cache (and friends) itself; the backend is + # passed in because the handler asserts flashinfer rather than switching. + {"enable_mis": True, "attention_backend": "flashinfer"}, +) + +# `declare_late_resolution` call sites whose keyword expansion is built +# dynamically; the written fields are spelled out here and drift-guarded. +_LATE_RESOLUTION_DYNAMIC_SITES = { + "parser/template_detection.py": frozenset({"reasoning_parser", "tool_call_parser"}), +} + +# Resolution also branches on ambient environment; those shapes are explicit +# entries so the written set is the same on every host. `SGLANG_IS_IN_CI` +# makes resolution fill `soft_watchdog_timeout`. +_ENV_MATRIX = (({}, {"SGLANG_IS_IN_CI": "true"}),) + +# Only true constructor inputs: `tokenizer_path` / `served_model_name` are +# resolution-written (filled from `model_path` when unset), so their readers +# are step-12 exposure like any other pair. +_PASSED = frozenset({"model_path", "device", "random_seed"}) + +_EXPOSED = { + ("configs/embedding_model_spec.py", "chunked_prefill_size"), + ("configs/embedding_model_spec.py", "cuda_graph_config"), + ("configs/embedding_model_spec.py", "disable_radix_cache"), + ("configs/embedding_model_spec.py", "is_embedding"), + ("configs/embedding_model_spec.py", "prefill_only_disable_kv_cache"), + ("configs/model_config.py", "_speculative_draft_quantization_explicitly_set"), + ("configs/model_config.py", "disable_hybrid_swa_memory"), + ("configs/model_config.py", "dtype"), + ("configs/model_config.py", "enable_multi_layer_eagle"), + ("configs/model_config.py", "is_embedding"), + ("configs/model_config.py", "model_path"), + ("configs/model_config.py", "quantization"), + ("configs/model_config.py", "speculative_algorithm"), + ("configs/model_config.py", "speculative_draft_model_quantization"), + ("constrained/base_grammar_backend.py", "grammar_backend"), + ("constrained/base_grammar_backend.py", "reasoning_parser"), + ("disaggregation/common/conn.py", "disaggregation_bootstrap_port"), + ("disaggregation/common/conn.py", "pp_size"), + ("disaggregation/decode_kvcache_offload_manager.py", "hicache_io_backend"), + ("disaggregation/decode_kvcache_offload_manager.py", "hicache_mem_layout"), + ("disaggregation/decode_kvcache_offload_manager.py", "served_model_name"), + ("disaggregation/encode_receiver.py", "disaggregation_ib_device"), + ("disaggregation/encode_receiver.py", "encoder_transfer_backend"), + ("disaggregation/encode_receiver.py", "mooncake_ib_device"), + ("disaggregation/encode_receiver.py", "tokenizer_path"), + ("disaggregation/encode_server.py", "device"), + ("disaggregation/encode_server.py", "dp_size"), + ("disaggregation/encode_server.py", "encoder_transfer_backend"), + ("disaggregation/encode_server.py", "load_format"), + ("disaggregation/encode_server.py", "mm_process_config"), + ("disaggregation/encode_server.py", "model_path"), + ("disaggregation/encode_server.py", "served_model_name"), + ("disaggregation/encode_server.py", "tokenizer_path"), + ("disaggregation/utils.py", "disaggregation_transfer_backend"), + ("distributed/bootstrap.py", "disable_custom_all_reduce"), + ("distributed/bootstrap.py", "enable_symm_mem"), + ("distributed/bootstrap.py", "enable_torch_symm_mem"), + ("distributed/bootstrap.py", "flashinfer_allreduce_fusion_backend"), + ("distributed/bootstrap.py", "moe_a2a_backend"), + ("distributed/bootstrap.py", "pre_warm_nccl"), + ( + "distributed/device_communicators/mooncake_transfer_engine.py", + "disaggregation_ib_device", + ), + ( + "distributed/device_communicators/mooncake_transfer_engine.py", + "disaggregation_mode", + ), + ( + "distributed/device_communicators/mooncake_transfer_engine.py", + "disaggregation_transfer_backend", + ), + ( + "distributed/device_communicators/mooncake_transfer_engine.py", + "enable_hierarchical_cache", + ), + ( + "distributed/device_communicators/mooncake_transfer_engine.py", + "encoder_transfer_backend", + ), + ( + "distributed/device_communicators/mooncake_transfer_engine.py", + "mooncake_ib_device", + ), + ("dllm/config.py", "max_running_requests"), + ("dllm/config.py", "model_path"), + ("elastic_ep/elastic_ep.py", "elastic_ep_initial_size"), + ("elastic_ep/elastic_ep.py", "ep_join_mode"), + ("elastic_ep/elastic_ep.py", "moe_a2a_backend"), + ("elastic_ep/expert_backup_manager.py", "disaggregation_ib_device"), + ("elastic_ep/expert_backup_manager.py", "load_format"), + ("elastic_ep/expert_backup_manager.py", "mooncake_ib_device"), + ("entrypoints/engine.py", "attn_cp_size"), + ("entrypoints/engine.py", "detokenizer_worker_num"), + ("entrypoints/engine.py", "dp_size"), + ("entrypoints/engine.py", "dtype"), + ("entrypoints/engine.py", "enable_dp_attention"), + ("entrypoints/engine.py", "enable_symm_mem"), + ("entrypoints/engine.py", "ep_join_mode"), + ("entrypoints/engine.py", "ep_size"), + ("entrypoints/engine.py", "load_format"), + ("entrypoints/engine.py", "model_path"), + ("entrypoints/engine.py", "moe_dp_size"), + ("entrypoints/engine.py", "pp_size"), + ("entrypoints/engine.py", "quantization"), + ("entrypoints/engine.py", "reasoning_parser"), + ( + "entrypoints/engine.py", + "remote_instance_weight_loader_start_seed_via_transfer_engine", + ), + ("entrypoints/engine.py", "tool_call_parser"), + ("entrypoints/http_server.py", "disaggregation_mode"), + ("entrypoints/http_server.py", "dp_size"), + ("entrypoints/http_server.py", "ep_join_mode"), + ("entrypoints/http_server.py", "grpc_port"), + ("entrypoints/http_server.py", "model_path"), + ("entrypoints/http_server.py", "served_model_name"), + ("entrypoints/http_server.py", "skip_server_warmup"), + ("entrypoints/sidecar.py", "grpc_port"), + ("eplb/eplb_manager.py", "ep_dispatch_algorithm"), + ("eplb/eplb_manager.py", "expert_distribution_recorder_buffer_size"), + ("eplb/expert_distribution.py", "deepep_mode"), + ("eplb/expert_distribution.py", "device"), + ("eplb/expert_distribution.py", "expert_distribution_recorder_mode"), + ("eplb/expert_distribution.py", "moe_a2a_backend"), + ("eplb/expert_location.py", "device"), + ("eplb/expert_location.py", "eplb_algorithm"), + ("kv_canary/api.py", "disaggregation_mode"), + ("kv_canary/api.py", "speculative_num_steps"), + ("kv_canary/capacities.py", "chunked_prefill_size"), + ("kv_canary/capacities.py", "cuda_graph_config"), + ("kv_canary/capacities.py", "speculative_num_draft_tokens"), + ("kv_canary/token_oracle/install.py", "sampling_backend"), + ("layers/attention/dsa/utils.py", "disaggregation_mode"), + ("layers/cp/base.py", "attn_cp_size"), + ("layers/cp/base.py", "cp_strategy"), + ("layers/cp/base.py", "enable_prefill_cp"), + ("layers/cp/bcg.py", "cp_strategy"), + ("layers/cp/bcg.py", "enable_prefill_cp"), + ("layers/dp_attention.py", "attn_cp_size"), + ("layers/dp_attention.py", "device"), + ("layers/dp_attention.py", "dp_size"), + ("layers/dp_attention.py", "enable_dp_attention"), + ("layers/flashinfer_comm_fusion.py", "flashinfer_allreduce_fusion_backend"), + ("layers/moe/kt_ep_wrapper.py", "chunked_prefill_size"), + ("layers/moe/utils.py", "deepep_mode"), + ("layers/moe/utils.py", "moe_a2a_backend"), + ("layers/moe/utils.py", "moe_runner_backend"), + ("layers/moe/utils.py", "quantization"), + ("layers/moe/utils.py", "speculative_moe_runner_backend"), + ("layers/quantization/unquant.py", "enable_deterministic_inference"), + ("lora/lora_manager.py", "enable_dp_attention"), + ("lora/lora_manager.py", "enable_lora_overlap_loading"), + ("lora/marlin_lora_temp/policy.py", "enable_lora"), + ("lora/marlin_lora_temp/policy.py", "lora_paths"), + ("managers/data_parallel_controller.py", "attn_cp_size"), + ("managers/data_parallel_controller.py", "disaggregation_mode"), + ("managers/data_parallel_controller.py", "dp_size"), + ("managers/data_parallel_controller.py", "enable_dp_attention"), + ( + "managers/data_parallel_controller.py", + "enable_dp_attention_local_control_broadcast", + ), + ("managers/data_parallel_controller.py", "ep_size"), + ("managers/data_parallel_controller.py", "load_balance_method"), + ("managers/data_parallel_controller.py", "moe_dp_size"), + ("managers/data_parallel_controller.py", "pp_size"), + ("managers/data_parallel_controller.py", "soft_watchdog_timeout"), + ("managers/detokenizer_manager.py", "soft_watchdog_timeout"), + ("managers/detokenizer_manager.py", "tokenizer_path"), + ("managers/detokenizer_manager.py", "tool_call_parser"), + ("managers/disagg_service.py", "disaggregation_bootstrap_port"), + ("managers/disagg_service.py", "disaggregation_mode"), + ("managers/disagg_service.py", "disaggregation_transfer_backend"), + ("managers/load_snapshot.py", "dp_size"), + ("managers/load_snapshot.py", "enable_dp_attention"), + ("managers/load_snapshot.py", "load_balance_method"), + ("managers/overlap_utils.py", "speculative_algorithm"), + ("managers/prefill_delayer.py", "disable_overlap_schedule"), + ("managers/prefill_delayer.py", "enable_dp_attention"), + ("managers/rust_server.py", "mm_process_config"), + ("managers/schedule_batch.py", "disaggregation_mode"), + ("managers/scheduler.py", "attn_cp_size"), + ("managers/scheduler.py", "disable_overlap_schedule"), + ("managers/scheduler.py", "disaggregation_mode"), + ("managers/scheduler.py", "dp_size"), + ("managers/scheduler.py", "enable_dp_attention"), + ("managers/scheduler.py", "enable_hierarchical_cache"), + ("managers/scheduler.py", "enable_lora"), + ("managers/scheduler.py", "enable_lora_overlap_loading"), + ("managers/scheduler.py", "ep_size"), + ("managers/scheduler.py", "moe_dp_size"), + ("managers/scheduler.py", "pp_size"), + ("managers/scheduler.py", "soft_watchdog_timeout"), + ("managers/scheduler.py", "speculative_algorithm"), + ( + "managers/scheduler_components/new_token_ratio_tracker.py", + "schedule_conservativeness", + ), + ("managers/scheduler_components/recv_skipper.py", "enable_dp_attention"), + ("managers/tokenizer_control_mixin.py", "dp_size"), + ("managers/tokenizer_manager.py", "disable_radix_cache"), + ("managers/tokenizer_manager.py", "disaggregation_mode"), + ("managers/tokenizer_manager.py", "disaggregation_transfer_backend"), + ("managers/tokenizer_manager.py", "dp_size"), + ("managers/tokenizer_manager.py", "enable_dp_attention"), + ("managers/tokenizer_manager.py", "enable_lora"), + ("managers/tokenizer_manager.py", "enable_tokenizer_batch_encode"), + ("managers/tokenizer_manager.py", "encoder_transfer_backend"), + ("managers/tokenizer_manager.py", "limit_mm_data_per_request"), + ("managers/tokenizer_manager.py", "lora_paths"), + ("managers/tokenizer_manager.py", "mm_feature_transport"), + ("managers/tokenizer_manager.py", "model_path"), + ("managers/tokenizer_manager.py", "preferred_sampling_params"), + ("managers/tokenizer_manager.py", "return_hidden_states_mode"), + ("managers/tokenizer_manager.py", "served_model_name"), + ("managers/tokenizer_manager.py", "soft_watchdog_timeout"), + ("managers/tokenizer_manager.py", "speculative_algorithm"), + ("managers/tokenizer_manager.py", "speculative_num_draft_tokens"), + ("managers/tokenizer_manager.py", "tokenizer_path"), + ("managers/tp_worker.py", "disable_overlap_schedule"), + ("managers/tp_worker.py", "model_path"), + ("managers/tp_worker.py", "random_seed"), + ("managers/tp_worker.py", "speculative_algorithm"), + ("managers/tp_worker.py", "tokenizer_path"), + ("managers/utils.py", "speculative_algorithm"), + ("managers/utils.py", "speculative_eagle_topk"), + ("managers/utils.py", "speculative_num_steps"), + ("mem_cache/allocation_sizing.py", "page_size"), + ("mem_cache/allocation_sizing.py", "speculative_algorithm"), + ("mem_cache/allocation_sizing.py", "speculative_eagle_topk"), + ("mem_cache/allocation_sizing.py", "speculative_num_steps"), + ("mem_cache/hiradix_cache.py", "hicache_io_backend"), + ("mem_cache/hiradix_cache.py", "hicache_mem_layout"), + ("mem_cache/hiradix_cache.py", "served_model_name"), + ("mem_cache/hybrid_cache/hybrid_pool_assembler.py", "hicache_io_backend"), + ("mem_cache/hybrid_cache/hybrid_pool_assembler.py", "hicache_mem_layout"), + ("mem_cache/hybrid_cache/hybrid_pool_assembler.py", "served_model_name"), + ("mem_cache/kv_cache_builder.py", "disable_radix_cache"), + ("mem_cache/kv_cache_builder.py", "disaggregation_mode"), + ("mem_cache/kv_cache_builder.py", "enable_dp_attention"), + ("mem_cache/kv_cache_builder.py", "hicache_mem_layout"), + ("mem_cache/radix_cache_cpp.py", "enable_hierarchical_cache"), + ("model_executor/forward_batch_info.py", "enable_return_hidden_states"), + ("model_executor/forward_batch_info.py", "return_hidden_states_mode"), + ("model_executor/model_runner.py", "device"), + ("model_executor/model_runner.py", "speculative_algorithm"), + ("model_executor/model_runner.py", "speculative_draft_attention_backend"), + ("model_executor/model_runner_components/load_model_utils.py", "load_format"), + ("model_executor/model_runner_components/load_model_utils.py", "quantization"), + ( + "model_executor/model_runner_components/spec_aux_hidden_state.py", + "speculative_draft_attention_backend", + ), + ( + "model_executor/model_runner_components/spec_aux_hidden_state.py", + "speculative_draft_model_path", + ), + ( + "model_executor/model_runner_components/spec_aux_hidden_state.py", + "speculative_draft_model_revision", + ), + ("model_executor/model_runner_components/startup_weight_load.py", "attn_cp_size"), + ( + "model_executor/model_runner_components/startup_weight_load.py", + "cuda_graph_config", + ), + ( + "model_executor/model_runner_components/startup_weight_load.py", + "custom_weight_loader", + ), + ("model_executor/model_runner_components/startup_weight_load.py", "device"), + ("model_executor/model_runner_components/startup_weight_load.py", "dp_size"), + ("model_executor/model_runner_components/startup_weight_load.py", "enable_lora"), + ("model_executor/model_runner_components/startup_weight_load.py", "ep_size"), + ("model_executor/model_runner_components/startup_weight_load.py", "lora_paths"), + ("model_executor/model_runner_components/startup_weight_load.py", "pp_size"), + ( + "model_executor/model_runner_components/startup_weight_load.py", + "speculative_algorithm", + ), + ( + "model_executor/runner_backend/tc_piecewise_cuda_graph_backend.py", + "cuda_graph_config", + ), + ("models/sarvam_moe.py", "attention_backend"), + ("models/sarvam_moe.py", "decode_attention_backend"), + ("models/sarvam_moe.py", "prefill_attention_backend"), + ("multimodal/cache/identity.py", "mm_process_config"), + ("multimodal/processors/base_processor.py", "image_processor_backend"), + ("multimodal/processors/base_processor.py", "mm_feature_transport"), + ("multimodal/processors/base_processor.py", "mm_process_config"), + ("multimodal/processors/mimo_v2.py", "device"), + ("observability/metrics_collector.py", "disaggregation_mode"), + ("observability/metrics_collector.py", "prefill_delayer_max_delay_passes"), + ("observability/metrics_collector.py", "served_model_name"), + ("parser/template_detection.py", "model_path"), + ("ray/data_parallel_controller.py", "attn_cp_size"), + ("ray/data_parallel_controller.py", "dp_size"), + ("ray/data_parallel_controller.py", "enable_dp_attention"), + ("ray/data_parallel_controller.py", "pp_size"), + ("ray/engine.py", "dp_size"), + ("ray/engine.py", "enable_dp_attention"), + ("ray/engine.py", "pp_size"), + ("speculative/adaptive_spec_params.py", "speculative_algorithm"), + ("speculative/adaptive_spec_params.py", "speculative_eagle_topk"), + ("speculative/dflash_worker_v2.py", "speculative_draft_window_size"), + ("speculative/dflash_worker_v2.py", "speculative_num_draft_tokens"), + ("speculative/draft_worker_common.py", "speculative_draft_attention_backend"), + ("speculative/dspark_components/dspark_config.py", "speculative_draft_model_path"), + ( + "speculative/dspark_components/dspark_config.py", + "speculative_draft_model_revision", + ), + ("speculative/dspark_components/dspark_worker_v2.py", "disable_cuda_graph"), + ("speculative/dspark_components/dspark_worker_v2.py", "disaggregation_mode"), + ("speculative/dspark_components/dspark_worker_v2.py", "enable_dp_attention"), + ( + "speculative/dspark_components/dspark_worker_v2.py", + "speculative_num_draft_tokens", + ), + ("speculative/eagle_disaggregation.py", "enable_multi_layer_eagle"), + ("speculative/eagle_disaggregation.py", "speculative_eagle_topk"), + ("speculative/eagle_disaggregation.py", "speculative_num_steps"), + ("speculative/eagle_worker_v2.py", "device"), + ("speculative/eagle_worker_v2.py", "enable_dp_attention"), + ("speculative/eagle_worker_v2.py", "speculative_adaptive"), + ("speculative/eagle_worker_v2.py", "speculative_algorithm"), + ("speculative/eagle_worker_v2.py", "speculative_eagle_topk"), + ("speculative/eagle_worker_v2.py", "speculative_num_draft_tokens"), + ("speculative/eagle_worker_v2.py", "speculative_num_steps"), + ("speculative/frozen_kv_mtp_worker_v2.py", "device"), + ("speculative/frozen_kv_mtp_worker_v2.py", "enable_dp_attention"), + ("speculative/frozen_kv_mtp_worker_v2.py", "speculative_adaptive"), + ("speculative/frozen_kv_mtp_worker_v2.py", "speculative_algorithm"), + ("speculative/frozen_kv_mtp_worker_v2.py", "speculative_eagle_topk"), + ("speculative/frozen_kv_mtp_worker_v2.py", "speculative_num_draft_tokens"), + ("speculative/frozen_kv_mtp_worker_v2.py", "speculative_num_steps"), + ("speculative/multi_layer_eagle_worker_v2.py", "device"), + ("speculative/multi_layer_eagle_worker_v2.py", "enable_dp_attention"), + ("speculative/multi_layer_eagle_worker_v2.py", "speculative_algorithm"), + ("speculative/multi_layer_eagle_worker_v2.py", "speculative_eagle_topk"), + ("speculative/multi_layer_eagle_worker_v2.py", "speculative_num_draft_tokens"), + ("speculative/multi_layer_eagle_worker_v2.py", "speculative_num_steps"), + ("speculative/ngram_worker.py", "device"), + ("speculative/ngram_worker.py", "disable_overlap_schedule"), + ("speculative/ngram_worker.py", "speculative_eagle_topk"), + ("speculative/ngram_worker.py", "speculative_num_draft_tokens"), + ("speculative/ngram_worker.py", "speculative_num_steps"), + ("speculative/spec_info.py", "enable_multi_layer_eagle"), + ("speculative/spec_info.py", "speculative_eagle_topk"), + ("speculative/spec_info.py", "speculative_num_draft_tokens"), + ("speculative/spec_info.py", "speculative_num_steps"), + ("speculative/spec_registry.py", "disable_overlap_schedule"), + ("speculative/spec_utils.py", "speculative_eagle_topk"), + ("speculative/spec_utils.py", "speculative_num_draft_tokens"), + ("speculative/standalone_worker_v2.py", "device"), + ("speculative/standalone_worker_v2.py", "enable_dp_attention"), + ("speculative/standalone_worker_v2.py", "speculative_algorithm"), + ("speculative/standalone_worker_v2.py", "speculative_eagle_topk"), + ("speculative/standalone_worker_v2.py", "speculative_num_draft_tokens"), + ("speculative/standalone_worker_v2.py", "speculative_num_steps"), + ("utils/common.py", "page_size"), + ("utils/common.py", "speculative_eagle_topk"), + ("utils/common.py", "speculative_num_draft_tokens"), + ("utils/common.py", "speculative_num_steps"), + ("utils/cuda_vmm_transport_utils.py", "dp_size"), + ("utils/cuda_vmm_transport_utils.py", "enable_dp_attention"), + ("utils/cuda_vmm_transport_utils.py", "mm_feature_transport"), + ("utils/hf_transformers/processor.py", "image_processor_backend"), + ("utils/offloader.py", "dp_size"), +} + +# Pairs whose resolution write only happens on a CUDA host (capability or +# `is_cuda()` gated): asserted on the CUDA registration, invisible to the CPU +# one. Empty today -- the current written sets coincide across the two hosts -- +# but this is where a GPU-only write's readers get pinned without breaking the +# CPU-exact assertion. +_EXPOSED_CUDA_ONLY: frozenset = frozenset() + + +# Axis two: (file, field) pairs where a supplied-instance read names a field that +# some code overrides post-publish. Each needs an ordering judgment, not a blanket +# conversion; the list exists so a new one is a decision made when it is written. +_OVERRIDDEN_AND_READ = { + ("configs/model_config.py", "dtype"), + ("configs/model_config.py", "model_path"), + ("constrained/base_grammar_backend.py", "grammar_backend"), + ("disaggregation/decode_kvcache_offload_manager.py", "hicache_storage_backend"), + ( + "disaggregation/decode_kvcache_offload_manager.py", + "hicache_storage_backend_extra_config", + ), + ("disaggregation/encode_server.py", "dp_size"), + ("disaggregation/encode_server.py", "load_format"), + ("disaggregation/encode_server.py", "model_path"), + ( + "distributed/device_communicators/mooncake_transfer_engine.py", + "hicache_storage_backend", + ), + ("dllm/config.py", "model_path"), + ("elastic_ep/expert_backup_manager.py", "load_format"), + ("entrypoints/engine.py", "dp_size"), + ("entrypoints/engine.py", "dtype"), + ("entrypoints/engine.py", "ep_size"), + ("entrypoints/engine.py", "load_format"), + ("entrypoints/engine.py", "model_path"), + ("entrypoints/http_server.py", "dp_size"), + ("entrypoints/http_server.py", "model_path"), + ("kv_canary/api.py", "speculative_num_steps"), + ("kv_canary/capacities.py", "speculative_num_draft_tokens"), + ("layers/dp_attention.py", "dp_size"), + ("managers/data_parallel_controller.py", "dp_size"), + ("managers/data_parallel_controller.py", "ep_size"), + ("managers/load_snapshot.py", "dp_size"), + ("managers/scheduler.py", "dp_size"), + ("managers/scheduler.py", "ep_size"), + ("managers/scheduler.py", "hicache_storage_backend"), + ("managers/tokenizer_control_mixin.py", "dp_size"), + ("managers/tokenizer_manager.py", "dp_size"), + ("managers/tokenizer_manager.py", "model_path"), + ("managers/tokenizer_manager.py", "speculative_num_draft_tokens"), + ("managers/tp_worker.py", "model_path"), + ("managers/utils.py", "speculative_num_steps"), + ("mem_cache/allocation_sizing.py", "speculative_num_steps"), + ("mem_cache/hiradix_cache.py", "hicache_storage_backend"), + ("mem_cache/hiradix_cache.py", "hicache_storage_backend_extra_config"), + ("mem_cache/hiradix_cache.py", "hicache_storage_prefetch_policy"), + ("mem_cache/hiradix_cache.py", "hicache_write_policy"), + ("mem_cache/hybrid_cache/hybrid_pool_assembler.py", "hicache_storage_backend"), + ("mem_cache/hybrid_cache/hybrid_pool_assembler.py", "hicache_write_policy"), + ("mem_cache/kv_cache_builder.py", "hicache_storage_backend"), + ("mem_cache/pool_host/common.py", "hicache_storage_backend"), + ("mem_cache/pool_host/common.py", "hicache_storage_backend_extra_config"), + ("mem_cache/radix_cache_cpp.py", "hicache_write_policy"), + ("mem_cache/unified_radix_cache.py", "hicache_storage_backend"), + ("mem_cache/unified_radix_cache.py", "hicache_storage_backend_extra_config"), + ("mem_cache/unified_radix_cache.py", "hicache_storage_prefetch_policy"), + ("mem_cache/unified_radix_cache.py", "hicache_write_policy"), + ("model_executor/model_runner_components/load_model_utils.py", "load_format"), + ("model_executor/model_runner_components/startup_weight_load.py", "dp_size"), + ("model_executor/model_runner_components/startup_weight_load.py", "ep_size"), + ("parser/template_detection.py", "model_path"), + ("ray/data_parallel_controller.py", "dp_size"), + ("ray/engine.py", "dp_size"), + ("speculative/dflash_worker_v2.py", "speculative_num_draft_tokens"), + ("speculative/dspark_components/dspark_worker_v2.py", "disable_cuda_graph"), + ( + "speculative/dspark_components/dspark_worker_v2.py", + "speculative_num_draft_tokens", + ), + ("speculative/eagle_disaggregation.py", "speculative_num_steps"), + ("speculative/eagle_worker_v2.py", "speculative_num_draft_tokens"), + ("speculative/eagle_worker_v2.py", "speculative_num_steps"), + ("speculative/frozen_kv_mtp_worker_v2.py", "speculative_num_draft_tokens"), + ("speculative/frozen_kv_mtp_worker_v2.py", "speculative_num_steps"), + ("speculative/multi_layer_eagle_worker_v2.py", "speculative_num_draft_tokens"), + ("speculative/multi_layer_eagle_worker_v2.py", "speculative_num_steps"), + ("speculative/ngram_worker.py", "speculative_num_draft_tokens"), + ("speculative/ngram_worker.py", "speculative_num_steps"), + ("speculative/spec_info.py", "speculative_num_draft_tokens"), + ("speculative/spec_info.py", "speculative_num_steps"), + ("speculative/spec_utils.py", "speculative_num_draft_tokens"), + ("speculative/standalone_worker_v2.py", "speculative_num_draft_tokens"), + ("speculative/standalone_worker_v2.py", "speculative_num_steps"), + ("utils/common.py", "speculative_num_draft_tokens"), + ("utils/common.py", "speculative_num_steps"), + ("utils/cuda_vmm_transport_utils.py", "dp_size"), + ("utils/offloader.py", "dp_size"), +} + + +def _expanded_override_keys(rel, tree, call, kw) -> set: + """The statically visible keys behind an ``override(..., **expr)``. + + Handles a dict literal, a conditional between dict literals, and a name + bound to a dict literal in the enclosing function (plus constant-subscript + stores onto it -- the HiCache attach shape). One expansion is unresolvable + by design and exempted by name: ``update_server_args`` forwards + operator-chosen fields, so its key set is the API's, not this file's. + Anything else unresolvable fails -- a silently skipped expansion would + shrink the written set. + """ + for a in call.args: + if isinstance(a, ast.Constant) and a.value == "update_server_args": + return set() + for k in call.keywords: + if ( + k.arg == "source" + and isinstance(k.value, ast.Constant) + and k.value.value == "update_server_args" + ): + return set() + + def dict_keys(node) -> set: + assert isinstance(node, ast.Dict) and all( + isinstance(key, ast.Constant) for key in node.keys + ), f"non-literal dict in override expansion at {rel}:{call.lineno}" + return {key.value for key in node.keys} + + if isinstance(kw.value, ast.Dict): + return dict_keys(kw.value) + if isinstance(kw.value, ast.IfExp): + keys = set() + for branch in (kw.value.body, kw.value.orelse): + if isinstance(branch, ast.Dict) and branch.keys: + keys |= dict_keys(branch) + elif isinstance(branch, ast.Dict): + pass + else: + raise AssertionError( + f"unresolvable override expansion at {rel}:{call.lineno}" + ) + return keys + assert isinstance( + kw.value, ast.Name + ), f"unresolvable override expansion at {rel}:{call.lineno}" + name = kw.value.id + enclosing = None + for fn in ast.walk(tree): + if isinstance(fn, (ast.FunctionDef, ast.AsyncFunctionDef)): + if ( + fn.lineno + <= call.lineno + <= max(getattr(fn, "end_lineno", fn.lineno), fn.lineno) + ): + if enclosing is None or fn.lineno > enclosing.lineno: + enclosing = fn + assert ( + enclosing is not None + ), f"override expansion outside any function at {rel}:{call.lineno}" + keys = set() + found = False + for node in ast.walk(enclosing): + if ( + isinstance(node, ast.Assign) + and len(node.targets) == 1 + and isinstance(node.targets[0], ast.Name) + and node.targets[0].id == name + and isinstance(node.value, ast.Dict) + ): + found = True + keys |= dict_keys(node.value) + elif ( + isinstance(node, ast.Assign) + and len(node.targets) == 1 + and isinstance(node.targets[0], ast.Subscript) + and isinstance(node.targets[0].value, ast.Name) + and node.targets[0].value.id == name + and isinstance(node.targets[0].slice, ast.Constant) + ): + keys.add(node.targets[0].slice.value) + assert found, ( + f"override expansion '{name}' at {rel}:{call.lineno} has no " + "dict-literal assignment in its function; extend the resolver" + ) + return keys + + +class TestSuppliedInstanceExposure(CustomTestCase): + def _callTestMethod(self, method): + # No CI retry: a failed first attempt has already resolved the matrix + # and mutated process state; a retry against that contamination could + # pass on a drifted written set or mask a real drift. + return unittest.TestCase._callTestMethod(self, method) + + def setUp(self): + # Resolving the matrix writes process state on the way through (the + # multimodal transport handler sets SGLANG_USE_CUDA_IPC_TRANSPORT, and + # `EnvField.set()` flips a descriptor flag `os.environ` does not carry). + # Leaking it makes *later* files in the same worker fail, which is how + # this was found -- so the case restores what it touched. + super().setUp() + state = (dict(os.environ), self._env_field_flags()) + self.addCleanup(self._restore_process_state, state) + + @staticmethod + def _env_field_flags() -> dict: + from sglang.srt.environ import EnvField, envs + + flags = {} + for klass in reversed(type(envs).__mro__): + for name, field in vars(klass).items(): + if isinstance(field, EnvField): + flags[name] = field._set_to_none + return flags + + @staticmethod + def _restore_process_state(state) -> None: + from sglang.srt.environ import envs + + saved_environ, saved_flags = state + os.environ.clear() + os.environ.update(saved_environ) + for name, was_none in saved_flags.items(): + getattr(type(envs), name)._set_to_none = was_none + + def _config_dir(self) -> str: + config_dir = tempfile.mkdtemp(prefix="supplied_instance_") + self.addCleanup(shutil.rmtree, config_dir, ignore_errors=True) + with open(os.path.join(config_dir, "config.json"), "w") as handle: + json.dump(_MINI_CONFIG, handle) + return config_dir + + def _resolution_written_fields(self) -> set: + """The union of what resolution fills in across the matrix. + + Every entry must resolve. A silently skipped one would shrink this set, + which makes pinned pairs look like they disappeared -- the list would + then drift by environment rather than by code, and the failure would + point at the wrong thing. Each entry resolves from the pristine + process snapshot (resolution writes env and EnvField flags on the way + through, and DWDP flips `SGLANG_SCHEDULER_SKIP_ALL_GATHER`), so the + union does not depend on matrix order; and the ambient CI marker is + cleared, so a runner's identity cannot leak into the measurement -- + the CI-conditioned writes come from `_ENV_MATRIX`'s explicit entry. + Late resolution counts too: `declare_late_resolution` writers run at + launcher stage (LoRA normalization, parser auto-detection), so their + target fields are collected statically from the call sites -- they are + resolution writes by definition, just staged after `__post_init__`. + """ + pristine = (dict(os.environ), self._env_field_flags()) + written = set() + + def resolve_one(extra, env): + self._restore_process_state(pristine) + os.environ.pop("SGLANG_IS_IN_CI", None) + os.environ.update(env) + model_path = self._config_dir() + try: + resolved = ServerArgs( + model_path=model_path, device="cuda", random_seed=42, **extra + ) + except Exception as exc: + self.fail( + f"the matrix entry {extra} (env={env}) did not resolve in " + f"this environment ({type(exc).__name__}: {exc}); the " + "written-field union would be short and the pinned list " + "would drift" + ) + defaults = {} + for field in dataclasses.fields(resolved): + if field.default is not dataclasses.MISSING: + defaults[field.name] = field.default + elif field.default_factory is not dataclasses.MISSING: + defaults[field.name] = field.default_factory() + for field_name, default in defaults.items(): + if field_name in _PASSED or field_name in extra: + continue + if getattr(resolved, field_name) != default: + written.add(field_name) + + for extra in _MATRIX: + resolve_one(extra, {}) + for extra, env in _ENV_MATRIX: + resolve_one(extra, env) + self._restore_process_state(pristine) + written |= self._late_resolution_written_fields() + written |= self._hook_assignment_targets() + written |= self._record_method_assignment_targets() + written |= self._declarative_override_fields() + return written + + def _hook_assignment_targets(self) -> set: + """Fields any resolution hook can write, collected statically. + + The matrix can only enumerate families someone thought to add -- the + DFLASH hole (its hook is the sole writer of + `speculative_draft_attention_backend`, and no entry ran it) showed + that a family nobody listed leaves its readers unpinned. The hook + modules under `arg_groups/` are the resolution pipeline's extension + points, and their assignment surface (`server_args.field = ...`) is + the may-write set, family-blind by construction. Collected like the + late-resolution keywords: statically, failing loudly on an + unparsable module. Underscore-prefixed targets are pipeline + bookkeeping, not config leaves. + """ + targets = set() + for path in sorted((_PACKAGE_ROOT / "arg_groups").glob("*.py")): + try: + tree = ast.parse(path.read_text(encoding="utf-8-sig")) + except SyntaxError: + self.fail(f"unparsable hook module in the census: {path.name}") + for node in ast.walk(tree): + if isinstance(node, ast.Assign): + tgts = node.targets + elif isinstance(node, (ast.AnnAssign, ast.AugAssign)): + tgts = [node.target] + else: + continue + for tgt in tgts: + if ( + isinstance(tgt, ast.Attribute) + and isinstance(tgt.value, ast.Name) + and tgt.value.id == "server_args" + and not tgt.attr.startswith("_") + ): + targets.add(tgt.attr) + return targets + + def _record_method_assignment_targets(self) -> set: + """Fields ``ServerArgs``'s own methods can write, collected statically. + + The record's handlers are as family-conditional as the hooks -- the + mooncake/layer_first layout rewrite, the deepseek-EP mode defaults, + the seed fill that only runs when the caller did *not* supply one (so + construct-and-diff can never see it: measuring requires supplying). + A write site that can never fire is a dead branch to delete upstream, + not a census exemption. Only names that are declared dataclass fields + count; underscore bookkeeping does not. + """ + tree = ast.parse( + (_PACKAGE_ROOT / "server_args.py").read_text(encoding="utf-8-sig") + ) + sa_class = next( + node + for node in tree.body + if isinstance(node, ast.ClassDef) and node.name == "ServerArgs" + ) + declared = { + node.target.id + for node in sa_class.body + if isinstance(node, ast.AnnAssign) and isinstance(node.target, ast.Name) + } + targets = set() + for node in ast.walk(sa_class): + if isinstance(node, ast.Assign): + tgts = node.targets + elif isinstance(node, (ast.AnnAssign, ast.AugAssign)): + tgts = [node.target] + else: + continue + for tgt in tgts: + if ( + isinstance(tgt, ast.Attribute) + and isinstance(tgt.value, ast.Name) + and tgt.value.id == "self" + and tgt.attr in declared + and not tgt.attr.startswith("_") + ): + targets.add(tgt.attr) + # The deprecated-alias normalization loop writes through a *name + # tuple* (`for attr in (...): setattr(self, attr, "dsv4")`), which no + # assignment scan sees; its field set is pinned here with a drift + # guard on the tuple itself. + alias_fields = { + "attention_backend", + "decode_attention_backend", + "prefill_attention_backend", + "speculative_draft_attention_backend", + } + found_tuples = [ + {elt.value for elt in node.iter.elts if isinstance(elt, ast.Constant)} + for node in ast.walk(sa_class) + if isinstance(node, ast.For) + and isinstance(node.iter, ast.Tuple) + and any( + isinstance(inner, ast.Call) + and isinstance(inner.func, ast.Name) + and inner.func.id == "setattr" + for inner in ast.walk(node) + ) + ] + self.assertIn( + alias_fields, + found_tuples, + "the deprecated-alias normalization loop moved or changed its " + "field tuple; update alias_fields to match", + ) + return targets | alias_fields + + def _declarative_override_fields(self) -> set: + """Fields the declarative override registry can write. + + ``MODEL_OVERRIDES`` maps arch -> {field: value}, and the + ``@register_model_override``(-``_predicate``) providers return (or + build by subscript) {field: value} dicts; ``materialize_declarations`` + applies them all via setattr, so no assignment scan sees these writes + and a llama-only matrix never triggers them. Keys must be + string literals; anything else fails loudly. + """ + tree = ast.parse( + (_PACKAGE_ROOT / "arg_groups" / "overrides.py").read_text( + encoding="utf-8-sig" + ) + ) + fields = set() + for node in tree.body: + target = None + if isinstance(node, ast.Assign) and isinstance(node.targets[0], ast.Name): + target = node.targets[0].id + elif isinstance(node, ast.AnnAssign) and isinstance(node.target, ast.Name): + target = node.target.id + if target != "MODEL_OVERRIDES" or node.value is None: + continue + for inner in ast.walk(node.value): + if not isinstance(inner, ast.Dict): + continue + for key, value in zip(inner.keys, inner.values): + if isinstance(value, ast.Dict): + continue # arch -> {…} outer layer + self.assertIsInstance(key, ast.Constant, "non-literal override key") + fields.add(key.value) + for node in ast.walk(tree): + if not isinstance(node, ast.FunctionDef): + continue + if not any( + isinstance(dec, ast.Call) + and isinstance(dec.func, ast.Name) + and dec.func.id.startswith("register_model_override") + for dec in node.decorator_list + ): + continue + for inner in ast.walk(node): + if isinstance(inner, ast.Assign) and isinstance( + inner.targets[0], ast.Subscript + ): + key = inner.targets[0].slice + self.assertIsInstance( + key, ast.Constant, f"non-literal override key in {node.name}" + ) + fields.add(key.value) + if isinstance(inner, ast.Dict): + for key in inner.keys: + self.assertIsInstance( + key, + ast.Constant, + f"non-literal override key in {node.name}", + ) + fields.add(key.value) + return fields + + def _late_resolution_written_fields(self) -> set: + """Fields `declare_late_resolution` writes, collected statically. + + These are resolution's launcher-stage writes (they need a tokenizer or + adapter load, so they cannot run in `__post_init__`), which the + construct-and-diff pass above never sees. The keywords at the call + sites are the written fields; an expansion this cannot resolve fails + loudly like the override collector's, except the named dynamic sites + below, whose field sets are spelled out and drift-guarded (each name + must still appear as a constant in the file).""" + written = set() + root = _PACKAGE_ROOT + for path in sorted(root.rglob("*.py")): + rel = path.relative_to(root).as_posix() + try: + tree = ast.parse(path.read_text(encoding="utf-8-sig")) + except SyntaxError: + self.fail(f"unparsable module in the census: {rel}") + for node in ast.walk(tree): + if not ( + isinstance(node, ast.Call) + and ( + ( + isinstance(node.func, ast.Name) + and node.func.id == "declare_late_resolution" + ) + or ( + isinstance(node.func, ast.Attribute) + and node.func.attr + in ("declare_late_resolution", "_late_resolution") + ) + ) + ): + continue + if all(kw.arg is None for kw in node.keywords) and any( + isinstance(kw.value, ast.Name) and kw.value.id == "fields" + for kw in node.keywords + ): + # The forwarding shim (`ServerArgs._late_resolution` / + # the helper's own body) re-expands its caller's kwargs; + # the write sites are the callers. + continue + for kw in node.keywords: + if kw.arg and kw.arg != "source": + written.add(kw.arg) + elif kw.arg is None: + dynamic = _LATE_RESOLUTION_DYNAMIC_SITES.get(rel) + if dynamic is not None: + constants = { + c.value + for c in ast.walk(tree) + if isinstance(c, ast.Constant) + } + missing = dynamic - constants + self.assertFalse( + missing, + f"{rel}: the declared dynamic field set drifted " + f"from the file ({sorted(missing)} not found)", + ) + written |= dynamic + else: + written |= _expanded_override_keys(rel, tree, node, kw) + return written + + def _supplied_instance_reads(self) -> set: + """Three spellings of the same read: ``server_args.field`` off the + parameter, ``getattr(server_args, "field", default)`` with a literal + name, and the *parked* form -- ``self.x = server_args`` in a method + that takes the parameter, read as ``self.x.field`` anywhere in the + class. Parking under a different object, a container, or a computed + name stays invisible, like in every census of this family -- the + loudest known boundary is the *chain* spelling, + ``model_runner.server_args.field`` off some other parameter, which + this census does not count (~150 reads tree-wide; extending the pin + to that spelling is its own step, not a by-product of this one).""" + pairs = set() + for path in sorted(_PACKAGE_ROOT.rglob("*.py")): + rel = path.relative_to(_PACKAGE_ROOT).as_posix() + if rel.startswith(_OWNERS): + continue + try: + tree = ast.parse(path.read_text(encoding="utf-8-sig")) + except SyntaxError: + # A silently dropped module shrinks `found` and reads as + # intentional surface shrinkage under the bidirectional pin. + self.fail(f"unparsable module in the census: {rel}") + for fn in ast.walk(tree): + if not isinstance(fn, (ast.FunctionDef, ast.AsyncFunctionDef)): + continue + params = {a.arg for a in list(fn.args.args) + list(fn.args.kwonlyargs)} + if "server_args" not in params: + continue + for node in ast.walk(fn): + if ( + isinstance(node, ast.Attribute) + and isinstance(node.value, ast.Name) + and node.value.id == "server_args" + and isinstance(node.ctx, ast.Load) + ): + pairs.add((rel, node.attr)) + elif ( + isinstance(node, ast.Call) + and isinstance(node.func, ast.Name) + and node.func.id == "getattr" + and len(node.args) >= 2 + and isinstance(node.args[0], ast.Name) + and node.args[0].id == "server_args" + and isinstance(node.args[1], ast.Constant) + and isinstance(node.args[1].value, str) + ): + # The same read in optional clothing. Only a literal + # name is censusable; a computed one is not. + pairs.add((rel, node.args[1].value)) + for cls in ast.walk(tree): + if not isinstance(cls, ast.ClassDef): + continue + # Parked: `self.x = server_args` in a method that takes the + # parameter, read as `self.x.field` anywhere in the class. + parked = set() + for fn in cls.body: + if not isinstance(fn, (ast.FunctionDef, ast.AsyncFunctionDef)): + continue + if "server_args" not in { + a.arg for a in list(fn.args.args) + list(fn.args.kwonlyargs) + }: + continue + for node in ast.walk(fn): + if ( + isinstance(node, ast.Assign) + and len(node.targets) == 1 + and isinstance(node.targets[0], ast.Attribute) + and isinstance(node.targets[0].value, ast.Name) + and node.targets[0].value.id == "self" + and isinstance(node.value, ast.Name) + and node.value.id == "server_args" + ): + parked.add(node.targets[0].attr) + for node in ast.walk(cls): + if ( + isinstance(node, ast.Attribute) + and isinstance(node.ctx, ast.Load) + and isinstance(node.value, ast.Attribute) + and node.value.attr in parked + and isinstance(node.value.value, ast.Name) + and node.value.value.id == "self" + ): + pairs.add((rel, node.attr)) + return pairs + + @staticmethod + def _override_written_fields() -> set: + """Fields written post-publish through ``get_context().override(...)``.""" + written = set() + for path in sorted(_PACKAGE_ROOT.rglob("*.py")): + rel = path.relative_to(_PACKAGE_ROOT).as_posix() + try: + tree = ast.parse(path.read_text(encoding="utf-8-sig")) + except SyntaxError: + raise AssertionError(f"unparsable module in the census: {rel}") + for node in ast.walk(tree): + if not ( + isinstance(node, ast.Call) + and isinstance(node.func, ast.Attribute) + and node.func.attr == "override" + ): + continue + base = node.func.value + if ( + isinstance(base, ast.Call) + and isinstance(base.func, ast.Name) + and base.func.id == "get_context" + ): + for kw in node.keywords: + if kw.arg == "source": + # Override metadata, not a config field. + continue + if kw.arg: + written.add(kw.arg) + else: + written |= _expanded_override_keys(rel, tree, node, kw) + return written + + def test_the_post_publish_override_surface_matches_the_pinned_list(self): + written = self._override_written_fields() + self.assertGreater( + len(written), 5, "found almost no override targets; the scan broke" + ) + found = {pair for pair in self._supplied_instance_reads() if pair[1] in written} + new = sorted(found - _OVERRIDDEN_AND_READ) + gone = sorted(_OVERRIDDEN_AND_READ - found) + self.assertEqual( + ([], []), + (new, gone), + "the post-publish override surface drifted. A read here answers with " + "the startup value once the override lands, so a new pair needs an " + "ordering judgment: copied before any override (fine), or read after " + "one (then it must come from the bags).\n" + f" new: {new}\n" + f" gone (delete from _OVERRIDDEN_AND_READ): {gone}", + ) + + def test_the_exposed_set_matches_the_pinned_list(self): + import torch + + written = self._resolution_written_fields() + found = {pair for pair in self._supplied_instance_reads() if pair[1] in written} + expected = set(_EXPOSED) + if torch.cuda.is_available(): + expected |= _EXPOSED_CUDA_ONLY + new = sorted(found - expected) + gone = sorted(expected - found) + self.assertEqual( + ([], []), + (new, gone), + "the supplied-instance step-12 surface drifted.\n" + f" new (decide where the resolved value comes from): {new}\n" + f" gone (delete from _EXPOSED / _EXPOSED_CUDA_ONLY): {gone}", + ) + + +if __name__ == "__main__": + unittest.main()