config: pin the step-12 debt on the supplied-instance surface

A callee that takes `server_args` keeps the supplied-instance contract, so no
ratchet counts its reads -- and that is right for the *object*. What it does not
cover is what the object will carry after step 12: the instance stays at the
user's raw input, so a callee reading a field resolution fills in starts seeing
the CLI default instead of the effective value.

Measured, not guessed: **297 distinct file/field pairs read one of the 125
fields resolution can write** -- what remains after the earlier members'
conversions (the census found 314 across the package; the page_size,
chunked_prefill_size and graph/limit families were converted by the members
below this one, so the list lands at the remaining debt with no churn). The
census counts three spellings of the read: `server_args.field` off the
parameter, `getattr(server_args, "field", default)` with a literal name, and
the *parked* form -- `self.x = server_args` in a method that takes the
parameter, read as `self.x.field` anywhere in the class. Parking under a
different object, a container, or a computed name stays invisible, like in
every census of this family.

The written-field set is derived in-test from resolved configs against the
dataclass defaults -- the same matrix the context repo's audit tool uses -- and
the union is only as complete as the matrix: fields a matrix entry passes in
are excluded, so each entry must let resolution make the decision the entry is
about. The DWDP shape is the loudest example: `_handle_dwdp` writes `dp_size`,
`enable_dp_attention`, `ep_size` and friends itself, and without a
`{tp_size: 2, dwdp_size: 2}` entry the whole DP/EP topology family (37 pairs
across the launcher, the controllers, the tokenizer and the spec workers)
never entered the written set at all. Multi-item scoring is the same shape in
miniature -- `_handle_multi_item_scoring` writes `disable_radix_cache` itself,
and the `{enable_mis, attention_backend=flashinfer}` entry (the backend is
passed because the handler asserts rather than switches) pins the radix-cache
builder and its friends.

The written set carries **may-write semantics**, statically collected from
every mechanism that can put a resolved value on the record, unioned with the
matrix (construct-and-diff still catches value-level writes statics cannot
name). The enumeration approach kept losing to review -- the DFLASH hook hole,
then the declarative registry (`MODEL_OVERRIDES` forces dtype for two arches
through a setattr applier no assignment scan sees), then the deprecated-alias
loop that writes through a name tuple -- because each round found one more
*mechanism*, not one more field. So all of them are collected now: hook
assignments under `arg_groups/`, the record's own method assignments (the
mooncake layout rewrite, the deepseek-EP mode defaults, the seed fill that
only fires when the caller did NOT supply one -- which construct-and-diff can
never see, since measuring requires supplying), the declarative override
registry, the alias-normalization tuple (drift-guarded), and the
late-resolution keywords. A statically-collected write site that can never
fire is a dead branch to delete upstream, not a reason to shrink the census
(maintainer's ruling). And the pin is split by host: `_EXPOSED` is asserted everywhere,
`_EXPOSED_CUDA_ONLY` (empty today) is where a capability-gated write's
readers go -- one shared exact list cannot hold such a pair at all, since
pinning it fails CPU as "gone" and omitting it fails CUDA as "new".

The resolve-once shape also undercounts on axes a construction call never
sees (each of these was a review catch, and each added pinned families):
resolution branches on the *environment*, so every matrix entry resolves under
the plain env and under the CI shape (`SGLANG_IS_IN_CI`), with the pristine
process state (environ plus the `EnvField` descriptor flags) restored between
entries -- that is where `soft_watchdog_timeout`'s four readers come from. Some
fields resolve *late*, at validation rather than construction
(`declare_late_resolution`): those writers are collected statically by keyword
-- `lora_paths`, `reasoning_parser`, `tool_call_parser` -- with the one dynamic
`**detected` site spelled out in a table guarded against drift. Fields holding
only a `default_factory` are materialized rather than skipped, and
`tokenizer_path` / `served_model_name` -- always filled from `model_path` --
leave the passed-inputs exemption and pin their twelve readers. A module the
census cannot parse fails the test instead of shrinking it, which immediately
caught a BOM-carrying file every previous census had silently skipped (the
scans read `utf-8-sig` now). And the test registers on the CUDA runner besides
the CPU suite, because capability-gated writes only open on real hardware;
AMD is intentionally not registered -- an exact pin cannot be verified from
any pinning host -- with the reasoning in the header.

**A second axis is already wrong today**, independent of step 12. Some config is
decided after publish and recorded with `get_context().override(...)` -- elastic
EP resizing `ep_size`, a weight update rewriting `model_path` / `load_format`,
HiCache attach naming a storage backend, adaptive speculative decoding moving
`speculative_num_steps`. That write reaches the bags and never the record, so a
supplied-instance read of one of those fields answers with the startup value from
the moment the override lands. **73 pairs over 13 fields** are in that position -- including the overrides
that arrive as `**kwargs`: the collector statically resolves dict-literal
expansions (the HiCache attach shape, whose write/read pairs on
`hicache_write_policy` / `hicache_storage_prefetch_policy` were invisible
before) and fails loudly on anything it cannot resolve, with
`update_server_args` exempted by name because its key set is the API's
caller's, not this file's.
Whether each is a defect depends on ordering -- a value copied at construction,
before any override, is fine -- so the axis is pinned as a measurement with the
same growth guard, not as a list of bugs.

One of them *was* a defect and is fixed at the base of this stack: the
linear-attn dispatch table rebuilt itself from the record after the SM100 GDN
prefill decision had been recorded in the bag, so a second runner's rebuild
dropped it. That choice is a per-runner stamp now and is not recorded
process-wide at all, which is why neither the read nor the field appears on this
axis.

The list is pinned both ways, on both axes. A new pair fails, because the moment
to decide where a resolved value comes from is when the read is written, not
during the flip; a disappeared pair fails too, naming the entry to delete, so the
registry stays a measurement rather than a memory of one. Both axes
reverse-verified: a new read of a written field is reported by file and field.

Per-field dispositions live in the plan doc; several of these are "should this
callee take a config at all?", which is a design call rather than a sweep.


test(step-12): tripwire on the EPD guard that a raw record would silence

`_reject_missing_dispatched_encoder_embedding` is one of the two reads the
step-12 audit calls a blocker: it keys on `encoder_transfer_backend`, a field
resolution fills in, off a handed record. Today that record carries the
resolved value; after the flip it stays at the argument default `"auto"`
(`ENCODER_TRANSFER_BACKEND_CHOICES[0]`) for every auto-resolved launch and the
503 stops firing -- a guard that goes quiet, which no existing case notices.

The tripwire resolves a real language-only Kimi-K3 TP2 launch (a mini config,
the shape whose auto pick is `"zmq_to_tokenizer"`) and asserts the guard
rejects with the record resolution produced. A fixed double cannot trip on the
flip -- it would keep handing the guard the resolved value by construction --
so the record has to come from resolution itself: when step 12 lands, this
same launch hands the guard `"auto"`, the rejection silently stops, and this
test fails, which is exactly the signal that this reader needs the resolved
value from somewhere else (the per-engine overlay or the bag).

The launch pins `mamba_radix_cache_strategy=no_buffer` (+ the overlap-off it
requires): resolution's hybrid state-cache sizing branches on the host device
and asserts a GPU stack for extra_buffer, which a CPU CI runner does not have,
while the guard under test reads a field independent of that branch. The case
restores env *and* the EnvField descriptor flags -- a real resolution leaves
state os.environ does not carry.
This commit is contained in:
Cheng Wan
2026-08-15 00:40:07 -07:00
committed by GitHub
parent 1ab713c334
commit d804b6bd98
2 changed files with 1269 additions and 0 deletions
@@ -86,6 +86,110 @@ def test_epd_language_only_rejects_missing_dispatched_embedding():
assert getattr(exc_info.value, "status_code", None) == 503
def test_epd_rejection_reads_the_resolved_transfer_backend():
"""Tripwire for step 12: this guard fires on the *resolved* backend.
The record is produced by actual resolution -- a language-only Kimi-K3
launch at TP2, whose `encoder_transfer_backend` starts at the argument
default `"auto"` (`ENCODER_TRANSFER_BACKEND_CHOICES[0]`) and is filled in
by `resolve_encoder_transfer_backend` to `"zmq_to_tokenizer"`. Today the
guard therefore rejects. When step 12 makes the instance raw, this same
launch hands the guard a record still at `"auto"`, the rejection silently
stops, and *this test fails* -- which is the signal to give this reader
the resolved value (per-engine overlay or bag) rather than the record.
Fixed doubles cannot trip on that change, so the record here must come
from resolution, not a SimpleNamespace.
"""
import json
import os
import shutil
import tempfile
from sglang.srt.server_args import ServerArgs
def env_field_flags():
from sglang.srt.environ import EnvField, envs
return {
name: field._set_to_none
for klass in reversed(type(envs).__mro__)
for name, field in vars(klass).items()
if isinstance(field, EnvField)
}
config_dir = tempfile.mkdtemp(prefix="epd_tripwire_")
try:
payload = {
"architectures": ["KimiK3ForConditionalGeneration"],
"model_type": "kimi_k3",
"text_config": {
"architectures": ["DeepseekV3ForCausalLM"],
"model_type": "deepseek_v3",
"hidden_size": 16,
"intermediate_size": 32,
"moe_intermediate_size": 32,
"num_attention_heads": 2,
"num_key_value_heads": 2,
"num_hidden_layers": 2,
"n_routed_experts": 8,
"n_shared_experts": 1,
"num_experts_per_tok": 2,
"first_k_dense_replace": 1,
"vocab_size": 128,
"max_position_embeddings": 2048,
"kv_lora_rank": 8,
"q_lora_rank": 8,
"qk_nope_head_dim": 8,
"qk_rope_head_dim": 8,
"v_head_dim": 8,
"topk_method": "greedy",
"scoring_func": "softmax",
},
"vision_config": {
"model_type": "kimi_k3_vision",
"hidden_size": 16,
"num_heads": 2,
"depth": 2,
"patch_size": 14,
"merge_kernel_size": [2, 2],
},
}
with open(os.path.join(config_dir, "config.json"), "w") as handle:
json.dump(payload, handle)
environ_before = dict(os.environ)
flags_before = env_field_flags()
try:
resolved = ServerArgs(
model_path=config_dir,
device="cuda",
random_seed=42,
language_only=True,
tp_size=2,
# Resolution branches on the host device for the hybrid
# state-cache sizing (extra_buffer asserts a GPU stack, which
# the CPU CI runner does not have); the guard under test reads
# `encoder_transfer_backend`, independent of that branch, so
# pin the strategy every host can resolve.
mamba_radix_cache_strategy="no_buffer",
disable_overlap_schedule=True,
)
finally:
os.environ.clear()
os.environ.update(environ_before)
from sglang.srt.environ import envs
for name, was_none in flags_before.items():
getattr(type(envs), name)._set_to_none = was_none
finally:
shutil.rmtree(config_dir, ignore_errors=True)
assert resolved.encoder_transfer_backend == "zmq_to_tokenizer"
request = SimpleNamespace(need_wait_for_mm_inputs=True)
with pytest.raises(HTTPException) as exc_info:
_reject_missing_dispatched_encoder_embedding(resolved, request, None)
assert getattr(exc_info.value, "status_code", None) == 503
def test_epd_allows_local_processing_when_request_was_not_dispatched():
server_args = SimpleNamespace(
language_only=True,
File diff suppressed because it is too large Load Diff