|
|
@@ -15,7 +15,7 @@ One container owns process-static runtime state: `sglang.srt.runtime_context.Run
|
|
|
|
| runtime flags | `get_flags()` | state that is *not* a pure function of config: `capture` (cuda-graph lifecycle), `moe` (ACTIVE backends, swappable), `dp` (DP-attention runtime flags) | materialized at subsystem init; groups offer `override()` for tests |
|
|
|
|
| runtime flags | `get_flags()` | state that is *not* a pure function of config: `capture` (cuda-graph lifecycle), `moe` (ACTIVE backends, swappable), `dp` (DP-attention runtime flags) | materialized at subsystem init; groups offer `override()` for tests |
|
|
|
|
| resources | `get_resources()`, `get_stream(name)`, `get_buffer(name, factory)` | process-level handles: graph pools, EPLB state, EP dispatcher state, named side streams, workspace buffers | lazy; cleared by `reset_context()` |
|
|
|
|
| resources | `get_resources()`, `get_stream(name)`, `get_buffer(name, factory)` | process-level handles: graph pools, EPLB state, EP dispatcher state, named side streams, workspace buffers | lazy; cleared by `reset_context()` |
|
|
|
|
| per-forward | `get_forward()` | forward-scoped flags (multi-stream switch, MoE output buffer, attn-TP inputs, extend-in-batch) | contextvar-backed; `scoped(**kw)` restores on exit; new threads see defaults |
|
|
|
|
| per-forward | `get_forward()` | forward-scoped flags (multi-stream switch, MoE output buffer, attn-TP inputs, extend-in-batch) | contextvar-backed; `scoped(**kw)` restores on exit; new threads see defaults |
|
|
|
|
| parallel | `get_parallel()` | one spelling per name: ranks and group handles are the live topology (`@property`, read-through); every other name, sizes included, is a leaf of the parallel config bag | ranks/groups: after dist init; leaves: after publish |
|
|
|
|
| parallel | `get_parallel()` | one spelling per name: scoped override, then permanent stamp, then published parallel config leaf | ranks: stamped at publish; group handles: stamped at distributed init; configured and derived widths: after publish |
|
|
|
|
|
|
|
|
|
|
|
|
`reset_context()` (unit-test teardown) drops the published config and installs fresh
|
|
|
|
`reset_context()` (unit-test teardown) drops the published config and installs fresh
|
|
|
|
flags/resources/forward tiers.
|
|
|
|
flags/resources/forward tiers.
|
|
|
@@ -87,13 +87,12 @@ with what the operator typed, not with what resolution decided.**
|
|
|
|
`ModelRunner.update_model_fields` overrides `model_path` / `load_format` for
|
|
|
|
`ModelRunner.update_model_fields` overrides `model_path` / `load_format` for
|
|
|
|
the target runner.
|
|
|
|
the target runner.
|
|
|
|
- **Late launcher-stage resolution (pre-publish)**: a few rules cannot run inside
|
|
|
|
- **Late launcher-stage resolution (pre-publish)**: a few rules cannot run inside
|
|
|
|
`__post_init__` — LoRA normalization, and the auto-parser detection that needs a
|
|
|
|
`ServerArgs.resolve_once` — LoRA normalization, and the auto-parser detection that needs a
|
|
|
|
tokenizer/chat-template load. They are resolution, not mutation, and they
|
|
|
|
tokenizer/chat-template load. They are resolution, not mutation, and they
|
|
|
|
**declare** via `arg_groups.overrides.declare_resolution(server_args, source,
|
|
|
|
**declare** via `arg_groups.overrides.declare_resolution(server_args, source,
|
|
|
|
**fields)`, the same call the rest of the pipeline makes; there is no
|
|
|
|
**fields)`, the same call the rest of the pipeline makes; there is no
|
|
|
|
`declare_late_resolution` any more. *When* a declaration is made is not
|
|
|
|
`declare_late_resolution` any more. *When* a declaration is made is not
|
|
|
|
something the code marks — the guardrails that used to read that marker
|
|
|
|
something the code marks. The declaration lands
|
|
|
|
cover these sites through the ordinary keyword scan instead. The declaration lands
|
|
|
|
|
|
|
|
in the stash on that very object, so every holder of it carries the decision —
|
|
|
|
in the stash on that very object, so every holder of it carries the decision —
|
|
|
|
the HTTP server, the multi-tokenizer workers it is serialized for, the
|
|
|
|
the HTTP server, the multi-tokenizer workers it is serialized for, the
|
|
|
|
schedulers it forks — and each of them publishes bags projected from it. The
|
|
|
|
schedulers it forks — and each of them publishes bags projected from it. The
|
|
|
@@ -185,8 +184,8 @@ bag to override at all.
|
|
|
|
|
|
|
|
|
|
|
|
### Four ways a config sweep breaks something no test runs
|
|
|
|
### Four ways a config sweep breaks something no test runs
|
|
|
|
|
|
|
|
|
|
|
|
Each of these shipped in a review round and cost a real defect; each now has a
|
|
|
|
Each of these shipped in a review round and cost a real defect. Where a test
|
|
|
|
guard, named here so the next sweep checks the same four things by hand first.
|
|
|
|
guards one it is named; the next sweep checks all four by hand first.
|
|
|
|
|
|
|
|
|
|
|
|
1. **The other implementations of an interface.** Dropping a parameter means
|
|
|
|
1. **The other implementations of an interface.** Dropping a parameter means
|
|
|
|
auditing implementers, not just callers: `CustomSpecAlgo` is the plugin
|
|
|
|
auditing implementers, not just callers: `CustomSpecAlgo` is the plugin
|
|
|
@@ -198,7 +197,8 @@ guard, named here so the next sweep checks the same four things by hand first.
|
|
|
|
`publish` says nothing about whether a given read runs before it. Spawned
|
|
|
|
`publish` says nothing about whether a given read runs before it. Spawned
|
|
|
|
workers (`MMEncoder` for encoder DP/TP, the Ray scheduler actor) start with
|
|
|
|
workers (`MMEncoder` for encoder DP/TP, the Ray scheduler actor) start with
|
|
|
|
an empty context, so a bag read above the publish raises only there.
|
|
|
|
an empty context, so a bag read above the publish raises only there.
|
|
|
|
Guard: `test_publish_precedes_bag_reads.py`.
|
|
|
|
Guarded on the launch path only (`test_pre_publish_readers.py`); other
|
|
|
|
|
|
|
|
entries, spawned workers included, are checked by review.
|
|
|
|
3. **The role namespace a process publishes under.** `ROLE_NAMESPACE_SETS`
|
|
|
|
3. **The role namespace a process publishes under.** `ROLE_NAMESPACE_SETS`
|
|
|
|
narrows what each role may read; the DP controller is audited for `exec`
|
|
|
|
narrows what each role may read; the DP controller is audited for `exec`
|
|
|
|
alone. A helper that reaches for another namespace passes every default-mode
|
|
|
|
alone. A helper that reaches for another namespace passes every default-mode
|
|
|
@@ -208,7 +208,9 @@ guard, named here so the next sweep checks the same four things by hand first.
|
|
|
|
means enumerating the others: HTTP, gRPC and in-process `Engine` each have
|
|
|
|
means enumerating the others: HTTP, gRPC and in-process `Engine` each have
|
|
|
|
their own server-info and model-info, and each passes its own tests while
|
|
|
|
their own server-info and model-info, and each passes its own tests while
|
|
|
|
its users lose the field.
|
|
|
|
its users lose the field.
|
|
|
|
Guard: `test_effective_state_surfaces.py`.
|
|
|
|
Guarded for server info across HTTP, both gRPC roles and the Rust frontend
|
|
|
|
|
|
|
|
(`TestServerInfoTransportParity` in `entrypoints/test_server_info.py`);
|
|
|
|
|
|
|
|
`Engine.get_server_info` and model info are checked by review.
|
|
|
|
|
|
|
|
|
|
|
|
A fifth, from the same rounds: the accessor **name itself**. Called as an
|
|
|
|
A fifth, from the same rounds: the accessor **name itself**. Called as an
|
|
|
|
object member (`manager.get_disagg()`), or shadowed by a same-named import
|
|
|
|
object member (`manager.get_disagg()`), or shadowed by a same-named import
|
|
|
@@ -230,12 +232,13 @@ slack (`>= len(...) - 1`) or compared key names instead of value sources.
|
|
|
|
|
|
|
|
|
|
|
|
### `get_parallel()`: one spelling per name
|
|
|
|
### `get_parallel()`: one spelling per name
|
|
|
|
|
|
|
|
|
|
|
|
**There is no `.config` hop.** Ranks and group handles are `@property`
|
|
|
|
**There is no `.config` hop.** `ParallelContext._read` answers every name in
|
|
|
|
read-through over the canonical getters, so they answer with the live process
|
|
|
|
one order: a scoped override, then a permanent stamp, then the published
|
|
|
|
groups. Everything else — `tp_size`, `pp_size`, `attn_cp_size`, `dcp_size`,
|
|
|
|
`parallel` bag leaf. `publish` stamps ranks from the launcher; distributed init
|
|
|
|
`moe_dp_size` included, alongside config-only leaves like `nccl_port`,
|
|
|
|
stamps group handles and validates their widths. Configured widths such as
|
|
|
|
`dp_size`, `ep_size`, `dwdp_size` — is answered from the
|
|
|
|
`tp_size`, `pp_size`, `attn_cp_size`, `dcp_size` and `moe_dp_size`, and config-only
|
|
|
|
published `parallel` bag. Reading a leaf before publish raises a `ValueError`
|
|
|
|
leaves such as `nccl_port`, `dp_size`, `ep_size` and `dwdp_size`, fall back to the
|
|
|
|
|
|
|
|
published bag. Reading a config leaf before publish raises a `ValueError`
|
|
|
|
naming the namespace; an unknown name is an `AttributeError`.
|
|
|
|
naming the namespace; an unknown name is an `AttributeError`.
|
|
|
|
|
|
|
|
|
|
|
|
A size reads from the configuration because the groups are built at exactly the
|
|
|
|
A size reads from the configuration because the groups are built at exactly the
|
|
|
@@ -252,38 +255,33 @@ configured widths — checked at every assignment to `_TP` / `_PP` / `_ATTN_CP`
|
|
|
|
coordinators keep their construction width. Those are different names, not two
|
|
|
|
coordinators keep their construction width. Those are different names, not two
|
|
|
|
answers to one name.
|
|
|
|
answers to one name.
|
|
|
|
|
|
|
|
|
|
|
|
DCP keeps its own pair: `get_parallel().attn_dcp_size` / `.dcp_enabled` answer the
|
|
|
|
DCP keeps its own pair: `get_parallel().attn_dcp_size` / `.dcp_enabled` are derived
|
|
|
|
*effective* topology (`1` / `False` with no group installed), while `dcp_size` is
|
|
|
|
from the published configuration unless explicitly overridden; `dcp_size` is
|
|
|
|
what the launch requested.
|
|
|
|
what the launch requested.
|
|
|
|
|
|
|
|
|
|
|
|
A process-global seed field-read of one of these sizes
|
|
|
|
A process-global seed field-read of one of these sizes
|
|
|
|
(`get_server_args().tp_size`, or an alias of it) is a read-ratchet failure. A
|
|
|
|
(`get_server_args().tp_size`, or an alias of it) is a read-ratchet failure. A
|
|
|
|
`server_args` the object was *handed* is a different thing and not a ratchet
|
|
|
|
`server_args` the object was *handed* is a different thing and not a ratchet
|
|
|
|
matter — see "Reads that legitimately stay on a ServerArgs instance".
|
|
|
|
matter — see "Reads that legitimately stay on a ServerArgs instance".
|
|
|
|
Fail-loud is narrower: before dist init, a live *rank/group* read raises. The six
|
|
|
|
Fail-loud is narrower: an unstamped rank or group handle with no declared default
|
|
|
|
|
|
|
|
raises; ranks are available after publish, before dist init. The six
|
|
|
|
parallel quotients are not live reads at all — `attn_tp_size`, `attn_dp_size`,
|
|
|
|
parallel quotients are not live reads at all — `attn_tp_size`, `attn_dp_size`,
|
|
|
|
`attn_dcp_size`, `moe_ep_size`, `moe_tp_size`, `dcp_enabled` are a function of the
|
|
|
|
`attn_dcp_size`, `moe_ep_size`, `moe_tp_size`, `dcp_enabled` are a function of the
|
|
|
|
configured leaves, computed once at publish into bag leaves, and answered
|
|
|
|
configured leaves, computed once at publish into bag leaves, and answered
|
|
|
|
override → stamp → published leaf. So `dcp_enabled` means "the launch configured
|
|
|
|
override → stamp → published leaf. So `dcp_enabled` means "the launch configured
|
|
|
|
DCP" (`dcp_size > 1`), not "a DCP group is installed here"; in a scheduler the
|
|
|
|
DCP" (`dcp_size > 1`), not "a DCP group is installed here". Normal distributed
|
|
|
|
stamp makes the two identical, in a process that publishes without dist init they
|
|
|
|
initialization builds the groups to match the configured widths; it does not
|
|
|
|
differ. `test_a_topology_is_stated_by_naming_the_width` and its neighbours in
|
|
|
|
restamp these DCP leaves. `test_a_topology_is_stated_by_naming_the_width` and its
|
|
|
|
`test_runtime_context.py` pin this; they replaced
|
|
|
|
neighbours in `test_runtime_context.py` pin this; they replaced
|
|
|
|
`test_attn_dcp_defaults_when_group_is_uninitialized`. One consequence for tests:
|
|
|
|
`test_attn_dcp_defaults_when_group_is_uninitialized`. One consequence for tests:
|
|
|
|
overriding a leaf no longer moves its quotient — state a topology by publishing a
|
|
|
|
overriding a leaf no longer moves its quotient — state a topology by publishing a
|
|
|
|
config, or by naming the width. After init,
|
|
|
|
config, or by naming the width. After init,
|
|
|
|
only the DCP group is optional (`_DCP` exists only when `dcp_size > 1`; attn-CP and
|
|
|
|
only the DCP group is optional (`_DCP` exists only when `dcp_size > 1`; attn-CP and
|
|
|
|
moe-DP always install, as size-1 aliases if unused). The `config` hop is
|
|
|
|
moe-DP always install, as size-1 aliases if unused). Bare parallel leaf reads are
|
|
|
|
deliberately dynamo-traceable (a plain property over a slot, no
|
|
|
|
dynamo-traceable; gate helpers like `enable_moe_dense_fully_dp()` run inside
|
|
|
|
`object.__getattribute__`); gate helpers like `enable_moe_dense_fully_dp()` run inside
|
|
|
|
|
|
|
|
compiled model forwards (`test_parallel_config_leaves_trace_under_torch_compile` pins
|
|
|
|
compiled model forwards (`test_parallel_config_leaves_trace_under_torch_compile` pins
|
|
|
|
this).
|
|
|
|
this).
|
|
|
|
|
|
|
|
|
|
|
|
A third surface carries the same names: `ParallelState` (`self.ps` / `mr.ps`), the
|
|
|
|
|
|
|
|
frozen per-process snapshot built once in `Scheduler.__init__` from these configured
|
|
|
|
|
|
|
|
sizes plus this process's ranks, and handed down (draft runners included). Prefer it
|
|
|
|
|
|
|
|
where an object was handed one; it is not a global accessor.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
### Reading config: the seed is off limits
|
|
|
|
### Reading config: the seed is off limits
|
|
|
|
|
|
|
|
|
|
|
|
`get_server_args().field` in business code is a ratchet failure. Read:
|
|
|
|
`get_server_args().field` in business code is a ratchet failure. Read:
|
|
|
@@ -329,9 +327,9 @@ where an object was handed one; it is not a global accessor.
|
|
|
|
follows a post-publish override. Two questions are *not* that, and have their
|
|
|
|
follows a post-publish override. Two questions are *not* that, and have their
|
|
|
|
own spelling: the width of the MoE communicator you are about to collectively
|
|
|
|
own spelling: the width of the MoE communicator you are about to collectively
|
|
|
|
operate on is `get_moe_cp_size()` (the `_MOE_DP = _ATTN_CP` alias makes it
|
|
|
|
operate on is `get_moe_cp_size()` (the `_MOE_DP = _ATTN_CP` alias makes it
|
|
|
|
differ), and the effective DCP topology is `get_parallel().attn_dcp_size` /
|
|
|
|
differ), and DCP's derived width and enablement are
|
|
|
|
`.dcp_enabled` (`1` / `False` when no group is installed), which does not need
|
|
|
|
`get_parallel().attn_dcp_size` / `.dcp_enabled` (published configuration
|
|
|
|
dist init to answer.
|
|
|
|
unless explicitly overridden).
|
|
|
|
- **this runner's resolved value** → the runner
|
|
|
|
- **this runner's resolved value** → the runner
|
|
|
|
(`prefill_attention_backend_str`, `kv_cache_dtype_str`,
|
|
|
|
(`prefill_attention_backend_str`, `kv_cache_dtype_str`,
|
|
|
|
`draft_attention_backend`); for what the model's layers bake in at
|
|
|
|
`draft_attention_backend`); for what the model's layers bake in at
|
|
|
@@ -343,10 +341,9 @@ per-instance boundaries and whole-object passes; there are no per-runner config
|
|
|
|
copies to read any more). The allow-list is `GrammarManager` and `MMEncoder`;
|
|
|
|
copies to read any more). The allow-list is `GrammarManager` and `MMEncoder`;
|
|
|
|
what sits beside it is residue, not a family — and not for one single reason:
|
|
|
|
what sits beside it is residue, not a family — and not for one single reason:
|
|
|
|
|
|
|
|
|
|
|
|
- the tokenizer-manager family and `entrypoints/` **read the bags**; what is
|
|
|
|
- the tokenizer-manager family and `entrypoints/` **read the bags**; review a new
|
|
|
|
left of them in the exposure ratchet is a handful of individually-dispositioned
|
|
|
|
instance read there against the raw-input contract rather than assuming a
|
|
|
|
pairs, not a family awaiting conversion. Read the ratchet for the current set
|
|
|
|
directory is off-limits;
|
|
|
|
rather than assuming a directory is off-limits;
|
|
|
|
|
|
|
|
- `GrammarManager` is a handed instance for its residual `self.server_args`
|
|
|
|
- `GrammarManager` is a handed instance for its residual `self.server_args`
|
|
|
|
reads, but backend selection is **not** on the instance any more:
|
|
|
|
reads, but backend selection is **not** on the instance any more:
|
|
|
|
`create_grammar_backend` reads `get_exec().kernel.grammar_backend`, and
|
|
|
|
`create_grammar_backend` reads `get_exec().kernel.grammar_backend`, and
|
|
|
@@ -392,15 +389,15 @@ if you do it, say so in the test.
|
|
|
|
|
|
|
|
|
|
|
|
### Mid-resolution reads (inside the pipeline only)
|
|
|
|
### Mid-resolution reads (inside the pipeline only)
|
|
|
|
|
|
|
|
|
|
|
|
Resolution runs in `__post_init__` and **writes nothing onto the record**: a
|
|
|
|
Resolution runs in `ServerArgs.resolve_once` and **writes nothing onto the record**: a
|
|
|
|
handler declares (`self._declare` / `declare_resolution`), the declaration goes
|
|
|
|
handler declares (`declare_resolution`), the declaration goes
|
|
|
|
into the stash, and the fields keep what the caller passed. So a mid-resolution
|
|
|
|
into the stash, and the fields keep what the caller passed. So a mid-resolution
|
|
|
|
read of a field answers with the *raw input* — every reader in the pipeline goes
|
|
|
|
read of a field answers with the *raw input* — every reader in the pipeline goes
|
|
|
|
through a view instead:
|
|
|
|
through a view instead:
|
|
|
|
|
|
|
|
|
|
|
|
- `resolving_view(server_args)` / `self._resolved()` — the live view (walks the
|
|
|
|
- `resolving_view(server_args)` — the live view (walks the
|
|
|
|
stash per read). This is what handlers and hooks bind, conventionally as
|
|
|
|
stash per read). This is what handlers and hooks bind, conventionally as
|
|
|
|
`cfg = resolving_view(self)` at the top of the handler.
|
|
|
|
`cfg = resolving_view(server_args)` at the top of the handler.
|
|
|
|
- `resolved_view(server_args)` — snapshots the overlay when built, which is what
|
|
|
|
- `resolved_view(server_args)` — snapshots the overlay when built, which is what
|
|
|
|
a post-process pass wants: it reads the state at *its* slot.
|
|
|
|
a post-process pass wants: it reads the state at *its* slot.
|
|
|
|
|
|
|
|
|
|
|
@@ -479,12 +476,12 @@ dataclass. A declaration against a non-whitelisted field fails at its slot.
|
|
|
|
|
|
|
|
|
|
|
|
### Load-time vs resolution-time (critical)
|
|
|
|
### Load-time vs resolution-time (critical)
|
|
|
|
|
|
|
|
|
|
|
|
`__post_init__` runs in the launcher process before any model/platform import. Logic that
|
|
|
|
`ServerArgs.resolve_once` runs in the launcher process before any model/platform import. Logic that
|
|
|
|
consults an **extensible registry** (e.g. out-of-tree platforms registering attention
|
|
|
|
consults an **extensible registry** (e.g. out-of-tree platforms registering attention
|
|
|
|
backends in `init_backend()`, which runs at `model_runner` import) must stay at load time
|
|
|
|
backends in `init_backend()`, which runs at `model_runner` import) must stay at load time
|
|
|
|
(ModelRunner init), writing through `get_context().override()`. Before moving any
|
|
|
|
(ModelRunner init), writing through `get_context().override()`. Before moving any
|
|
|
|
load-time logic into resolution, verify everything it reads is already complete at
|
|
|
|
load-time logic into resolution, verify everything it reads is already complete at
|
|
|
|
construction time.
|
|
|
|
resolution time.
|
|
|
|
|
|
|
|
|
|
|
|
## Runtime flags (`get_flags()`)
|
|
|
|
## Runtime flags (`get_flags()`)
|
|
|
|
|
|
|
|
|
|
|
@@ -556,8 +553,9 @@ ONE thread — do not design for TBO threads that don't exist.
|
|
|
|
them explicitly on the mock; `MagicMock(spec=...)` raises on attributes that only
|
|
|
|
them explicitly on the mock; `MagicMock(spec=...)` raises on attributes that only
|
|
|
|
exist post-`__init__`, which is the fastest way to find a missed stub.
|
|
|
|
exist post-`__init__`, which is the fastest way to find a missed stub.
|
|
|
|
- `reset_context()` in teardown when a test publishes outside a scoped override.
|
|
|
|
- `reset_context()` in teardown when a test publishes outside a scoped override.
|
|
|
|
- `ServerArgs(model_path="dummy")` early-returns the pipeline (few declarations, no
|
|
|
|
- `ServerArgs(model_path="dummy")` resolves through the pipeline's early return (few
|
|
|
|
strict guard) — fine for lightweight fixtures.
|
|
|
|
declarations) — fine for lightweight fixtures. The strict guard still applies once
|
|
|
|
|
|
|
|
`resolve_once` has run, as it does at `publish`.
|
|
|
|
- **Asserting what resolution decided reads `resolution_result(sa, "field")`**, not
|
|
|
|
- **Asserting what resolution decided reads `resolution_result(sa, "field")`**, not
|
|
|
|
`sa.field`: the field is the raw input. Assert the field only when the point of
|
|
|
|
`sa.field`: the field is the raw input. Assert the field only when the point of
|
|
|
|
the case *is* that the record stayed pristine (the FA4 page-size and waterfill
|
|
|
|
the case *is* that the record stayed pristine (the FA4 page-size and waterfill
|
|
|
@@ -566,7 +564,7 @@ ONE thread — do not design for TBO threads that don't exist.
|
|
|
|
pytest run lets a context published by an earlier file mask a missing-publish bug in a
|
|
|
|
pytest run lets a context published by an earlier file mask a missing-publish bug in a
|
|
|
|
later one.
|
|
|
|
later one.
|
|
|
|
|
|
|
|
|
|
|
|
## Guardrails (these fail CI; what to do when they fire)
|
|
|
|
## Guardrails and review rules
|
|
|
|
|
|
|
|
|
|
|
|
1. **Strict mutation guard** (always on, and with no exception): bare
|
|
|
|
1. **Strict mutation guard** (always on, and with no exception): bare
|
|
|
|
`server_args.x = ...` after resolution raises unconditionally in
|
|
|
|
`server_args.x = ...` after resolution raises unconditionally in
|
|
|
@@ -576,10 +574,9 @@ ONE thread — do not design for TBO threads that don't exist.
|
|
|
|
resolved config with `get_context().override`; hand a per-runner value to its
|
|
|
|
resolved config with `get_context().override`; hand a per-runner value to its
|
|
|
|
runner as a constructor argument. Projected bags are sealed the same way (leaf
|
|
|
|
runner as a constructor argument. Projected bags are sealed the same way (leaf
|
|
|
|
assignment raises).
|
|
|
|
assignment raises).
|
|
|
|
2. **Mutation ratchet** (`test_server_args_mutation_ratchet.py`, exact pin 0 over the whole
|
|
|
|
2. **No textual mutation ratchet**: the strict guard above is the only mutation
|
|
|
|
package minus the pipeline / multimodal_gen): textual scan for assignment forms. Never
|
|
|
|
check.
|
|
|
|
raise the baseline.
|
|
|
|
3. **No-copy contract** (review only, no automated guard): neither
|
|
|
|
3. **No-copy contract** (`test_server_args_no_instance_mutation_entry.py`): neither
|
|
|
|
|
|
|
|
`ServerArgs.override` nor `ServerArgs.derive` exists, and nothing in the package
|
|
|
|
`ServerArgs.override` nor `ServerArgs.derive` exists, and nothing in the package
|
|
|
|
calls either form. Rerouting a writer to the bags means flipping **all its readers
|
|
|
|
calls either form. Rerouting a writer to the bags means flipping **all its readers
|
|
|
|
in the same commit** (no transitional dual-write).
|
|
|
|
in the same commit** (no transitional dual-write).
|
|
|
@@ -590,23 +587,23 @@ ONE thread — do not design for TBO threads that don't exist.
|
|
|
|
runner's stamp — not `get_server_args().field`, which the read ratchet below pins
|
|
|
|
runner's stamp — not `get_server_args().field`, which the read ratchet below pins
|
|
|
|
at zero. `runtime_context.get_server_args()` is only for the
|
|
|
|
at zero. `runtime_context.get_server_args()` is only for the
|
|
|
|
whole-object shapes (dumps, provenance, a hand-off to a callee that takes a config).
|
|
|
|
whole-object shapes (dumps, provenance, a hand-off to a callee that takes a config).
|
|
|
|
5. **Global config read ratchet** (`test_global_config_read_ratchet.py`): baselines are
|
|
|
|
5. **Global config read check** (`test_global_config_read_ratchet.py`): no business code
|
|
|
|
**0** for both the direct `get_server_args().field` and the alias form (function-local
|
|
|
|
reads a field off `get_server_args()`, directly or through an alias (function-local
|
|
|
|
— including local copies of an alias, `cfg = sa` — module-level, or parked on an
|
|
|
|
— including local copies of an alias, `cfg = sa` — module-level, or parked on an
|
|
|
|
instance attribute, plus the `getattr(..., "field")` spelling of each; a name
|
|
|
|
instance attribute, plus the `getattr(..., "field")` spelling of each; a name
|
|
|
|
computed at runtime or indirection deeper than a local name copy is census-tool
|
|
|
|
computed at runtime or indirection deeper than a local name copy is outside the
|
|
|
|
territory, per the test's docstring). The scanner matches `get_server_args` by its
|
|
|
|
check, per the test's docstring). The scanner matches `get_server_args` by its
|
|
|
|
literal name — bare or module-qualified (`ctx.get_server_args()`) — and
|
|
|
|
literal name — bare or module-qualified (`ctx.get_server_args()`) — and
|
|
|
|
`TestNoRenamedAccessorImports` in the same file *bans* `import ... as` renames of it,
|
|
|
|
`TestNoRenamedAccessorImports` in the same file *bans* `import ... as` renames of it,
|
|
|
|
which is what makes literal-name matching sound. Exempt by owner
|
|
|
|
which is what makes literal-name matching sound. Exempt by owner
|
|
|
|
module only (`runtime_context.py`, `server_args.py`, `arg_groups/`). Two classes,
|
|
|
|
module only (`runtime_context.py`, `server_args.py`, `arg_groups/`). Two classes,
|
|
|
|
no more: `TestGlobalConfigReadRatchet` holds the two baselines and
|
|
|
|
no more: `TestGlobalConfigReads` holds the zero check and
|
|
|
|
`TestNoRenamedAccessorImports` holds the ban. There is no configured-size registry
|
|
|
|
`TestNoRenamedAccessorImports` holds the ban. There is no configured-size registry
|
|
|
|
here any longer — `get_parallel()` has one spelling per name, so a size read is not a
|
|
|
|
here any longer — `get_parallel()` has one spelling per name, so a size read is not a
|
|
|
|
choice between two answers and nothing needs registering.
|
|
|
|
choice between two answers and nothing needs registering.
|
|
|
|
6. **Module-state ratchet** (`test_module_state_ratchet.py`): `global` statements in the
|
|
|
|
6. **Module-state rule** (review only, no automated guard): a new module-level
|
|
|
|
flag-owning layers are pinned by name. A new module-level runtime global belongs on a
|
|
|
|
runtime global in the flag-owning layers belongs on a flags group / resources slot
|
|
|
|
flags group / resources slot instead; migrating a pinned survivor must shrink the pin.
|
|
|
|
instead.
|
|
|
|
7. **Namespace coverage** (`test_server_args_namespaces.py`,
|
|
|
|
7. **Namespace coverage** (`test_server_args_namespaces.py`,
|
|
|
|
`test_runtime_context_config_bags.py`): every `ServerArgs` field resolves to a
|
|
|
|
`test_runtime_context_config_bags.py`): every `ServerArgs` field resolves to a
|
|
|
|
namespace — from the `arg_groups/fields/` class that declares it — and the
|
|
|
|
namespace — from the `arg_groups/fields/` class that declares it — and the
|
|
|
@@ -662,8 +659,7 @@ Key source files: `python/sglang/srt/runtime_context.py` (the container, every t
|
|
|
|
`python/sglang/srt/arg_groups/overrides.py` (override registry, passes,
|
|
|
|
`python/sglang/srt/arg_groups/overrides.py` (override registry, passes,
|
|
|
|
`declare_resolution` and the spellings around it), `python/sglang/srt/server_args.py` (`NS` metadata,
|
|
|
|
`declare_resolution` and the spellings around it), `python/sglang/srt/server_args.py` (`NS` metadata,
|
|
|
|
`Arg(..., resolvable=True)`, `__setattr__` strict guard), and the guardrail tests under
|
|
|
|
`Arg(..., resolvable=True)`, `__setattr__` strict guard), and the guardrail tests under
|
|
|
|
`test/registered/unit/` (`test_server_args_mutation_ratchet.py`,
|
|
|
|
`test/registered/unit/` (`test_global_config_read_ratchet.py`,
|
|
|
|
`test_global_config_read_ratchet.py`,
|
|
|
|
`test_server_args_namespaces.py`,
|
|
|
|
`test_module_state_ratchet.py`, `test_server_args_namespaces.py`,
|
|
|
|
|
|
|
|
`test_runtime_context.py` — the last one doubles
|
|
|
|
`test_runtime_context.py` — the last one doubles
|
|
|
|
as executable documentation of every tier's semantics).
|
|
|
|
as executable documentation of every tier's semantics).
|
|
|
|