config: retire the last process-global config field reads (#33338)
`get_server_args().<field>` reads one process's startup record. Nine sites still did that for a value that has a namespace: the attention backend (5), `skip_tokenizer_init` (2), the draft-aware `load_format`, and a chunked-prefill size in `sglang.kernels`. They now read `get_exec().kernel` / `get_serving()` / `get_model()` / `get_schedule()`, so they see the resolved value including post-publish overrides. The multimodal processor's device selection moves to the instance it was constructed with rather than to a namespace: `base_gpu_id` differs per worker (the encode-server DP workers each specialise their own copy), so no process-global value can stand in for it, and engines sharing a tokenizer process each need their own. Branch order, the NPU preprocess patches, and the case that leaves "device" unset are unchanged. What stays on `get_server_args()` is the derived API — `@property` and method members computed from several fields plus the HF config (`mamba_cache_chunk_size`, `get_model_config()`, `enable_mamba_extra_buffer*`) — plus three config-intent reads of live-shadowed sizes, each of which needs an answer the live topology property cannot give (the DSA indexer's PP gate must short-circuit before touching the PP group, `allocation`'s DCP gate asks whether DCP was configured at all, and the CUDA-IPC recycler runs where no group exists). A new AST ratchet pins both shapes it can see — the direct call and an alias bound from it in the same function — at 0 and 12 respectively, exempting the derived APIs and those three sites by name. The alias-form baseline is not zero: those reads are mostly per-runner fields in model code, and lowering them is the next slice. Two fixtures stopped faking config: `test_dllm_fdfo_kv_reuse` rebound `allocation.get_server_args` to a SimpleNamespace, which silently stops intercepting the moment a reader migrates; it publishes a real config instead.
This commit is contained in:
@@ -69,10 +69,10 @@ resolved configuration lives in the namespace bags.**
|
||||
- **Per-instance boundaries** — the tokenizer-manager family, everything under
|
||||
`entrypoints/`, and the tokenizer-process multimodal processors read
|
||||
`self.server_args`: several `Engine`s can share one process, and the process-global
|
||||
bags are last-publish-wins across engines. (The mm-processor boundary is not yet
|
||||
airtight: `BaseMultimodalProcessor.process_mm_data` still reads `base_gpu_id` /
|
||||
`rl_on_policy_target` through `get_server_args()` — a known last-publish-wins gap,
|
||||
not a pattern to copy.)
|
||||
bags are last-publish-wins across engines. `base_gpu_id` also differs per worker
|
||||
(the encode-server DP workers each specialize their own copy), so no process-global
|
||||
value can stand in for it — `BaseMultimodalProcessor._fast_image_processor_device`
|
||||
is the shape to copy.
|
||||
- **Whole-object passes** (`f(server_args)` handing the instance along) keep the
|
||||
supplied-instance contract; don't rewrite the parameter reads to bag reads unless the
|
||||
field is runtime-mutated (see the elastic-EP `ep_size` case in
|
||||
|
||||
Reference in New Issue
Block a user