config: retire the last process-global config field reads (#33338)

`get_server_args().<field>` reads one process's startup record. Nine sites still
did that for a value that has a namespace: the attention backend (5),
`skip_tokenizer_init` (2), the draft-aware `load_format`, and a chunked-prefill
size in `sglang.kernels`. They now read `get_exec().kernel` / `get_serving()` /
`get_model()` / `get_schedule()`, so they see the resolved value including
post-publish overrides.

The multimodal processor's device selection moves to the instance it was
constructed with rather than to a namespace: `base_gpu_id` differs per worker (the
encode-server DP workers each specialise their own copy), so no process-global
value can stand in for it, and engines sharing a tokenizer process each need their
own. Branch order, the NPU preprocess patches, and the case that leaves "device"
unset are unchanged.

What stays on `get_server_args()` is the derived API — `@property` and method
members computed from several fields plus the HF config
(`mamba_cache_chunk_size`, `get_model_config()`, `enable_mamba_extra_buffer*`) —
plus three config-intent reads of live-shadowed sizes, each of which needs an
answer the live topology property cannot give (the DSA indexer's PP gate must
short-circuit before touching the PP group, `allocation`'s DCP gate asks whether
DCP was configured at all, and the CUDA-IPC recycler runs where no group exists).

A new AST ratchet pins both shapes it can see — the direct call and an alias
bound from it in the same function — at 0 and 12 respectively, exempting the
derived APIs and those three sites by name. The alias-form baseline is not zero:
those reads are mostly per-runner fields in model code, and lowering them is the
next slice.

Two fixtures stopped faking config: `test_dllm_fdfo_kv_reuse` rebound
`allocation.get_server_args` to a SimpleNamespace, which silently stops
intercepting the moment a reader migrates; it publishes a real config instead.
This commit is contained in:
Cheng Wan
2026-08-02 21:24:42 -07:00
committed by GitHub
parent aa3bbbc6e8
commit b8109b5d63
13 changed files with 307 additions and 57 deletions
@@ -7,9 +7,9 @@ from types import SimpleNamespace
import torch
from sglang.srt.dllm.mixin.scheduler import DllmManager
from sglang.srt.mem_cache import allocation
from sglang.srt.mem_cache.allocation import alloc_for_extend
from sglang.srt.mem_cache.memory_pool import ReqToTokenPool
from sglang.srt.runtime_context import get_context
from sglang.test.ci.ci_register import register_cpu_ci
register_cpu_ci(est_time=5, suite="base-a-test-cpu")
@@ -117,16 +117,11 @@ class TestDllmFdfoKvReuse(unittest.TestCase):
self.pool = ReqToTokenPool(
size=8, max_context_len=64, device="cpu", enable_memory_saver=False
)
self._old_support_triton = allocation.support_triton
self._old_get_server_args = allocation.get_server_args
allocation.support_triton = lambda _: False
allocation.get_server_args = lambda: SimpleNamespace(
override = get_context().override_server_args(
attention_backend="torch_native", dcp_size=1
)
def tearDown(self):
allocation.support_triton = self._old_support_triton
allocation.get_server_args = self._old_get_server_args
override.install()
self.addCleanup(override.restore)
def test_alloc_for_extend_mixed_reuse_allocates_only_fresh_and_writes_rows(self):
allocator = _FakeAllocator(base=200)