config: retire the last process-global config field reads (#33338)
`get_server_args().<field>` reads one process's startup record. Nine sites still did that for a value that has a namespace: the attention backend (5), `skip_tokenizer_init` (2), the draft-aware `load_format`, and a chunked-prefill size in `sglang.kernels`. They now read `get_exec().kernel` / `get_serving()` / `get_model()` / `get_schedule()`, so they see the resolved value including post-publish overrides. The multimodal processor's device selection moves to the instance it was constructed with rather than to a namespace: `base_gpu_id` differs per worker (the encode-server DP workers each specialise their own copy), so no process-global value can stand in for it, and engines sharing a tokenizer process each need their own. Branch order, the NPU preprocess patches, and the case that leaves "device" unset are unchanged. What stays on `get_server_args()` is the derived API — `@property` and method members computed from several fields plus the HF config (`mamba_cache_chunk_size`, `get_model_config()`, `enable_mamba_extra_buffer*`) — plus three config-intent reads of live-shadowed sizes, each of which needs an answer the live topology property cannot give (the DSA indexer's PP gate must short-circuit before touching the PP group, `allocation`'s DCP gate asks whether DCP was configured at all, and the CUDA-IPC recycler runs where no group exists). A new AST ratchet pins both shapes it can see — the direct call and an alias bound from it in the same function — at 0 and 12 respectively, exempting the derived APIs and those three sites by name. The alias-form baseline is not zero: those reads are mostly per-runner fields in model code, and lowering them is the next slice. Two fixtures stopped faking config: `test_dllm_fdfo_kv_reuse` rebound `allocation.get_server_args` to a SimpleNamespace, which silently stops intercepting the moment a reader migrates; it publishes a real config instead.
This commit is contained in:
@@ -7,9 +7,9 @@ from types import SimpleNamespace
|
||||
import torch
|
||||
|
||||
from sglang.srt.dllm.mixin.scheduler import DllmManager
|
||||
from sglang.srt.mem_cache import allocation
|
||||
from sglang.srt.mem_cache.allocation import alloc_for_extend
|
||||
from sglang.srt.mem_cache.memory_pool import ReqToTokenPool
|
||||
from sglang.srt.runtime_context import get_context
|
||||
from sglang.test.ci.ci_register import register_cpu_ci
|
||||
|
||||
register_cpu_ci(est_time=5, suite="base-a-test-cpu")
|
||||
@@ -117,16 +117,11 @@ class TestDllmFdfoKvReuse(unittest.TestCase):
|
||||
self.pool = ReqToTokenPool(
|
||||
size=8, max_context_len=64, device="cpu", enable_memory_saver=False
|
||||
)
|
||||
self._old_support_triton = allocation.support_triton
|
||||
self._old_get_server_args = allocation.get_server_args
|
||||
allocation.support_triton = lambda _: False
|
||||
allocation.get_server_args = lambda: SimpleNamespace(
|
||||
override = get_context().override_server_args(
|
||||
attention_backend="torch_native", dcp_size=1
|
||||
)
|
||||
|
||||
def tearDown(self):
|
||||
allocation.support_triton = self._old_support_triton
|
||||
allocation.get_server_args = self._old_get_server_args
|
||||
override.install()
|
||||
self.addCleanup(override.restore)
|
||||
|
||||
def test_alloc_for_extend_mixed_reuse_allocates_only_fresh_and_writes_rows(self):
|
||||
allocator = _FakeAllocator(base=200)
|
||||
|
||||
Reference in New Issue
Block a user