config: the post-publish consumers of the supplied-instance surface read the bags
config: the speculative workers take page_size from the bags Seven worker constructors stored `self.page_size = server_args.page_size` off the handed record. They all run after publish and all keep a copy of a process-level value, which is the first row of the plan doc's supplied-instance disposition table -- so they read `get_schedule().page_size`, and a post-publish override now reaches them like it reaches every other consumer. The supplied-instance census named the seven pairs; the exposure ratchet in the next member pins what remains after this batch of conversions. config: the post-publish chunked_prefill_size consumers read the bags Four of the ten supplied-instance `chunked_prefill_size` reads are plain post-publish consumers -- the EPLB recorder's buffer sizing, the deep-gemm compile warmup (five reads), the KV-cache builder's effective size, and the ngram embedding manager's assert. All are reached from runner init, so they read `get_schedule()`. Two are deliberately left: `create_kt_config_from_server_args` builds a config *from a supplied record* by name and contract, and `CanaryLaunchCapacities.from_args` is the same shape. Converting those would change what the function is, not where it reads -- the plan doc's disposition table says so per field. config: the remaining post-publish graph/limit consumers read the bags Three more of the census's supplied-instance debts are plain post-publish reads: the dspark worker's cuda-graph decode sizes, the dspark planner's SPS table bound (`max_running_requests`), and the LoRA manager's cuda-graph moe buffers. The dspark worker is the clearest of them -- it already read `get_exec().graph.cuda_graph_config.decode.bs` thirty lines below the instance read, so the file disagreed with itself about where the same value comes from. Left where the function's contract is "build a config from the record you are handed" rather than "read this process's config": `create_kt_config_from_server_args`, `DllmConfig.from_server_args`, `CanaryLaunchCapacities.from_args`, `build_compilation_config`. Changing those would change what the function is. config: the runner, scheduler and offload manager take page_size from the bags The same `self.page_size = server_args.page_size` shape as the speculative workers, in the three remaining process-owned constructors: `ModelRunner`, `Scheduler`, and the decode-side KV offload manager. The scheduler process publishes before any of them run. The one path that did not is `ModelRunner` constructed standalone -- `python -m sglang.benchmark.one_batch` and the manual runner tests build it with no prior publish, and the constructor's own publish sat below this read -- so that publish moves above the constructor's first bag read instead of leaving a window where the runner half-exists unpublished. Left where the read belongs to something else: `utils/common`'s predicates are called only from the resolution pipeline with a `resolved_view`, `allocation_sizing` takes the config its callers supply by contract, and `CudaVmmFeatureTransport` is tokenizer-owned -- one per tokenizer worker, which is the per-instance boundary. The conversion left the offload manager parking a record it no longer reads; the parked copy goes with the read (the constructor parameter stays -- its hicache sizing still reads it directly).
This commit is contained in:
@@ -1,7 +1,6 @@
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
|
||||
from sglang.srt.speculative.dspark_components.dspark_sps import (
|
||||
SpsAdditiveCostTable,
|
||||
@@ -143,22 +142,30 @@ class TestProfileSpsTable(CustomTestCase):
|
||||
self.assertEqual(table.max_batch_tokens, 256)
|
||||
|
||||
|
||||
def _build_sps_cost_table_for(*, sps_table_path):
|
||||
def _build_sps_cost_table_for(testcase, *, sps_table_path):
|
||||
from sglang.srt.runtime_context import get_context, get_server_args
|
||||
from sglang.srt.speculative.dspark_components.dspark_planner import (
|
||||
build_sps_cost_table,
|
||||
)
|
||||
|
||||
server_args = SimpleNamespace(
|
||||
# The table bound reads `max_running_requests` from the published bags, so
|
||||
# the case publishes it; the table path stays on the handed record, which is
|
||||
# what `build_sps_cost_table` takes.
|
||||
override = get_context().override_server_args(
|
||||
speculative_dspark_sps_table_path=sps_table_path,
|
||||
max_running_requests=4,
|
||||
)
|
||||
return build_sps_cost_table(server_args=server_args, verify_num_draft_tokens=5)
|
||||
override.install()
|
||||
testcase.addCleanup(override.restore)
|
||||
return build_sps_cost_table(
|
||||
server_args=get_server_args(), verify_num_draft_tokens=5
|
||||
)
|
||||
|
||||
|
||||
class TestBuildSpsCostTableContract(CustomTestCase):
|
||||
def test_unset_table_path_returns_flat_table(self):
|
||||
for sps_table_path in (None, ""):
|
||||
table = _build_sps_cost_table_for(sps_table_path=sps_table_path)
|
||||
table = _build_sps_cost_table_for(self, sps_table_path=sps_table_path)
|
||||
self.assertEqual(table.sample_batch_tokens, [1])
|
||||
self.assertEqual(table.sample_steps_per_sec, [1.0])
|
||||
self.assertEqual(table.max_batch_tokens, 20)
|
||||
@@ -168,7 +175,7 @@ class TestBuildSpsCostTableContract(CustomTestCase):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
path = Path(tmp) / "sps.json"
|
||||
path.write_text(table.to_json(), encoding="utf-8")
|
||||
loaded = _build_sps_cost_table_for(sps_table_path=str(path))
|
||||
loaded = _build_sps_cost_table_for(self, sps_table_path=str(path))
|
||||
self.assertEqual(loaded.sample_batch_tokens, table.sample_batch_tokens)
|
||||
self.assertEqual(loaded.sample_steps_per_sec, table.sample_steps_per_sec)
|
||||
self.assertEqual(loaded.max_batch_tokens, table.max_batch_tokens)
|
||||
|
||||
Reference in New Issue
Block a user