config: the post-publish consumers of the supplied-instance surface read the bags

config: the speculative workers take page_size from the bags

Seven worker constructors stored `self.page_size = server_args.page_size` off
the handed record. They all run after publish and all keep a copy of a
process-level value, which is the first row of the plan doc's supplied-instance
disposition table -- so they read `get_schedule().page_size`, and a post-publish
override now reaches them like it reaches every other consumer.

The supplied-instance census named the seven pairs; the exposure ratchet in the
next member pins what remains after this batch of conversions.


config: the post-publish chunked_prefill_size consumers read the bags

Four of the ten supplied-instance `chunked_prefill_size` reads are plain
post-publish consumers -- the EPLB recorder's buffer sizing, the deep-gemm
compile warmup (five reads), the KV-cache builder's effective size, and the
ngram embedding manager's assert. All are reached from runner init, so they read
`get_schedule()`.

Two are deliberately left: `create_kt_config_from_server_args` builds a config
*from a supplied record* by name and contract, and `CanaryLaunchCapacities.from_args`
is the same shape. Converting those would change what the function is, not where
it reads -- the plan doc's disposition table says so per field.


config: the remaining post-publish graph/limit consumers read the bags

Three more of the census's supplied-instance debts are plain post-publish reads: the dspark worker's
cuda-graph decode sizes, the dspark planner's SPS table bound
(`max_running_requests`), and the LoRA manager's cuda-graph moe buffers. The
dspark worker is the clearest of them -- it already read
`get_exec().graph.cuda_graph_config.decode.bs` thirty lines below the instance
read, so the file disagreed with itself about where the same value comes from.

Left where the function's contract is "build a config from the record you are
handed" rather than "read this process's config":
`create_kt_config_from_server_args`, `DllmConfig.from_server_args`,
`CanaryLaunchCapacities.from_args`, `build_compilation_config`. Changing those
would change what the function is.


config: the runner, scheduler and offload manager take page_size from the bags

The same `self.page_size = server_args.page_size` shape as the speculative
workers, in the three remaining process-owned constructors: `ModelRunner`,
`Scheduler`, and the decode-side KV offload manager. The scheduler process
publishes before any of them run. The one path that did not is `ModelRunner`
constructed standalone -- `python -m sglang.benchmark.one_batch` and the manual
runner tests build it with no prior publish, and the constructor's own publish
sat below this read -- so that publish moves above the constructor's first bag
read instead of leaving a window where the runner half-exists unpublished.

Left where the read belongs to something else: `utils/common`'s predicates are
called only from the resolution pipeline with a `resolved_view`,
`allocation_sizing` takes the config its callers supply by contract, and
`CudaVmmFeatureTransport` is tokenizer-owned -- one per tokenizer worker, which
is the per-instance boundary.

The conversion left the offload manager parking a record it no longer
reads; the parked copy goes with the read (the constructor parameter stays
-- its hicache sizing still reads it directly).
This commit is contained in:
Cheng Wan
2026-08-15 00:39:35 -07:00
committed by GitHub
parent f2ab6e306b
commit 1ab713c334
17 changed files with 57 additions and 40 deletions
@@ -1,7 +1,6 @@
import tempfile
import unittest
from pathlib import Path
from types import SimpleNamespace
from sglang.srt.speculative.dspark_components.dspark_sps import (
SpsAdditiveCostTable,
@@ -143,22 +142,30 @@ class TestProfileSpsTable(CustomTestCase):
self.assertEqual(table.max_batch_tokens, 256)
def _build_sps_cost_table_for(*, sps_table_path):
def _build_sps_cost_table_for(testcase, *, sps_table_path):
from sglang.srt.runtime_context import get_context, get_server_args
from sglang.srt.speculative.dspark_components.dspark_planner import (
build_sps_cost_table,
)
server_args = SimpleNamespace(
# The table bound reads `max_running_requests` from the published bags, so
# the case publishes it; the table path stays on the handed record, which is
# what `build_sps_cost_table` takes.
override = get_context().override_server_args(
speculative_dspark_sps_table_path=sps_table_path,
max_running_requests=4,
)
return build_sps_cost_table(server_args=server_args, verify_num_draft_tokens=5)
override.install()
testcase.addCleanup(override.restore)
return build_sps_cost_table(
server_args=get_server_args(), verify_num_draft_tokens=5
)
class TestBuildSpsCostTableContract(CustomTestCase):
def test_unset_table_path_returns_flat_table(self):
for sps_table_path in (None, ""):
table = _build_sps_cost_table_for(sps_table_path=sps_table_path)
table = _build_sps_cost_table_for(self, sps_table_path=sps_table_path)
self.assertEqual(table.sample_batch_tokens, [1])
self.assertEqual(table.sample_steps_per_sec, [1.0])
self.assertEqual(table.max_batch_tokens, 20)
@@ -168,7 +175,7 @@ class TestBuildSpsCostTableContract(CustomTestCase):
with tempfile.TemporaryDirectory() as tmp:
path = Path(tmp) / "sps.json"
path.write_text(table.to_json(), encoding="utf-8")
loaded = _build_sps_cost_table_for(sps_table_path=str(path))
loaded = _build_sps_cost_table_for(self, sps_table_path=str(path))
self.assertEqual(loaded.sample_batch_tokens, table.sample_batch_tokens)
self.assertEqual(loaded.sample_steps_per_sec, table.sample_steps_per_sec)
self.assertEqual(loaded.max_batch_tokens, table.max_batch_tokens)