config: the draft runner carries its own attention backend
`build_draft_tp_worker` built a `ServerArgs` variant whose only job was to make four config reads answer with the draft's backend instead of the target's, and published it for the duration of the build so the bags agreed. The backend is a per-runner fact — target and draft coexist in one process — so it moves onto the runner, and the variant and the construction-time publish both go away. `ModelRunner` takes `draft_attention_backend` and resolves the runner's effective value once (`resolve_draft_attention_backend`: the algorithm's resolved backend, else `--speculative-draft-attention-backend`, else None for a target runner); `TpModelWorker` threads it to both runner constructions. `resolve_attention_backend_strs` reads it off the runner, and `ModelRunner` stamps the resolved pair *before* building backends so a backend can read it while it constructs — which is what the FlashInfer KV-access check needs now that it no longer asks the config. `configure_kv_cache_dtype` and the draft backend factory read the runner too. One latent bug falls out: the non-hybrid branch of the backend build ignored the resolved pair and re-read `server_args.attention_backend`, which is why the variant had to set that field as well as the split pair. It now uses the value that was resolved for the runner. `draft_server_args_overrides` and the `preserve_config()` publish switch are deleted; with them goes the last production `ServerArgs.derive` outside pre-publish config building, and the last construction-time publish. The chunked-prefix gate the target resolved simply stays in the bags, since nothing re-projects them.
This commit is contained in:
@@ -51,17 +51,15 @@ class TestChunkedPrefixCacheGate(CustomTestCase):
|
||||
get_context().set_server_args(sa) # what a later republish would do
|
||||
self.assertFalse(get_schedule().disable_chunked_prefix_cache)
|
||||
|
||||
def test_draft_variant_fields_carry_the_gate(self):
|
||||
# Publishing the draft variant re-projects the bags from it, so the
|
||||
# gate — which lives in the bags only — has to travel on the variant.
|
||||
from sglang.srt.speculative.draft_worker_common import (
|
||||
draft_server_args_overrides,
|
||||
)
|
||||
|
||||
def test_the_gate_survives_a_draft_build(self):
|
||||
# The draft build no longer publishes a config of its own, so the gate
|
||||
# the target resolved stays in the bags for the rest of the process.
|
||||
self._seed(attention_backend="triton")
|
||||
maybe_disable_chunked_prefix_cache(use_mla_backend=True, is_draft_worker=False)
|
||||
fields = draft_server_args_overrides(draft_backend="fa3")
|
||||
self.assertTrue(fields["disable_chunked_prefix_cache"])
|
||||
self.assertTrue(get_schedule().disable_chunked_prefix_cache)
|
||||
|
||||
maybe_disable_chunked_prefix_cache(use_mla_backend=False, is_draft_worker=True)
|
||||
self.assertTrue(get_schedule().disable_chunked_prefix_cache)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
Reference in New Issue
Block a user