config: the draft runner carries its own attention backend
`build_draft_tp_worker` built a `ServerArgs` variant whose only job was to make four config reads answer with the draft's backend instead of the target's, and published it for the duration of the build so the bags agreed. The backend is a per-runner fact — target and draft coexist in one process — so it moves onto the runner, and the variant and the construction-time publish both go away. `ModelRunner` takes `draft_attention_backend` and resolves the runner's effective value once (`resolve_draft_attention_backend`: the algorithm's resolved backend, else `--speculative-draft-attention-backend`, else None for a target runner); `TpModelWorker` threads it to both runner constructions. `resolve_attention_backend_strs` reads it off the runner, and `ModelRunner` stamps the resolved pair *before* building backends so a backend can read it while it constructs — which is what the FlashInfer KV-access check needs now that it no longer asks the config. `configure_kv_cache_dtype` and the draft backend factory read the runner too. One latent bug falls out: the non-hybrid branch of the backend build ignored the resolved pair and re-read `server_args.attention_backend`, which is why the variant had to set that field as well as the split pair. It now uses the value that was resolved for the runner. `draft_server_args_overrides` and the `preserve_config()` publish switch are deleted; with them goes the last production `ServerArgs.derive` outside pre-publish config building, and the last construction-time publish. The chunked-prefix gate the target resolved simply stays in the bags, since nothing re-projects them.
This commit is contained in:
@@ -75,6 +75,7 @@ class TestKVCacheQuantRegistry(CustomTestCase):
|
||||
|
||||
runner = object.__new__(ModelRunner)
|
||||
runner.server_args = SimpleNamespace(kv_cache_dtype="fp4_e2m1")
|
||||
runner.draft_attention_backend = None
|
||||
with self.assertRaisesRegex(ValueError, "fp4_mx_block16"):
|
||||
runner.configure_kv_cache_dtype()
|
||||
|
||||
|
||||
Reference in New Issue
Block a user