Last of four; stacked on #38048. The record is the operator's input; the bags are what is in effect. A reader that takes the record and reads a field off it gets the input, which is the wrong one of the two whenever resolution decided something -- and the mistake is silent, because for most fields and most launches the two agree. Several of these files already read both ways, sometimes in the same expression: ```python get_tokenizer( get_serving().tokenizer_path, tokenizer_mode=server_args.tokenizer_mode, # the input, not the decision ... ) ``` Sixty-odd files convert. Record field reads in runtime code go from 199 to 11. Nine parameters that the conversion emptied are dropped along with the argument at every call site -- the dead-parameter ratchet is what names them. ### "Runs after its process publishes" is a per-entry-point claim Most converted reads sit in the serving and model-executor layers, which only exist after publication, or in the two subprocess entry points, which publish first thing. Three places are not like that, and they keep reading the record they were handed: - **`HttpServerEngineAdapter`** launches the server as a *child*. The parent resolves the record and never publishes, so the adapter's own reads -- the launch banner, the API key in its readiness loop, the TP width in `update_weights_from_tensor` -- are of `self.server_args`. A bag read here fails closed in a bare process, or answers for an unrelated engine in one that happens to have published. - **`serve_grpc`** reads its sidecar port before the integrated servicer builds the `Engine` that publishes. The comment above that line already said so and already bound `cfg = resolving_view(server_args)` for it; the sidecar port and the port it derives from read `cfg`. - **`initialize_dp_attention`** runs from callers whose publish is not guaranteed, so its one predicate stays on the resolution view. `ROLE_NAMESPACE_SETS["dp_controller"]` gains `observability` and `serving`, because the controller's metrics gate, tracing setup and worker-port broadcast now read those namespaces. Under `SGLANG_ROLE_NAMESPACES=enforce` that set is what the process may read, so a conversion that reaches a new namespace has to widen it in the same change. ## Three things worth a reviewer's attention **Eleven reads were `getattr(record, "field", default)`.** An AST scan for attribute access does not see those, so the census that said "43 readers" was counting the shape it could match rather than the thing it was after. `incremental_streaming_output` was read that way twice, and the transcription tests were the only reason it surfaced. **Not every record read is a bag read waiting to happen.** A multimodal processor's `base_gpu_id` is the instance's, not the process's: two engines in one process keep different ones, and `test_publishing_another_config_does_not_move_the_device` exists to say so. It stays on the record while `rl_on_policy_target` beside it moves. `RequestMetricsExporter` is the same shape -- it is handed the directory it writes to, and a test builds several with different ones. `configure_logger` is a third: 17 call sites, one of which passes an `argparse.Namespace`, so it is not a global-context reader at all. Those eleven remaining reads are the ones with a reason. **The fixtures move with the code.** Tests that hung config off a mock manager now publish a record, which is what the serving layer reads; where a test states a value it says so with `override_server_args` instead of assigning through the mock. `test_hisparse_unit` is the last of them: it stubbed a `server_args` onto a fake scheduler to say the decode radix cache was off, and the value it was standing in for is the published default, so the stub goes and the class publishes. ## Two things CI caught that a local sweep could not **`unittest.TestCase.enterContext` is Python 3.11+.** The converted fixtures used it at 18 sites; `requires-python` is `>=3.10` and CI runs 3.10, so every one of them raised `AttributeError` there while passing on a newer local interpreter. They call `enter_override(self, ...)` now -- a four-line helper in `sglang/test/test_utils.py` over the override's own `install()` / `restore()`. **A batched sweep cannot see a missing publish.** Three fixtures needed a published config and did not have one; each *passed* inside a shard where some other file had published, and failed when run alone. The affected cases are `test_serving_completions` (which set `incremental_streaming_output` on the mock manager's record, where nothing reads it now), `test_qwen3_vl_feature_materialization` (same shape for `mm_enable_dp_encoder`), and the two Qwen Rust tests -- whose fixture already carried the comment `# Non-auto: get_resolved_model_impl would choke on a SimpleNamespace` next to the `model_impl` it sets, which is exactly what happened once `get_mm_processor_cls` started reading that value from the bag. Its `publish` mirrors `model_impl` now, like the four fields it already mirrored. ## Verification A full registered-unit sweep (648 files) against this stack's merge-base: 19 failures on both sides, the same 19, none of them config. That sweep is what caught 23 failures the file-scoped runs missed -- and, later, that the narrower 139-file list did not even contain the files this change reaches. It is also what caught the `test_hisparse_unit` fixture above: the file passes inside a shard where something else published, and fails when it is run on its own, which is why every failing file is re-run alone before it is counted.
647 lines
24 KiB
Python
647 lines
24 KiB
Python
"""Unit tests for token embedding override support.
|
|
|
|
Covers:
|
|
- PositionalEmbeds dataclass (embed_types.py)
|
|
- convert_embeds_to_tensors (utils.py)
|
|
- TokenizerManager._resolve_embed_overrides (tokenizer_manager.py)
|
|
- positional_embed_overrides on GenerateReqInput/EmbeddingReqInput (io_struct.py)
|
|
- Score mixin override resolution (tokenizer_manager_score_mixin.py)
|
|
"""
|
|
|
|
import unittest
|
|
from unittest.mock import AsyncMock, MagicMock
|
|
|
|
import torch
|
|
|
|
from sglang.srt.constants import MIS_DELIMITER_TOKEN_ID
|
|
from sglang.srt.entrypoints.openai.utils import convert_embeds_to_tensors
|
|
from sglang.srt.managers.embed_types import PositionalEmbeds
|
|
from sglang.srt.managers.io_struct import EmbeddingReqInput, GenerateReqInput
|
|
from sglang.srt.managers.tokenizer_manager import TokenizerManager
|
|
from sglang.srt.managers.tokenizer_manager_score_mixin import (
|
|
TokenizerManagerScoreMixin,
|
|
)
|
|
from sglang.srt.runtime_context import publish, reset_context
|
|
from sglang.srt.server_args import ServerArgs
|
|
from sglang.test.ci.ci_register import register_cpu_ci
|
|
from sglang.test.test_utils import CustomTestCase
|
|
|
|
register_cpu_ci(est_time=11, suite="base-a-test-cpu")
|
|
|
|
HIDDEN_DIM = 4
|
|
|
|
|
|
def _vec(val: float = 1.0) -> torch.Tensor:
|
|
"""Create a 1-D tensor of size HIDDEN_DIM."""
|
|
return torch.full((HIDDEN_DIM,), val, dtype=torch.float32)
|
|
|
|
|
|
def _vec2d(val: float = 1.0) -> torch.Tensor:
|
|
"""Create a [1, HIDDEN_DIM] tensor."""
|
|
return torch.full((1, HIDDEN_DIM), val, dtype=torch.float32)
|
|
|
|
|
|
# ========================================================================
|
|
# PositionalEmbeds
|
|
# ========================================================================
|
|
|
|
|
|
class TestPositionalEmbeds(CustomTestCase):
|
|
def setUp(self):
|
|
# The code under test reads its config from the bags.
|
|
reset_context()
|
|
self.addCleanup(reset_context)
|
|
publish(ServerArgs(model_path="dummy"), role="tokenizer")
|
|
|
|
def test_from_list_of_1d_tensors(self):
|
|
pe = PositionalEmbeds(embeds=[_vec(1), _vec(2)], positions=[0, 5])
|
|
self.assertEqual(pe.embeds.shape, (2, HIDDEN_DIM))
|
|
self.assertAlmostEqual(pe.embeds[0, 0].item(), 1.0)
|
|
self.assertAlmostEqual(pe.embeds[1, 0].item(), 2.0)
|
|
|
|
def test_from_list_of_2d_tensors(self):
|
|
pe = PositionalEmbeds(embeds=[_vec2d(3), _vec2d(4)], positions=[1, 2])
|
|
self.assertEqual(pe.embeds.shape, (2, HIDDEN_DIM))
|
|
|
|
def test_from_pre_stacked_tensor(self):
|
|
stacked = torch.zeros(3, HIDDEN_DIM)
|
|
pe = PositionalEmbeds(embeds=stacked, positions=[0, 1, 2])
|
|
self.assertIs(pe.embeds, stacked)
|
|
|
|
def test_length_mismatch_raises(self):
|
|
with self.assertRaises(ValueError):
|
|
PositionalEmbeds(embeds=[_vec()], positions=[0, 1])
|
|
|
|
def test_empty(self):
|
|
pe = PositionalEmbeds(embeds=torch.zeros(0, HIDDEN_DIM), positions=[])
|
|
self.assertEqual(pe.embeds.shape[0], 0)
|
|
|
|
|
|
# ========================================================================
|
|
# convert_embeds_to_tensors
|
|
# ========================================================================
|
|
|
|
|
|
class TestConvertEmbedsToTensors(CustomTestCase):
|
|
def setUp(self):
|
|
# The code under test reads its config from the bags.
|
|
reset_context()
|
|
self.addCleanup(reset_context)
|
|
publish(ServerArgs(model_path="dummy"), role="tokenizer")
|
|
|
|
def test_none_returns_none(self):
|
|
self.assertIsNone(convert_embeds_to_tensors(None))
|
|
|
|
def test_empty_list(self):
|
|
self.assertEqual(convert_embeds_to_tensors([]), [])
|
|
|
|
def test_single_input(self):
|
|
"""[num_replacements][hidden_size] -> [[tensor, ...]]"""
|
|
result = convert_embeds_to_tensors([[1.0, 2.0], [3.0, 4.0]])
|
|
self.assertEqual(len(result), 1) # wrapped in outer list
|
|
self.assertEqual(len(result[0]), 2) # two replacement vectors
|
|
self.assertTrue(torch.is_tensor(result[0][0]))
|
|
self.assertEqual(result[0][0].tolist(), [1.0, 2.0])
|
|
self.assertEqual(result[0][0].dtype, torch.float32)
|
|
self.assertEqual(result[0][0].dim(), 1) # each vector is 1-D
|
|
|
|
def test_batch_input(self):
|
|
"""[num_inputs][num_replacements][hidden_size] -> [[tensor, ...], ...]"""
|
|
result = convert_embeds_to_tensors(
|
|
[
|
|
[[1.0, 2.0]],
|
|
[[3.0, 4.0], [5.0, 6.0]],
|
|
]
|
|
)
|
|
self.assertEqual(len(result), 2)
|
|
self.assertEqual(len(result[0]), 1)
|
|
self.assertEqual(len(result[1]), 2)
|
|
|
|
|
|
# ========================================================================
|
|
# TokenizerManager._resolve_embed_overrides
|
|
# ========================================================================
|
|
|
|
|
|
class TestResolveEmbedOverrides(CustomTestCase):
|
|
def setUp(self):
|
|
# The code under test reads its config from the bags.
|
|
reset_context()
|
|
self.addCleanup(reset_context)
|
|
publish(ServerArgs(model_path="dummy"), role="tokenizer")
|
|
|
|
def test_basic_resolution(self):
|
|
embeds = [_vec(1), _vec(2)]
|
|
pe = TokenizerManager._resolve_embed_overrides(
|
|
input_ids=[10, 50, 20, 50, 30],
|
|
token_id=50,
|
|
embeds=embeds,
|
|
)
|
|
self.assertIsInstance(pe, PositionalEmbeds)
|
|
self.assertEqual(pe.positions, [1, 3])
|
|
self.assertEqual(pe.embeds.shape, (2, HIDDEN_DIM))
|
|
|
|
def test_no_placeholders_raises(self):
|
|
with self.assertRaises(ValueError):
|
|
TokenizerManager._resolve_embed_overrides(
|
|
input_ids=[10, 20, 30],
|
|
token_id=50,
|
|
embeds=[_vec()],
|
|
)
|
|
|
|
def test_count_mismatch_raises(self):
|
|
with self.assertRaises(ValueError):
|
|
TokenizerManager._resolve_embed_overrides(
|
|
input_ids=[10, 50, 20],
|
|
token_id=50,
|
|
embeds=[_vec(1), _vec(2)],
|
|
)
|
|
|
|
|
|
# ========================================================================
|
|
# io_struct: positional_embed_overrides on GenerateReqInput
|
|
# ========================================================================
|
|
|
|
|
|
class TestGenerateReqInputEmbedOverride(CustomTestCase):
|
|
def setUp(self):
|
|
# The code under test reads its config from the bags.
|
|
reset_context()
|
|
self.addCleanup(reset_context)
|
|
publish(ServerArgs(model_path="dummy"), role="tokenizer")
|
|
|
|
def test_single_override_in_getitem(self):
|
|
"""Single PositionalEmbeds is shared across all items in __getitem__."""
|
|
pe = PositionalEmbeds(embeds=[_vec()], positions=[0])
|
|
req = GenerateReqInput(
|
|
input_ids=[[1, 2], [3, 4]],
|
|
sampling_params=[{}, {}],
|
|
positional_embed_overrides=pe,
|
|
)
|
|
req.normalize_batch_and_arguments()
|
|
item = req[0]
|
|
self.assertIs(item.positional_embed_overrides, pe)
|
|
|
|
def test_batch_override_in_getitem(self):
|
|
"""List[Optional[PositionalEmbeds]] is indexed per-item."""
|
|
pe0 = PositionalEmbeds(embeds=[_vec(1)], positions=[0])
|
|
pe1 = None
|
|
req = GenerateReqInput(
|
|
input_ids=[[1, 2], [3, 4]],
|
|
sampling_params=[{}, {}],
|
|
positional_embed_overrides=[pe0, pe1],
|
|
)
|
|
req.normalize_batch_and_arguments()
|
|
self.assertEqual(req[0].positional_embed_overrides, pe0)
|
|
self.assertIsNone(req[1].positional_embed_overrides)
|
|
|
|
|
|
# ========================================================================
|
|
# io_struct: embed override fields on EmbeddingReqInput
|
|
# ========================================================================
|
|
|
|
|
|
class TestEmbeddingReqInputEmbedOverride(CustomTestCase):
|
|
def setUp(self):
|
|
# The code under test reads its config from the bags.
|
|
reset_context()
|
|
self.addCleanup(reset_context)
|
|
publish(ServerArgs(model_path="dummy"), role="tokenizer")
|
|
|
|
def test_override_fields_in_getitem(self):
|
|
"""embed_override_token_id, embed_overrides, and positional_embed_overrides
|
|
are correctly sliced in __getitem__."""
|
|
pe0 = PositionalEmbeds(embeds=[_vec(1)], positions=[0])
|
|
pe1 = PositionalEmbeds(embeds=[_vec(2)], positions=[1])
|
|
req = EmbeddingReqInput(
|
|
input_ids=[[50, 10], [20, 50]],
|
|
sampling_params=[{}, {}],
|
|
embed_override_token_id=50,
|
|
embed_overrides=[[_vec(1)], [_vec(2)]],
|
|
positional_embed_overrides=[pe0, pe1],
|
|
)
|
|
req.normalize_batch_and_arguments()
|
|
item0 = req[0]
|
|
item1 = req[1]
|
|
self.assertEqual(item0.embed_override_token_id, 50)
|
|
self.assertEqual(len(item0.embed_overrides), 1)
|
|
self.assertEqual(item0.positional_embed_overrides, pe0)
|
|
self.assertEqual(item1.positional_embed_overrides, pe1)
|
|
|
|
|
|
# ========================================================================
|
|
# Score mixin: _resolve_overrides_for_sequence
|
|
# ========================================================================
|
|
|
|
|
|
class _FakeServerArgs:
|
|
"""Minimal stub for server_args."""
|
|
|
|
def __init__(self, enable_mis=False):
|
|
self.enable_mis = enable_mis
|
|
|
|
|
|
class _FakeMixin(TokenizerManagerScoreMixin):
|
|
"""Minimal stub to call mixin methods without a full TokenizerManager."""
|
|
|
|
def __init__(self, enable_mis=False):
|
|
self.server_args = _FakeServerArgs(enable_mis)
|
|
self.tokenizer = None
|
|
self.is_generation = True
|
|
|
|
|
|
class TestResolveOverridesForSequence(CustomTestCase):
|
|
def setUp(self):
|
|
reset_context()
|
|
self.addCleanup(reset_context)
|
|
publish(ServerArgs(model_path="dummy"), role="tokenizer")
|
|
self.mixin = _FakeMixin()
|
|
|
|
def test_none_embeds_returns_empty(self):
|
|
embeds, positions = self.mixin._resolve_overrides_for_sequence(
|
|
token_ids=[10, 50, 20],
|
|
embeds=None,
|
|
embed_override_token_id=50,
|
|
)
|
|
self.assertEqual(embeds, [])
|
|
self.assertEqual(positions, [])
|
|
|
|
def test_basic_resolution(self):
|
|
e1, e2 = _vec(1), _vec(2)
|
|
embeds, positions = self.mixin._resolve_overrides_for_sequence(
|
|
token_ids=[50, 10, 50],
|
|
embeds=[e1, e2],
|
|
embed_override_token_id=50,
|
|
)
|
|
self.assertEqual(len(embeds), 2)
|
|
self.assertEqual(positions, [0, 2])
|
|
|
|
def test_with_offset(self):
|
|
embeds, positions = self.mixin._resolve_overrides_for_sequence(
|
|
token_ids=[10, 50],
|
|
embeds=[_vec()],
|
|
embed_override_token_id=50,
|
|
position_offset=100,
|
|
)
|
|
self.assertEqual(positions, [101])
|
|
|
|
def test_empty_embeds_list(self):
|
|
"""Empty embeds list with no placeholders succeeds."""
|
|
embeds, positions = self.mixin._resolve_overrides_for_sequence(
|
|
token_ids=[10, 20],
|
|
embeds=[],
|
|
embed_override_token_id=50,
|
|
)
|
|
self.assertEqual(embeds, [])
|
|
self.assertEqual(positions, [])
|
|
|
|
def test_count_mismatch_raises(self):
|
|
with self.assertRaises(ValueError):
|
|
self.mixin._resolve_overrides_for_sequence(
|
|
token_ids=[50, 50],
|
|
embeds=[_vec()],
|
|
embed_override_token_id=50,
|
|
)
|
|
|
|
|
|
# ========================================================================
|
|
# Score mixin: _resolve_embed_overrides_for_request
|
|
# ========================================================================
|
|
|
|
|
|
class TestResolveEmbedOverridesForRequest(CustomTestCase):
|
|
def setUp(self):
|
|
reset_context()
|
|
self.addCleanup(reset_context)
|
|
publish(ServerArgs(model_path="dummy"), role="tokenizer")
|
|
self.mixin = _FakeMixin()
|
|
|
|
def test_no_overrides_returns_none(self):
|
|
result = self.mixin._resolve_embed_overrides_for_request(
|
|
query=[10, 20],
|
|
item=[30, 40],
|
|
embed_override_token_id=50,
|
|
query_embed_overrides=None,
|
|
item_embeds=None,
|
|
item_position_offset=2,
|
|
item_label="items[0]",
|
|
)
|
|
self.assertIsNone(result)
|
|
|
|
def test_query_only_overrides(self):
|
|
pe = self.mixin._resolve_embed_overrides_for_request(
|
|
query=[50, 20],
|
|
item=[30, 40],
|
|
embed_override_token_id=50,
|
|
query_embed_overrides=[_vec(1)],
|
|
item_embeds=None,
|
|
item_position_offset=2,
|
|
item_label="items[0]",
|
|
)
|
|
self.assertIsInstance(pe, PositionalEmbeds)
|
|
self.assertEqual(pe.positions, [0])
|
|
self.assertEqual(pe.embeds.shape, (1, HIDDEN_DIM))
|
|
|
|
def test_item_only_overrides(self):
|
|
pe = self.mixin._resolve_embed_overrides_for_request(
|
|
query=[10, 20],
|
|
item=[50, 40],
|
|
embed_override_token_id=50,
|
|
query_embed_overrides=None,
|
|
item_embeds=[_vec(2)],
|
|
item_position_offset=2,
|
|
item_label="items[0]",
|
|
)
|
|
self.assertEqual(pe.positions, [2]) # offset applied
|
|
|
|
def test_query_and_item_overrides(self):
|
|
pe = self.mixin._resolve_embed_overrides_for_request(
|
|
query=[50, 20],
|
|
item=[30, 50],
|
|
embed_override_token_id=50,
|
|
query_embed_overrides=[_vec(1)],
|
|
item_embeds=[_vec(2)],
|
|
item_position_offset=2,
|
|
item_label="items[0]",
|
|
)
|
|
self.assertEqual(pe.positions, [0, 3]) # query pos 0, item pos 1+offset 2
|
|
self.assertEqual(pe.embeds.shape, (2, HIDDEN_DIM))
|
|
|
|
|
|
# ========================================================================
|
|
# Score mixin: _build_token_id_inputs
|
|
# ========================================================================
|
|
|
|
DELIM_TOKEN = MIS_DELIMITER_TOKEN_ID
|
|
|
|
|
|
class TestBuildTokenIdInputs(CustomTestCase):
|
|
def setUp(self):
|
|
reset_context()
|
|
self.addCleanup(reset_context)
|
|
publish(ServerArgs(model_path="dummy"), role="tokenizer")
|
|
self.mixin = _FakeMixin(enable_mis=True)
|
|
|
|
# --- single-item mode, no embeds ---
|
|
|
|
def test_single_item_no_embeds(self):
|
|
_, input_ids, positional_embed_overrides, _ = self.mixin._build_token_id_inputs(
|
|
query=[1, 2],
|
|
items=[[3, 4], [5, 6]],
|
|
item_first=False,
|
|
use_multi_item_scoring=False,
|
|
embed_override_token_id=None,
|
|
query_embed_overrides=None,
|
|
item_embed_overrides=None,
|
|
)
|
|
self.assertEqual(input_ids, [[1, 2, 3, 4], [1, 2, 5, 6]])
|
|
self.assertIsNone(positional_embed_overrides)
|
|
|
|
def test_single_item_no_embeds_item_first(self):
|
|
_, input_ids, positional_embed_overrides, _ = self.mixin._build_token_id_inputs(
|
|
query=[1, 2],
|
|
items=[[3, 4]],
|
|
item_first=True,
|
|
use_multi_item_scoring=False,
|
|
embed_override_token_id=None,
|
|
query_embed_overrides=None,
|
|
item_embed_overrides=None,
|
|
)
|
|
self.assertEqual(input_ids, [[3, 4, 1, 2]])
|
|
self.assertIsNone(positional_embed_overrides)
|
|
|
|
# --- multi-item mode, no embeds ---
|
|
|
|
def test_multi_item_no_embeds(self):
|
|
_, input_ids, positional_embed_overrides, _ = self.mixin._build_token_id_inputs(
|
|
query=[1, 2],
|
|
items=[[3, 4], [5, 6]],
|
|
item_first=False,
|
|
use_multi_item_scoring=True,
|
|
embed_override_token_id=None,
|
|
query_embed_overrides=None,
|
|
item_embed_overrides=None,
|
|
)
|
|
# query<D>item1<D>item2<D>
|
|
self.assertEqual(
|
|
input_ids, [[1, 2, DELIM_TOKEN, 3, 4, DELIM_TOKEN, 5, 6, DELIM_TOKEN]]
|
|
)
|
|
self.assertIsNone(positional_embed_overrides)
|
|
|
|
# --- single-item mode, with embeds ---
|
|
|
|
def test_single_item_query_embeds(self):
|
|
"""Query placeholder overrides are resolved per item."""
|
|
_, input_ids, positional_embed_overrides, _ = self.mixin._build_token_id_inputs(
|
|
query=[50, 10],
|
|
items=[[20, 30], [40, 50]],
|
|
item_first=False,
|
|
use_multi_item_scoring=False,
|
|
embed_override_token_id=50,
|
|
query_embed_overrides=[_vec(1)],
|
|
item_embed_overrides=None,
|
|
)
|
|
self.assertEqual(input_ids, [[50, 10, 20, 30], [50, 10, 40, 50]])
|
|
self.assertIsNotNone(positional_embed_overrides)
|
|
self.assertEqual(len(positional_embed_overrides), 2)
|
|
# Each item gets its own PositionalEmbeds with query override at pos 0
|
|
self.assertEqual(positional_embed_overrides[0].positions, [0])
|
|
self.assertEqual(positional_embed_overrides[1].positions, [0])
|
|
|
|
def test_single_item_item_embeds(self):
|
|
"""Per-item overrides with correct position offsets."""
|
|
_, input_ids, positional_embed_overrides, _ = self.mixin._build_token_id_inputs(
|
|
query=[10, 20],
|
|
items=[[50, 30]],
|
|
item_first=False,
|
|
use_multi_item_scoring=False,
|
|
embed_override_token_id=50,
|
|
query_embed_overrides=None,
|
|
item_embed_overrides=[[_vec(2)]],
|
|
)
|
|
self.assertEqual(input_ids, [[10, 20, 50, 30]])
|
|
self.assertIsNotNone(positional_embed_overrides)
|
|
# item placeholder at index 0 of item, offset by query length 2
|
|
self.assertEqual(positional_embed_overrides[0].positions, [2])
|
|
|
|
def test_single_item_no_override_positions_returns_none_injection(self):
|
|
"""When no items have placeholders, positional_embed_overrides should be None."""
|
|
_, input_ids, positional_embed_overrides, _ = self.mixin._build_token_id_inputs(
|
|
query=[10, 20],
|
|
items=[[30, 40]],
|
|
item_first=False,
|
|
use_multi_item_scoring=False,
|
|
embed_override_token_id=50,
|
|
query_embed_overrides=None,
|
|
item_embed_overrides=[None],
|
|
)
|
|
self.assertIsNone(positional_embed_overrides)
|
|
|
|
def test_single_item_query_and_item_embeds(self):
|
|
"""Single-item mode with both query and item overrides in one request."""
|
|
_, input_ids, positional_embed_overrides, _ = self.mixin._build_token_id_inputs(
|
|
query=[50, 10],
|
|
items=[[20, 50]],
|
|
item_first=False,
|
|
use_multi_item_scoring=False,
|
|
embed_override_token_id=50,
|
|
query_embed_overrides=[_vec(1)],
|
|
item_embed_overrides=[[_vec(2)]],
|
|
)
|
|
self.assertEqual(input_ids, [[50, 10, 20, 50]])
|
|
self.assertIsNotNone(positional_embed_overrides)
|
|
pe = positional_embed_overrides[0]
|
|
# query override at pos 0, item override at pos 3 (query_len=2 + idx=1)
|
|
self.assertEqual(pe.positions, [0, 3])
|
|
self.assertEqual(pe.embeds.shape, (2, HIDDEN_DIM))
|
|
|
|
def test_single_item_empty_query(self):
|
|
"""Empty query with item-only overrides (valid from score_prompts)."""
|
|
_, input_ids, positional_embed_overrides, _ = self.mixin._build_token_id_inputs(
|
|
query=[],
|
|
items=[[50, 10]],
|
|
item_first=False,
|
|
use_multi_item_scoring=False,
|
|
embed_override_token_id=50,
|
|
query_embed_overrides=None,
|
|
item_embed_overrides=[[_vec(1)]],
|
|
)
|
|
self.assertEqual(input_ids, [[50, 10]])
|
|
self.assertIsNotNone(positional_embed_overrides)
|
|
# item placeholder at absolute pos 0 (offset=len([])=0)
|
|
self.assertEqual(positional_embed_overrides[0].positions, [0])
|
|
|
|
# --- multi-item mode, with embeds ---
|
|
|
|
def test_multi_item_with_query_and_item_embeds(self):
|
|
"""Multi-item mode resolves query overrides once and item overrides per item."""
|
|
_, input_ids, positional_embed_overrides, _ = self.mixin._build_token_id_inputs(
|
|
query=[50, 10],
|
|
items=[[20, 50], [30, 40]],
|
|
item_first=False,
|
|
use_multi_item_scoring=True,
|
|
embed_override_token_id=50,
|
|
query_embed_overrides=[_vec(1)],
|
|
item_embed_overrides=[[_vec(2)], None],
|
|
)
|
|
# query<D>item1<D>item2<D> = [50,10, DELIM, 20,50, DELIM, 30,40, DELIM]
|
|
self.assertEqual(len(input_ids), 1)
|
|
self.assertIsNotNone(positional_embed_overrides)
|
|
self.assertEqual(
|
|
len(positional_embed_overrides), 1
|
|
) # single PositionalEmbeds for combined sequence
|
|
pe = positional_embed_overrides[0]
|
|
# query override at pos 0, item[0] override at pos 4 (query_len=2 + delim=1 + idx=1)
|
|
self.assertIn(0, pe.positions)
|
|
self.assertIn(4, pe.positions)
|
|
self.assertEqual(pe.embeds.shape[0], 2)
|
|
|
|
|
|
# ========================================================================
|
|
# Score mixin: score_request validation
|
|
# ========================================================================
|
|
|
|
|
|
class TestScoreRequestValidation(CustomTestCase):
|
|
"""Test validation guards in score_request without running full pipeline."""
|
|
|
|
def setUp(self):
|
|
reset_context()
|
|
self.addCleanup(reset_context)
|
|
publish(ServerArgs(model_path="dummy"), role="tokenizer")
|
|
self.mixin = _FakeMixin()
|
|
|
|
def _call(self, **kwargs):
|
|
"""Wrapper to call score_request synchronously."""
|
|
import asyncio
|
|
|
|
return asyncio.run(self.mixin.score_request(**kwargs))
|
|
|
|
def test_generation_requires_label_token_ids(self):
|
|
self.mixin.is_generation = True
|
|
with self.assertRaisesRegex(ValueError, "label_token_ids is required"):
|
|
self._call(
|
|
query=[1, 2],
|
|
items=[[3, 4]],
|
|
label_token_ids=None,
|
|
)
|
|
|
|
def test_seq_classification_allows_none_label_token_ids(self):
|
|
"""SequenceClassification models should not require label_token_ids.
|
|
Verify it passes validation and reaches generate_request."""
|
|
self.mixin.is_generation = False
|
|
mock_result = AsyncMock()
|
|
mock_result.__anext__ = AsyncMock(
|
|
return_value=[{"embedding": [0.1, 0.9], "meta_info": {"prompt_tokens": 2}}]
|
|
)
|
|
self.mixin.generate_request = MagicMock(return_value=mock_result)
|
|
result = self._call(
|
|
query=[1, 2],
|
|
items=[[3, 4]],
|
|
label_token_ids=None,
|
|
)
|
|
self.mixin.generate_request.assert_called_once()
|
|
self.assertEqual(len(result.scores), 1)
|
|
|
|
def test_items_none_raises(self):
|
|
with self.assertRaisesRegex(ValueError, "items must be provided"):
|
|
self._call(
|
|
query=[1, 2],
|
|
items=None,
|
|
label_token_ids=[100],
|
|
)
|
|
|
|
def test_empty_items_returns_empty(self):
|
|
result = self._call(
|
|
query=[1, 2],
|
|
items=[],
|
|
label_token_ids=[100],
|
|
)
|
|
self.assertEqual(result.scores, [])
|
|
self.assertEqual(result.prompt_tokens, 0)
|
|
|
|
def test_embed_override_token_id_required_with_query_embeds(self):
|
|
with self.assertRaisesRegex(ValueError, "embed_override_token_id is required"):
|
|
self._call(
|
|
query=[1, 2],
|
|
items=[[3, 4]],
|
|
label_token_ids=[100],
|
|
query_embed_overrides=[_vec(1)],
|
|
embed_override_token_id=None,
|
|
)
|
|
|
|
def test_embed_override_token_id_required_with_item_embeds(self):
|
|
with self.assertRaisesRegex(ValueError, "embed_override_token_id is required"):
|
|
self._call(
|
|
query=[1, 2],
|
|
items=[[3, 4]],
|
|
label_token_ids=[100],
|
|
item_embed_overrides=[[_vec(1)]],
|
|
embed_override_token_id=None,
|
|
)
|
|
|
|
def test_item_first_with_embeds_raises(self):
|
|
with self.assertRaisesRegex(ValueError, "item_first is not supported"):
|
|
self._call(
|
|
query=[1, 2],
|
|
items=[[3, 4]],
|
|
label_token_ids=[100],
|
|
item_first=True,
|
|
embed_override_token_id=50,
|
|
query_embed_overrides=[_vec(1)],
|
|
)
|
|
|
|
def test_item_embed_overrides_length_mismatch_raises(self):
|
|
with self.assertRaisesRegex(ValueError, "must match items length"):
|
|
self._call(
|
|
query=[1, 2],
|
|
items=[[3, 4], [5, 6]],
|
|
label_token_ids=[100],
|
|
embed_override_token_id=50,
|
|
item_embed_overrides=[[_vec(1)]], # 1 override for 2 items
|
|
)
|
|
|
|
|
|
if __name__ == "__main__":
|
|
unittest.main()
|