Files
sglang/test/registered/lora/test_lora_openai_api.py
T
Cheng Wan b99175dc7d [Config] Round 6.4: the runtime reads the bags, not the record (#38049)
Last of four; stacked on #38048.

The record is the operator's input; the bags are what is in effect. A reader
that takes the record and reads a field off it gets the input, which is the
wrong one of the two whenever resolution decided something -- and the mistake is
silent, because for most fields and most launches the two agree. Several of
these files already read both ways, sometimes in the same expression:

```python
get_tokenizer(
    get_serving().tokenizer_path,
    tokenizer_mode=server_args.tokenizer_mode,   # the input, not the decision
    ...
)
```

Sixty-odd files convert. Record field reads in runtime code go from 199 to 11.
Nine parameters that the conversion emptied are dropped along with the argument
at every call site -- the dead-parameter ratchet is what names them.

### "Runs after its process publishes" is a per-entry-point claim

Most converted reads sit in the serving and model-executor layers, which only
exist after publication, or in the two subprocess entry points, which publish
first thing. Three places are not like that, and they keep reading the record
they were handed:

- **`HttpServerEngineAdapter`** launches the server as a *child*. The parent
  resolves the record and never publishes, so the adapter's own reads -- the
  launch banner, the API key in its readiness loop, the TP width in
  `update_weights_from_tensor` -- are of `self.server_args`. A bag read here
  fails closed in a bare process, or answers for an unrelated engine in one that
  happens to have published.
- **`serve_grpc`** reads its sidecar port before the integrated servicer builds
  the `Engine` that publishes. The comment above that line already said so and
  already bound `cfg = resolving_view(server_args)` for it; the sidecar port and
  the port it derives from read `cfg`.
- **`initialize_dp_attention`** runs from callers whose publish is not
  guaranteed, so its one predicate stays on the resolution view.

`ROLE_NAMESPACE_SETS["dp_controller"]` gains `observability` and `serving`,
because the controller's metrics gate, tracing setup and worker-port broadcast
now read those namespaces. Under `SGLANG_ROLE_NAMESPACES=enforce` that set is
what the process may read, so a conversion that reaches a new namespace has to
widen it in the same change.

## Three things worth a reviewer's attention

**Eleven reads were `getattr(record, "field", default)`.** An AST scan for
attribute access does not see those, so the census that said "43 readers" was
counting the shape it could match rather than the thing it was after.
`incremental_streaming_output` was read that way twice, and the transcription
tests were the only reason it surfaced.

**Not every record read is a bag read waiting to happen.** A multimodal
processor's `base_gpu_id` is the instance's, not the process's: two engines in
one process keep different ones, and
`test_publishing_another_config_does_not_move_the_device` exists to say so. It
stays on the record while `rl_on_policy_target` beside it moves.
`RequestMetricsExporter` is the same shape -- it is handed the directory it
writes to, and a test builds several with different ones. `configure_logger` is
a third: 17 call sites, one of which passes an `argparse.Namespace`, so it is
not a global-context reader at all. Those eleven remaining reads are the ones
with a reason.

**The fixtures move with the code.** Tests that hung config off a mock manager
now publish a record, which is what the serving layer reads; where a test states
a value it says so with `override_server_args` instead of assigning through the
mock. `test_hisparse_unit` is the last of them: it stubbed a `server_args` onto
a fake scheduler to say the decode radix cache was off, and the value it was
standing in for is the published default, so the stub goes and the class
publishes.

## Two things CI caught that a local sweep could not

**`unittest.TestCase.enterContext` is Python 3.11+.** The converted fixtures used
it at 18 sites; `requires-python` is `>=3.10` and CI runs 3.10, so every one of
them raised `AttributeError` there while passing on a newer local interpreter.
They call `enter_override(self, ...)` now -- a four-line helper in
`sglang/test/test_utils.py` over the override's own `install()` / `restore()`.

**A batched sweep cannot see a missing publish.** Three fixtures needed a
published config and did not have one; each *passed* inside a shard where some
other file had published, and failed when run alone. The affected cases are
`test_serving_completions` (which set `incremental_streaming_output` on the mock
manager's record, where nothing reads it now), `test_qwen3_vl_feature_materialization`
(same shape for `mm_enable_dp_encoder`), and the two Qwen Rust tests -- whose
fixture already carried the comment `# Non-auto: get_resolved_model_impl would
choke on a SimpleNamespace` next to the `model_impl` it sets, which is exactly
what happened once `get_mm_processor_cls` started reading that value from the
bag. Its `publish` mirrors `model_impl` now, like the four fields it already
mirrored.

## Verification

A full registered-unit sweep (648 files) against this stack's merge-base:
19 failures on both sides, the same 19, none of them config. That sweep is what
caught 23 failures the file-scoped runs missed -- and, later, that the narrower
139-file list did not even contain the files this change reaches. It is also
what caught the `test_hisparse_unit` fixture above: the file passes inside a
shard where something else published, and fails when it is run on its own,
which is why every failing file is re-run alone before it is counted.
2026-09-06 21:41:46 -07:00

272 lines
11 KiB
Python

"""
Unit tests for OpenAI-compatible LoRA API support.
Tests the model parameter parsing and LoRA adapter resolution logic
that enables OpenAI-compatible LoRA adapter selection.
"""
import unittest
from unittest.mock import MagicMock
from sglang.srt.entrypoints.openai.serving_base import OpenAIServingBase
from sglang.srt.runtime_context import publish, reset_context
from sglang.srt.server_args import ServerArgs
from sglang.test.ci.ci_register import register_amd_ci, register_cpu_ci
register_amd_ci(est_time=30, suite="nightly-amd-1-gpu", nightly=True)
register_cpu_ci(est_time=7, suite="stage-b-test-cpu-intel")
def publish_config(case):
"""`OpenAIServingBase.__init__` reads `get_observability()`, so a case that
builds one needs a published config. The mock manager cannot stand in for
it: `MagicMock(spec=ServerArgs)` passes the `isinstance` guard, so the read
happens and there is no bag to answer from."""
reset_context()
case.addCleanup(reset_context)
publish(ServerArgs(model_path="dummy"), role="tokenizer")
class MockTokenizerManager:
"""Mock TokenizerManager for testing."""
def __init__(self, enable_lora=False):
self.server_args = MagicMock(spec=ServerArgs)
self.server_args.enable_lora = enable_lora
class ConcreteServingBase(OpenAIServingBase):
"""Concrete implementation for testing abstract base class."""
def _request_id_prefix(self) -> str:
return "test-"
def _convert_to_internal_request(self, request, raw_request=None):
pass
def _validate_request(self, request):
pass
class TestParseModelParameter(unittest.TestCase):
"""Test _parse_model_parameter method."""
def setUp(self):
publish_config(self)
self.tokenizer_manager = MockTokenizerManager(enable_lora=True)
self.serving = ConcreteServingBase(self.tokenizer_manager)
def test_model_without_adapter(self):
"""Test parsing model without adapter returns None for adapter."""
base_model, adapter = self.serving._parse_model_parameter("llama-3.1-8B")
self.assertEqual(base_model, "llama-3.1-8B")
self.assertIsNone(adapter)
def test_model_with_adapter(self):
"""Test parsing model with adapter extracts both parts."""
base_model, adapter = self.serving._parse_model_parameter(
"llama-3.1-8B:sql-expert"
)
self.assertEqual(base_model, "llama-3.1-8B")
self.assertEqual(adapter, "sql-expert")
def test_model_with_path_and_adapter(self):
"""Test parsing model path with slashes and adapter."""
base_model, adapter = self.serving._parse_model_parameter(
"meta-llama/Llama-3.1-8B-Instruct:adapter-name"
)
self.assertEqual(base_model, "meta-llama/Llama-3.1-8B-Instruct")
self.assertEqual(adapter, "adapter-name")
def test_model_with_multiple_colons(self):
"""Test that only first colon is used for splitting."""
base_model, adapter = self.serving._parse_model_parameter("model:adapter:extra")
self.assertEqual(base_model, "model")
self.assertEqual(adapter, "adapter:extra")
def test_model_with_whitespace(self):
"""Test that whitespace is stripped from both parts."""
base_model, adapter = self.serving._parse_model_parameter(
" model-name : adapter-name "
)
self.assertEqual(base_model, "model-name")
self.assertEqual(adapter, "adapter-name")
def test_model_with_empty_adapter(self):
"""Test model ending with colon returns None for adapter."""
base_model, adapter = self.serving._parse_model_parameter("model-name:")
self.assertEqual(base_model, "model-name")
self.assertIsNone(adapter)
def test_model_with_only_spaces_after_colon(self):
"""Test model with only whitespace after colon returns None for adapter."""
base_model, adapter = self.serving._parse_model_parameter("model-name: ")
self.assertEqual(base_model, "model-name")
self.assertIsNone(adapter)
class TestResolveLoraPath(unittest.TestCase):
"""Test _resolve_lora_path method."""
def setUp(self):
publish_config(self)
self.tokenizer_manager = MockTokenizerManager(enable_lora=True)
self.serving = ConcreteServingBase(self.tokenizer_manager)
def test_no_adapter_specified(self):
"""Test when neither model nor explicit lora_path has adapter."""
result = self.serving._resolve_lora_path("model-name", None)
self.assertIsNone(result)
def test_adapter_in_model_only(self):
"""Test adapter from model parameter when no explicit path."""
result = self.serving._resolve_lora_path("model:sql-expert", None)
self.assertEqual(result, "sql-expert")
def test_adapter_in_explicit_only(self):
"""Test adapter from explicit lora_path when not in model."""
result = self.serving._resolve_lora_path("model-name", "python-expert")
self.assertEqual(result, "python-expert")
def test_model_parameter_takes_precedence(self):
"""Test model parameter adapter takes precedence over explicit."""
result = self.serving._resolve_lora_path("model:sql-expert", "python-expert")
self.assertEqual(result, "sql-expert")
def test_with_list_explicit_lora_path(self):
"""Test that explicit list is returned when no model adapter."""
explicit = ["adapter1", "adapter2", None]
result = self.serving._resolve_lora_path("model-name", explicit)
self.assertEqual(result, explicit)
def test_model_adapter_overrides_list(self):
"""Test model adapter overrides even when explicit is a list."""
result = self.serving._resolve_lora_path(
"model:sql-expert", ["adapter1", "adapter2"]
)
self.assertEqual(result, "sql-expert")
def test_complex_model_name_with_adapter(self):
"""Test resolution with complex model name."""
result = self.serving._resolve_lora_path(
"org/model-v2.1:adapter-name", "other-adapter"
)
self.assertEqual(result, "adapter-name")
class TestIntegrationScenarios(unittest.TestCase):
"""Integration tests for common usage scenarios."""
def setUp(self):
publish_config(self)
self.tokenizer_manager = MockTokenizerManager(enable_lora=True)
self.serving = ConcreteServingBase(self.tokenizer_manager)
def test_openai_compatible_usage(self):
"""Test typical OpenAI-compatible usage pattern."""
# User specifies adapter in model parameter
model = "meta-llama/Llama-3.1-8B:sql-expert"
explicit_lora = None
lora_path = self.serving._resolve_lora_path(model, explicit_lora)
self.assertEqual(lora_path, "sql-expert")
def test_backward_compatible_usage(self):
"""Test backward-compatible usage with explicit lora_path."""
model = "meta-llama/Llama-3.1-8B"
explicit_lora = "sql-expert"
lora_path = self.serving._resolve_lora_path(model, explicit_lora)
self.assertEqual(lora_path, "sql-expert")
def test_base_model_usage(self):
"""Test using base model without any adapter."""
model = "meta-llama/Llama-3.1-8B"
explicit_lora = None
lora_path = self.serving._resolve_lora_path(model, explicit_lora)
self.assertIsNone(lora_path)
# No validation needed when no adapter
def test_batch_request_scenario(self):
"""Test batch request with list of adapters."""
model = "meta-llama/Llama-3.1-8B" # No adapter in model
explicit_lora = ["sql-expert", "python-expert", None]
lora_path = self.serving._resolve_lora_path(model, explicit_lora)
self.assertEqual(lora_path, explicit_lora)
def test_adapter_in_model_overrides_batch_list(self):
"""Test that adapter in model parameter overrides batch list."""
model = "meta-llama/Llama-3.1-8B:preferred-adapter"
explicit_lora = ["adapter1", "adapter2"]
lora_path = self.serving._resolve_lora_path(model, explicit_lora)
self.assertEqual(lora_path, "preferred-adapter")
class TestEdgeCases(unittest.TestCase):
"""Test edge cases and error conditions."""
def setUp(self):
publish_config(self)
self.tokenizer_manager = MockTokenizerManager(enable_lora=True)
self.serving = ConcreteServingBase(self.tokenizer_manager)
def test_empty_string_model(self):
"""Test handling of empty string model."""
base, adapter = self.serving._parse_model_parameter("")
self.assertEqual(base, "")
self.assertIsNone(adapter)
def test_only_colon(self):
"""Test model parameter that is just a colon."""
base, adapter = self.serving._parse_model_parameter(":")
self.assertEqual(base, "")
self.assertIsNone(adapter)
def test_empty_list_lora_path(self):
"""Test validation with empty list doesn't crash."""
lora_path = self.serving._resolve_lora_path("model-name", [])
# Empty list is falsy, so validation won't be called
self.assertEqual(lora_path, [])
def test_list_with_none_first(self):
"""Test validation finds first non-None adapter in list."""
lora_path = self.serving._resolve_lora_path("model-name", [None, "adapter2"])
self.assertEqual(lora_path, [None, "adapter2"])
# In actual usage, validation would find "adapter2"
def test_list_all_none(self):
"""Test validation with list of all None values."""
lora_path = self.serving._resolve_lora_path("model-name", [None, None])
self.assertEqual(lora_path, [None, None])
# In actual usage, no validation would occur (no non-None adapters)
def test_unicode_in_adapter_name(self):
"""Test Unicode characters in adapter name."""
base, adapter = self.serving._parse_model_parameter("model:adapter-名前")
self.assertEqual(base, "model")
self.assertEqual(adapter, "adapter-名前")
def test_special_characters_in_adapter(self):
"""Test special characters in adapter name."""
base, adapter = self.serving._parse_model_parameter("model:adapter_v2.1-final")
self.assertEqual(base, "model")
self.assertEqual(adapter, "adapter_v2.1-final")
def test_none_as_explicit_lora_path(self):
"""Test None as explicit lora_path is handled correctly."""
result = self.serving._resolve_lora_path("model:adapter", None)
self.assertEqual(result, "adapter")
def test_empty_string_as_explicit_lora_path(self):
"""Test empty string as explicit lora_path."""
result = self.serving._resolve_lora_path("model-name", "")
self.assertEqual(result, "")
if __name__ == "__main__":
unittest.main()