Cheng Wan
b99175dc7d
Last of four; stacked on #38048. The record is the operator's input; the bags are what is in effect. A reader that takes the record and reads a field off it gets the input, which is the wrong one of the two whenever resolution decided something -- and the mistake is silent, because for most fields and most launches the two agree. Several of these files already read both ways, sometimes in the same expression: ```python get_tokenizer( get_serving().tokenizer_path, tokenizer_mode=server_args.tokenizer_mode, # the input, not the decision ... ) ``` Sixty-odd files convert. Record field reads in runtime code go from 199 to 11. Nine parameters that the conversion emptied are dropped along with the argument at every call site -- the dead-parameter ratchet is what names them. ### "Runs after its process publishes" is a per-entry-point claim Most converted reads sit in the serving and model-executor layers, which only exist after publication, or in the two subprocess entry points, which publish first thing. Three places are not like that, and they keep reading the record they were handed: - **`HttpServerEngineAdapter`** launches the server as a *child*. The parent resolves the record and never publishes, so the adapter's own reads -- the launch banner, the API key in its readiness loop, the TP width in `update_weights_from_tensor` -- are of `self.server_args`. A bag read here fails closed in a bare process, or answers for an unrelated engine in one that happens to have published. - **`serve_grpc`** reads its sidecar port before the integrated servicer builds the `Engine` that publishes. The comment above that line already said so and already bound `cfg = resolving_view(server_args)` for it; the sidecar port and the port it derives from read `cfg`. - **`initialize_dp_attention`** runs from callers whose publish is not guaranteed, so its one predicate stays on the resolution view. `ROLE_NAMESPACE_SETS["dp_controller"]` gains `observability` and `serving`, because the controller's metrics gate, tracing setup and worker-port broadcast now read those namespaces. Under `SGLANG_ROLE_NAMESPACES=enforce` that set is what the process may read, so a conversion that reaches a new namespace has to widen it in the same change. ## Three things worth a reviewer's attention **Eleven reads were `getattr(record, "field", default)`.** An AST scan for attribute access does not see those, so the census that said "43 readers" was counting the shape it could match rather than the thing it was after. `incremental_streaming_output` was read that way twice, and the transcription tests were the only reason it surfaced. **Not every record read is a bag read waiting to happen.** A multimodal processor's `base_gpu_id` is the instance's, not the process's: two engines in one process keep different ones, and `test_publishing_another_config_does_not_move_the_device` exists to say so. It stays on the record while `rl_on_policy_target` beside it moves. `RequestMetricsExporter` is the same shape -- it is handed the directory it writes to, and a test builds several with different ones. `configure_logger` is a third: 17 call sites, one of which passes an `argparse.Namespace`, so it is not a global-context reader at all. Those eleven remaining reads are the ones with a reason. **The fixtures move with the code.** Tests that hung config off a mock manager now publish a record, which is what the serving layer reads; where a test states a value it says so with `override_server_args` instead of assigning through the mock. `test_hisparse_unit` is the last of them: it stubbed a `server_args` onto a fake scheduler to say the decode radix cache was off, and the value it was standing in for is the published default, so the stub goes and the class publishes. ## Two things CI caught that a local sweep could not **`unittest.TestCase.enterContext` is Python 3.11+.** The converted fixtures used it at 18 sites; `requires-python` is `>=3.10` and CI runs 3.10, so every one of them raised `AttributeError` there while passing on a newer local interpreter. They call `enter_override(self, ...)` now -- a four-line helper in `sglang/test/test_utils.py` over the override's own `install()` / `restore()`. **A batched sweep cannot see a missing publish.** Three fixtures needed a published config and did not have one; each *passed* inside a shard where some other file had published, and failed when run alone. The affected cases are `test_serving_completions` (which set `incremental_streaming_output` on the mock manager's record, where nothing reads it now), `test_qwen3_vl_feature_materialization` (same shape for `mm_enable_dp_encoder`), and the two Qwen Rust tests -- whose fixture already carried the comment `# Non-auto: get_resolved_model_impl would choke on a SimpleNamespace` next to the `model_impl` it sets, which is exactly what happened once `get_mm_processor_cls` started reading that value from the bag. Its `publish` mirrors `model_impl` now, like the four fields it already mirrored. ## Verification A full registered-unit sweep (648 files) against this stack's merge-base: 19 failures on both sides, the same 19, none of them config. That sweep is what caught 23 failures the file-scoped runs missed -- and, later, that the narrower 139-file list did not even contain the files this change reaches. It is also what caught the `test_hisparse_unit` fixture above: the file passes inside a shard where something else published, and fails when it is run on its own, which is why every failing file is re-run alone before it is counted.
Test and Continuous Integration (CI) System in SGLang
This page covers principles and essentials: folder layout, how to run tests, registration, and suite selection. For complete references, see the skill guides:
- Writing tests — templates, fixtures, model selection, complete suite tables, checklist:
.claude/skills/write-sglang-test/SKILL.md - CI pipeline internals — stage flow diagrams, fast-fail layers, gating, partitioning, execution modes, debugging failures:
.claude/skills/ci-workflow-guide/SKILL.md
CI Pipeline Overview
The CI pipeline runs in three sequential stages: A (pre-flight, ~3 min) → B (basic, ~30 min) → C (advanced, ~30 min). Kernel and multimodal-gen tests run in parallel with stage B. For details on stage gating, fast-fail mechanisms, execution modes (PR vs scheduled vs manual dispatch), and debugging CI failures, see the CI workflow guide.
Folder Organization
registered/: CI test files, auto-discovered byrun_suite.py. Most tests live here. JIT kernel tests are an exception (see below).manual/: Non-CI tests for local debugging or special setups.run_suite.py: CI runner — scansregistered/and JIT kernel directories.
The system supports both unittest and pytest. The launcher runs python filename.py -f with failfast enabled by default.
Make sure your file ends with exactly one of:
# for unittest
if __name__ == "__main__":
unittest.main()
# for pytest
if __name__ == "__main__":
import sys
sys.exit(pytest.main([__file__]))
Do not add custom argparse or modify sys.argv before these calls — the CI runner appends -f for failfast.
Run Tests Locally
# Single file
python3 test/registered/core/test_srt_endpoint.py
# Single test method
python3 test/registered/core/test_srt_endpoint.py TestSRTEndpoint.test_simple_decode
# Single JIT kernel test
python3 test/registered/jit/test_add_constant.py
# Run a suite
python3 test/run_suite.py --hw cpu --suite base-a-test-cpu
python3 test/run_suite.py --hw cuda --suite base-a-test-1-gpu-small
# Nightly tests (CUDA nightly suites take no --nightly; the stage is in the name)
python3 test/run_suite.py --hw cuda --suite nightly-test-1-gpu-large
# With auto-partitioning (for parallel CI jobs)
python3 test/run_suite.py --hw cuda --suite base-b-test-1-gpu-small \
--auto-partition-id 0 --auto-partition-size 4
CI Registration
Every CI-discovered test file must call a registration function at module level:
from sglang.test.ci.ci_register import register_cuda_ci
register_cuda_ci(est_time=80, stage="base-b", runner_config="1-gpu-small")
Parameters: est_time (seconds), stage + runner_config (target stage and runner pool from scripts/ci/runner_configs.yml), nightly=True (nightly-only), disabled="reason" (temporarily disable).
Keep est_time, stage, runner_config as literal values — run_suite.py collects them by AST parsing.
JIT kernel correctness tests and benchmarks live under test/registered/jit/, same as other registered tests (their helpers stay alongside the kernel source under python/sglang/kernels/jit/ and are imported by absolute path):
- Correctness tests:
test/registered/jit/test_*.py→base-b-kernel-unit-test-1-gpu-large - Benchmarks:
test/registered/jit/benchmark/bench_*.py→base-b-kernel-benchmark-test-1-gpu-large
Choosing a Suite
Use the lightest suite that meets your test's needs. Full suite tables are in the write-sglang-test skill.
| Need | Suite |
|---|---|
| No GPU required | base-a-test-cpu |
| Small GPU (fits 5090, 32GB) | base-b-test-1-gpu-small (most tests go here) |
| Large GPU memory or Hopper features | base-b-test-1-gpu-large |
| JIT kernel correctness | base-b-kernel-unit-test-1-gpu-large |
| JIT kernel benchmarks | base-b-kernel-benchmark-test-1-gpu-large |
| Multi-GPU (2/4/8) | base-b-test-2-gpu-large, base-c-test-* |
| Long-running or experimental | nightly-* suites |
Steps for Adding a Test
See the write-sglang-test skill for templates, fixtures, model selection, and a complete checklist.
Multi-Hardware Backends
This README mostly describes the NVIDIA GPU CI pipeline. Other hardware backends (AMD, NPU) follow the same practices and use the multi-backend registry system. A scheduled job summarizes test coverage across all backends; here is an example run.
Tips
- Learn from existing examples in test/registered.
- Reuse servers — launching is expensive. Share one server across many test methods via
setUpClass. - Use as few GPUs as possible. Prefer 1-GPU runners.
- Each test file should take < 500 seconds; split if longer.
- Each GitHub Actions job should take < 30 minutes; split if longer.
- If tests are too slow for per-commit, consider nightly suites.
Other Notes
Adding New Models to Nightly CI
- Text models: Extend the global model list variables in
test_utils.py. - VLMs: Extend the
MODEL_THRESHOLDSdictionary intest/registered/eval/test_vlms_mmmu_eval.py.