Sender and executor threads can enter synchronous IntraNode transfers with another current CUDA device. Keep the explicit bootstrap ordinal, check it against the published model device, and scope the native batch call to that device while restoring the calling thread on success or failure. Ordinary transports, force-TCP and non-CUDA devices keep their existing path. Missing or inconsistent CUDA identity is rejected before native IO. Add 13 CPU regressions importing the production wrapper, including real constructor calls with a mocked SDK, concurrent thread-local device states, error restoration and transport/device compatibility. All 13 pass; two selected tests fail on the unchanged wrapper at behavior assertions. Native and CUDA boundaries are mocked, so this does not certify physical CUDA IPC or attribute the separate container-PID collision to this change.
Test and Continuous Integration (CI) System in SGLang
This page covers principles and essentials: folder layout, how to run tests, registration, and suite selection. For complete references, see the skill guides:
- Writing tests — templates, fixtures, model selection, complete suite tables, checklist:
.claude/skills/write-sglang-test/SKILL.md - CI pipeline internals — stage flow diagrams, fail-fast layers, gating, partitioning, execution modes, debugging failures:
.claude/skills/ci-workflow-guide/SKILL.md
CI Pipeline Overview
The CI pipeline runs in three sequential stages: A (pre-flight, ~3 min) → B (basic, ~30 min) → C (advanced, ~30 min). Kernel and multimodal-gen tests run in parallel with stage B. For details on stage gating, fail-fast mechanisms, execution modes (PR vs scheduled vs manual dispatch), and debugging CI failures, see the CI workflow guide.
Folder Organization
registered/: CI test files, including kernel tests and benchmarks, auto-discovered byrun_suite.py.manual/: Non-CI tests for local debugging or special setups.run_suite.py: CI runner — scansregistered/recursively.
test/srt/ is retired. Put CI tests under registered/ and local-only tests
under manual/; lint rejects files in the retired directory.
The system supports both unittest and pytest. The launcher runs python filename.py -f with failfast enabled by default.
Make sure your file ends with exactly one of:
# for unittest
if __name__ == "__main__":
unittest.main()
# for pytest
if __name__ == "__main__":
import sys
sys.exit(pytest.main([__file__]))
Do not add custom argparse or modify sys.argv before these calls — the CI runner appends -f for failfast.
Run Tests Locally
# Single file
python3 test/registered/core/test_srt_endpoint.py
# Single test method
python3 test/registered/core/test_srt_endpoint.py TestSRTEndpoint.test_simple_decode
# Single JIT kernel test
python3 test/registered/kernels/ops/elementwise/test_add_constant.py
# Run a suite
python3 test/run_suite.py --hw cpu --suite base-a-test-cpu
python3 test/run_suite.py --hw cuda --suite base-a-test-1-gpu-small
# Nightly tests (CUDA nightly suites take no --nightly; the stage is in the name)
python3 test/run_suite.py --hw cuda --suite nightly-test-1-gpu-large
# With auto-partitioning (for parallel CI jobs)
python3 test/run_suite.py --hw cuda --suite base-b-test-1-gpu-small \
--auto-partition-id 0 --auto-partition-size 4
CI Registration
Every CI-discovered test file must call a registration function at module level:
from sglang.test.ci.ci_register import register_cuda_ci
register_cuda_ci(est_time=80, stage="base-b", runner_config="1-gpu-small")
Parameters: est_time (seconds), stage + runner_config (target stage and runner pool from scripts/ci/runner_configs.yml), nightly=True (nightly-only), disabled="reason" (temporarily disable).
Keep est_time, stage, runner_config as literal values — run_suite.py collects them by AST parsing.
Directories under test/registered/ group tests by topic and are free-form
(lora/, hicache/, disaggregation/, perf/, ...); unit tests cover one srt
module, so they mirror the source tree under unit/. What a test costs, which
stage gates it and which runner it needs are declared by its register_*_ci
call -- including hardware, which is expressed by one or more register_*_ci
calls and never by a new top-level directory. Kernel tests use
test/registered/kernels/{ops,benchmark}/<group>/, retaining the established
plural kernels root.
Diffusion workflows also enter through test/run_suite.py; registered bridge
files preserve their case-level pytest partitioning until the remaining
diffusion cases are moved out of the package test-support tree.
Kernel correctness tests and benchmarks use the established plural kernels
root and mirror the operator group under python/sglang/kernels/ops/. Helpers
stay alongside the kernel source under python/sglang/kernels/jit/ and are
imported by absolute path:
- Correctness tests:
test/registered/kernels/ops/<group>/test_*.py→base-b-kernel-unit-test-1-gpu-large - Benchmarks:
test/registered/kernels/benchmark/<group>/bench_*.py→base-b-kernel-benchmark-test-1-gpu-large
Choosing a Suite
Use the lightest suite that meets your test's needs. Full suite tables are in the write-sglang-test skill.
| Need | Suite |
|---|---|
| No GPU required | base-a-test-cpu |
| Small GPU (fits 5090, 32GB) | base-b-test-1-gpu-small (most tests go here) |
| Large GPU memory or Hopper features | base-b-test-1-gpu-large |
| JIT kernel correctness | base-b-kernel-unit-test-1-gpu-large |
| JIT kernel benchmarks | base-b-kernel-benchmark-test-1-gpu-large |
| Multi-GPU (2/4/8) | base-b-test-2-gpu-large, base-c-test-* |
| Long-running or experimental | nightly-* suites |
Steps for Adding a Test
See the write-sglang-test skill for templates, fixtures, model selection, and a complete checklist.
Before adding a registered test, identify the production change that would make it fail. Prefer extending an existing fixture/server launch over adding another file. The incremental admission check applies these ratchets to new or modified registered tests:
- Temporary
disabled=registrations and unconditional skips must reference an issue and includeuntil YYYY-MM-DD; expired entries fail lint. - A file registered on CUDA plus another accelerator must place a nearby
backend-specific:comment above the extra registration and name the path or failure mode that only that backend can catch. - Default PR registrations are limited to 1,200 estimated weighted accelerator-seconds
per backend (
est_time * GPU count). Move larger matrices to extra/nightly, or document a nearbyci-cost-override:rationale.
Multi-Hardware Backends
This README mostly describes the NVIDIA GPU CI pipeline. Other hardware backends (AMD, NPU) follow the same practices and use the multi-backend registry system. A scheduled job summarizes test coverage across all backends; here is an example run.
Tips
- Learn from existing examples in test/registered.
- Reuse servers — launching is expensive. Share one server across many test methods via
setUpClass. - Use as few GPUs as possible. Prefer 1-GPU runners.
- Each test file should take < 500 seconds; split if longer.
- Each GitHub Actions job should take < 30 minutes; split if longer.
- If tests are too slow for per-commit, consider nightly suites.
- New GSM8K or MMLU accuracy checks should use sgl-eval (
sglang.test.sgl_eval_utils.run_sgl_evalorsgl-eval run) instead of adding another dataset or evaluator copy.
Other Notes
Adding New Models to Nightly CI
- Text models: Extend the global model list variables in
test_utils.py. - VLMs: Extend the
MODEL_THRESHOLDSdictionary intest/registered/accuracy/models/test_vlms_mmmu_eval.py.