Files
sglang/test
zicai.pu ee9170e7bb fix(mooncake): bind IntraNode batches to the bootstrap CUDA device
Sender and executor threads can enter synchronous IntraNode transfers with
another current CUDA device. Keep the explicit bootstrap ordinal, check it
against the published model device, and scope the native batch call to that
device while restoring the calling thread on success or failure.

Ordinary transports, force-TCP and non-CUDA devices keep their existing
path. Missing or inconsistent CUDA identity is rejected before native IO.

Add 13 CPU regressions importing the production wrapper, including real
constructor calls with a mocked SDK, concurrent thread-local device states,
error restoration and transport/device compatibility. All 13 pass; two
selected tests fail on the unchanged wrapper at behavior assertions. Native
and CUDA boundaries are mocked, so this does not certify physical CUDA IPC
or attribute the separate container-PID collision to this change.
2026-10-30 11:42:19 +08:00
..

Test and Continuous Integration (CI) System in SGLang

This page covers principles and essentials: folder layout, how to run tests, registration, and suite selection. For complete references, see the skill guides:

CI Pipeline Overview

The CI pipeline runs in three sequential stages: A (pre-flight, ~3 min) → B (basic, ~30 min) → C (advanced, ~30 min). Kernel and multimodal-gen tests run in parallel with stage B. For details on stage gating, fail-fast mechanisms, execution modes (PR vs scheduled vs manual dispatch), and debugging CI failures, see the CI workflow guide.

Folder Organization

  • registered/: CI test files, including kernel tests and benchmarks, auto-discovered by run_suite.py.
  • manual/: Non-CI tests for local debugging or special setups.
  • run_suite.py: CI runner — scans registered/ recursively.

test/srt/ is retired. Put CI tests under registered/ and local-only tests under manual/; lint rejects files in the retired directory.

The system supports both unittest and pytest. The launcher runs python filename.py -f with failfast enabled by default.

Make sure your file ends with exactly one of:

# for unittest
if __name__ == "__main__":
    unittest.main()
# for pytest
if __name__ == "__main__":
    import sys
    sys.exit(pytest.main([__file__]))

Do not add custom argparse or modify sys.argv before these calls — the CI runner appends -f for failfast.

Run Tests Locally

# Single file
python3 test/registered/core/test_srt_endpoint.py

# Single test method
python3 test/registered/core/test_srt_endpoint.py TestSRTEndpoint.test_simple_decode

# Single JIT kernel test
python3 test/registered/kernels/ops/elementwise/test_add_constant.py

# Run a suite
python3 test/run_suite.py --hw cpu --suite base-a-test-cpu
python3 test/run_suite.py --hw cuda --suite base-a-test-1-gpu-small

# Nightly tests (CUDA nightly suites take no --nightly; the stage is in the name)
python3 test/run_suite.py --hw cuda --suite nightly-test-1-gpu-large

# With auto-partitioning (for parallel CI jobs)
python3 test/run_suite.py --hw cuda --suite base-b-test-1-gpu-small \
    --auto-partition-id 0 --auto-partition-size 4

CI Registration

Every CI-discovered test file must call a registration function at module level:

from sglang.test.ci.ci_register import register_cuda_ci

register_cuda_ci(est_time=80, stage="base-b", runner_config="1-gpu-small")

Parameters: est_time (seconds), stage + runner_config (target stage and runner pool from scripts/ci/runner_configs.yml), nightly=True (nightly-only), disabled="reason" (temporarily disable).

Keep est_time, stage, runner_config as literal values — run_suite.py collects them by AST parsing.

Directories under test/registered/ group tests by topic and are free-form (lora/, hicache/, disaggregation/, perf/, ...); unit tests cover one srt module, so they mirror the source tree under unit/. What a test costs, which stage gates it and which runner it needs are declared by its register_*_ci call -- including hardware, which is expressed by one or more register_*_ci calls and never by a new top-level directory. Kernel tests use test/registered/kernels/{ops,benchmark}/<group>/, retaining the established plural kernels root.

Diffusion workflows also enter through test/run_suite.py; registered bridge files preserve their case-level pytest partitioning until the remaining diffusion cases are moved out of the package test-support tree.

Kernel correctness tests and benchmarks use the established plural kernels root and mirror the operator group under python/sglang/kernels/ops/. Helpers stay alongside the kernel source under python/sglang/kernels/jit/ and are imported by absolute path:

  • Correctness tests: test/registered/kernels/ops/<group>/test_*.py → base-b-kernel-unit-test-1-gpu-large
  • Benchmarks: test/registered/kernels/benchmark/<group>/bench_*.py → base-b-kernel-benchmark-test-1-gpu-large

Choosing a Suite

Use the lightest suite that meets your test's needs. Full suite tables are in the write-sglang-test skill.

Need Suite
No GPU required base-a-test-cpu
Small GPU (fits 5090, 32GB) base-b-test-1-gpu-small (most tests go here)
Large GPU memory or Hopper features base-b-test-1-gpu-large
JIT kernel correctness base-b-kernel-unit-test-1-gpu-large
JIT kernel benchmarks base-b-kernel-benchmark-test-1-gpu-large
Multi-GPU (2/4/8) base-b-test-2-gpu-large, base-c-test-*
Long-running or experimental nightly-* suites

Steps for Adding a Test

See the write-sglang-test skill for templates, fixtures, model selection, and a complete checklist.

Before adding a registered test, identify the production change that would make it fail. Prefer extending an existing fixture/server launch over adding another file. The incremental admission check applies these ratchets to new or modified registered tests:

  • Temporary disabled= registrations and unconditional skips must reference an issue and include until YYYY-MM-DD; expired entries fail lint.
  • A file registered on CUDA plus another accelerator must place a nearby backend-specific: comment above the extra registration and name the path or failure mode that only that backend can catch.
  • Default PR registrations are limited to 1,200 estimated weighted accelerator-seconds per backend (est_time * GPU count). Move larger matrices to extra/nightly, or document a nearby ci-cost-override: rationale.

Multi-Hardware Backends

This README mostly describes the NVIDIA GPU CI pipeline. Other hardware backends (AMD, NPU) follow the same practices and use the multi-backend registry system. A scheduled job summarizes test coverage across all backends; here is an example run.

Tips

  • Learn from existing examples in test/registered.
  • Reuse servers — launching is expensive. Share one server across many test methods via setUpClass.
  • Use as few GPUs as possible. Prefer 1-GPU runners.
  • Each test file should take < 500 seconds; split if longer.
  • Each GitHub Actions job should take < 30 minutes; split if longer.
  • If tests are too slow for per-commit, consider nightly suites.
  • New GSM8K or MMLU accuracy checks should use sgl-eval (sglang.test.sgl_eval_utils.run_sgl_eval or sgl-eval run) instead of adding another dataset or evaluator copy.

Other Notes

Adding New Models to Nightly CI

  • Text models: Extend the global model list variables in test_utils.py.
  • VLMs: Extend the MODEL_THRESHOLDS dictionary in test/registered/accuracy/models/test_vlms_mmmu_eval.py.