Files
sglang/test
冯骐 8c6cbd8c40 Support DSpark decoder bounded replay with DSV4.1 vision CP
Use tail-aware CP gathers and logits metadata, preserve original CP ownership of canonical routing IDs, and pass global token indices to DSpark. Adapt mHC state slicing and register CPU regression tests. Based on PR40219; tail handling informed by PR40161.
2026-10-20 10:20:57 +08:00
..

Test and Continuous Integration (CI) System in SGLang

This page covers principles and essentials: folder layout, how to run tests, registration, and suite selection. For complete references, see the skill guides:

CI Pipeline Overview

The CI pipeline runs in three sequential stages: A (pre-flight, ~3 min) → B (basic, ~30 min) → C (advanced, ~30 min). Kernel and multimodal-gen tests run in parallel with stage B. For details on stage gating, fail-fast mechanisms, execution modes (PR vs scheduled vs manual dispatch), and debugging CI failures, see the CI workflow guide.

Folder Organization

  • registered/: CI test files, including kernel tests and benchmarks, auto-discovered by run_suite.py.
  • manual/: Non-CI tests for local debugging or special setups.
  • run_suite.py: CI runner — scans registered/ recursively.

The system supports both unittest and pytest. The launcher runs python filename.py -f with failfast enabled by default.

Make sure your file ends with exactly one of:

# for unittest
if __name__ == "__main__":
    unittest.main()
# for pytest
if __name__ == "__main__":
    import sys
    sys.exit(pytest.main([__file__]))

Do not add custom argparse or modify sys.argv before these calls — the CI runner appends -f for failfast.

Run Tests Locally

# Single file
python3 test/registered/core/test_srt_endpoint.py

# Single test method
python3 test/registered/core/test_srt_endpoint.py TestSRTEndpoint.test_simple_decode

# Single JIT kernel test
python3 test/registered/kernels/ops/elementwise/test_add_constant.py

# Run a suite
python3 test/run_suite.py --hw cpu --suite base-a-test-cpu
python3 test/run_suite.py --hw cuda --suite base-a-test-1-gpu-small

# Nightly tests (CUDA nightly suites take no --nightly; the stage is in the name)
python3 test/run_suite.py --hw cuda --suite nightly-test-1-gpu-large

# With auto-partitioning (for parallel CI jobs)
python3 test/run_suite.py --hw cuda --suite base-b-test-1-gpu-small \
    --auto-partition-id 0 --auto-partition-size 4

CI Registration

Every CI-discovered test file must call a registration function at module level:

from sglang.test.ci.ci_register import register_cuda_ci

register_cuda_ci(est_time=80, stage="base-b", runner_config="1-gpu-small")

Parameters: est_time (seconds), stage + runner_config (target stage and runner pool from scripts/ci/runner_configs.yml), nightly=True (nightly-only), disabled="reason" (temporarily disable).

Keep est_time, stage, runner_config as literal values — run_suite.py collects them by AST parsing.

Directories under test/registered/ group tests by topic and are free-form (lora/, hicache/, disaggregation/, perf/, ...); unit tests cover one srt module, so they mirror the source tree under unit/. What a test costs, which stage gates it and which runner it needs are declared by its register_*_ci call -- including hardware, which is expressed by one or more register_*_ci calls and never by a new top-level directory. Kernel tests use test/registered/kernels/{ops,benchmark}/<group>/, retaining the established plural kernels root.

Diffusion workflows also enter through test/run_suite.py; registered bridge files preserve their case-level pytest partitioning until the remaining diffusion cases are moved out of the package test-support tree.

Kernel correctness tests and benchmarks use the established plural kernels root and mirror the operator group under python/sglang/kernels/ops/. Helpers stay alongside the kernel source under python/sglang/kernels/jit/ and are imported by absolute path:

  • Correctness tests: test/registered/kernels/ops/<group>/test_*.py → base-b-kernel-unit-test-1-gpu-large
  • Benchmarks: test/registered/kernels/benchmark/<group>/bench_*.py → base-b-kernel-benchmark-test-1-gpu-large

Choosing a Suite

Use the lightest suite that meets your test's needs. Full suite tables are in the write-sglang-test skill.

Need Suite
No GPU required base-a-test-cpu
Small GPU (fits 5090, 32GB) base-b-test-1-gpu-small (most tests go here)
Large GPU memory or Hopper features base-b-test-1-gpu-large
JIT kernel correctness base-b-kernel-unit-test-1-gpu-large
JIT kernel benchmarks base-b-kernel-benchmark-test-1-gpu-large
Multi-GPU (2/4/8) base-b-test-2-gpu-large, base-c-test-*
Long-running or experimental nightly-* suites

Steps for Adding a Test

See the write-sglang-test skill for templates, fixtures, model selection, and a complete checklist.

Before adding a registered test, identify the production change that would make it fail. Prefer extending an existing fixture/server launch over adding another file. The incremental admission check applies these ratchets to new or modified registered tests:

  • Temporary disabled= registrations and unconditional skips must reference an issue and include until YYYY-MM-DD; expired entries fail lint.
  • A file registered on CUDA plus another accelerator must place a nearby backend-specific: comment above the extra registration and name the path or failure mode that only that backend can catch.
  • Default PR registrations are limited to 1,200 estimated weighted accelerator-seconds per backend (est_time * GPU count). Move larger matrices to extra/nightly, or document a nearby ci-cost-override: rationale.

Multi-Hardware Backends

This README mostly describes the NVIDIA GPU CI pipeline. Other hardware backends (AMD, NPU) follow the same practices and use the multi-backend registry system. A scheduled job summarizes test coverage across all backends; here is an example run.

Tips

  • Learn from existing examples in test/registered.
  • Reuse servers — launching is expensive. Share one server across many test methods via setUpClass.
  • Use as few GPUs as possible. Prefer 1-GPU runners.
  • Each test file should take < 500 seconds; split if longer.
  • Each GitHub Actions job should take < 30 minutes; split if longer.
  • If tests are too slow for per-commit, consider nightly suites.
  • New GSM8K or MMLU accuracy checks should use sgl-eval (sglang.test.sgl_eval_utils.run_sgl_eval or sgl-eval run) instead of adding another dataset or evaluator copy.

Other Notes

Adding New Models to Nightly CI

  • Text models: Extend the global model list variables in test_utils.py.
  • VLMs: Extend the MODEL_THRESHOLDS dictionary in test/registered/accuracy/models/test_vlms_mmmu_eval.py.