Use tail-aware CP gathers and logits metadata, preserve original CP ownership of canonical routing IDs, and pass global token indices to DSpark. Adapt mHC state slicing and register CPU regression tests. Based on PR40219; tail handling informed by PR40161.
Test and Continuous Integration (CI) System in SGLang
This page covers principles and essentials: folder layout, how to run tests, registration, and suite selection. For complete references, see the skill guides:
- Writing tests — templates, fixtures, model selection, complete suite tables, checklist:
.claude/skills/write-sglang-test/SKILL.md - CI pipeline internals — stage flow diagrams, fail-fast layers, gating, partitioning, execution modes, debugging failures:
.claude/skills/ci-workflow-guide/SKILL.md
CI Pipeline Overview
The CI pipeline runs in three sequential stages: A (pre-flight, ~3 min) → B (basic, ~30 min) → C (advanced, ~30 min). Kernel and multimodal-gen tests run in parallel with stage B. For details on stage gating, fail-fast mechanisms, execution modes (PR vs scheduled vs manual dispatch), and debugging CI failures, see the CI workflow guide.
Folder Organization
registered/: CI test files, including kernel tests and benchmarks, auto-discovered byrun_suite.py.manual/: Non-CI tests for local debugging or special setups.run_suite.py: CI runner — scansregistered/recursively.
The system supports both unittest and pytest. The launcher runs python filename.py -f with failfast enabled by default.
Make sure your file ends with exactly one of:
# for unittest
if __name__ == "__main__":
unittest.main()
# for pytest
if __name__ == "__main__":
import sys
sys.exit(pytest.main([__file__]))
Do not add custom argparse or modify sys.argv before these calls — the CI runner appends -f for failfast.
Run Tests Locally
# Single file
python3 test/registered/core/test_srt_endpoint.py
# Single test method
python3 test/registered/core/test_srt_endpoint.py TestSRTEndpoint.test_simple_decode
# Single JIT kernel test
python3 test/registered/kernels/ops/elementwise/test_add_constant.py
# Run a suite
python3 test/run_suite.py --hw cpu --suite base-a-test-cpu
python3 test/run_suite.py --hw cuda --suite base-a-test-1-gpu-small
# Nightly tests (CUDA nightly suites take no --nightly; the stage is in the name)
python3 test/run_suite.py --hw cuda --suite nightly-test-1-gpu-large
# With auto-partitioning (for parallel CI jobs)
python3 test/run_suite.py --hw cuda --suite base-b-test-1-gpu-small \
--auto-partition-id 0 --auto-partition-size 4
CI Registration
Every CI-discovered test file must call a registration function at module level:
from sglang.test.ci.ci_register import register_cuda_ci
register_cuda_ci(est_time=80, stage="base-b", runner_config="1-gpu-small")
Parameters: est_time (seconds), stage + runner_config (target stage and runner pool from scripts/ci/runner_configs.yml), nightly=True (nightly-only), disabled="reason" (temporarily disable).
Keep est_time, stage, runner_config as literal values — run_suite.py collects them by AST parsing.
Directories under test/registered/ group tests by topic and are free-form
(lora/, hicache/, disaggregation/, perf/, ...); unit tests cover one srt
module, so they mirror the source tree under unit/. What a test costs, which
stage gates it and which runner it needs are declared by its register_*_ci
call -- including hardware, which is expressed by one or more register_*_ci
calls and never by a new top-level directory. Kernel tests use
test/registered/kernels/{ops,benchmark}/<group>/, retaining the established
plural kernels root.
Diffusion workflows also enter through test/run_suite.py; registered bridge
files preserve their case-level pytest partitioning until the remaining
diffusion cases are moved out of the package test-support tree.
Kernel correctness tests and benchmarks use the established plural kernels
root and mirror the operator group under python/sglang/kernels/ops/. Helpers
stay alongside the kernel source under python/sglang/kernels/jit/ and are
imported by absolute path:
- Correctness tests:
test/registered/kernels/ops/<group>/test_*.py→base-b-kernel-unit-test-1-gpu-large - Benchmarks:
test/registered/kernels/benchmark/<group>/bench_*.py→base-b-kernel-benchmark-test-1-gpu-large
Choosing a Suite
Use the lightest suite that meets your test's needs. Full suite tables are in the write-sglang-test skill.
| Need | Suite |
|---|---|
| No GPU required | base-a-test-cpu |
| Small GPU (fits 5090, 32GB) | base-b-test-1-gpu-small (most tests go here) |
| Large GPU memory or Hopper features | base-b-test-1-gpu-large |
| JIT kernel correctness | base-b-kernel-unit-test-1-gpu-large |
| JIT kernel benchmarks | base-b-kernel-benchmark-test-1-gpu-large |
| Multi-GPU (2/4/8) | base-b-test-2-gpu-large, base-c-test-* |
| Long-running or experimental | nightly-* suites |
Steps for Adding a Test
See the write-sglang-test skill for templates, fixtures, model selection, and a complete checklist.
Before adding a registered test, identify the production change that would make it fail. Prefer extending an existing fixture/server launch over adding another file. The incremental admission check applies these ratchets to new or modified registered tests:
- Temporary
disabled=registrations and unconditional skips must reference an issue and includeuntil YYYY-MM-DD; expired entries fail lint. - A file registered on CUDA plus another accelerator must place a nearby
backend-specific:comment above the extra registration and name the path or failure mode that only that backend can catch. - Default PR registrations are limited to 1,200 estimated weighted accelerator-seconds
per backend (
est_time * GPU count). Move larger matrices to extra/nightly, or document a nearbyci-cost-override:rationale.
Multi-Hardware Backends
This README mostly describes the NVIDIA GPU CI pipeline. Other hardware backends (AMD, NPU) follow the same practices and use the multi-backend registry system. A scheduled job summarizes test coverage across all backends; here is an example run.
Tips
- Learn from existing examples in test/registered.
- Reuse servers — launching is expensive. Share one server across many test methods via
setUpClass. - Use as few GPUs as possible. Prefer 1-GPU runners.
- Each test file should take < 500 seconds; split if longer.
- Each GitHub Actions job should take < 30 minutes; split if longer.
- If tests are too slow for per-commit, consider nightly suites.
- New GSM8K or MMLU accuracy checks should use sgl-eval (
sglang.test.sgl_eval_utils.run_sgl_evalorsgl-eval run) instead of adding another dataset or evaluator copy.
Other Notes
Adding New Models to Nightly CI
- Text models: Extend the global model list variables in
test_utils.py. - VLMs: Extend the
MODEL_THRESHOLDSdictionary intest/registered/accuracy/models/test_vlms_mmmu_eval.py.