Files
sglang/python/sglang
minke.yu b91137ab98
build-sglang-image / build (push) Successful in 27m52s
scheduler: CP-symmetric idle check for health-check admission
Health-check admit/skip used is_fully_idle(), which includes rank-local
hicache drain queues; ranks diverge right after activity, so one rank
dispatched the health-check generate while others piggyback-skipped,
deadlocking CP (hicache drain all_reduce vs CP request broadcast).
Seen on cp2/cp4 + hicache L3 after router health checks.

Recovered from b300 /data/ymk/sglang working copy (uncommitted WIP).
2026-09-24 12:01:57 +08:00
..
2026-09-22 14:56:35 +08:00

Code Structure

Folders

  • benchmark/: Benchmark implementations and dataset utilities.
  • cli/: Command-line interface commands and entrypoints.
  • kernels/: Kernel interfaces, implementations, selection, and debugging utilities shared by the runtimes.
  • lang/: Deprecated language frontend that is no longer actively maintained.
  • multimodal_gen/: Core runtime for image, video, and audio generation models, most of which are diffusion models.
  • srt/: Core runtime for autoregressive language models. (SRT = SGLang Runtime.)
  • test/: Shared test and evaluation utilities.

Files

  • README.md: This package structure overview.
  • __init__.py: Package initialization and public Python APIs.
  • bench_offline_throughput.py: Deprecated wrapper for the offline throughput benchmark.
  • bench_one_batch.py: Deprecated wrapper for the one-batch benchmark.
  • bench_one_batch_server.py: Deprecated wrapper for the server-based one-batch benchmark.
  • bench_serving.py: Deprecated wrapper for the online serving benchmark.
  • check_env.py: Environment and dependency diagnostics.
  • compile_deep_gemm.py: DeepGEMM kernel precompilation entrypoint.
  • launch_server.py: Compatibility entrypoint for launching an inference server.
  • profiler.py: Client entrypoint for collecting server profiling traces.
  • utils.py: Common package utilities.
  • version.py: Public package version resolution.