Files
sglang/python/sglang
minke.yu f9056f0fcd fix(dsv4 fp4_indexer): 128B-align SM90 paged score rows for deep_select
fp4_index_logits_paged padded rows to 8 floats (32B) and
fp4_index_logits_candidates did not pad at all, but the tvm-ffi
deep_select topk kernel enforces kInputAlignmentBytes=128 on the input
row stride. bs>1 decode/CUDA-graph capture died with
'Tensor stride for dimension 0 is not aligned to 128 bytes'.
Candidates kernel now takes a separate OUT_STRIDE constexpr.
2026-10-19 20:55:01 +08:00
..

Code Structure

Folders

  • benchmark/: Benchmark implementations and dataset utilities.
  • cli/: Command-line interface commands and entrypoints.
  • kernels/: Kernel interfaces, implementations, selection, and debugging utilities shared by the runtimes.
  • lang/: Deprecated language frontend that is no longer actively maintained.
  • multimodal_gen/: Core runtime for image, video, and audio generation models, most of which are diffusion models.
  • srt/: Core runtime for autoregressive language models. (SRT = SGLang Runtime.)
  • test/: Shared test and evaluation utilities.

Files

  • README.md: This package structure overview.
  • __init__.py: Package initialization and public Python APIs.
  • bench_one_batch_server.py: Deprecated wrapper for the server-based one-batch benchmark.
  • bench_serving.py: Deprecated wrapper for the online serving benchmark.
  • check_env.py: Environment and dependency diagnostics.
  • compile_deep_gemm.py: DeepGEMM kernel precompilation entrypoint.
  • launch_server.py: Compatibility entrypoint for launching an inference server.
  • profiler.py: Client entrypoint for collecting server profiling traces.
  • utils.py: Common package utilities.
  • version.py: Public package version resolution.