minke.yu/sglang

Fork 0

Files

T

History

2969ab3d41 [MLX] Window-bounded SWA KV storage and in-graph sampling (#34166 )

Co-authored-by: Siming Deng <siming_deng_stat@163.com>
Co-authored-by: R0CKSTAR <yeahdongcn@gmail.com>
Co-authored-by: Jiminator <Jiminator@users.noreply.github.com>
Co-authored-by: damahua <damahua@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

2026-08-09 21:28:51 -07:00

benchmark

config: derive the runner's DCP topology from its ParallelState (#34133 )

2026-08-09 01:18:24 -07:00

cli

[diffusion] fix: fix local-path detection for MiniMax-H3 and other non-diffusers models (#33365 )

2026-08-04 14:03:35 +08:00

eval

…

kernels

[JIT Kernel][DSv4] Optimize epilogue of c128 (#26671 )

2026-08-10 10:32:43 +08:00

lang

config: retire ServerArgs.override in favour of derive()

2026-08-05 19:30:53 -07:00

multimodal_gen

[diffusion] chore: make torch.compile opt-in for speed mode (#34173 )

2026-08-10 10:22:16 +08:00

srt

[MLX] Window-bounded SWA KV storage and in-graph sampling (#34166 )

2026-08-09 21:28:51 -07:00

test

[NPU] Increase the retry count to 3 for the GSM8K. (#34103 )

2026-08-10 10:55:41 +08:00

__init__.py

[core] Consolidate compiled-kernel caches under SGLANG_CACHE_DIR (#32434 )

2026-08-05 13:54:27 -07:00

_mps_stub.py

[lint] Enable Ruff UP037 to drop redundant quoted annotations (#27984 )

2026-06-11 17:38:05 -07:00

_triton_stub.py

…

bench_offline_throughput.py

[misc] Add bench_serving compatibility shim (#28997 )

2026-06-23 19:49:53 -07:00

bench_one_batch_server.py

[misc] Unify benchmark deprecation shims and one_batch_server CLI entrypoint (#29108 )

2026-06-23 21:10:25 -07:00

bench_one_batch.py

[misc] Add bench_serving compatibility shim (#28997 )

2026-06-23 19:49:53 -07:00

bench_serving.py

[misc] Unify benchmark deprecation shims and one_batch_server CLI entrypoint (#29108 )

2026-06-23 21:10:25 -07:00

check_env.py

…

compile_deep_gemm.py

config: retire ServerArgs.override in favour of derive()

2026-08-05 19:30:53 -07:00

global_config.py

…

kernel_api_logging.py

…

launch_server.py

[gRPC] Native server: launcher + HTTP + server args wiring (3/4) (#23508 )

2026-07-07 14:57:25 -07:00

profiler.py

…

README.md

…

utils.py

[diffusion] fix: fix local-path detection for MiniMax-H3 and other non-diffusers models (#33365 )

2026-08-04 14:03:35 +08:00

version.py

…

README.md

Code Structure

eval: The evaluation utilities.
lang: The frontend language.
multimodal_gen: Inference framework for accelerated image/video generation.
srt: The backend engine for running local models. (SRT = SGLang Runtime).
test: The test utilities.
api.py: The public APIs.
bench_offline_throughput.py: Benchmark the performance in the offline mode.
bench_one_batch.py: Benchmark the latency of running a single static batch without a server.
bench_one_batch_server.py: Benchmark the latency of running a single batch with a server.
bench_serving.py: Benchmark online serving with dynamic requests.
check_env.py: Check the environment variables and dependencies.
global_config.py: The global configs and constants.
launch_server.py: The entry point for launching a local server.
profiler.py: The profiling entry point to send profile requests.
utils.py: Common utilities.
version.py: Version info.