--- title: Welcome to SGLang description: High-performance serving framework for large language and multimodal models. keywords: - sglang - llm serving - multimodal - inference runtime mode: wide --- import { popularModels } from "/src/snippets/configs/popular-models.jsx"; import { PopularModels } from "/src/snippets/_popular_models.jsx"; {/* One hero per model, rotating. The Cookbook landing page renders the same list as a compact strip. Edit the list, not this page. */} Star Fork

Designed for low-latency, high-throughput inference with RadixAttention, prefix caching, and multi-GPU parallelism. Broad support for Llama, Qwen, DeepSeek, and more. Compatible with Hugging Face and OpenAI APIs. Native support across Hardware Platforms including NVIDIA, AMD, Intel Xeon, Google TPU, and Ascend NPU accelerators. Open-source with widespread adoption, powering 400k+ GPUs and integrated with major RL frameworks. SGLang powers large-scale production deployments, generating trillions of tokens each day across more than 400,000 GPUs worldwide. It is hosted under the non-profit open-source organization [LMSYS](https://lmsys.org/about/). --- ## Get Started SGLang is an inference framework meant for production level serving. It is designed to deliver low-latency and high-throughput inference across a wide range of setups, from a single GPU to large distributed clusters. Install SGLang with pip, from source, or via Docker on your preferred hardware platform. Launch your first model server and send requests in minutes with OpenAI-compatible APIs. ## News and latest blogs {/* BEGIN_LMSYS_SGLANG_BLOG_CARDS */}
Towards Blackwell-Native 8-bit and 4-bit RL: End-to-End MXFP8 and NVFP4 RL in Miles

{"Towards Blackwell-Native 8-bit and 4-bit RL: End-to-End MXFP8 and NVFP4 RL in Miles"}

{"July 29, 2026"}

SGLang and Miles Add Day-0 Support for Kimi K3

{"SGLang and Miles Add Day-0 Support for Kimi K3"}

{"July 27, 2026"}

OPD Support in Miles

{"OPD Support in Miles"}

{"July 18, 2026"}

SGLang and Miles Add Day-0 Support for Inkling, a Frontier Multimodal Model

{"SGLang and Miles Add Day-0 Support for Inkling, a Frontier Multimodal Model"}

{"July 15, 2026"}

Serving GLM5.2 NVFP4 Agentic Workload with SGLang: Reaching 500 TPS in 2 Weeks

{"Serving GLM5.2 NVFP4 Agentic Workload with SGLang: Reaching 500 TPS in 2 Weeks"}

{"July 14, 2026"}

Bringing DeepSeek-V4 Flash RL Training to AMD Instinct MI355X GPUs with Miles

{"Bringing DeepSeek-V4 Flash RL Training to AMD Instinct MI355X GPUs with Miles"}

{"July 10, 2026"}

{/* END_LMSYS_SGLANG_BLOG_CARDS */} --- ## Learn more and join the community

Stay connected

{" "} Development roadmap to follow current priorities and upcoming work.
{" "} Weekly public development meeting to hear updates and join open discussions.
{" "} Slack for questions, feedback, and community support.
X Twitter and {" "} LinkedIn for project updates.
{" "} LMSYS blog for release notes, benchmarks, and technical deep dives.
{" "} Learning materials for blogs, slides, and videos.