minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

Add new Mintlify documentation site (docs_new/) (#23001)

Browse Source 
Co-authored-by: AdityaVKochar <adityavardhankochar@gmail.com>
Co-authored-by: mintlify[bot] <109931778+mintlify[bot]@users.noreply.github.com>
Co-authored-by: adhyan-jain <adhyanjain2006@gmail.com>
Co-authored-by: Adhyan Jain <71976554+adhyan-jain@users.noreply.github.com>
Co-authored-by: Maitri-shah29 <maitrirajivshah@gmail.com>
Co-authored-by: Adarsh Shirawalmath <114558126+adarshxs@users.noreply.github.com>
Co-authored-by: Maitri Shah <shah29maitri@gmail.com>
Co-authored-by: Aditya Vardhan Kochar <80113212+AdityaVKochar@users.noreply.github.com>
Co-authored-by: Rishit Shivam <164783543+pokymono@users.noreply.github.com>
Co-authored-by: Rishitshivam <164783543+Rishitshivam@users.noreply.github.com>
Co-authored-by: IshhanKheria <ishhankheria06@gmail.com>
Co-authored-by: Ishita Joshi <ishitata.joshi@gmail.com>
Co-authored-by: Richard Chen <104477092+Richardczl98@users.noreply.github.com>
Co-authored-by: longGGGGGG <553746008@qq.com>
Co-authored-by: Richard <richardchen@radixark.ai>
Co-authored-by: Nakul Sinha <nakul.new4socials@gmail.com>
Co-authored-by: Divyam Agrawal <ludicrouslytrue@gmail.com>
Co-authored-by: Richardczl98 <Zhenlinc@stanford.edu>
Co-authored-by: Krishang Zinzuwadia <krishangzinzuwadia@gmail.com>
Co-authored-by: nimeshas <nimesha.s106@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Jignas Paturu <86356085+JignasP@users.noreply.github.com>
Co-authored-by: zijiexia <37504505+zijiexia@users.noreply.github.com>
This commit is contained in:
Mingyi
2026-04-20 15:10:22 -07:00
committed by GitHub
co-authored by AdityaVKochar mintlify[bot] adhyan-jain Adhyan Jain Maitri-shah29 Adarsh Shirawalmath Maitri Shah Aditya Vardhan Kochar Rishit Shivam Rishitshivam IshhanKheria Ishita Joshi Richard Chen longGGGGGG Richard Nakul Sinha Divyam Agrawal Richardczl98 Krishang Zinzuwadia nimeshas Claude Opus 4.6 github-actions[bot] Jignas Paturu zijiexia
parent 575fdc2c4c
commit a3291b5654
330 changed files with 100371 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs_new/index.mdx
+497
View File
@@ -0,0 +1,497 @@
---
title: Welcome to SGLang
description: High-performance serving framework for large language and multimodal models.
keywords:
- sglang
- llm serving
- multimodal
- inference runtime
mode: wide
---
<a
class="github-button"
href="https://github.com/sgl-project/sglang"
data-size="large"
data-show-count="true"
aria-label="Star sgl-project/sglang on GitHub"
>
Star
</a>
<a
class="github-button"
href="https://github.com/sgl-project/sglang/fork"
data-icon="octicon-repo-forked"
data-size="large"
data-show-count="true"
aria-label="Fork sgl-project/sglang on GitHub"
>
Fork
</a>
<script async defer src="https://buttons.github.io/buttons.js"></script>
<br></br>
<CardGroup cols={2}>
<Card title="Performance & Runtime" icon="arrow-trend-up">
Designed for low-latency, high-throughput inference with RadixAttention, prefix caching, and multi-GPU parallelism.
</Card>
<Card title="Models & Ecosystem" icon="hexagon-nodes">
Broad support for Llama, Qwen, DeepSeek, and more. Compatible with Hugging
Face and OpenAI APIs.
</Card>
<Card title="Extensive Hardware Support" icon="microchip">
Native support across <a href="./docs/hardware-platforms/overview">Hardware Platforms</a>
including NVIDIA, AMD, Intel Xeon, Google TPU, and Ascend NPU accelerators.
</Card>
<Card title="Community & Training" icon="users">
Open-source with widespread adoption, powering 400k+ GPUs and integrated with major RL frameworks.
</Card>
</CardGroup>
SGLang powers large-scale production deployments, generating trillions of tokens each day across more than 400,000 GPUs worldwide. It is hosted under the non-profit open-source organization [LMSYS](https://lmsys.org/about/).
---
## Get Started
SGLang is an inference framework meant for production level serving.
It is designed to deliver low-latency and high-throughput inference across a wide range of setups, from a single GPU to large distributed clusters.
<CardGroup cols={2}>
<Card title="Install SGLang" icon="angles-down" href="./docs/get-started/installation">
Install SGLang with pip, from source, or via Docker on your preferred hardware platform.
</Card>
<Card title="Quickstart" icon="zap" href="./docs/get-started/quickstart">
Launch your first model server and send requests in minutes with OpenAI-compatible APIs.
</Card>
</CardGroup>
## News and latest blogs
{/* BEGIN_LMSYS_SGLANG_BLOG_CARDS */}
<div className="not-prose">
<div
style={{
display: "grid",
gridTemplateColumns: "repeat(auto-fit, minmax(300px, 1fr))",
gap: "1rem",
alignItems: "stretch",
}}
>
<a
href="https://lmsys.org/blog/2026-03-25-gtc2026/"
target="_blank"
rel="noopener noreferrer"
style={{
display: "block",
border: "1px solid rgba(128, 128, 128, 0.3)",
borderRadius: "0.75rem",
overflow: "hidden",
textDecoration: "none",
color: "inherit",
height: "100%",
}}
>
<div
style={{
aspectRatio: "16 / 9",
overflow: "hidden",
background: "rgba(128, 128, 128, 0.15)",
}}
>
<img
src="https://lmsys.org/images/blog/gtc2026/happyhour-crowd.jpg"
alt="Highlights of SGLang at NVIDIA GTC 2026"
style={{
width: "100%",
height: "100%",
objectFit: "cover",
objectPosition: "center",
display: "block",
}}
/>
</div>
<div style={{ padding: "0.9rem 1rem 1rem" }}>
<p
style={{
margin: 0,
fontWeight: 600,
lineHeight: 1.35,
fontSize: "0.98rem",
}}
>
{"Highlights of SGLang at NVIDIA GTC 2026"}
</p>
<p
style={{
margin: "0.55rem 0 0",
fontSize: "0.85rem",
opacity: 0.75,
}}
>
{"March 31, 2026"}
</p>
</div>
</a>
<a
href="https://lmsys.org/blog/2026-03-25-eep-partial-failure-tolerance/"
target="_blank"
rel="noopener noreferrer"
style={{
display: "block",
border: "1px solid rgba(128, 128, 128, 0.3)",
borderRadius: "0.75rem",
overflow: "hidden",
textDecoration: "none",
color: "inherit",
height: "100%",
}}
>
<div
style={{
aspectRatio: "16 / 9",
overflow: "hidden",
background: "rgba(128, 128, 128, 0.15)",
}}
>
<img
src="https://lmsys.org/images/blog/eep-partial-failure-tolerance/figure.png"
alt="Elastic EP in SGLang: Achieving Partial Failure Tolerance for DeepSeek MoE Deployments"
style={{
width: "100%",
height: "100%",
objectFit: "cover",
objectPosition: "center",
display: "block",
}}
/>
</div>
<div style={{ padding: "0.9rem 1rem 1rem" }}>
<p
style={{
margin: 0,
fontWeight: 600,
lineHeight: 1.35,
fontSize: "0.98rem",
}}
>
{"Elastic EP in SGLang: Achieving Partial Failure Tolerance for DeepSeek MoE Deployments"}
</p>
<p
style={{
margin: "0.55rem 0 0",
fontSize: "0.85rem",
opacity: 0.75,
}}
>
{"March 25, 2026"}
</p>
</div>
</a>
<a
href="https://lmsys.org/blog/2026-03-17-rocm-miles-rl-amd/"
target="_blank"
rel="noopener noreferrer"
style={{
display: "block",
border: "1px solid rgba(128, 128, 128, 0.3)",
borderRadius: "0.75rem",
overflow: "hidden",
textDecoration: "none",
color: "inherit",
height: "100%",
}}
>
<div
style={{
aspectRatio: "16 / 9",
overflow: "hidden",
background: "rgba(128, 128, 128, 0.15)",
}}
>
<img
src="https://lmsys.org/images/blog/rocm_miles_rl/fig_1.png"
alt="ROCm Support for Miles: Large-Scale RL Post-Training on AMD Instinct\u2122 GPUs"
style={{
width: "100%",
height: "100%",
objectFit: "cover",
objectPosition: "center",
display: "block",
}}
/>
</div>
<div style={{ padding: "0.9rem 1rem 1rem" }}>
<p
style={{
margin: 0,
fontWeight: 600,
lineHeight: 1.35,
fontSize: "0.98rem",
}}
>
{"ROCm Support for Miles: Large-Scale RL Post-Training on AMD Instinct\u2122 GPUs"}
</p>
<p
style={{
margin: "0.55rem 0 0",
fontSize: "0.85rem",
opacity: 0.75,
}}
>
{"March 17, 2026"}
</p>
</div>
</a>
<a
href="https://lmsys.org/blog/2026-03-11-run-nvidia-nemotron-3-super/"
target="_blank"
rel="noopener noreferrer"
style={{
display: "block",
border: "1px solid rgba(128, 128, 128, 0.3)",
borderRadius: "0.75rem",
overflow: "hidden",
textDecoration: "none",
color: "inherit",
height: "100%",
}}
>
<div
style={{
aspectRatio: "16 / 9",
overflow: "hidden",
background: "rgba(128, 128, 128, 0.15)",
}}
>
<img
src="https://lmsys.org/images/blog/nemotron-3-super/figure_1.svg"
alt="SGLang Adds Day-0 Support for NVIDIA Nemotron 3 Super for building High-Efficiency Multi-Agent Systems"
style={{
width: "100%",
height: "100%",
objectFit: "cover",
objectPosition: "center",
display: "block",
}}
/>
</div>
<div style={{ padding: "0.9rem 1rem 1rem" }}>
<p
style={{
margin: 0,
fontWeight: 600,
lineHeight: 1.35,
fontSize: "0.98rem",
}}
>
{"SGLang Adds Day-0 Support for NVIDIA Nemotron 3 Super for building High-Efficiency Multi-Agent Systems"}
</p>
<p
style={{
margin: "0.55rem 0 0",
fontSize: "0.85rem",
opacity: 0.75,
}}
>
{"March 11, 2026"}
</p>
</div>
</a>
<a
href="https://lmsys.org/blog/2026-02-20-gb300-inferencex/"
target="_blank"
rel="noopener noreferrer"
style={{
display: "block",
border: "1px solid rgba(128, 128, 128, 0.3)",
borderRadius: "0.75rem",
overflow: "hidden",
textDecoration: "none",
color: "inherit",
height: "100%",
}}
>
<div
style={{
aspectRatio: "16 / 9",
overflow: "hidden",
background: "rgba(128, 128, 128, 0.15)",
}}
>
<img
src="https://lmsys.org/images/blog/gb300_inferencex/img-1.png"
alt="Unlocking 25x Inference Performance with SGLang on NVIDIA GB300 NVL72"
style={{
width: "100%",
height: "100%",
objectFit: "cover",
objectPosition: "center",
display: "block",
}}
/>
</div>
<div style={{ padding: "0.9rem 1rem 1rem" }}>
<p
style={{
margin: 0,
fontWeight: 600,
lineHeight: 1.35,
fontSize: "0.98rem",
}}
>
{"Unlocking 25x Inference Performance with SGLang on NVIDIA GB300 NVL72"}
</p>
<p
style={{
margin: "0.55rem 0 0",
fontSize: "0.85rem",
opacity: 0.75,
}}
>
{"February 20, 2026"}
</p>
</div>
</a>
<a
href="https://lmsys.org/blog/2026-02-19-gb300-longctx/"
target="_blank"
rel="noopener noreferrer"
style={{
display: "block",
border: "1px solid rgba(128, 128, 128, 0.3)",
borderRadius: "0.75rem",
overflow: "hidden",
textDecoration: "none",
color: "inherit",
height: "100%",
}}
>
<div
style={{
aspectRatio: "16 / 9",
overflow: "hidden",
background: "rgba(128, 128, 128, 0.15)",
}}
>
<img
src="https://lmsys.org/images/blog/gb300_longctx/cover.png"
alt="Deploying DeepSeek on GB300 NVL72: Big Wins in Long-Context Inference"
style={{
width: "100%",
height: "100%",
objectFit: "cover",
objectPosition: "center",
display: "block",
}}
/>
</div>
<div style={{ padding: "0.9rem 1rem 1rem" }}>
<p
style={{
margin: 0,
fontWeight: 600,
lineHeight: 1.35,
fontSize: "0.98rem",
}}
>
{"Deploying DeepSeek on GB300 NVL72: Big Wins in Long-Context Inference"}
</p>
<p
style={{
margin: "0.55rem 0 0",
fontSize: "0.85rem",
opacity: 0.75,
}}
>
{"February 19, 2026"}
</p>
</div>
</a>
</div>
</div>
{/* END_LMSYS_SGLANG_BLOG_CARDS */}
---
## Learn more and join the community
<div className="not-prose">
<div
style={{
padding: "0.9rem 0",
borderTop: "1px solid rgba(128, 128, 128, 0.24)",
borderBottom: "1px solid rgba(128, 128, 128, 0.24)",
}}
>
<p
style={{
margin: "0 0 0.35rem",
fontSize: "0.82rem",
fontWeight: 700,
letterSpacing: "0.08em",
textTransform: "uppercase",
opacity: 0.72,
}}
>
Stay connected
</p>
<div
style={{
display: "grid",
gap: "0.55rem",
fontSize: "0.97rem",
lineHeight: 1.7,
}}
>
<div>
<span style={{ display: "inline-flex", alignItems: "center", verticalAlign: "-0.125em" }}>
<Icon icon="map" size={14} />
</span>{" "}
<a href="https://roadmap.sglang.io">Development roadmap</a>
<span style={{ opacity: 0.62 }}> to follow current priorities and upcoming work.</span>
</div>
<div>
<span style={{ display: "inline-flex", alignItems: "center", verticalAlign: "-0.125em" }}>
<Icon icon="calendar-days" size={14} />
</span>{" "}
<a href="https://meet.sglang.io">Weekly public development meeting</a>
<span style={{ opacity: 0.62 }}> to hear updates and join open discussions.</span>
</div>
<div>
<span style={{ display: "inline-flex", alignItems: "center", verticalAlign: "-0.125em" }}>
<Icon icon="slack" size={14} />
</span>{" "}
<a href="https://slack.sglang.io/">Slack</a>
<span style={{ opacity: 0.62 }}> for questions, feedback, and community support.</span>
</div>
<div>
<a href="https://x.com/lmsysorg">X Twitter</a>
<span style={{ opacity: 0.62 }}> and </span>
<span style={{ display: "inline-flex", alignItems: "center", verticalAlign: "-0.125em" }}>
<Icon icon="linkedin" size={14} />
</span>{" "}
<a href="https://www.linkedin.com/company/sgl-project/">LinkedIn</a>
<span style={{ opacity: 0.62 }}> for project updates.</span>
</div>
<div>
<span style={{ display: "inline-flex", alignItems: "center", verticalAlign: "-0.125em" }}>
<Icon icon="newspaper" size={14} />
</span>{" "}
<a href="https://lmsys.org/blog/">LMSYS blog</a>
<span style={{ opacity: 0.62 }}> for release notes, benchmarks, and technical deep dives.</span>
</div>
<div>
<span style={{ display: "inline-flex", alignItems: "center", verticalAlign: "-0.125em" }}>
<Icon icon="book-open" size={14} />
</span>{" "}
<a href="https://github.com/sgl-project/sgl-learning-materials">Learning materials</a>
<span style={{ opacity: 0.62 }}> for blogs, slides, and videos.</span>
</div>
</div>
</div>
</div>