minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 13 Packages Projects Releases Wiki Activity

[Deps] Retire the CUDA 12 lane (#38404)

Browse Source 
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
This commit is contained in:
Mohammad Miadh Angkad
2026-09-10 16:58:09 -07:00
committed by GitHub
co-authored by Mohammad Angkad
parent fae8cd84cb
commit 52c191da52
38 changed files with 170 additions and 737 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/docs/developer_guide/contribution_guide.mdx
+4 -8
View File
@@ -206,9 +206,9 @@ Develop SGLang's customized DeepGEMM package on the [`dev` branch of `sgl-projec
### Update sgl-deep-ep
Develop `sgl-deep-ep` in [`sgl-project/DeepEP`](https://github.com/sgl-project/DeepEP). Use the implementation branch for the target platform: `sgl-deepep` for CUDA 13 on x86_64 or aarch64, `sgl-deepep-cu12-x86` for CUDA 12.9 on x86_64, or `sgl-deepep-cu12-arm` for CUDA 12.9 on aarch64. Merge packaging changes into `sgl-deepep-packaging`. The [`sgl-deep-ep` README](https://github.com/sgl-project/DeepEP/blob/sgl-deepep-packaging/sgl_deep_ep/README.md) describes the platform prerequisites and release matrix.
Develop `sgl-deep-ep` in [`sgl-project/DeepEP`](https://github.com/sgl-project/DeepEP). Use `sgl-deepep`, the implementation branch for CUDA 13 on x86_64 or aarch64. Merge packaging changes into `sgl-deepep-packaging`. The [`sgl-deep-ep` README](https://github.com/sgl-project/DeepEP/blob/sgl-deepep-packaging/sgl_deep_ep/README.md) describes the platform prerequisites and release matrix.
To validate locally, check out the selected implementation branch as `DeepEP-source` and the packaging branch as `DeepEP-packaging`. Install the required build dependencies first; CUDA 12.9 builds also require GDRCopy. The following CUDA 13 example builds a wheel for the host architecture, installs that exact wheel, and verifies that its guarded package import succeeds:
To validate locally, check out the selected implementation branch as `DeepEP-source` and the packaging branch as `DeepEP-packaging`. Install the required build dependencies first. The following example builds a wheel for the host architecture, installs that exact wheel, and verifies that its guarded package import succeeds:
```bash
DEEPEP_OUTPUT_DIR="$(mktemp -d)"
@@ -223,17 +223,13 @@ python3 -m pip install --force-reinstall --no-deps \
python3 -c "import deep_ep; print(deep_ep.__file__)"
```
Use `12.9` instead of `13.0` for a CUDA 12.9 build. The import check validates packaging and binary loading, but not communication correctness. On a configured multi-GPU host, also run the test appropriate for the implementation branch:
The import check validates packaging and binary loading, but not communication correctness. On a configured multi-GPU host, also run:
```bash
# CUDA 13 implementation branch
python3 DeepEP-source/tests/elastic/test_ep.py --num-processes 8
# CUDA 12.9 implementation branches
python3 DeepEP-source/tests/test_intranode.py --num-processes 8
```
Adjust `--num-processes` to the available GPUs and run the internode or low-latency tests when those transports changed. After local validation, ask the SGLang team to run the [sgl-deep-ep release workflow](https://github.com/sgl-project/sglang/actions/workflows/release-whl-deepep.yml) with the new version, CUDA target, and packaging ref. After verifying the published wheels for the supported Python versions and architectures, update the `sgl-deep-ep` pin in `python/pyproject.toml` before landing dependent SGLang changes.
Adjust `--num-processes` to the available GPUs and run the internode or low-latency tests when those transports changed. After local validation, ask the SGLang team to run the [sgl-deep-ep release workflow](https://github.com/sgl-project/sglang/actions/workflows/release-whl-deepep.yml) with the new version and packaging ref. After verifying the published wheels for the supported Python versions and architectures, update the `sgl-deep-ep` pin in `python/pyproject.toml` before landing dependent SGLang changes.
## Tips for newcomers
docs/docs/get-started/install.mdx
+6 -17
View File
@@ -29,15 +29,11 @@ uv pip install --prerelease=allow sglang
Some of SGLang's dependencies only publish pre-releases on PyPI, so without `--prerelease=allow` uv older than 0.12.0 silently installs SGLang 0.5.9. On [uv 0.12.0](https://github.com/astral-sh/uv/releases/tag/0.12.0) and newer the flag is a harmless no-op.
</Note>
The major version of Cuda is 13 by default. To install sglang under Cuda 12 with pip or uv, please try the following commands:
```bash Command
pip install --upgrade pip
pip install uv
uv pip install --prerelease=allow sglang
uv pip install --force-reinstall torch==2.13.0 torchaudio==2.11.0 torchvision --index-url https://download.pytorch.org/whl/cu129
uv pip install --force-reinstall sglang-kernel --index-url https://docs.sglang.ai/whl/cu129/
uv pip install --force-reinstall sgl-deep-gemm --index-url https://docs.sglang.ai/whl/cu129/ --no-deps
```
<Note>
SGLang requires CUDA 13. The CUDA 12 (`cu129`) wheels and images are retired,
because PyTorch 2.14 publishes no CUDA 12.9 builds. SGLang 0.5.19 is the last
release with a CUDA 12 lane.
</Note>
### Nightly builds
@@ -49,13 +45,6 @@ pip install uv
uv pip install --prerelease=allow --index-strategy unsafe-best-match --extra-index-url https://docs.sglang.ai/whl/cu130/ sglang
```
To install a nightly build under Cuda 12, swap the index to `cu129`:
```bash Command
pip install --upgrade pip
pip install uv
uv pip install --prerelease=allow --index-strategy unsafe-best-match --extra-index-url https://docs.sglang.ai/whl/cu129/ sglang
```
### Quick fixes to common problems
- If you encounter `OSError: CUDA_HOME environment variable is not set`. Please set it to your CUDA install root with either of the following solutions:
1. Use `export CUDA_HOME=/usr/local/cuda-<your-cuda-version>` to set the `CUDA_HOME` environment variable.
@@ -113,7 +102,7 @@ docker run --gpus all \
You can also find the nightly docker images [here](https://hub.docker.com/r/lmsysorg/sglang/tags?name=nightly).
Notes:
- SGLang is shipped with CUDA 13 environment by default. To run SGLang on CUDA 12 environment, please use images with `-cu12` or `-cu129` suffix, such as `lmsysorg/sglang:latest-cu129` or `lmsysorg/sglang:dev-cu12`.
- SGLang images ship a CUDA 13 environment. The `-cu12` / `-cu129` images are no longer published; `lmsysorg/sglang:v0.5.19-cu129` is the last CUDA 12 tag.
## Method 4: Using Kubernetes