[Doc] Fix instruction on Cuda 13 environments (#24516)
This commit is contained in:
@@ -25,35 +25,6 @@ pip install uv
|
|||||||
uv pip install sglang
|
uv pip install sglang
|
||||||
```
|
```
|
||||||
|
|
||||||
### For CUDA 13
|
|
||||||
|
|
||||||
Docker is recommended (see Method 3 note on B300/GB300/CUDA 13). If you do not have Docker access, follow these steps:
|
|
||||||
|
|
||||||
1. Install PyTorch with CUDA 13 support first:
|
|
||||||
```bash Command
|
|
||||||
# Replace X.Y.Z with the version by your SGLang install
|
|
||||||
uv pip install torch==X.Y.Z torchvision torchaudio --index-url https://download.pytorch.org/whl/cu130
|
|
||||||
```
|
|
||||||
|
|
||||||
2. Install sglang:
|
|
||||||
```bash Command
|
|
||||||
uv pip install sglang
|
|
||||||
```
|
|
||||||
|
|
||||||
3. Install the `sglang-kernel` wheel for CUDA 13 from [the sgl-project whl releases](https://github.com/sgl-project/whl/blob/gh-pages/cu130/sglang-kernel/index.html). Replace `X.Y.Z` with the `sglang-kernel` version required by your SGLang install (you can find this by running `uv pip show sglang-kernel`). Examples:
|
|
||||||
```bash Command
|
|
||||||
# x86_64
|
|
||||||
uv pip install "https://github.com/sgl-project/whl/releases/download/vX.Y.Z/sglang_kernel-X.Y.Z+cu130-cp310-abi3-manylinux2014_x86_64.whl"
|
|
||||||
|
|
||||||
# aarch64
|
|
||||||
uv pip install "https://github.com/sgl-project/whl/releases/download/vX.Y.Z/sglang_kernel-X.Y.Z+cu130-cp310-abi3-manylinux2014_aarch64.whl"
|
|
||||||
```
|
|
||||||
|
|
||||||
4. If you encounter `ptxas fatal : Value 'sm_103a' is not defined for option 'gpu-name'` on B300/GB300, fix it with:
|
|
||||||
```bash Command
|
|
||||||
export TRITON_PTXAS_PATH=/usr/local/cuda/bin/ptxas
|
|
||||||
```
|
|
||||||
|
|
||||||
### Quick fixes to common problems
|
### Quick fixes to common problems
|
||||||
- If you encounter `OSError: CUDA_HOME environment variable is not set`. Please set it to your CUDA install root with either of the following solutions:
|
- If you encounter `OSError: CUDA_HOME environment variable is not set`. Please set it to your CUDA install root with either of the following solutions:
|
||||||
1. Use `export CUDA_HOME=/usr/local/cuda-<your-cuda-version>` to set the `CUDA_HOME` environment variable.
|
1. Use `export CUDA_HOME=/usr/local/cuda-<your-cuda-version>` to set the `CUDA_HOME` environment variable.
|
||||||
@@ -107,7 +78,7 @@ docker run --gpus all \
|
|||||||
You can also find the nightly docker images [here](https://hub.docker.com/r/lmsysorg/sglang/tags?name=nightly).
|
You can also find the nightly docker images [here](https://hub.docker.com/r/lmsysorg/sglang/tags?name=nightly).
|
||||||
|
|
||||||
Notes:
|
Notes:
|
||||||
- On B300/GB300 (SM103) or CUDA 13 environment, we recommend using the nightly image at `lmsysorg/sglang:dev-cu13` or stable image at `lmsysorg/sglang:latest-cu130-runtime`. Please, do not re-install the project as editable inside the docker image, since it will override the version of libraries specified by the cu13 docker image.
|
- SGLang is shipped with CUDA 13 environment by default. To run SGLang on CUDA 12 environment, please use images with `-cu12` or `-cu129` suffix, such as `lmsysorg/sglang:latest-cu129` or `lmsysorg/sglang:dev-cu12`.
|
||||||
|
|
||||||
## Method 4: Using Kubernetes
|
## Method 4: Using Kubernetes
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user