[Apple Silicon] Add Metal kernel support in sgl-kernel (#23449)
Signed-off-by: Xiaodong Ye <yeahdongcn@gmail.com>
This commit is contained in:
@@ -6,6 +6,19 @@ metatags:
|
||||
|
||||
This document describes how run SGLang on Apple Silicon using [Metal (MLX)](https://opensource.apple.com/projects/mlx/). If you encounter issues or have questions, please [open an issue](https://github.com/sgl-project/sglang/issues).
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Building the native Metal kernels in `sgl-kernel` requires the Apple
|
||||
toolchain (`clang++`, the Metal framework headers, and `xcrun`). These ship
|
||||
with the **Xcode Command Line Tools**, which cannot be installed via `pip`:
|
||||
|
||||
```bash
|
||||
xcode-select --install
|
||||
```
|
||||
|
||||
If you have the full Xcode app installed, the Command Line Tools are already
|
||||
available. You can verify with `xcode-select -p && xcrun --find metal`.
|
||||
|
||||
## Install SGLang
|
||||
|
||||
You can install SGLang using one of the methods below.
|
||||
@@ -17,8 +30,15 @@ You can install SGLang using one of the methods below.
|
||||
git clone https://github.com/sgl-project/sglang.git
|
||||
cd sglang
|
||||
|
||||
# Install sglang python package
|
||||
pip install --upgrade pip
|
||||
# Create and activate a virtual environment
|
||||
uv venv -p 3.12 sglang-metal
|
||||
source sglang-metal/bin/activate
|
||||
|
||||
# (Optional) Compile sgl-kernel
|
||||
uv pip install --upgrade pip
|
||||
uv run sgl-kernel/setup_metal.py install
|
||||
|
||||
# Install sglang python package along with diffusion support
|
||||
rm -f python/pyproject.toml && mv python/pyproject_other.toml python/pyproject.toml
|
||||
uv pip install -e "python[all_mps]"
|
||||
```
|
||||
|
||||
@@ -22,7 +22,7 @@ pip install --upgrade pip
|
||||
cd sgl-kernel
|
||||
python setup_musa.py install
|
||||
|
||||
# Install sglang python package
|
||||
# Install sglang python package along with diffusion support
|
||||
cd ..
|
||||
rm -f python/pyproject.toml && mv python/pyproject_other.toml python/pyproject.toml
|
||||
pip install -e "python[all_musa]"
|
||||
|
||||
@@ -120,7 +120,7 @@ git clone https://github.com/sgl-project/sglang.git
|
||||
cd sglang
|
||||
|
||||
# Create and activate a virtual environment
|
||||
uv venv -p 3.11 sglang-diffusion
|
||||
uv venv -p 3.12 sglang-diffusion
|
||||
source sglang-diffusion/bin/activate
|
||||
|
||||
# Install the Python packages
|
||||
|
||||
Reference in New Issue
Block a user