minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

Deepseek V4 (#23882)

Browse Source 
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: fzyzcjy <ch271828n@outlook.com>
Co-authored-by: ispobock <ispobaoke@gmail.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
Co-authored-by: yueming-yuan <yym022502@gmail.com>
Co-authored-by: DarkSharpness <2040703891@qq.com>
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: yhyang201 <yhyang201@users.noreply.github.com>
Co-authored-by: yhyang201 <yhyang201@gmail.com>
Co-authored-by: Qiaolin Yu <90088090+qiaolin-yu@users.noreply.github.com>
Co-authored-by: Ethan (Yusheng) Su <11704492+yushengsu-thu@users.noreply.github.com>
Co-authored-by: Mingyi <27337995+wisclmy0611@users.noreply.github.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: Yihao Wang <42559837+againstentropy@users.noreply.github.com>
This commit is contained in:
Liangsheng Yin
2026-05-07 18:32:21 -07:00
committed by GitHub
co-authored by Baizhou Zhang Claude Opus 4.7 fzyzcjy ispobock Zhiqiang Xie yueming-yuan DarkSharpness Yuhao Yang yhyang201 yhyang201 Qiaolin Yu Ethan Su Mingyi Cheng Wan Yihao Wang
parent 55224fff08
commit 35870d55ac
154 changed files with 24970 additions and 759 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docker/Dockerfile
+10
View File
@@ -13,6 +13,7 @@ ARG DEEPEP_COMMIT=9af0e0d0e74f3577af1979c9b9e1ac2cad0104ee
ARG BUILD_AND_DOWNLOAD_PARALLEL=8
ARG SGL_KERNEL_VERSION=0.4.2.post1
ARG SGL_VERSION
ARG SGL_DEEP_GEMM_VERSION=0.0.1
ARG USE_LATEST_SGLANG=0
ARG GDRCOPY_VERSION=2.5.1
ARG PIP_DEFAULT_INDEX
@@ -244,6 +245,7 @@ RUN --mount=type=cache,target=/root/.cache/pip \
| xargs -r python3 -m pip uninstall -y && \
python3 -m pip install --index-url https://download.pytorch.org/whl/cu${CUINDEX} \
torch torchvision torchaudio --force-reinstall; \
python3 -m pip install https://github.com/sgl-project/whl/releases/download/v${SGL_DEEP_GEMM_VERSION}/sgl_deep_gemm-${SGL_DEEP_GEMM_VERSION}+cu129-py3-none-manylinux2014_$(uname -m).whl --force-reinstall; \
fi \
&& cd /sgl-workspace \
&& rm -rf /tmp/sglang_deps \
@@ -539,12 +541,20 @@ RUN --mount=type=cache,target=/root/.cache/pip \
# the `nixl` import path) but unconditionally requires nixl-cu12, so we install
# it with --no-deps and pair it with the matching nixl-cu12 / nixl-cu13 binary
# to avoid shipping wrong-CUDA libs on cu13 images.
# The upstream flash-mla packages are required for running deepseek-v4 models
RUN --mount=type=cache,target=/root/.cache/pip if [ "${CUDA_VERSION%%.*}" = "12" ]; then \
python3 -m pip install nixl nixl-cu12 --no-deps ; \
python3 -m pip install cuda-python==12.9 ; \
cd /sgl-workspace && git clone https://github.com/deepseek-ai/FlashMLA.git flash-mla \
&& cd flash-mla && git submodule update --init --recursive \
&& pip install --no-build-isolation -v . ; \
elif [ "${CUDA_VERSION%%.*}" = "13" ]; then \
python3 -m pip install nixl nixl-cu13 --no-deps ; \
python3 -m pip install cuda-python==13.2.0 ; \
cd /sgl-workspace && git clone https://github.com/deepseek-ai/FlashMLA.git flash-mla \
&& ln -s /usr/local/cuda/include/cccl/cuda /usr/local/cuda/include/cuda \
&& cd flash-mla && git submodule update --init --recursive \
&& pip install --no-build-isolation -v . ; \
fi
# Add yank script