minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 10 Packages Projects Releases Wiki Activity

[AMD] Enable FP4 indexer for Deepseek V4 (#37353)

Browse Source 
Co-authored-by: 1am9trash <1am9trash@gmail.com>
Co-authored-by: AMD-yanfeiwang <256076023+AMD-yanfeiwang@users.noreply.github.com>
Co-authored-by: Thomas Wang <thomawan@amd.com>
This commit is contained in:
Xinyi Song
2026-09-02 09:45:08 -07:00
committed by GitHub
co-authored by 1am9trash AMD-yanfeiwang Thomas Wang
parent f6aed6ec53
commit f8cbf000f4
21 changed files with 1727 additions and 75 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docker/rocm.Dockerfile
+3
View File
@@ -569,10 +569,13 @@ RUN pip uninstall -y aiter
# produced by a fresh `git clone` above, so there are no real user changes to
# preserve.
# cherry pick 8578af1 commit for v4 fp4 indexer kv-cache fix, may be removed in next aiter upgrade
# apply fix for v4 fp4 indexer, may be removed in next aiter upgrade
RUN git clone ${AITER_REPO} \
&& cd aiter \
&& git checkout -f ${AITER_COMMIT} \
&& git cherry-pick --no-commit 8578af153f4fa1e007fede7e3c1e1b373f07af4c \
&& sed -i 's/from functools import lru_cache/from functools import cache/' aiter/ops/flydsl/kernels/mqa_logits/pa_mqa_logits_fp4_prefill.py \
&& sed -i 's/@lru_cache(maxsize=32)/@cache/' aiter/ops/flydsl/kernels/mqa_logits/pa_mqa_logits_fp4_prefill.py \
&& git submodule update --init --recursive \
&& pip install -r requirements.txt \
&& if [ "${GPU_ARCH_LIST}" = "gfx1250" ]; then \