minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 13 Packages Projects Releases Wiki Activity

[kernel] Integrate flashinfer's rope with higher precision and better perf (#3134)

Browse Source
This commit is contained in:
Byron Hsu
2025-01-27 15:28:00 +08:00
committed by GitHub
parent af02f99b7c
commit fb11a43981
8 changed files with 244 additions and 98 deletions
Download Patch File Download Diff File Expand all files Collapse all files
sgl-kernel/setup.py
+1
View File
@@ -94,6 +94,7 @@ sources = [
"3rdparty/flashinfer/csrc/norm.cu",
"3rdparty/flashinfer/csrc/sampling.cu",
"3rdparty/flashinfer/csrc/renorm.cu",
"3rdparty/flashinfer/csrc/rope.cu",
]
enable_bf16 = os.getenv("SGL_KERNEL_ENABLE_BF16", "0") == "1"