minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

[Kernel] Move sgl-kernel under sglang.kernels.aot (#32648)

Browse Source
This commit is contained in:
Xiaoyu Zhang
2026-07-29 17:25:00 +08:00
committed by GitHub
parent 1b9dfa14e6
commit c32c4ef79c
370 changed files with 300 additions and 269 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.claude/skills/add-jit-kernel/SKILL.md
+1 -1
View File
@@ -18,7 +18,7 @@ Add a new operation that scales each element of a tensor by a scalar factor:
## When to use JIT vs AOT (`sgl-kernel`) ## When to use JIT vs AOT (`sgl-kernel`)
- **JIT (`jit_kernel`)**: prefer this first for kernels that do **not** depend on CUTLASS or another large C++ project. It is the default choice for lightweight kernels that benefit from rapid iteration and first-use compilation. - **JIT (`jit_kernel`)**: prefer this first for kernels that do **not** depend on CUTLASS or another large C++ project. It is the default choice for lightweight kernels that benefit from rapid iteration and first-use compilation.
- **AOT (`sgl-kernel`)**: prefer this when the kernel **does** depend on CUTLASS or another large C++ project, or when it should live in `sgl-kernel/` and participate in the wheel build / torch op registration flow. - **AOT (`sgl-kernel`)**: prefer this when the kernel **does** depend on CUTLASS or another large C++ project, or when it should live in `python/sglang/kernels/aot/` and participate in the wheel build / torch op registration flow.
- **Exception**: kernels that depend on `flashinfer`, or on CUTLASS that is already provided through `flashinfer`, can still be implemented as `jit_kernel`. - **Exception**: kernels that depend on `flashinfer`, or on CUTLASS that is already provided through `flashinfer`, can still be implemented as `jit_kernel`.
--- ---
.claude/skills/add-sgl-kernel/SKILL.md
+38 -38
View File
@@ -14,7 +14,7 @@ Add a new operation that scales each element of a tensor by a scalar factor:
- Input: tensor `x` (CUDA) and scalar `factor` (float) - Input: tensor `x` (CUDA) and scalar `factor` (float)
- Output: `x * factor` (element-wise, in-place or into pre-allocated `out`) - Output: `x * factor` (element-wise, in-place or into pre-allocated `out`)
- Supported dtypes: **FP16 (`torch.float16`), BF16 (`torch.bfloat16`), FP32 (`torch.float32`)** - Supported dtypes: **FP16 (`torch.float16`), BF16 (`torch.bfloat16`), FP32 (`torch.float32`)**
- Dispatched via `DISPATCH_PYTORCH_DTYPE_TO_CTYPE_FLOAT_FP16` macro (defined in `sgl-kernel/include/utils.h`) - Dispatched via `DISPATCH_PYTORCH_DTYPE_TO_CTYPE_FLOAT_FP16` macro (defined in `python/sglang/kernels/aot/include/utils.h`)
## Two rules of thumb (must follow) ## Two rules of thumb (must follow)
@@ -33,13 +33,13 @@ In addition, every new kernel must ship with:
You will typically touch these files/areas: You will typically touch these files/areas:
- Implementation: `sgl-kernel/csrc/elementwise/scale.cu` (pick the right subdirectory) - Implementation: `python/sglang/kernels/aot/csrc/elementwise/scale.cu` (pick the right subdirectory)
- Public declarations: `sgl-kernel/include/sgl_kernel_ops.h` - Public declarations: `python/sglang/kernels/aot/include/sgl_kernel_ops.h`
- Torch extension registration: `sgl-kernel/csrc/common_extension.cc` - Torch extension registration: `python/sglang/kernels/aot/csrc/common_extension.cc`
- Build: `sgl-kernel/CMakeLists.txt` (`set(SOURCES ...)`) - Build: `python/sglang/kernels/aot/CMakeLists.txt` (`set(SOURCES ...)`)
- Python API: `sgl-kernel/python/sgl_kernel/` and `sgl-kernel/python/sgl_kernel/__init__.py` - Python API: `python/sglang/kernels/aot/python/sgl_kernel/` and `python/sglang/kernels/aot/python/sgl_kernel/__init__.py`
- Tests: `sgl-kernel/tests/test_scale.py` - Tests: `python/sglang/kernels/aot/tests/test_scale.py`
- Benchmarks: `sgl-kernel/benchmark/bench_scale.py` - Benchmarks: `python/sglang/kernels/aot/benchmark/bench_scale.py`
--- ---
@@ -50,7 +50,7 @@ Pick the right subdirectory:
- `csrc/elementwise/` — for element-wise ops (our example) - `csrc/elementwise/` — for element-wise ops (our example)
- `csrc/gemm/`, `csrc/attention/`, `csrc/moe/` — for other categories - `csrc/gemm/`, `csrc/attention/`, `csrc/moe/` — for other categories
Create `sgl-kernel/csrc/elementwise/scale.cu`: Create `python/sglang/kernels/aot/csrc/elementwise/scale.cu`:
```cpp ```cpp
#include <ATen/cuda/CUDAContext.h> #include <ATen/cuda/CUDAContext.h>
@@ -115,7 +115,7 @@ void scale(at::Tensor& out, const at::Tensor& input, double factor) {
## Step 2: Add a C++ declaration in `include/sgl_kernel_ops.h` ## Step 2: Add a C++ declaration in `include/sgl_kernel_ops.h`
Edit `sgl-kernel/include/sgl_kernel_ops.h`, add to the elementwise section: Edit `python/sglang/kernels/aot/include/sgl_kernel_ops.h`, add to the elementwise section:
```cpp ```cpp
void scale(at::Tensor& out, const at::Tensor& input, double factor); void scale(at::Tensor& out, const at::Tensor& input, double factor);
@@ -125,7 +125,7 @@ void scale(at::Tensor& out, const at::Tensor& input, double factor);
## Step 3: Register the op in `csrc/common_extension.cc` ## Step 3: Register the op in `csrc/common_extension.cc`
Edit `sgl-kernel/csrc/common_extension.cc`, inside `TORCH_LIBRARY_FRAGMENT(sgl_kernel, m)`: Edit `python/sglang/kernels/aot/csrc/common_extension.cc`, inside `TORCH_LIBRARY_FRAGMENT(sgl_kernel, m)`:
```cpp ```cpp
// From csrc/elementwise // From csrc/elementwise
@@ -143,7 +143,7 @@ m.impl("scale", torch::kCUDA, &scale);
## Step 4: Add the new source file to `CMakeLists.txt` ## Step 4: Add the new source file to `CMakeLists.txt`
Edit `sgl-kernel/CMakeLists.txt`, add to `set(SOURCES ...)`: Edit `python/sglang/kernels/aot/CMakeLists.txt`, add to `set(SOURCES ...)`:
```cmake ```cmake
csrc/elementwise/scale.cu csrc/elementwise/scale.cu
@@ -156,14 +156,14 @@ csrc/elementwise/scale.cu
--- ---
## Step 5: Expose a Python API under `sgl-kernel/python/sgl_kernel/` ## Step 5: Expose a Python API under `python/sglang/kernels/aot/python/sgl_kernel/`
Prefer following the existing module organization first. For elementwise kernels, the usual pattern is: Prefer following the existing module organization first. For elementwise kernels, the usual pattern is:
- implement the Python wrapper in `sgl-kernel/python/sgl_kernel/elementwise.py` - implement the Python wrapper in `python/sglang/kernels/aot/python/sgl_kernel/elementwise.py`
- then re-export it from `sgl-kernel/python/sgl_kernel/__init__.py` - then re-export it from `python/sglang/kernels/aot/python/sgl_kernel/__init__.py`
For example, in `sgl-kernel/python/sgl_kernel/elementwise.py`, add: For example, in `python/sglang/kernels/aot/python/sgl_kernel/elementwise.py`, add:
```python ```python
import torch import torch
@@ -190,13 +190,13 @@ def scale(
return out return out
``` ```
Then re-export it from `sgl-kernel/python/sgl_kernel/__init__.py` following the existing import style used by other kernels. Then re-export it from `python/sglang/kernels/aot/python/sgl_kernel/__init__.py` following the existing import style used by other kernels.
--- ---
## Step 6: Write tests (required) ## Step 6: Write tests (required)
Create `sgl-kernel/tests/test_scale.py`: Create `python/sglang/kernels/aot/tests/test_scale.py`:
```python ```python
import pytest import pytest
@@ -241,7 +241,7 @@ if __name__ == "__main__":
## Step 7: Add a benchmark (required) ## Step 7: Add a benchmark (required)
Create `sgl-kernel/benchmark/bench_scale.py`: Create `python/sglang/kernels/aot/benchmark/bench_scale.py`:
```python ```python
import itertools import itertools
@@ -306,14 +306,14 @@ if __name__ == "__main__":
Build: Build:
```bash ```bash
cd sgl-kernel cd python/sglang/kernels/aot
make build -j16 make build -j16
``` ```
If you need to limit host resource usage: If you need to limit host resource usage:
```bash ```bash
cd sgl-kernel cd python/sglang/kernels/aot
make build -j1 MAX_JOBS=2 CMAKE_ARGS="-DSGL_KERNEL_COMPILE_THREADS=1" make build -j1 MAX_JOBS=2 CMAKE_ARGS="-DSGL_KERNEL_COMPILE_THREADS=1"
``` ```
@@ -324,8 +324,8 @@ make build -j1 MAX_JOBS=2 CMAKE_ARGS="-DSGL_KERNEL_COMPILE_THREADS=1"
After building successfully, run the test and benchmark: After building successfully, run the test and benchmark:
```bash ```bash
pytest sgl-kernel/tests/test_scale.py -q pytest python/sglang/kernels/aot/tests/test_scale.py -q
python sgl-kernel/benchmark/bench_scale.py python python/sglang/kernels/aot/benchmark/bench_scale.py
``` ```
PR CI also runs `pr-test-sgl-kernel.yml`, including the B200 job PR CI also runs `pr-test-sgl-kernel.yml`, including the B200 job
@@ -339,29 +339,29 @@ Blackwell coverage signal for AOT `sgl-kernel` changes.
- **Async CUDA errors**: `CUDA_LAUNCH_BLOCKING=1` - **Async CUDA errors**: `CUDA_LAUNCH_BLOCKING=1`
- **Memory errors**: `compute-sanitizer --tool memcheck python ...` - **Memory errors**: `compute-sanitizer --tool memcheck python ...`
- **Build is too slow / OOM**: reduce `MAX_JOBS` and `SGL_KERNEL_COMPILE_THREADS` - **Build is too slow / OOM**: reduce `MAX_JOBS` and `SGL_KERNEL_COMPILE_THREADS`
- **Binary bloat**: use `sgl-kernel/analyze_whl_kernel_sizes.py` - **Binary bloat**: use `python/sglang/kernels/aot/analyze_whl_kernel_sizes.py`
- **CMake sources list**: if your `.cu` file is missing from `SOURCES`, the symbol will be undefined at link time - **CMake sources list**: if your `.cu` file is missing from `SOURCES`, the symbol will be undefined at link time
--- ---
## References ## References
- `sgl-kernel/README.md` - `python/sglang/kernels/aot/README.md`
- `sgl-kernel/include/sgl_kernel_ops.h` - `python/sglang/kernels/aot/include/sgl_kernel_ops.h`
- `sgl-kernel/csrc/common_extension.cc` - `python/sglang/kernels/aot/csrc/common_extension.cc`
- `sgl-kernel/CMakeLists.txt` - `python/sglang/kernels/aot/CMakeLists.txt`
- `sgl-kernel/include/utils.h` — `DISPATCH_PYTORCH_DTYPE_TO_CTYPE_FLOAT_FP16` macro and friends - `python/sglang/kernels/aot/include/utils.h` — `DISPATCH_PYTORCH_DTYPE_TO_CTYPE_FLOAT_FP16` macro and friends
- `sgl-kernel/csrc/elementwise/activation.cu` — reference for the FP16/BF16/FP32 dispatch pattern - `python/sglang/kernels/aot/csrc/elementwise/activation.cu` — reference for the FP16/BF16/FP32 dispatch pattern
## Summary of Files Created/Modified ## Summary of Files Created/Modified
``` ```
sgl-kernel/csrc/elementwise/scale.cu # NEW: CUDA kernel + launcher python/sglang/kernels/aot/csrc/elementwise/scale.cu # NEW: CUDA kernel + launcher
sgl-kernel/include/sgl_kernel_ops.h # MODIFIED: C++ declaration python/sglang/kernels/aot/include/sgl_kernel_ops.h # MODIFIED: C++ declaration
sgl-kernel/csrc/common_extension.cc # MODIFIED: schema + dispatch registration python/sglang/kernels/aot/csrc/common_extension.cc # MODIFIED: schema + dispatch registration
sgl-kernel/CMakeLists.txt # MODIFIED: add source file (alphabetical) python/sglang/kernels/aot/CMakeLists.txt # MODIFIED: add source file (alphabetical)
sgl-kernel/python/sgl_kernel/elementwise.py # MODIFIED: Python wrapper python/sglang/kernels/aot/python/sgl_kernel/elementwise.py # MODIFIED: Python wrapper
sgl-kernel/python/sgl_kernel/__init__.py # MODIFIED: re-export Python API python/sglang/kernels/aot/python/sgl_kernel/__init__.py # MODIFIED: re-export Python API
sgl-kernel/tests/test_scale.py # NEW: tests python/sglang/kernels/aot/tests/test_scale.py # NEW: tests
sgl-kernel/benchmark/bench_scale.py # NEW: benchmark python/sglang/kernels/aot/benchmark/bench_scale.py # NEW: benchmark
``` ```
.claude/skills/llm-torch-profiler-analysis/references/fuse-overlap-catalog.md
+1 -1
View File
@@ -145,7 +145,7 @@ Stable entries should be folded into the mainline family rows above.
| PR `#21491` FlashInfer TRTLLM FP8 MoE with fused shared experts | `num_fused_shared_experts`<br>`trtllm_fp8_block_scale_moe` | `PR #21491`<br>`python/sglang/srt/layers/moe/fused_moe_triton/fused_moe.py`<br>`python/sglang/srt/models/deepseek_v2.py` | FlashInfer TRTLLM FP8 MoE path can fuse shared experts inside the routed MoE kernel | On FP8 TRTLLM MoE discussions, treat fused shared experts as an upstream pattern that already has a concrete PR. | | PR `#21491` FlashInfer TRTLLM FP8 MoE with fused shared experts | `num_fused_shared_experts`<br>`trtllm_fp8_block_scale_moe` | `PR #21491`<br>`python/sglang/srt/layers/moe/fused_moe_triton/fused_moe.py`<br>`python/sglang/srt/models/deepseek_v2.py` | FlashInfer TRTLLM FP8 MoE path can fuse shared experts inside the routed MoE kernel | On FP8 TRTLLM MoE discussions, treat fused shared experts as an upstream pattern that already has a concrete PR. |
| PR `#22005` fused add + RMSNorm + per-token FP8 quant | `fused_add_rmsnorm_per_token_quant`<br>`per_token_quant_fp8` | `PR #22005`<br>`python/sglang/kernels/jit/csrc/elementwise/fused_add_rmsnorm_per_token_quant.cuh`<br>`python/sglang/kernels/jit/fused_add_rmsnorm_per_token_quant.py` | CUDA JIT kernel keeps normed values in registers and emits BF16 + FP8 outputs plus per-token scales | If FP8 online-quant traces show add+norm followed by per-token quant, treat this as an in-flight upstream CUDA fuse family. | | PR `#22005` fused add + RMSNorm + per-token FP8 quant | `fused_add_rmsnorm_per_token_quant`<br>`per_token_quant_fp8` | `PR #22005`<br>`python/sglang/kernels/jit/csrc/elementwise/fused_add_rmsnorm_per_token_quant.cuh`<br>`python/sglang/kernels/jit/fused_add_rmsnorm_per_token_quant.py` | CUDA JIT kernel keeps normed values in registers and emits BF16 + FP8 outputs plus per-token scales | If FP8 online-quant traces show add+norm followed by per-token quant, treat this as an in-flight upstream CUDA fuse family. |
| PR `#20667` Qwen3.5 fused QK norm + RoPE + KV cache write | `fused_qk_norm_rope_cache_pts_quant_shuffle`<br>`fused_qk_norm_mrope_3d_cache_pts_quant_shuffle`<br>`rotary_dim` | `PR #20667`<br>`python/sglang/srt/models/qwen3_5.py`<br>`python/sglang/srt/models/utils.py` | ROCm / AITER path fuses Q / K RMSNorm, partial or 3D RoPE, and direct KV cache write for Qwen3.5 attention | Treat split QK-norm + RoPE + cache-store on Qwen3.5 as a concrete in-flight upstream family, not a novel idea. | | PR `#20667` Qwen3.5 fused QK norm + RoPE + KV cache write | `fused_qk_norm_rope_cache_pts_quant_shuffle`<br>`fused_qk_norm_mrope_3d_cache_pts_quant_shuffle`<br>`rotary_dim` | `PR #20667`<br>`python/sglang/srt/models/qwen3_5.py`<br>`python/sglang/srt/models/utils.py` | ROCm / AITER path fuses Q / K RMSNorm, partial or 3D RoPE, and direct KV cache write for Qwen3.5 attention | Treat split QK-norm + RoPE + cache-store on Qwen3.5 as a concrete in-flight upstream family, not a novel idea. |
| PR `#22392` CUTLASS FP8 GEMM replacing nvjet | `cutlass_scaled_mm`<br>`fp8_scaled_mm`<br>`nvjet`<br>`cudaMemsetAsync` | `PR #22392`<br>`sgl-kernel/python/sgl_kernel/gemm.py`<br>`python/sglang/srt/layers/quantization/fp8_utils.py` | Runtime replacement swaps nvjet FP8 GEMMs for CUTLASS kernels, removing per-launch memset bubbles and extra output-copy kernels | Treat nvjet GEMM + memset bubble ladders as an in-flight SGLang linear-kernel family before calling them novel. | | PR `#22392` CUTLASS FP8 GEMM replacing nvjet | `cutlass_scaled_mm`<br>`fp8_scaled_mm`<br>`nvjet`<br>`cudaMemsetAsync` | `PR #22392`<br>`python/sglang/kernels/aot/python/sgl_kernel/gemm.py`<br>`python/sglang/srt/layers/quantization/fp8_utils.py` | Runtime replacement swaps nvjet FP8 GEMMs for CUTLASS kernels, removing per-launch memset bubbles and extra output-copy kernels | Treat nvjet GEMM + memset bubble ladders as an in-flight SGLang linear-kernel family before calling them novel. |
| PR `#18612` NVFP4 CUTLASS MoE fused SiLU+Mul+quant | `silu_and_mul_scaled_nvfp4`<br>`nvfp4 expert quant`<br>`cutlass moe` | `PR #18612`<br>`python/sglang/srt/layers/moe/cutlass_w4a8_moe.py`<br>`python/sglang/kernels/ops/quantization/nvfp4_gemm_swiglu_nvfp4_quant.py` | Fuses MoE activation epilogue and NVFP4 expert quantization before the CUTLASS MoE second GEMM | Treat split SiLU+Mul then NVFP4 expert quant in CUTLASS MoE traces as an in-flight upstream SGLang family. | | PR `#18612` NVFP4 CUTLASS MoE fused SiLU+Mul+quant | `silu_and_mul_scaled_nvfp4`<br>`nvfp4 expert quant`<br>`cutlass moe` | `PR #18612`<br>`python/sglang/srt/layers/moe/cutlass_w4a8_moe.py`<br>`python/sglang/kernels/ops/quantization/nvfp4_gemm_swiglu_nvfp4_quant.py` | Fuses MoE activation epilogue and NVFP4 expert quantization before the CUTLASS MoE second GEMM | Treat split SiLU+Mul then NVFP4 expert quant in CUTLASS MoE traces as an in-flight upstream SGLang family. |
| PR `#22918` FlashInfer per-token NVFP4 MoE | `per_token_nvfp4`<br>`trtllm_fp4_block_scale_moe`<br>`FlashInfer MoE` | `PR #22918`<br>`python/sglang/srt/layers/moe/fused_moe_triton/fused_moe.py` | Adds FlashInfer-backed per-token NVFP4 MoE execution so expert quant/dequant work can move into the fused MoE backend | Treat standalone per-token NVFP4 MoE support kernels as a candidate missing backend-selection path, not an automatically novel kernel idea. | | PR `#22918` FlashInfer per-token NVFP4 MoE | `per_token_nvfp4`<br>`trtllm_fp4_block_scale_moe`<br>`FlashInfer MoE` | `PR #22918`<br>`python/sglang/srt/layers/moe/fused_moe_triton/fused_moe.py` | Adds FlashInfer-backed per-token NVFP4 MoE execution so expert quant/dequant work can move into the fused MoE backend | Treat standalone per-token NVFP4 MoE support kernels as a candidate missing backend-selection path, not an automatically novel kernel idea. |
| PR `#22851` NSA top-k backend and FlashInfer / PyTorch top-k split | `nsa topk`<br>`flashinfer_topk`<br>`pytorch_topk`<br>`fast_topk_transform` | `PR #22851`<br>`python/sglang/srt/layers/attention/nsa_backend.py` | Makes NSA top-k backend selection explicit and aligns fused top-k transform with FlashInfer / PyTorch fallbacks | When NSA top-k dominates decode, first classify it as backend selection or fused-transform eligibility work. | | PR `#22851` NSA top-k backend and FlashInfer / PyTorch top-k split | `nsa topk`<br>`flashinfer_topk`<br>`pytorch_topk`<br>`fast_topk_transform` | `PR #22851`<br>`python/sglang/srt/layers/attention/nsa_backend.py` | Makes NSA top-k backend selection explicit and aligns fused top-k transform with FlashInfer / PyTorch fallbacks | When NSA top-k dominates decode, first classify it as backend selection or fused-transform eligibility work. |
.claude/skills/llm-torch-profiler-analysis/scripts/triage_kernel_helpers.py
+2 -2
View File
@@ -840,7 +840,7 @@ FUSION_PATTERN_REGISTRY: Tuple[FusionPatternSpec, ...] = (
pattern="PR #22392 CUTLASS FP8 scaled MM replacing nvjet", pattern="PR #22392 CUTLASS FP8 scaled MM replacing nvjet",
candidate_path=( candidate_path=(
"PR #22392" "PR #22392"
"<br>sgl-kernel/python/sgl_kernel/gemm.py" "<br>python/sglang/kernels/aot/python/sgl_kernel/gemm.py"
"<br>python/sglang/srt/layers/quantization/fp8_utils.py" "<br>python/sglang/srt/layers/quantization/fp8_utils.py"
), ),
active_keywords=("cutlass_scaled_mm", "fp8_scaled_mm"), active_keywords=("cutlass_scaled_mm", "fp8_scaled_mm"),
@@ -2632,7 +2632,7 @@ def fusion_framework_hints(spec: FusionPatternSpec) -> set[str]:
hints.add("tokenspeed") hints.add("tokenspeed")
if "tensorrt_llm/" in text: if "tensorrt_llm/" in text:
hints.add("trtllm") hints.add("trtllm")
if any(token in text for token in ("python/sglang/", "sgl-kernel/", "sgl_kernel/")): if any(token in text for token in ("python/sglang/", "sgl_kernel/")):
hints.add("sglang") hints.add("sglang")
return hints return hints
.github/CODEOWNERS
+2 -2
View File
@@ -67,8 +67,8 @@
/python/sglang/srt/speculative @Ying1123 @merrymercy @hnyls2002 @Qiaolin-Yu /python/sglang/srt/speculative @Ying1123 @merrymercy @hnyls2002 @Qiaolin-Yu
/python/sglang/srt/utils/hf_transformers @JustinTong0323 /python/sglang/srt/utils/hf_transformers @JustinTong0323
/python/sglang/srt/weight_cache @liusy58 @QiuMike @alexnails /python/sglang/srt/weight_cache @liusy58 @QiuMike @alexnails
/sgl-kernel @ispobock @BBuf @yizhang2077 @merrymercy @FlamingoPg @HaiShaw /python/sglang/kernels/aot @ispobock @BBuf @yizhang2077 @merrymercy @FlamingoPg @HaiShaw
/sgl-kernel/csrc/musa @yeahdongcn /python/sglang/kernels/aot/csrc/musa @yeahdongcn
/sgl-model-gateway @slin1237 @CatherineSue /sgl-model-gateway @slin1237 @CatherineSue
/sgl-model-gateway/benches @slin1237 /sgl-model-gateway/benches @slin1237
/sgl-model-gateway/bindings/python @CatherineSue @key4ng @slin1237 /sgl-model-gateway/bindings/python @CatherineSue @key4ng @slin1237
.github/labeler.yml
+5 -3
View File
@@ -9,12 +9,14 @@ model-gateway:
# Kernel specific # Kernel specific
sgl-kernel: sgl-kernel:
- changed-files: - changed-files:
- any-glob-to-any-file: 'sgl-kernel/**/*' - any-glob-to-any-file: 'python/sglang/kernels/aot/**/*'
# JIT kernel specific # JIT kernel specific
jit-kernel: jit-kernel:
- changed-files: - changed-files:
- any-glob-to-any-file: 'python/sglang/kernels/**/*' - any-glob-to-any-file:
- 'python/sglang/kernels/!(*.md)'
- 'python/sglang/kernels/!(aot)/**/*'
# Documentation # Documentation
documentation: documentation:
@@ -90,7 +92,7 @@ blackwell:
- changed-files: - changed-files:
- any-glob-to-any-file: - any-glob-to-any-file:
- '**/*nvfp4*' - '**/*nvfp4*'
- 'sgl-kernel/csrc/attention/cutlass_sm100_mla/**/*' - 'python/sglang/kernels/aot/csrc/attention/cutlass_sm100_mla/**/*'
- 'python/sglang/srt/layers/attention/trtllm_mla_backend.py' - 'python/sglang/srt/layers/attention/trtllm_mla_backend.py'
- 'python/sglang/srt/layers/attention/trtllm_mha_backend.py' - 'python/sglang/srt/layers/attention/trtllm_mha_backend.py'
.github/workflows/_pr-test-check-changes.yml
+6 -3
View File
@@ -82,7 +82,9 @@ jobs:
- ".github/workflows/pr-gate.yml" - ".github/workflows/pr-gate.yml"
- ".github/actions/**" - ".github/actions/**"
- "python/pyproject.toml" - "python/pyproject.toml"
- "python/sglang/!(multimodal_gen)/**/!(*.md)" - "python/sglang/!(multimodal_gen|kernels)/**/!(*.md)"
- "python/sglang/kernels/!(*.md)"
- "python/sglang/kernels/!(aot)/**/!(*.md)"
- "scripts/ci/cuda/*" - "scripts/ci/cuda/*"
- "scripts/ci/utils/*" - "scripts/ci/utils/*"
- "test/**/!(*.md)" - "test/**/!(*.md)"
@@ -104,11 +106,12 @@ jobs:
- "test/registered/kernels/**" - "test/registered/kernels/**"
# sglang.kernels is the migrated kernel namespace (RFC #29630 / #30044); the # sglang.kernels is the migrated kernel namespace (RFC #29630 / #30044); the
# base-b-kernel suites import it directly, so kernel edits must run them. # base-b-kernel suites import it directly, so kernel edits must run them.
- "python/sglang/kernels/**" - "python/sglang/kernels/!(*.md)"
- "python/sglang/kernels/!(aot)/**"
sgl_kernel: sgl_kernel:
# Intentionally excludes ".github/workflows/pr-test-sgl-kernel.yml" — # Intentionally excludes ".github/workflows/pr-test-sgl-kernel.yml" —
# see API-side detector below for rationale. # see API-side detector below for rationale.
- "sgl-kernel/**/!(*.md|THIRDPARTYNOTICES.txt|LICENSE)" - "python/sglang/kernels/aot/**/!(*.md|THIRDPARTYNOTICES.txt|LICENSE)"
- name: Determine full-parallel mode - name: Determine full-parallel mode
id: parallel-mode id: parallel-mode
.github/workflows/_pr-test-sgl-kernel-build.yml
+4 -4
View File
@@ -100,19 +100,19 @@ jobs:
- name: Build wheel for Python ${{ matrix.python-version }} and CUDA ${{ matrix.cuda-version }} - name: Build wheel for Python ${{ matrix.python-version }} and CUDA ${{ matrix.cuda-version }}
run: | run: |
cd sgl-kernel cd python/sglang/kernels/aot
./build.sh "${{ matrix.python-version }}" "${{ matrix.cuda-version }}" ./build.sh "${{ matrix.python-version }}" "${{ matrix.cuda-version }}"
env: env:
USE_CCACHE: 1 USE_CCACHE: 1
- name: Verify wheel artifacts - name: Verify wheel artifacts
run: | run: |
ls -alh sgl-kernel/dist ls -alh python/sglang/kernels/aot/dist
ls -alh sgl-kernel/dist/*.whl ls -alh python/sglang/kernels/aot/dist/*.whl
- name: Upload artifacts - name: Upload artifacts
uses: actions/upload-artifact@v4 uses: actions/upload-artifact@v4
with: with:
name: wheel-python${{ matrix.python-version }}-cuda${{ matrix.cuda-version }}${{ inputs.arch_suffix }} name: wheel-python${{ matrix.python-version }}-cuda${{ matrix.cuda-version }}${{ inputs.arch_suffix }}
path: sgl-kernel/dist/* path: python/sglang/kernels/aot/dist/*
if-no-files-found: error if-no-files-found: error
.github/workflows/_pr-test-stage.yml
+2 -2
View File
@@ -108,7 +108,7 @@ jobs:
if: ${{ fromJson(inputs.check_changes).sgl_kernel == 'true' && steps.rc.outputs.artifact_version == 'v4' }} if: ${{ fromJson(inputs.check_changes).sgl_kernel == 'true' && steps.rc.outputs.artifact_version == 'v4' }}
uses: actions/download-artifact@v4 uses: actions/download-artifact@v4
with: with:
path: sgl-kernel/dist/ path: python/sglang/kernels/aot/dist/
merge-multiple: true merge-multiple: true
pattern: wheel-python3.10-cuda* pattern: wheel-python3.10-cuda*
@@ -116,7 +116,7 @@ jobs:
if: ${{ fromJson(inputs.check_changes).sgl_kernel == 'true' && steps.rc.outputs.artifact_version == 'v6' }} if: ${{ fromJson(inputs.check_changes).sgl_kernel == 'true' && steps.rc.outputs.artifact_version == 'v6' }}
uses: actions/download-artifact@v6 uses: actions/download-artifact@v6
with: with:
path: sgl-kernel/dist/ path: python/sglang/kernels/aot/dist/
merge-multiple: true merge-multiple: true
pattern: wheel-python3.10-cuda* pattern: wheel-python3.10-cuda*
.github/workflows/diffusion-ci-gt-gen.yml
+16 -16
View File
@@ -186,17 +186,17 @@ jobs:
diffusion-ci/consistency_gt/official_generated/case_map.json diffusion-ci/consistency_gt/official_generated/case_map.json
sparse-checkout-cone-mode: false sparse-checkout-cone-mode: false
- name: Prepare sgl-kernel/dist for prebuilt wheel - name: Prepare AOT dist for prebuilt wheel
if: inputs.kernel_artifact_run_id != '' if: inputs.kernel_artifact_run_id != ''
run: | run: |
ls -alh sgl-kernel/dist || true ls -alh python/sglang/kernels/aot/dist || true
rm -rf sgl-kernel/dist/* || true rm -rf python/sglang/kernels/aot/dist/* || true
- name: Download prebuilt sgl-kernel wheel - name: Download prebuilt sgl-kernel wheel
if: inputs.kernel_artifact_run_id != '' if: inputs.kernel_artifact_run_id != ''
uses: actions/download-artifact@v4 uses: actions/download-artifact@v4
with: with:
path: sgl-kernel/dist/ path: python/sglang/kernels/aot/dist/
merge-multiple: true merge-multiple: true
name: wheel-python3.10-cuda13.0 name: wheel-python3.10-cuda13.0
run-id: ${{ inputs.kernel_artifact_run_id }} run-id: ${{ inputs.kernel_artifact_run_id }}
@@ -382,17 +382,17 @@ jobs:
with: with:
ref: ${{ inputs.ref || github.ref }} ref: ${{ inputs.ref || github.ref }}
- name: Prepare sgl-kernel/dist for prebuilt wheel - name: Prepare AOT dist for prebuilt wheel
if: inputs.kernel_artifact_run_id != '' if: inputs.kernel_artifact_run_id != ''
run: | run: |
ls -alh sgl-kernel/dist || true ls -alh python/sglang/kernels/aot/dist || true
rm -rf sgl-kernel/dist/* || true rm -rf python/sglang/kernels/aot/dist/* || true
- name: Download prebuilt sgl-kernel wheel - name: Download prebuilt sgl-kernel wheel
if: inputs.kernel_artifact_run_id != '' if: inputs.kernel_artifact_run_id != ''
uses: actions/download-artifact@v4 uses: actions/download-artifact@v4
with: with:
path: sgl-kernel/dist/ path: python/sglang/kernels/aot/dist/
merge-multiple: true merge-multiple: true
name: wheel-python3.10-cuda13.0 name: wheel-python3.10-cuda13.0
run-id: ${{ inputs.kernel_artifact_run_id }} run-id: ${{ inputs.kernel_artifact_run_id }}
@@ -457,17 +457,17 @@ jobs:
with: with:
ref: ${{ inputs.ref || github.ref }} ref: ${{ inputs.ref || github.ref }}
- name: Prepare sgl-kernel/dist for prebuilt wheel - name: Prepare AOT dist for prebuilt wheel
if: inputs.kernel_artifact_run_id != '' if: inputs.kernel_artifact_run_id != ''
run: | run: |
ls -alh sgl-kernel/dist || true ls -alh python/sglang/kernels/aot/dist || true
rm -rf sgl-kernel/dist/* || true rm -rf python/sglang/kernels/aot/dist/* || true
- name: Download prebuilt sgl-kernel wheel - name: Download prebuilt sgl-kernel wheel
if: inputs.kernel_artifact_run_id != '' if: inputs.kernel_artifact_run_id != ''
uses: actions/download-artifact@v4 uses: actions/download-artifact@v4
with: with:
path: sgl-kernel/dist/ path: python/sglang/kernels/aot/dist/
merge-multiple: true merge-multiple: true
name: wheel-python3.10-cuda13.0 name: wheel-python3.10-cuda13.0
run-id: ${{ inputs.kernel_artifact_run_id }} run-id: ${{ inputs.kernel_artifact_run_id }}
@@ -532,17 +532,17 @@ jobs:
with: with:
ref: ${{ inputs.ref || github.ref }} ref: ${{ inputs.ref || github.ref }}
- name: Prepare sgl-kernel/dist for prebuilt wheel - name: Prepare AOT dist for prebuilt wheel
if: inputs.kernel_artifact_run_id != '' if: inputs.kernel_artifact_run_id != ''
run: | run: |
ls -alh sgl-kernel/dist || true ls -alh python/sglang/kernels/aot/dist || true
rm -rf sgl-kernel/dist/* || true rm -rf python/sglang/kernels/aot/dist/* || true
- name: Download prebuilt sgl-kernel wheel - name: Download prebuilt sgl-kernel wheel
if: inputs.kernel_artifact_run_id != '' if: inputs.kernel_artifact_run_id != ''
uses: actions/download-artifact@v4 uses: actions/download-artifact@v4
with: with:
path: sgl-kernel/dist/ path: python/sglang/kernels/aot/dist/
merge-multiple: true merge-multiple: true
name: wheel-python3.10-cuda13.0 name: wheel-python3.10-cuda13.0
run-id: ${{ inputs.kernel_artifact_run_id }} run-id: ${{ inputs.kernel_artifact_run_id }}
.github/workflows/lint.yml
+1 -1
View File
@@ -59,7 +59,7 @@ jobs:
- name: Run sgl-kernel clang-format checks - name: Run sgl-kernel clang-format checks
uses: DoozyX/clang-format-lint-action@v0.20 uses: DoozyX/clang-format-lint-action@v0.20
with: with:
source: sgl-kernel source: python/sglang/kernels/aot
extensions: h,c,cpp,hpp,cu,cuh,cc extensions: h,c,cpp,hpp,cu,cuh,cc
clangFormatVersion: 20 clangFormatVersion: 20
style: file style: file
.github/workflows/nightly-72-gpu-gb200.yml
+1 -1
View File
@@ -132,7 +132,7 @@ jobs:
image_tag: ${{ steps.build.outputs.image_tag }} image_tag: ${{ steps.build.outputs.image_tag }}
steps: steps:
# Self-hosted runners retain the workspace across jobs. Prior `docker buildx` # Self-hosted runners retain the workspace across jobs. Prior `docker buildx`
# runs on this node leave root-owned build artifacts (e.g. sgl-kernel/build/) # runs on this node leave root-owned build artifacts (e.g. python/sglang/kernels/aot/build/)
# that actions/checkout cannot remove, causing EACCES on rmdir. Wipe them # that actions/checkout cannot remove, causing EACCES on rmdir. Wipe them
# via a throwaway root container before checkout recreates the workspace. # via a throwaway root container before checkout recreates the workspace.
- name: Clean workspace (remove root-owned files from prior runs) - name: Clean workspace (remove root-owned files from prior runs)
.github/workflows/nightly-test-musa.yml
+5 -5
View File
@@ -74,11 +74,11 @@ jobs:
if: steps.gate.outputs.run_job == 'true' if: steps.gate.outputs.run_job == 'true'
timeout-minutes: 30 timeout-minutes: 30
run: | run: |
pytest sgl-kernel/tests/test_per_token_quant_fp8.py pytest python/sglang/kernels/aot/tests/test_per_token_quant_fp8.py
pytest sgl-kernel/tests/speculative/test_eagle_utils.py pytest python/sglang/kernels/aot/tests/speculative/test_eagle_utils.py
pytest sgl-kernel/tests/speculative/test_ngram_utils.py pytest python/sglang/kernels/aot/tests/speculative/test_ngram_utils.py
pytest sgl-kernel/tests/speculative/test_speculative_sampling.py pytest python/sglang/kernels/aot/tests/speculative/test_speculative_sampling.py
pytest sgl-kernel/tests/test_torch_defaults_reset.py pytest python/sglang/kernels/aot/tests/test_torch_defaults_reset.py
# ==================== General: multimodal layer ==================== # ==================== General: multimodal layer ====================
nightly-test-musa-general-multimodal-layer: nightly-test-musa-general-multimodal-layer:
.github/workflows/pr-test-amd-rocm720.yml
+20 -17
View File
@@ -15,7 +15,7 @@ on:
# - "python/**" # - "python/**"
# - "scripts/ci/**" # - "scripts/ci/**"
# - "test/**" # - "test/**"
# - "sgl-kernel/**" # - "python/sglang/kernels/aot/**"
# - ".github/workflows/pr-test-amd-rocm720.yml" # - ".github/workflows/pr-test-amd-rocm720.yml"
# - "docker/rocm.Dockerfile" # - "docker/rocm.Dockerfile"
# pull_request: # pull_request:
@@ -23,7 +23,7 @@ on:
# - "python/**" # - "python/**"
# - "scripts/ci/**" # - "scripts/ci/**"
# - "test/**" # - "test/**"
# - "sgl-kernel/**" # - "python/sglang/kernels/aot/**"
# - ".github/workflows/pr-test-amd-rocm720.yml" # - ".github/workflows/pr-test-amd-rocm720.yml"
# - "docker/rocm.Dockerfile" # - "docker/rocm.Dockerfile"
workflow_dispatch: workflow_dispatch:
@@ -187,7 +187,9 @@ jobs:
with: with:
filters: | filters: |
main_package: main_package:
- "python/sglang/!(multimodal_gen)/**/!(*.md)" - "python/sglang/!(multimodal_gen|kernels)/**/!(*.md)"
- "python/sglang/kernels/!(*.md)"
- "python/sglang/kernels/!(aot)/**/!(*.md)"
- "python/pyproject_rocm.toml" - "python/pyproject_rocm.toml"
- "python/pyproject_other.toml" - "python/pyproject_other.toml"
- "scripts/ci/amd/*" - "scripts/ci/amd/*"
@@ -195,10 +197,11 @@ jobs:
- "test/**/!(*.md)" - "test/**/!(*.md)"
- ".github/workflows/pr-test-amd-rocm720.yml" - ".github/workflows/pr-test-amd-rocm720.yml"
sgl_kernel: sgl_kernel:
- "sgl-kernel/**/!(*.md|THIRDPARTYNOTICES.txt|LICENSE)" - "python/sglang/kernels/aot/**/!(*.md|THIRDPARTYNOTICES.txt|LICENSE)"
- ".github/workflows/pr-test-amd-rocm720.yml" - ".github/workflows/pr-test-amd-rocm720.yml"
jit_kernel: jit_kernel:
- "python/sglang/kernels/**" - "python/sglang/kernels/!(*.md)"
- "python/sglang/kernels/!(aot)/**"
- "test/registered/kernels/**" - "test/registered/kernels/**"
- ".github/workflows/pr-test-amd-rocm720.yml" - ".github/workflows/pr-test-amd-rocm720.yml"
multimodal_gen: multimodal_gen:
@@ -268,15 +271,15 @@ jobs:
- name: Run test - name: Run test
timeout-minutes: 30 timeout-minutes: 30
run: | run: |
docker exec -w /sglang-checkout/sgl-kernel/tests ci_sglang python3 -m pytest test_moe_align.py docker exec -w /sglang-checkout/python/sglang/kernels/aot/tests ci_sglang python3 -m pytest test_moe_align.py
docker exec -w /sglang-checkout/sgl-kernel/tests ci_sglang python3 -m pytest test_moe_topk_softmax.py docker exec -w /sglang-checkout/python/sglang/kernels/aot/tests ci_sglang python3 -m pytest test_moe_topk_softmax.py
docker exec -w /sglang-checkout/sgl-kernel/tests/speculative ci_sglang python3 -m pytest test_eagle_utils.py docker exec -w /sglang-checkout/python/sglang/kernels/aot/tests/speculative ci_sglang python3 -m pytest test_eagle_utils.py
docker exec -w /sglang-checkout/sgl-kernel/tests ci_sglang python3 -m pytest test_apply_token_bitmask_inplace.py docker exec -w /sglang-checkout/python/sglang/kernels/aot/tests ci_sglang python3 -m pytest test_apply_token_bitmask_inplace.py
docker exec -w /sglang-checkout/sgl-kernel/tests ci_sglang python3 -m pytest test_activation.py docker exec -w /sglang-checkout/python/sglang/kernels/aot/tests ci_sglang python3 -m pytest test_activation.py
docker exec -w /sglang-checkout/sgl-kernel/tests ci_sglang python3 -m pytest test_topk.py docker exec -w /sglang-checkout/python/sglang/kernels/aot/tests ci_sglang python3 -m pytest test_topk.py
docker exec -w /sglang-checkout/sgl-kernel/tests ci_sglang python3 -m pytest test_kvcacheio.py docker exec -w /sglang-checkout/python/sglang/kernels/aot/tests ci_sglang python3 -m pytest test_kvcacheio.py
docker exec -w /sglang-checkout/sgl-kernel/tests ci_sglang python3 -m pytest test_moe_topk_sigmoid.py docker exec -w /sglang-checkout/python/sglang/kernels/aot/tests ci_sglang python3 -m pytest test_moe_topk_sigmoid.py
docker exec -w /sglang-checkout/sgl-kernel/tests ci_sglang python3 -m pytest test_torch_defaults_reset.py docker exec -w /sglang-checkout/python/sglang/kernels/aot/tests ci_sglang python3 -m pytest test_torch_defaults_reset.py
sgl-kernel-unit-test-2-gpu-amd-rocm720: sgl-kernel-unit-test-2-gpu-amd-rocm720:
needs: [check-changes] needs: [check-changes]
@@ -657,7 +660,7 @@ jobs:
if: needs.check-changes.outputs.sgl_kernel == 'true' if: needs.check-changes.outputs.sgl_kernel == 'true'
uses: actions/download-artifact@v4 uses: actions/download-artifact@v4
with: with:
path: sgl-kernel/dist/ path: python/sglang/kernels/aot/dist/
merge-multiple: true merge-multiple: true
pattern: wheel-python3.10-cuda12.9 pattern: wheel-python3.10-cuda12.9
@@ -797,7 +800,7 @@ jobs:
if: needs.check-changes.outputs.sgl_kernel == 'true' if: needs.check-changes.outputs.sgl_kernel == 'true'
uses: actions/download-artifact@v4 uses: actions/download-artifact@v4
with: with:
path: sgl-kernel/dist/ path: python/sglang/kernels/aot/dist/
merge-multiple: true merge-multiple: true
pattern: wheel-python3.10-cuda12.9 pattern: wheel-python3.10-cuda12.9
@@ -934,7 +937,7 @@ jobs:
if: needs.check-changes.outputs.sgl_kernel == 'true' if: needs.check-changes.outputs.sgl_kernel == 'true'
uses: actions/download-artifact@v4 uses: actions/download-artifact@v4
with: with:
path: sgl-kernel/dist/ path: python/sglang/kernels/aot/dist/
merge-multiple: true merge-multiple: true
pattern: wheel-python3.10-cuda12.9 pattern: wheel-python3.10-cuda12.9
.github/workflows/pr-test-amd.yml
+19 -16
View File
@@ -11,7 +11,7 @@ on:
- "python/**" - "python/**"
- "scripts/ci/**" - "scripts/ci/**"
- "test/**" - "test/**"
- "sgl-kernel/**" - "python/sglang/kernels/aot/**"
- ".github/workflows/pr-test-amd.yml" - ".github/workflows/pr-test-amd.yml"
- "docker/rocm.Dockerfile" - "docker/rocm.Dockerfile"
workflow_dispatch: workflow_dispatch:
@@ -175,7 +175,9 @@ jobs:
with: with:
filters: | filters: |
main_package: main_package:
- "python/sglang/!(multimodal_gen)/**/!(*.md)" - "python/sglang/!(multimodal_gen|kernels)/**/!(*.md)"
- "python/sglang/kernels/!(*.md)"
- "python/sglang/kernels/!(aot)/**/!(*.md)"
- "python/pyproject_rocm.toml" - "python/pyproject_rocm.toml"
- "python/pyproject_other.toml" - "python/pyproject_other.toml"
- "scripts/ci/amd/*" - "scripts/ci/amd/*"
@@ -183,10 +185,11 @@ jobs:
- "test/**/!(*.md)" - "test/**/!(*.md)"
- ".github/workflows/pr-test-amd.yml" - ".github/workflows/pr-test-amd.yml"
sgl_kernel: sgl_kernel:
- "sgl-kernel/**/!(*.md|THIRDPARTYNOTICES.txt|LICENSE)" - "python/sglang/kernels/aot/**/!(*.md|THIRDPARTYNOTICES.txt|LICENSE)"
- ".github/workflows/pr-test-amd.yml" - ".github/workflows/pr-test-amd.yml"
jit_kernel: jit_kernel:
- "python/sglang/kernels/**" - "python/sglang/kernels/!(*.md)"
- "python/sglang/kernels/!(aot)/**"
- "test/registered/kernels/**" - "test/registered/kernels/**"
- ".github/workflows/pr-test-amd.yml" - ".github/workflows/pr-test-amd.yml"
multimodal_gen: multimodal_gen:
@@ -266,15 +269,15 @@ jobs:
"$@" "$@"
fi fi
} }
run_pytest docker exec -w /sglang-checkout/sgl-kernel/tests ci_sglang python3 -m pytest test_moe_align.py run_pytest docker exec -w /sglang-checkout/python/sglang/kernels/aot/tests ci_sglang python3 -m pytest test_moe_align.py
run_pytest docker exec -w /sglang-checkout/sgl-kernel/tests ci_sglang python3 -m pytest test_moe_topk_softmax.py run_pytest docker exec -w /sglang-checkout/python/sglang/kernels/aot/tests ci_sglang python3 -m pytest test_moe_topk_softmax.py
run_pytest docker exec -w /sglang-checkout/sgl-kernel/tests/speculative ci_sglang python3 -m pytest test_eagle_utils.py run_pytest docker exec -w /sglang-checkout/python/sglang/kernels/aot/tests/speculative ci_sglang python3 -m pytest test_eagle_utils.py
run_pytest docker exec -w /sglang-checkout/sgl-kernel/tests ci_sglang python3 -m pytest test_apply_token_bitmask_inplace.py run_pytest docker exec -w /sglang-checkout/python/sglang/kernels/aot/tests ci_sglang python3 -m pytest test_apply_token_bitmask_inplace.py
run_pytest docker exec -w /sglang-checkout/sgl-kernel/tests ci_sglang python3 -m pytest test_activation.py run_pytest docker exec -w /sglang-checkout/python/sglang/kernels/aot/tests ci_sglang python3 -m pytest test_activation.py
run_pytest docker exec -w /sglang-checkout/sgl-kernel/tests ci_sglang python3 -m pytest test_topk.py run_pytest docker exec -w /sglang-checkout/python/sglang/kernels/aot/tests ci_sglang python3 -m pytest test_topk.py
run_pytest docker exec -w /sglang-checkout/sgl-kernel/tests ci_sglang python3 -m pytest test_kvcacheio.py run_pytest docker exec -w /sglang-checkout/python/sglang/kernels/aot/tests ci_sglang python3 -m pytest test_kvcacheio.py
run_pytest docker exec -w /sglang-checkout/sgl-kernel/tests ci_sglang python3 -m pytest test_moe_topk_sigmoid.py run_pytest docker exec -w /sglang-checkout/python/sglang/kernels/aot/tests ci_sglang python3 -m pytest test_moe_topk_sigmoid.py
run_pytest docker exec -w /sglang-checkout/sgl-kernel/tests ci_sglang python3 -m pytest test_torch_defaults_reset.py run_pytest docker exec -w /sglang-checkout/python/sglang/kernels/aot/tests ci_sglang python3 -m pytest test_torch_defaults_reset.py
exit $failures exit $failures
sgl-kernel-unit-test-2-gpu-amd: sgl-kernel-unit-test-2-gpu-amd:
@@ -681,7 +684,7 @@ jobs:
if: needs.check-changes.outputs.sgl_kernel == 'true' if: needs.check-changes.outputs.sgl_kernel == 'true'
uses: actions/download-artifact@v4 uses: actions/download-artifact@v4
with: with:
path: sgl-kernel/dist/ path: python/sglang/kernels/aot/dist/
merge-multiple: true merge-multiple: true
pattern: wheel-python3.10-cuda12.9 pattern: wheel-python3.10-cuda12.9
@@ -821,7 +824,7 @@ jobs:
if: needs.check-changes.outputs.sgl_kernel == 'true' if: needs.check-changes.outputs.sgl_kernel == 'true'
uses: actions/download-artifact@v4 uses: actions/download-artifact@v4
with: with:
path: sgl-kernel/dist/ path: python/sglang/kernels/aot/dist/
merge-multiple: true merge-multiple: true
pattern: wheel-python3.10-cuda12.9 pattern: wheel-python3.10-cuda12.9
@@ -960,7 +963,7 @@ jobs:
if: needs.check-changes.outputs.sgl_kernel == 'true' if: needs.check-changes.outputs.sgl_kernel == 'true'
uses: actions/download-artifact@v4 uses: actions/download-artifact@v4
with: with:
path: sgl-kernel/dist/ path: python/sglang/kernels/aot/dist/
merge-multiple: true merge-multiple: true
pattern: wheel-python3.10-cuda12.9 pattern: wheel-python3.10-cuda12.9
.github/workflows/pr-test-arm64.yml
+1 -1
View File
@@ -55,7 +55,7 @@ jobs:
- "python/sglang/!(multimodal_gen)/**/!(*.md)" - "python/sglang/!(multimodal_gen)/**/!(*.md)"
- "python/pyproject_cpu.toml" - "python/pyproject_cpu.toml"
- "test/**/!(*.md)" - "test/**/!(*.md)"
- "sgl-kernel/**/*.!(md|txt)" - "python/sglang/kernels/aot/**/*.!(md|txt)"
- ".github/workflows/pr-test-arm64.yml" - ".github/workflows/pr-test-arm64.yml"
- "docker/arm64.Dockerfile" - "docker/arm64.Dockerfile"
.github/workflows/pr-test-jit-kernel.yml
+12 -12
View File
@@ -56,14 +56,14 @@ jobs:
- name: Cleanup - name: Cleanup
if: inputs.sgl_kernel == 'true' if: inputs.sgl_kernel == 'true'
run: | run: |
ls -alh sgl-kernel/dist || true ls -alh python/sglang/kernels/aot/dist || true
rm -rf sgl-kernel/dist/* || true rm -rf python/sglang/kernels/aot/dist/* || true
- name: Download artifacts - name: Download artifacts
if: inputs.sgl_kernel == 'true' if: inputs.sgl_kernel == 'true'
uses: actions/download-artifact@v4 uses: actions/download-artifact@v4
with: with:
path: sgl-kernel/dist/ path: python/sglang/kernels/aot/dist/
merge-multiple: true merge-multiple: true
pattern: wheel-python3.10-cuda13.0 pattern: wheel-python3.10-cuda13.0
@@ -94,14 +94,14 @@ jobs:
- name: Cleanup - name: Cleanup
if: inputs.sgl_kernel == 'true' if: inputs.sgl_kernel == 'true'
run: | run: |
ls -alh sgl-kernel/dist || true ls -alh python/sglang/kernels/aot/dist || true
rm -rf sgl-kernel/dist/* || true rm -rf python/sglang/kernels/aot/dist/* || true
- name: Download artifacts - name: Download artifacts
if: inputs.sgl_kernel == 'true' if: inputs.sgl_kernel == 'true'
uses: actions/download-artifact@v4 uses: actions/download-artifact@v4
with: with:
path: sgl-kernel/dist/ path: python/sglang/kernels/aot/dist/
merge-multiple: true merge-multiple: true
pattern: wheel-python3.10-cuda13.0 pattern: wheel-python3.10-cuda13.0
@@ -134,14 +134,14 @@ jobs:
- name: Cleanup - name: Cleanup
if: inputs.sgl_kernel == 'true' if: inputs.sgl_kernel == 'true'
run: | run: |
ls -alh sgl-kernel/dist || true ls -alh python/sglang/kernels/aot/dist || true
rm -rf sgl-kernel/dist/* || true rm -rf python/sglang/kernels/aot/dist/* || true
- name: Download artifacts - name: Download artifacts
if: inputs.sgl_kernel == 'true' if: inputs.sgl_kernel == 'true'
uses: actions/download-artifact@v4 uses: actions/download-artifact@v4
with: with:
path: sgl-kernel/dist/ path: python/sglang/kernels/aot/dist/
merge-multiple: true merge-multiple: true
pattern: wheel-python3.10-cuda13.0 pattern: wheel-python3.10-cuda13.0
@@ -174,14 +174,14 @@ jobs:
- name: Cleanup - name: Cleanup
if: inputs.sgl_kernel == 'true' if: inputs.sgl_kernel == 'true'
run: | run: |
ls -alh sgl-kernel/dist || true ls -alh python/sglang/kernels/aot/dist || true
rm -rf sgl-kernel/dist/* || true rm -rf python/sglang/kernels/aot/dist/* || true
- name: Download artifacts - name: Download artifacts
if: inputs.sgl_kernel == 'true' if: inputs.sgl_kernel == 'true'
uses: actions/download-artifact@v4 uses: actions/download-artifact@v4
with: with:
path: sgl-kernel/dist/ path: python/sglang/kernels/aot/dist/
merge-multiple: true merge-multiple: true
pattern: wheel-python3.10-cuda13.0 pattern: wheel-python3.10-cuda13.0
.github/workflows/pr-test-mlx.yml
+4 -2
View File
@@ -43,12 +43,14 @@ jobs:
base: main base: main
filters: | filters: |
main_package: main_package:
- "python/sglang/!(multimodal_gen)/**/!(*.md)" - "python/sglang/!(multimodal_gen|kernels)/**/!(*.md)"
- "python/sglang/kernels/!(*.md)"
- "python/sglang/kernels/!(aot)/**/!(*.md)"
- "python/pyproject_other.toml" - "python/pyproject_other.toml"
- "test/**/!(*.md)" - "test/**/!(*.md)"
- ".github/workflows/pr-test-mlx.yml" - ".github/workflows/pr-test-mlx.yml"
sgl_kernel: sgl_kernel:
- "sgl-kernel/**/!(*.md|THIRDPARTYNOTICES.txt|LICENSE)" - "python/sglang/kernels/aot/**/!(*.md|THIRDPARTYNOTICES.txt|LICENSE)"
- ".github/workflows/pr-test-mlx.yml" - ".github/workflows/pr-test-mlx.yml"
# ==================== PR Gate ==================== # # ==================== PR Gate ==================== #
.github/workflows/pr-test-multimodal-gen.yml
+7 -7
View File
@@ -98,7 +98,7 @@ jobs:
if: inputs.sgl_kernel == 'true' if: inputs.sgl_kernel == 'true'
uses: actions/download-artifact@v4 uses: actions/download-artifact@v4
with: with:
path: sgl-kernel/dist/ path: python/sglang/kernels/aot/dist/
merge-multiple: true merge-multiple: true
pattern: wheel-python3.10-cuda* pattern: wheel-python3.10-cuda*
@@ -164,7 +164,7 @@ jobs:
if: inputs.sgl_kernel == 'true' if: inputs.sgl_kernel == 'true'
uses: actions/download-artifact@v4 uses: actions/download-artifact@v4
with: with:
path: sgl-kernel/dist/ path: python/sglang/kernels/aot/dist/
merge-multiple: true merge-multiple: true
pattern: wheel-python3.10-cuda* pattern: wheel-python3.10-cuda*
@@ -227,7 +227,7 @@ jobs:
if: inputs.sgl_kernel == 'true' if: inputs.sgl_kernel == 'true'
uses: actions/download-artifact@v4 uses: actions/download-artifact@v4
with: with:
path: sgl-kernel/dist/ path: python/sglang/kernels/aot/dist/
merge-multiple: true merge-multiple: true
pattern: wheel-python3.10-cuda* pattern: wheel-python3.10-cuda*
@@ -289,7 +289,7 @@ jobs:
if: inputs.sgl_kernel == 'true' if: inputs.sgl_kernel == 'true'
uses: actions/download-artifact@v4 uses: actions/download-artifact@v4
with: with:
path: sgl-kernel/dist/ path: python/sglang/kernels/aot/dist/
merge-multiple: true merge-multiple: true
pattern: wheel-python3.10-cuda* pattern: wheel-python3.10-cuda*
@@ -356,7 +356,7 @@ jobs:
if: inputs.sgl_kernel == 'true' if: inputs.sgl_kernel == 'true'
uses: actions/download-artifact@v4 uses: actions/download-artifact@v4
with: with:
path: sgl-kernel/dist/ path: python/sglang/kernels/aot/dist/
merge-multiple: true merge-multiple: true
pattern: wheel-python3.10-cuda* pattern: wheel-python3.10-cuda*
@@ -401,7 +401,7 @@ jobs:
if: inputs.sgl_kernel == 'true' if: inputs.sgl_kernel == 'true'
uses: actions/download-artifact@v4 uses: actions/download-artifact@v4
with: with:
path: sgl-kernel/dist/ path: python/sglang/kernels/aot/dist/
merge-multiple: true merge-multiple: true
pattern: wheel-python3.10-cuda* pattern: wheel-python3.10-cuda*
@@ -455,7 +455,7 @@ jobs:
if: inputs.sgl_kernel == 'true' if: inputs.sgl_kernel == 'true'
uses: actions/download-artifact@v4 uses: actions/download-artifact@v4
with: with:
path: sgl-kernel/dist/ path: python/sglang/kernels/aot/dist/
merge-multiple: true merge-multiple: true
pattern: wheel-python3.10-cuda* pattern: wheel-python3.10-cuda*
.github/workflows/pr-test-musa.yml
+6 -6
View File
@@ -83,7 +83,7 @@ jobs:
- "python/sglang/multimodal_gen/test/server/musa/**" - "python/sglang/multimodal_gen/test/server/musa/**"
sgl_kernel: sgl_kernel:
- ".github/workflows/pr-test-musa.yml" - ".github/workflows/pr-test-musa.yml"
- "sgl-kernel/csrc/musa/**" - "python/sglang/kernels/aot/csrc/musa/**"
# ==================== PR Gate ==================== # # ==================== PR Gate ==================== #
pr-gate: pr-gate:
@@ -217,11 +217,11 @@ jobs:
- name: Run sgl-kernel test - name: Run sgl-kernel test
timeout-minutes: 20 timeout-minutes: 20
run: | run: |
pytest sgl-kernel/tests/test_per_token_quant_fp8.py pytest python/sglang/kernels/aot/tests/test_per_token_quant_fp8.py
pytest sgl-kernel/tests/speculative/test_eagle_utils.py pytest python/sglang/kernels/aot/tests/speculative/test_eagle_utils.py
pytest sgl-kernel/tests/speculative/test_ngram_utils.py pytest python/sglang/kernels/aot/tests/speculative/test_ngram_utils.py
pytest sgl-kernel/tests/speculative/test_speculative_sampling.py pytest python/sglang/kernels/aot/tests/speculative/test_speculative_sampling.py
pytest sgl-kernel/tests/test_torch_defaults_reset.py pytest python/sglang/kernels/aot/tests/test_torch_defaults_reset.py
pr-test-musa-finish: pr-test-musa-finish:
.github/workflows/pr-test-sgl-kernel.yml
+16 -16
View File
@@ -44,13 +44,13 @@ jobs:
- name: Cleanup - name: Cleanup
run: | run: |
ls -alh sgl-kernel/dist || true ls -alh python/sglang/kernels/aot/dist || true
rm -rf sgl-kernel/dist/* || true rm -rf python/sglang/kernels/aot/dist/* || true
- name: Download artifacts - name: Download artifacts
uses: actions/download-artifact@v4 uses: actions/download-artifact@v4
with: with:
path: sgl-kernel/dist/ path: python/sglang/kernels/aot/dist/
merge-multiple: true merge-multiple: true
pattern: wheel-python3.10-cuda* pattern: wheel-python3.10-cuda*
@@ -62,7 +62,7 @@ jobs:
- name: Run test - name: Run test
timeout-minutes: 30 timeout-minutes: 30
run: | run: |
cd sgl-kernel cd python/sglang/kernels/aot
pytest tests/ pytest tests/
sgl-kernel-benchmark-test: sgl-kernel-benchmark-test:
@@ -79,13 +79,13 @@ jobs:
- name: Cleanup - name: Cleanup
run: | run: |
ls -alh sgl-kernel/dist || true ls -alh python/sglang/kernels/aot/dist || true
rm -rf sgl-kernel/dist/* || true rm -rf python/sglang/kernels/aot/dist/* || true
- name: Download artifacts - name: Download artifacts
uses: actions/download-artifact@v4 uses: actions/download-artifact@v4
with: with:
path: sgl-kernel/dist/ path: python/sglang/kernels/aot/dist/
merge-multiple: true merge-multiple: true
pattern: wheel-python3.10-cuda* pattern: wheel-python3.10-cuda*
@@ -97,7 +97,7 @@ jobs:
- name: Run benchmark tests - name: Run benchmark tests
timeout-minutes: 45 timeout-minutes: 45
run: | run: |
cd sgl-kernel/benchmark cd python/sglang/kernels/aot/benchmark
echo "Running sgl-kernel benchmark tests in CI mode..." echo "Running sgl-kernel benchmark tests in CI mode..."
echo "CI environment variable: $CI" echo "CI environment variable: $CI"
@@ -126,13 +126,13 @@ jobs:
- name: Cleanup - name: Cleanup
run: | run: |
ls -alh sgl-kernel/dist || true ls -alh python/sglang/kernels/aot/dist || true
rm -rf sgl-kernel/dist/* || true rm -rf python/sglang/kernels/aot/dist/* || true
- name: Download artifacts - name: Download artifacts
uses: actions/download-artifact@v4 uses: actions/download-artifact@v4
with: with:
path: sgl-kernel/dist/ path: python/sglang/kernels/aot/dist/
merge-multiple: true merge-multiple: true
pattern: wheel-python3.10-cuda* pattern: wheel-python3.10-cuda*
@@ -144,7 +144,7 @@ jobs:
- name: Run sgl-kernel unit tests on B200 - name: Run sgl-kernel unit tests on B200
timeout-minutes: 30 timeout-minutes: 30
run: | run: |
cd sgl-kernel cd python/sglang/kernels/aot
pytest tests/ pytest tests/
# Adding a single CUDA13 build-and-run check for the kernel # Adding a single CUDA13 build-and-run check for the kernel
@@ -157,13 +157,13 @@ jobs:
# - name: Cleanup # - name: Cleanup
# run: | # run: |
# ls -alh sgl-kernel/dist || true # ls -alh python/sglang/kernels/aot/dist || true
# rm -rf sgl-kernel/dist/* || true # rm -rf python/sglang/kernels/aot/dist/* || true
# - name: Download CUDA 13.0 artifacts # - name: Download CUDA 13.0 artifacts
# uses: actions/download-artifact@v4 # uses: actions/download-artifact@v4
# with: # with:
# path: sgl-kernel/dist/ # path: python/sglang/kernels/aot/dist/
# merge-multiple: true # merge-multiple: true
# pattern: wheel-python3.10-cuda* # pattern: wheel-python3.10-cuda*
@@ -174,5 +174,5 @@ jobs:
# - name: Run kernel unit tests # - name: Run kernel unit tests
# timeout-minutes: 30 # timeout-minutes: 30
# run: | # run: |
# cd sgl-kernel # cd python/sglang/kernels/aot
# pytest tests/ # pytest tests/
.github/workflows/pr-test-xeon.yml
+1 -1
View File
@@ -66,7 +66,7 @@ jobs:
- "python/sglang/!(multimodal_gen)/**/!(*.md)" - "python/sglang/!(multimodal_gen)/**/!(*.md)"
- "python/pyproject_cpu.toml" - "python/pyproject_cpu.toml"
- "test/**/!(*.md)" - "test/**/!(*.md)"
- "sgl-kernel/**/!(*.md|THIRDPARTYNOTICES.txt|LICENSE)" - "python/sglang/kernels/aot/**/!(*.md|THIRDPARTYNOTICES.txt|LICENSE)"
- ".github/workflows/pr-test-xeon.yml" - ".github/workflows/pr-test-xeon.yml"
- "docker/xeon.Dockerfile" - "docker/xeon.Dockerfile"
.github/workflows/pr-test-xpu.yml
+1 -1
View File
@@ -57,7 +57,7 @@ jobs:
- "python/sglang/!(multimodal_gen)/**/!(*.md)" - "python/sglang/!(multimodal_gen)/**/!(*.md)"
- "python/pyproject_xpu.toml" - "python/pyproject_xpu.toml"
- "test/**/!(*.md)" - "test/**/!(*.md)"
- "sgl-kernel/**/!(*.md|THIRDPARTYNOTICES.txt|LICENSE)" - "python/sglang/kernels/aot/**/!(*.md|THIRDPARTYNOTICES.txt|LICENSE)"
- ".github/workflows/pr-test-xpu.yml" - ".github/workflows/pr-test-xpu.yml"
- "docker/xpu.Dockerfile" - "docker/xpu.Dockerfile"
.github/workflows/release-whl-kernel.yml
+32 -32
View File
@@ -5,7 +5,7 @@ on:
branches: branches:
- main - main
paths: paths:
- sgl-kernel/python/sgl_kernel/version.py - python/sglang/kernels/aot/python/sgl_kernel/version.py
workflow_dispatch: workflow_dispatch:
inputs: inputs:
target: target:
@@ -54,7 +54,7 @@ jobs:
runs-on: ${{ matrix.runner }} runs-on: ${{ matrix.runner }}
steps: steps:
# Self-hosted build nodes retain the workspace across jobs. Prior builds # Self-hosted build nodes retain the workspace across jobs. Prior builds
# leave root-owned artifacts under sgl-kernel/build/ that actions/checkout # leave root-owned artifacts under python/sglang/kernels/aot/build/ that actions/checkout
# cannot remove, causing EACCES on rmdir. Wipe them via a throwaway root # cannot remove, causing EACCES on rmdir. Wipe them via a throwaway root
# container before checkout recreates the workspace. # container before checkout recreates the workspace.
- name: Clean workspace (remove root-owned files from prior runs) - name: Clean workspace (remove root-owned files from prior runs)
@@ -74,7 +74,7 @@ jobs:
- name: Build wheels - name: Build wheels
run: | run: |
cd sgl-kernel cd python/sglang/kernels/aot
chmod +x ./build.sh chmod +x ./build.sh
./build.sh "${{ matrix.python-version }}" "${{ matrix.cuda-version }}" ${{ matrix.arch == 'aarch64' && 'aarch64' || '' }} ./build.sh "${{ matrix.python-version }}" "${{ matrix.cuda-version }}" ${{ matrix.arch == 'aarch64' && 'aarch64' || '' }}
env: env:
@@ -85,7 +85,7 @@ jobs:
uses: actions/upload-artifact@v4 uses: actions/upload-artifact@v4
with: with:
name: wheel-python${{ matrix.python-version }}-cuda${{ matrix.cuda-version }}${{ matrix.arch == 'aarch64' && '-aarch64' || '' }} name: wheel-python${{ matrix.python-version }}-cuda${{ matrix.cuda-version }}${{ matrix.arch == 'aarch64' && '-aarch64' || '' }}
path: sgl-kernel/dist/* path: python/sglang/kernels/aot/dist/*
release-cu129: release-cu129:
needs: build-cu129-matrix needs: build-cu129-matrix
@@ -98,7 +98,7 @@ jobs:
- name: Download artifacts - name: Download artifacts
uses: actions/download-artifact@v4 uses: actions/download-artifact@v4
with: with:
path: sgl-kernel/dist/ path: python/sglang/kernels/aot/dist/
merge-multiple: true merge-multiple: true
pattern: wheel-* pattern: wheel-*
@@ -106,7 +106,7 @@ jobs:
id: set_tag_name id: set_tag_name
run: | run: |
if [ -z "${{ inputs.tag_name }}" ]; then if [ -z "${{ inputs.tag_name }}" ]; then
TAG_NAME="v$(cat sgl-kernel/python/sgl_kernel/version.py | cut -d'"' -f2)" TAG_NAME="v$(cat python/sglang/kernels/aot/python/sgl_kernel/version.py | cut -d'"' -f2)"
echo "tag_name=$TAG_NAME" >> $GITHUB_OUTPUT echo "tag_name=$TAG_NAME" >> $GITHUB_OUTPUT
else else
echo "tag_name=${{ inputs.tag_name }}" >> $GITHUB_OUTPUT echo "tag_name=${{ inputs.tag_name }}" >> $GITHUB_OUTPUT
@@ -119,7 +119,7 @@ jobs:
repository: sgl-project/whl repository: sgl-project/whl
token: ${{ secrets.GH_PAT_FOR_WHL_RELEASE }} token: ${{ secrets.GH_PAT_FOR_WHL_RELEASE }}
files: | files: |
sgl-kernel/dist/* python/sglang/kernels/aot/dist/*
- name: Clone wheel index - name: Clone wheel index
run: git clone https://oauth2:${WHL_TOKEN}@github.com/sgl-project/whl.git sgl-whl run: git clone https://oauth2:${WHL_TOKEN}@github.com/sgl-project/whl.git sgl-whl
@@ -155,7 +155,7 @@ jobs:
runs-on: ${{ matrix.runner }} runs-on: ${{ matrix.runner }}
steps: steps:
# Self-hosted build nodes retain the workspace across jobs. Prior builds # Self-hosted build nodes retain the workspace across jobs. Prior builds
# leave root-owned artifacts under sgl-kernel/build/ that actions/checkout # leave root-owned artifacts under python/sglang/kernels/aot/build/ that actions/checkout
# cannot remove, causing EACCES on rmdir. Wipe them via a throwaway root # cannot remove, causing EACCES on rmdir. Wipe them via a throwaway root
# container before checkout recreates the workspace. # container before checkout recreates the workspace.
- name: Clean workspace (remove root-owned files from prior runs) - name: Clean workspace (remove root-owned files from prior runs)
@@ -175,7 +175,7 @@ jobs:
- name: Build wheels - name: Build wheels
run: | run: |
cd sgl-kernel cd python/sglang/kernels/aot
chmod +x ./build.sh chmod +x ./build.sh
./build.sh "${{ matrix.python-version }}" "${{ matrix.cuda-version }}" ${{ matrix.arch == 'aarch64' && 'aarch64' || '' }} ./build.sh "${{ matrix.python-version }}" "${{ matrix.cuda-version }}" ${{ matrix.arch == 'aarch64' && 'aarch64' || '' }}
env: env:
@@ -183,7 +183,7 @@ jobs:
NVCC_THREADS: 8 NVCC_THREADS: 8
- name: Strip +cu130 local version for PyPI upload - name: Strip +cu130 local version for PyPI upload
working-directory: sgl-kernel working-directory: python/sglang/kernels/aot
run: | run: |
set -eux set -eux
pip install wheel pip install wheel
@@ -208,7 +208,7 @@ jobs:
ls -lh dist-pypi/ ls -lh dist-pypi/
- name: Upload to PyPI - name: Upload to PyPI
working-directory: sgl-kernel working-directory: python/sglang/kernels/aot
run: | run: |
pip install twine pip install twine
python3 -m twine upload --skip-existing dist-pypi/* -u __token__ -p ${{ secrets.PYPI_TOKEN_SGLANG_KERNEL }} python3 -m twine upload --skip-existing dist-pypi/* -u __token__ -p ${{ secrets.PYPI_TOKEN_SGLANG_KERNEL }}
@@ -217,7 +217,7 @@ jobs:
uses: actions/upload-artifact@v4 uses: actions/upload-artifact@v4
with: with:
name: wheel-python${{ matrix.python-version }}-cuda${{ matrix.cuda-version }}${{ matrix.arch == 'aarch64' && '-aarch64' || '' }} name: wheel-python${{ matrix.python-version }}-cuda${{ matrix.cuda-version }}${{ matrix.arch == 'aarch64' && '-aarch64' || '' }}
path: sgl-kernel/dist/* path: python/sglang/kernels/aot/dist/*
release-cu130: release-cu130:
needs: build-cu130-matrix needs: build-cu130-matrix
@@ -230,7 +230,7 @@ jobs:
- name: Download artifacts - name: Download artifacts
uses: actions/download-artifact@v4 uses: actions/download-artifact@v4
with: with:
path: sgl-kernel/dist/ path: python/sglang/kernels/aot/dist/
merge-multiple: true merge-multiple: true
pattern: wheel-* pattern: wheel-*
@@ -238,7 +238,7 @@ jobs:
id: set_tag_name id: set_tag_name
run: | run: |
if [ -z "${{ inputs.tag_name }}" ]; then if [ -z "${{ inputs.tag_name }}" ]; then
TAG_NAME="v$(cat sgl-kernel/python/sgl_kernel/version.py | cut -d'"' -f2)" TAG_NAME="v$(cat python/sglang/kernels/aot/python/sgl_kernel/version.py | cut -d'"' -f2)"
echo "tag_name=$TAG_NAME" >> $GITHUB_OUTPUT echo "tag_name=$TAG_NAME" >> $GITHUB_OUTPUT
else else
echo "tag_name=${{ inputs.tag_name }}" >> $GITHUB_OUTPUT echo "tag_name=${{ inputs.tag_name }}" >> $GITHUB_OUTPUT
@@ -251,7 +251,7 @@ jobs:
repository: sgl-project/whl repository: sgl-project/whl
token: ${{ secrets.GH_PAT_FOR_WHL_RELEASE }} token: ${{ secrets.GH_PAT_FOR_WHL_RELEASE }}
files: | files: |
sgl-kernel/dist/* python/sglang/kernels/aot/dist/*
- name: Clone wheel index - name: Clone wheel index
run: git clone https://oauth2:${WHL_TOKEN}@github.com/sgl-project/whl.git sgl-whl run: git clone https://oauth2:${WHL_TOKEN}@github.com/sgl-project/whl.git sgl-whl
@@ -281,7 +281,7 @@ jobs:
rocm-version: ["700", "720"] rocm-version: ["700", "720"]
steps: steps:
# Self-hosted build nodes retain the workspace across jobs. Prior builds # Self-hosted build nodes retain the workspace across jobs. Prior builds
# leave root-owned artifacts under sgl-kernel/build/ that actions/checkout # leave root-owned artifacts under python/sglang/kernels/aot/build/ that actions/checkout
# cannot remove, causing EACCES on rmdir. Wipe them via a throwaway root # cannot remove, causing EACCES on rmdir. Wipe them via a throwaway root
# container before checkout recreates the workspace. # container before checkout recreates the workspace.
- name: Clean workspace (remove root-owned files from prior runs) - name: Clean workspace (remove root-owned files from prior runs)
@@ -301,8 +301,8 @@ jobs:
- name: Build wheels - name: Build wheels
run: | run: |
cp 3rdparty/amd/wheel/sgl-kernel/* sgl-kernel/ cp 3rdparty/amd/wheel/sgl-kernel/* python/sglang/kernels/aot/
cd sgl-kernel cd python/sglang/kernels/aot
chmod +x ./build_rocm.sh chmod +x ./build_rocm.sh
./build_rocm.sh "${{ matrix.rocm-version }}" ./build_rocm.sh "${{ matrix.rocm-version }}"
@@ -310,7 +310,7 @@ jobs:
uses: actions/upload-artifact@v4 uses: actions/upload-artifact@v4
with: with:
name: wheel-python${{ matrix.python-version }}-rocm${{ matrix.rocm-version }} name: wheel-python${{ matrix.python-version }}-rocm${{ matrix.rocm-version }}
path: sgl-kernel/dist/* path: python/sglang/kernels/aot/dist/*
release-rocm700: release-rocm700:
needs: build-rocm-matrix needs: build-rocm-matrix
@@ -323,7 +323,7 @@ jobs:
- name: Download artifacts - name: Download artifacts
uses: actions/download-artifact@v4 uses: actions/download-artifact@v4
with: with:
path: sgl-kernel/dist/ path: python/sglang/kernels/aot/dist/
merge-multiple: true merge-multiple: true
pattern: wheel-*-rocm700 pattern: wheel-*-rocm700
@@ -331,7 +331,7 @@ jobs:
id: set_tag_name id: set_tag_name
run: | run: |
if [ -z "${{ inputs.tag_name }}" ]; then if [ -z "${{ inputs.tag_name }}" ]; then
TAG_NAME="v$(cat sgl-kernel/python/sgl_kernel/version.py | cut -d'"' -f2)" TAG_NAME="v$(cat python/sglang/kernels/aot/python/sgl_kernel/version.py | cut -d'"' -f2)"
echo "tag_name=$TAG_NAME" >> $GITHUB_OUTPUT echo "tag_name=$TAG_NAME" >> $GITHUB_OUTPUT
else else
echo "tag_name=${{ inputs.tag_name }}" >> $GITHUB_OUTPUT echo "tag_name=${{ inputs.tag_name }}" >> $GITHUB_OUTPUT
@@ -344,7 +344,7 @@ jobs:
repository: sgl-project/whl repository: sgl-project/whl
token: ${{ secrets.GH_PAT_FOR_WHL_RELEASE }} token: ${{ secrets.GH_PAT_FOR_WHL_RELEASE }}
files: | files: |
sgl-kernel/dist/* python/sglang/kernels/aot/dist/*
- name: Clone wheel index - name: Clone wheel index
run: git clone https://oauth2:${WHL_TOKEN}@github.com/sgl-project/whl.git sgl-whl run: git clone https://oauth2:${WHL_TOKEN}@github.com/sgl-project/whl.git sgl-whl
@@ -374,7 +374,7 @@ jobs:
- name: Download artifacts - name: Download artifacts
uses: actions/download-artifact@v4 uses: actions/download-artifact@v4
with: with:
path: sgl-kernel/dist/ path: python/sglang/kernels/aot/dist/
merge-multiple: true merge-multiple: true
pattern: wheel-*-rocm720 pattern: wheel-*-rocm720
@@ -382,7 +382,7 @@ jobs:
id: set_tag_name id: set_tag_name
run: | run: |
if [ -z "${{ inputs.tag_name }}" ]; then if [ -z "${{ inputs.tag_name }}" ]; then
TAG_NAME="v$(cat sgl-kernel/python/sgl_kernel/version.py | cut -d'"' -f2)" TAG_NAME="v$(cat python/sglang/kernels/aot/python/sgl_kernel/version.py | cut -d'"' -f2)"
echo "tag_name=$TAG_NAME" >> $GITHUB_OUTPUT echo "tag_name=$TAG_NAME" >> $GITHUB_OUTPUT
else else
echo "tag_name=${{ inputs.tag_name }}" >> $GITHUB_OUTPUT echo "tag_name=${{ inputs.tag_name }}" >> $GITHUB_OUTPUT
@@ -395,7 +395,7 @@ jobs:
repository: sgl-project/whl repository: sgl-project/whl
token: ${{ secrets.GH_PAT_FOR_WHL_RELEASE }} token: ${{ secrets.GH_PAT_FOR_WHL_RELEASE }}
files: | files: |
sgl-kernel/dist/* python/sglang/kernels/aot/dist/*
- name: Clone wheel index - name: Clone wheel index
run: git clone https://oauth2:${WHL_TOKEN}@github.com/sgl-project/whl.git sgl-whl run: git clone https://oauth2:${WHL_TOKEN}@github.com/sgl-project/whl.git sgl-whl
@@ -425,7 +425,7 @@ jobs:
musa-version: ["43"] musa-version: ["43"]
steps: steps:
# Self-hosted build nodes retain the workspace across jobs. Prior builds # Self-hosted build nodes retain the workspace across jobs. Prior builds
# leave root-owned artifacts under sgl-kernel/build/ that actions/checkout # leave root-owned artifacts under python/sglang/kernels/aot/build/ that actions/checkout
# cannot remove, causing EACCES on rmdir. Wipe them via a throwaway root # cannot remove, causing EACCES on rmdir. Wipe them via a throwaway root
# container before checkout recreates the workspace. # container before checkout recreates the workspace.
- name: Clean workspace (remove root-owned files from prior runs) - name: Clean workspace (remove root-owned files from prior runs)
@@ -451,19 +451,19 @@ jobs:
- name: Build wheels - name: Build wheels
run: | run: |
cd sgl-kernel cd python/sglang/kernels/aot
mv pyproject_musa.toml pyproject.toml mv pyproject_musa.toml pyproject.toml
python setup_musa.py sdist bdist_wheel python setup_musa.py sdist bdist_wheel
- name: Rename MUSA wheels - name: Rename MUSA wheels
run: | run: |
bash scripts/ci/musa/rename_wheels_musa.sh ${{ matrix.musa-version }} sgl-kernel/dist bash scripts/ci/musa/rename_wheels_musa.sh ${{ matrix.musa-version }} python/sglang/kernels/aot/dist
- name: Upload artifacts - name: Upload artifacts
uses: actions/upload-artifact@v4 uses: actions/upload-artifact@v4
with: with:
name: wheel-python${{ matrix.python-version }}-musa${{ matrix.musa-version }} name: wheel-python${{ matrix.python-version }}-musa${{ matrix.musa-version }}
path: sgl-kernel/dist/* path: python/sglang/kernels/aot/dist/*
release-musa43: release-musa43:
needs: build-musa43 needs: build-musa43
@@ -474,7 +474,7 @@ jobs:
- name: Download artifacts - name: Download artifacts
uses: actions/download-artifact@v4 uses: actions/download-artifact@v4
with: with:
path: sgl-kernel/dist/ path: python/sglang/kernels/aot/dist/
merge-multiple: true merge-multiple: true
pattern: wheel-* pattern: wheel-*
@@ -482,7 +482,7 @@ jobs:
id: set_tag_name id: set_tag_name
run: | run: |
if [ -z "${{ inputs.tag_name }}" ]; then if [ -z "${{ inputs.tag_name }}" ]; then
TAG_NAME="v$(cat sgl-kernel/python/sgl_kernel/version.py | cut -d'"' -f2)" TAG_NAME="v$(cat python/sglang/kernels/aot/python/sgl_kernel/version.py | cut -d'"' -f2)"
echo "tag_name=$TAG_NAME" >> $GITHUB_OUTPUT echo "tag_name=$TAG_NAME" >> $GITHUB_OUTPUT
else else
echo "tag_name=${{ inputs.tag_name }}" >> $GITHUB_OUTPUT echo "tag_name=${{ inputs.tag_name }}" >> $GITHUB_OUTPUT
@@ -495,7 +495,7 @@ jobs:
repository: sgl-project/whl repository: sgl-project/whl
token: ${{ secrets.GH_PAT_FOR_WHL_RELEASE }} token: ${{ secrets.GH_PAT_FOR_WHL_RELEASE }}
files: | files: |
sgl-kernel/dist/* python/sglang/kernels/aot/dist/*
- name: Clone wheel index - name: Clone wheel index
run: git clone https://oauth2:${WHL_TOKEN}@github.com/sgl-project/whl.git sgl-whl run: git clone https://oauth2:${WHL_TOKEN}@github.com/sgl-project/whl.git sgl-whl
.gitignore
+3 -3
View File
@@ -263,9 +263,9 @@ python/kernel.lock
# MUSA section # MUSA section
# Generated source files by torchada # Generated source files by torchada
sgl-kernel/csrc_musa/ python/sglang/kernels/aot/csrc_musa/
sgl-kernel/include_musa/ python/sglang/kernels/aot/include_musa/
sgl-kernel/csrc/**/*_musa/ python/sglang/kernels/aot/csrc/**/*_musa/
# MUSA core dump files # MUSA core dump files
*.mudmp *.mudmp
.pre-commit-config.yaml
+1
View File
@@ -42,6 +42,7 @@ repos:
(?x)^( (?x)^(
.*/__init__\.py$| .*/__init__\.py$|
.*\.ipynb$| .*\.ipynb$|
python/sglang/kernels/aot/.*|
python/sglang/srt/grpc/.*_pb2\.py$| python/sglang/srt/grpc/.*_pb2\.py$|
python/sglang/srt/grpc/.*_pb2_grpc\.py$| python/sglang/srt/grpc/.*_pb2_grpc\.py$|
python/sglang/srt/grpc/.*_pb2\.pyi$| python/sglang/srt/grpc/.*_pb2\.pyi$|
docker/arm64.Dockerfile
+1 -1
View File
@@ -42,7 +42,7 @@ RUN source $HOME/.local/bin/env && \
cd python && \ cd python && \
cp pyproject_cpu.toml pyproject.toml && \ cp pyproject_cpu.toml pyproject.toml && \
uv pip install . && \ uv pip install . && \
cd ../sgl-kernel && \ cd sglang/kernels/aot && \
cp pyproject_cpu.toml pyproject.toml && \ cp pyproject_cpu.toml pyproject.toml && \
uv pip install . uv pip install .
docker/rocm.Dockerfile
+2 -2
View File
@@ -317,11 +317,11 @@ RUN if [ "$BRANCH_TYPE" = "local" ]; then \
fi \ fi \
&& rm -rf /tmp/local_src \ && rm -rf /tmp/local_src \
&& cd sglang \ && cd sglang \
&& cd sgl-kernel \ && cd python/sglang/kernels/aot \
&& rm -f pyproject.toml \ && rm -f pyproject.toml \
&& mv pyproject_rocm.toml pyproject.toml \ && mv pyproject_rocm.toml pyproject.toml \
&& AMDGPU_TARGET=$GPU_ARCH_LIST python setup_rocm.py install \ && AMDGPU_TARGET=$GPU_ARCH_LIST python setup_rocm.py install \
&& cd .. \ && cd ../../../.. \
&& rm -rf python/pyproject.toml && mv python/pyproject_other.toml python/pyproject.toml \ && rm -rf python/pyproject.toml && mv python/pyproject_other.toml python/pyproject.toml \
&& if [ "$BUILD_TYPE" = "srt" ]; then \ && if [ "$BUILD_TYPE" = "srt" ]; then \
export SETUPTOOLS_SCM_PRETEND_VERSION="${SETUPTOOLS_SCM_PRETEND_VERSION}" && python -m pip --no-cache-dir install -e "python[srt_hip,diffusion_hip]"; \ export SETUPTOOLS_SCM_PRETEND_VERSION="${SETUPTOOLS_SCM_PRETEND_VERSION}" && python -m pip --no-cache-dir install -e "python[srt_hip,diffusion_hip]"; \
docker/xeon.Dockerfile
+1 -1
View File
@@ -40,7 +40,7 @@ RUN source /opt/.venv/bin/activate && \
cd python && \ cd python && \
cp pyproject_cpu.toml pyproject.toml && \ cp pyproject_cpu.toml pyproject.toml && \
uv pip install . && \ uv pip install . && \
cd ../sgl-kernel && \ cd sglang/kernels/aot && \
cp pyproject_cpu.toml pyproject.toml && \ cp pyproject_cpu.toml pyproject.toml && \
uv pip install . uv pip install .
docs_new/docs/developer_guide/contribution_guide.mdx
+1 -1
View File
@@ -164,7 +164,7 @@ Users listed in [CI_PERMISSIONS.json](https://github.com/sgl-project/sglang/blob
## How to update sgl-kernel ## How to update sgl-kernel
Since sglang and the `sglang-kernel` (prior `sgl-kernel`) distribution are separate Python packages, our current GitHub CI infrastructure does not support updating a kernel and using it immediately within the same pull request (PR). Since sglang and the `sglang-kernel` (prior `sgl-kernel`) distribution are separate Python packages, our current GitHub CI infrastructure does not support updating a kernel and using it immediately within the same pull request (PR).
To add a new kernel or modify an existing one in the `sgl-kernel/` source tree, you must use multiple PRs. To add a new kernel or modify an existing one in the `python/sglang/kernels/aot/` source tree, you must use multiple PRs.
Follow these steps: Follow these steps:
docs_new/docs/hardware-platforms/amd_gpu.mdx
+2 -2
View File
@@ -50,11 +50,11 @@ cd sglang
# Compile sgl-kernel # Compile sgl-kernel
pip install --upgrade pip pip install --upgrade pip
cd sgl-kernel cd python/sglang/kernels/aot
python setup_rocm.py install python setup_rocm.py install
# Install sglang python package along with diffusion support # Install sglang python package along with diffusion support
cd .. cd ../../../..
rm -rf python/pyproject.toml && mv python/pyproject_other.toml python/pyproject.toml rm -rf python/pyproject.toml && mv python/pyproject_other.toml python/pyproject.toml
pip install -e "python[all_hip]" pip install -e "python[all_hip]"
``` ```
docs_new/docs/hardware-platforms/apple_metal.mdx
+1 -1
View File
@@ -36,7 +36,7 @@ source sglang-metal/bin/activate
# (Optional) Compile sgl-kernel # (Optional) Compile sgl-kernel
uv pip install --upgrade pip uv pip install --upgrade pip
uv run sgl-kernel/setup_metal.py install uv run python/sglang/kernels/aot/setup_metal.py install
# Install sglang python package along with diffusion support # Install sglang python package along with diffusion support
rm -f python/pyproject.toml && mv python/pyproject_other.toml python/pyproject.toml rm -f python/pyproject.toml && mv python/pyproject_other.toml python/pyproject.toml
docs_new/docs/hardware-platforms/cpu_server.mdx
+1 -1
View File
@@ -126,7 +126,7 @@ uv pip install --upgrade pip setuptools
uv pip install . uv pip install .
# Build the CPU backend kernels # Build the CPU backend kernels
cd ../sgl-kernel cd sglang/kernels/aot
cp pyproject_cpu.toml pyproject.toml cp pyproject_cpu.toml pyproject.toml
uv pip install . uv pip install .
``` ```
docs_new/docs/hardware-platforms/mthreads_gpu.mdx
+2 -2
View File
@@ -19,11 +19,11 @@ cd sglang
# Compile sgl-kernel # Compile sgl-kernel
pip install --upgrade pip pip install --upgrade pip
cd sgl-kernel cd python/sglang/kernels/aot
python setup_musa.py install python setup_musa.py install
# Install sglang python package along with diffusion support # Install sglang python package along with diffusion support
cd .. cd ../../../..
rm -f python/pyproject.toml && mv python/pyproject_other.toml python/pyproject.toml rm -f python/pyproject.toml && mv python/pyproject_other.toml python/pyproject.toml
pip install -e "python[all_musa]" pip install -e "python[all_musa]"
``` ```
python/MANIFEST.in
+1
View File
@@ -0,0 +1 @@
prune sglang/kernels/aot
python/pyproject.toml
+8
View File
@@ -199,6 +199,12 @@ killall_sglang = "sglang.cli.killall:main"
"multimodal_gen/apps/realtime_webui/**/*" "multimodal_gen/apps/realtime_webui/**/*"
] ]
[tool.setuptools.exclude-package-data]
"sglang" = [
"kernels/aot/*",
"kernels/aot/**/*",
]
[tool.setuptools.packages.find] [tool.setuptools.packages.find]
exclude = [ exclude = [
"assets*", "assets*",
@@ -207,6 +213,7 @@ exclude = [
"dist*", "dist*",
"playground*", "playground*",
"scripts*", "scripts*",
"sglang.kernels.aot*",
"tests*", "tests*",
] ]
@@ -218,6 +225,7 @@ exclude = [
"dist*", "dist*",
"playground*", "playground*",
"scripts*", "scripts*",
"sglang/kernels/aot*",
"tests*", "tests*",
] ]
sgl-kernel/.clang-format → python/sglang/kernels/aot/.clang-format
View File
sgl-kernel/CMakeLists.txt → python/sglang/kernels/aot/CMakeLists.txt
View File
sgl-kernel/Dockerfile → python/sglang/kernels/aot/Dockerfile
View File
sgl-kernel/LICENSE → python/sglang/kernels/aot/LICENSE
View File
sgl-kernel/Makefile → python/sglang/kernels/aot/Makefile
View File
sgl-kernel/README.md → python/sglang/kernels/aot/README.md
+9 -9
View File
@@ -1,6 +1,6 @@
# sglang-kernel (prior sgl-kernel) # sglang-kernel (prior sgl-kernel)
[Kernel Library](https://github.com/sgl-project/sglang/tree/main/sgl-kernel) for LLM inference engines [Kernel Library](https://github.com/sgl-project/sglang/tree/main/python/sglang/kernels/aot) for LLM inference engines
<div align="center"> <div align="center">
@@ -9,7 +9,7 @@
</div> </div>
`sglang-kernel` provides optimized compute primitives for LLM inference engines, enabling efficient inference for large language models and vision-language models through custom kernel operations. The source tree remains under the `sgl-kernel/` directory and the Python import path remains `sgl_kernel`. `sglang-kernel` provides optimized compute primitives for LLM inference engines, enabling efficient inference for large language models and vision-language models through custom kernel operations. The source tree lives under the `python/sglang/kernels/aot/` directory and the Python import path remains `sgl_kernel`.
## Installation ## Installation
Requires torch == 2.11.0 Requires torch == 2.11.0
@@ -48,11 +48,11 @@ make build MAX_JOBS=2 CMAKE_ARGS="-DSGL_KERNEL_COMPILE_THREADS=1"
### Steps to add a new kernel: ### Steps to add a new kernel:
1. Implement the kernel in [csrc](https://github.com/sgl-project/sglang/tree/main/sgl-kernel/csrc) 1. Implement the kernel in [csrc](https://github.com/sgl-project/sglang/tree/main/python/sglang/kernels/aot/csrc)
2. Expose the interface in [include/sgl_kernel_ops.h](https://github.com/sgl-project/sglang/blob/main/sgl-kernel/include/sgl_kernel_ops.h) 2. Expose the interface in [include/sgl_kernel_ops.h](https://github.com/sgl-project/sglang/blob/main/python/sglang/kernels/aot/include/sgl_kernel_ops.h)
3. Create torch extension in [csrc/common_extension.cc](https://github.com/sgl-project/sglang/blob/main/sgl-kernel/csrc/common_extension.cc) 3. Create torch extension in [csrc/common_extension.cc](https://github.com/sgl-project/sglang/blob/main/python/sglang/kernels/aot/csrc/common_extension.cc)
4. Update [CMakeLists.txt](https://github.com/sgl-project/sglang/blob/main/sgl-kernel/CMakeLists.txt) to include new CUDA source 4. Update [CMakeLists.txt](https://github.com/sgl-project/sglang/blob/main/python/sglang/kernels/aot/CMakeLists.txt) to include new CUDA source
5. Expose Python interface in [python](https://github.com/sgl-project/sglang/blob/main/sgl-kernel/python/sgl_kernel) 5. Expose Python interface in [python](https://github.com/sgl-project/sglang/blob/main/python/sglang/kernels/aot/python/sgl_kernel)
6. Add test and benchmark 6. Add test and benchmark
### Development Tips ### Development Tips
@@ -95,7 +95,7 @@ m.impl("fwd", torch::kCUDA, make_pytorch_shim(&mha_fwd));
### Testing & Benchmarking ### Testing & Benchmarking
1. Add pytest tests in [tests/](https://github.com/sgl-project/sglang/tree/main/sgl-kernel/tests), if you need to skip some test, please use `@pytest.mark.skipif` 1. Add pytest tests in [tests/](https://github.com/sgl-project/sglang/tree/main/python/sglang/kernels/aot/tests), if you need to skip some test, please use `@pytest.mark.skipif`
```python ```python
@pytest.mark.skipif( @pytest.mark.skipif(
@@ -103,7 +103,7 @@ m.impl("fwd", torch::kCUDA, make_pytorch_shim(&mha_fwd));
) )
``` ```
2. Add benchmarks using [triton benchmark](https://triton-lang.org/main/python-api/generated/triton.testing.Benchmark.html) in [benchmark/](https://github.com/sgl-project/sglang/tree/main/sgl-kernel/benchmark) 2. Add benchmarks using [triton benchmark](https://triton-lang.org/main/python-api/generated/triton.testing.Benchmark.html) in [benchmark/](https://github.com/sgl-project/sglang/tree/main/python/sglang/kernels/aot/benchmark)
**We recommend using `triton.testing.do_bench_cudagraph` for kernel benchmarking**: **We recommend using `triton.testing.do_bench_cudagraph` for kernel benchmarking**:
sgl-kernel/THIRDPARTYNOTICES.txt → python/sglang/kernels/aot/THIRDPARTYNOTICES.txt
View File
sgl-kernel/analyze_whl_kernel_sizes.py → python/sglang/kernels/aot/analyze_whl_kernel_sizes.py
View File
sgl-kernel/benchmark/bench_activation.py → python/sglang/kernels/aot/benchmark/bench_activation.py
View File
sgl-kernel/benchmark/bench_amd_deterministic_allreduce.py → python/sglang/kernels/aot/benchmark/bench_amd_deterministic_allreduce.py
View File
sgl-kernel/benchmark/bench_awq_dequant.py → python/sglang/kernels/aot/benchmark/bench_awq_dequant.py
View File
sgl-kernel/benchmark/bench_cutlass_mla.py → python/sglang/kernels/aot/benchmark/bench_cutlass_mla.py
View File
sgl-kernel/benchmark/bench_dsv4_norm_rope.py → python/sglang/kernels/aot/benchmark/bench_dsv4_norm_rope.py
View File
sgl-kernel/benchmark/bench_es_fp8_blockwise_grouped_gemm.py → python/sglang/kernels/aot/benchmark/bench_es_fp8_blockwise_grouped_gemm.py
View File
sgl-kernel/benchmark/bench_fp4_gemm.py → python/sglang/kernels/aot/benchmark/bench_fp4_gemm.py
View File
sgl-kernel/benchmark/bench_fp8_blockwise_group_gemm.py → python/sglang/kernels/aot/benchmark/bench_fp8_blockwise_group_gemm.py
View File
sgl-kernel/benchmark/bench_fp8_gemm.py → python/sglang/kernels/aot/benchmark/bench_fp8_gemm.py
View File
sgl-kernel/benchmark/bench_fp8_gemm_swap_ab.py → python/sglang/kernels/aot/benchmark/bench_fp8_gemm_swap_ab.py
View File
sgl-kernel/benchmark/bench_int8_gemm.py → python/sglang/kernels/aot/benchmark/bench_int8_gemm.py
View File
sgl-kernel/benchmark/bench_moe_align_block_size.py → python/sglang/kernels/aot/benchmark/bench_moe_align_block_size.py
View File
sgl-kernel/benchmark/bench_moe_ep_post_reorder.py → python/sglang/kernels/aot/benchmark/bench_moe_ep_post_reorder.py
View File
sgl-kernel/benchmark/bench_moe_topk_sigmoid.py → python/sglang/kernels/aot/benchmark/bench_moe_topk_sigmoid.py
View File
sgl-kernel/benchmark/bench_moe_topk_softmax.py → python/sglang/kernels/aot/benchmark/bench_moe_topk_softmax.py
View File
sgl-kernel/benchmark/bench_mrope.py → python/sglang/kernels/aot/benchmark/bench_mrope.py
View File
sgl-kernel/benchmark/bench_per_tensor_quant_fp8.py → python/sglang/kernels/aot/benchmark/bench_per_tensor_quant_fp8.py
View File
sgl-kernel/benchmark/bench_per_token_group_quant_8bit.py → python/sglang/kernels/aot/benchmark/bench_per_token_group_quant_8bit.py
View File
sgl-kernel/benchmark/bench_per_token_quant_fp8.py → python/sglang/kernels/aot/benchmark/bench_per_token_quant_fp8.py
View File
sgl-kernel/benchmark/bench_rmsnorm.py → python/sglang/kernels/aot/benchmark/bench_rmsnorm.py
View File
sgl-kernel/benchmark/bench_rotary_embedding.py → python/sglang/kernels/aot/benchmark/bench_rotary_embedding.py
View File
sgl-kernel/benchmark/bench_sum_scale.py → python/sglang/kernels/aot/benchmark/bench_sum_scale.py
View File
sgl-kernel/benchmark/bench_top_k_top_p_sampling.py → python/sglang/kernels/aot/benchmark/bench_top_k_top_p_sampling.py
View File
sgl-kernel/build.sh → python/sglang/kernels/aot/build.sh
View File
sgl-kernel/cmake/flashmla.cmake → python/sglang/kernels/aot/cmake/flashmla.cmake
View File
sgl-kernel/cmake/utils.cmake → python/sglang/kernels/aot/cmake/utils.cmake
View File
sgl-kernel/csrc/allreduce/custom_all_reduce.cu → python/sglang/kernels/aot/csrc/allreduce/custom_all_reduce.cu
View File
sgl-kernel/csrc/allreduce/custom_all_reduce.cuh → python/sglang/kernels/aot/csrc/allreduce/custom_all_reduce.cuh
View File
sgl-kernel/csrc/allreduce/custom_all_reduce.hip → python/sglang/kernels/aot/csrc/allreduce/custom_all_reduce.hip
View File
sgl-kernel/csrc/allreduce/custom_all_reduce_hip.cuh → python/sglang/kernels/aot/csrc/allreduce/custom_all_reduce_hip.cuh
View File
sgl-kernel/csrc/allreduce/deterministic_all_reduce.hip → python/sglang/kernels/aot/csrc/allreduce/deterministic_all_reduce.hip
View File
sgl-kernel/csrc/allreduce/quick_all_reduce.cu → python/sglang/kernels/aot/csrc/allreduce/quick_all_reduce.cu
View File
sgl-kernel/csrc/allreduce/quick_all_reduce.cuh → python/sglang/kernels/aot/csrc/allreduce/quick_all_reduce.cuh
View File
sgl-kernel/csrc/allreduce/quick_all_reduce.h → python/sglang/kernels/aot/csrc/allreduce/quick_all_reduce.h
View File
sgl-kernel/csrc/allreduce/quick_all_reduce_base.h → python/sglang/kernels/aot/csrc/allreduce/quick_all_reduce_base.h
View File
sgl-kernel/csrc/attention/cutlass_mla_kernel.cu → python/sglang/kernels/aot/csrc/attention/cutlass_mla_kernel.cu
View File
sgl-kernel/csrc/attention/cutlass_sm100_mla/device/sm100_mla.hpp → python/sglang/kernels/aot/csrc/attention/cutlass_sm100_mla/device/sm100_mla.hpp
View File
sgl-kernel/csrc/attention/cutlass_sm100_mla/kernel/sm100_fmha_mla_reduction.hpp → python/sglang/kernels/aot/csrc/attention/cutlass_sm100_mla/kernel/sm100_fmha_mla_reduction.hpp
View File
sgl-kernel/csrc/attention/cutlass_sm100_mla/kernel/sm100_fmha_mla_tma_warpspecialized.hpp → python/sglang/kernels/aot/csrc/attention/cutlass_sm100_mla/kernel/sm100_fmha_mla_tma_warpspecialized.hpp
View File
sgl-kernel/csrc/attention/cutlass_sm100_mla/kernel/sm100_mla_tile_scheduler.hpp → python/sglang/kernels/aot/csrc/attention/cutlass_sm100_mla/kernel/sm100_mla_tile_scheduler.hpp
View File
sgl-kernel/csrc/attention/merge_attn_states.cu → python/sglang/kernels/aot/csrc/attention/merge_attn_states.cu
View File
sgl-kernel/csrc/attention/vertical_slash_index.cu → python/sglang/kernels/aot/csrc/attention/vertical_slash_index.cu
View File
sgl-kernel/csrc/common_extension.cc → python/sglang/kernels/aot/csrc/common_extension.cc
View File
sgl-kernel/csrc/common_extension_musa.cc → python/sglang/kernels/aot/csrc/common_extension_musa.cc
View File
sgl-kernel/csrc/common_extension_rocm.cc → python/sglang/kernels/aot/csrc/common_extension_rocm.cc
View File
sgl-kernel/csrc/cpu/CMakeLists.txt → python/sglang/kernels/aot/csrc/cpu/CMakeLists.txt
View File
sgl-kernel/csrc/cpu/aarch64/gemm_int8.cpp → python/sglang/kernels/aot/csrc/cpu/aarch64/gemm_int8.cpp
View File
sgl-kernel/csrc/cpu/aarch64/moe.cpp → python/sglang/kernels/aot/csrc/cpu/aarch64/moe.cpp
View File
sgl-kernel/csrc/cpu/aarch64/op.h → python/sglang/kernels/aot/csrc/cpu/aarch64/op.h
View File
sgl-kernel/csrc/cpu/aarch64/shm.h → python/sglang/kernels/aot/csrc/cpu/aarch64/shm.h
View File
sgl-kernel/csrc/cpu/activation.cpp → python/sglang/kernels/aot/csrc/cpu/activation.cpp
View File
sgl-kernel/csrc/cpu/bmm.cpp → python/sglang/kernels/aot/csrc/cpu/bmm.cpp
View File
sgl-kernel/csrc/cpu/common.h → python/sglang/kernels/aot/csrc/cpu/common.h
View File
sgl-kernel/csrc/cpu/conv3d.cpp → python/sglang/kernels/aot/csrc/cpu/conv3d.cpp
View File
sgl-kernel/csrc/cpu/decode.cpp → python/sglang/kernels/aot/csrc/cpu/decode.cpp
View File
sgl-kernel/csrc/cpu/extend.cpp → python/sglang/kernels/aot/csrc/cpu/extend.cpp
View File

Some files were not shown because too many files have changed in this diff Show More