[Kernel] Move sgl-kernel under sglang.kernels.aot (#32648)

.claude/skills/add-jit-kernel/SKILL.md

@@ -18,7 +18,7 @@ Add a new operation that scales each element of a tensor by a scalar factor:
 ## When to use JIT vs AOT (`sgl-kernel`)
 
 - **JIT (`jit_kernel`)**: prefer this first for kernels that do **not** depend on CUTLASS or another large C++ project. It is the default choice for lightweight kernels that benefit from rapid iteration and first-use compilation.
-- **AOT (`sgl-kernel`)**: prefer this when the kernel **does** depend on CUTLASS or another large C++ project, or when it should live in `sgl-kernel/` and participate in the wheel build / torch op registration flow.
+- **AOT (`sgl-kernel`)**: prefer this when the kernel **does** depend on CUTLASS or another large C++ project, or when it should live in `python/sglang/kernels/aot/` and participate in the wheel build / torch op registration flow.
 - **Exception**: kernels that depend on `flashinfer`, or on CUTLASS that is already provided through `flashinfer`, can still be implemented as `jit_kernel`.
 
 ---

.claude/skills/add-sgl-kernel/SKILL.md

@@ -14,7 +14,7 @@ Add a new operation that scales each element of a tensor by a scalar factor:
 - Input: tensor `x` (CUDA) and scalar `factor` (float)
 - Output: `x * factor` (element-wise, in-place or into pre-allocated `out`)
 - Supported dtypes: **FP16 (`torch.float16`), BF16 (`torch.bfloat16`), FP32 (`torch.float32`)**
-  - Dispatched via `DISPATCH_PYTORCH_DTYPE_TO_CTYPE_FLOAT_FP16` macro (defined in `sgl-kernel/include/utils.h`)
+  - Dispatched via `DISPATCH_PYTORCH_DTYPE_TO_CTYPE_FLOAT_FP16` macro (defined in `python/sglang/kernels/aot/include/utils.h`)
 
 ## Two rules of thumb (must follow)
 
@@ -33,13 +33,13 @@ In addition, every new kernel must ship with:
 
 You will typically touch these files/areas:
 
-- Implementation: `sgl-kernel/csrc/elementwise/scale.cu` (pick the right subdirectory)
-- Public declarations: `sgl-kernel/include/sgl_kernel_ops.h`
-- Torch extension registration: `sgl-kernel/csrc/common_extension.cc`
-- Build: `sgl-kernel/CMakeLists.txt` (`set(SOURCES ...)`)
-- Python API: `sgl-kernel/python/sgl_kernel/` and `sgl-kernel/python/sgl_kernel/__init__.py`
-- Tests: `sgl-kernel/tests/test_scale.py`
-- Benchmarks: `sgl-kernel/benchmark/bench_scale.py`
+- Implementation: `python/sglang/kernels/aot/csrc/elementwise/scale.cu` (pick the right subdirectory)
+- Public declarations: `python/sglang/kernels/aot/include/sgl_kernel_ops.h`
+- Torch extension registration: `python/sglang/kernels/aot/csrc/common_extension.cc`
+- Build: `python/sglang/kernels/aot/CMakeLists.txt` (`set(SOURCES ...)`)
+- Python API: `python/sglang/kernels/aot/python/sgl_kernel/` and `python/sglang/kernels/aot/python/sgl_kernel/__init__.py`
+- Tests: `python/sglang/kernels/aot/tests/test_scale.py`
+- Benchmarks: `python/sglang/kernels/aot/benchmark/bench_scale.py`
 
 ---
 
@@ -50,7 +50,7 @@ Pick the right subdirectory:
 - `csrc/elementwise/` — for element-wise ops (our example)
 - `csrc/gemm/`, `csrc/attention/`, `csrc/moe/` — for other categories
 
-Create `sgl-kernel/csrc/elementwise/scale.cu`:
+Create `python/sglang/kernels/aot/csrc/elementwise/scale.cu`:
 
 ```cpp
 #include <ATen/cuda/CUDAContext.h>
@@ -115,7 +115,7 @@ void scale(at::Tensor& out, const at::Tensor& input, double factor) {
 
 ## Step 2: Add a C++ declaration in `include/sgl_kernel_ops.h`
 
-Edit `sgl-kernel/include/sgl_kernel_ops.h`, add to the elementwise section:
+Edit `python/sglang/kernels/aot/include/sgl_kernel_ops.h`, add to the elementwise section:
 
 ```cpp
 void scale(at::Tensor& out, const at::Tensor& input, double factor);
@@ -125,7 +125,7 @@ void scale(at::Tensor& out, const at::Tensor& input, double factor);
 
 ## Step 3: Register the op in `csrc/common_extension.cc`
 
-Edit `sgl-kernel/csrc/common_extension.cc`, inside `TORCH_LIBRARY_FRAGMENT(sgl_kernel, m)`:
+Edit `python/sglang/kernels/aot/csrc/common_extension.cc`, inside `TORCH_LIBRARY_FRAGMENT(sgl_kernel, m)`:
 
 ```cpp
 // From csrc/elementwise
@@ -143,7 +143,7 @@ m.impl("scale", torch::kCUDA, &scale);
 
 ## Step 4: Add the new source file to `CMakeLists.txt`
 
-Edit `sgl-kernel/CMakeLists.txt`, add to `set(SOURCES ...)`:
+Edit `python/sglang/kernels/aot/CMakeLists.txt`, add to `set(SOURCES ...)`:
 
 ```cmake
 csrc/elementwise/scale.cu
@@ -156,14 +156,14 @@ csrc/elementwise/scale.cu
 
 ---
 
-## Step 5: Expose a Python API under `sgl-kernel/python/sgl_kernel/`
+## Step 5: Expose a Python API under `python/sglang/kernels/aot/python/sgl_kernel/`
 
 Prefer following the existing module organization first. For elementwise kernels, the usual pattern is:
 
-- implement the Python wrapper in `sgl-kernel/python/sgl_kernel/elementwise.py`
-- then re-export it from `sgl-kernel/python/sgl_kernel/__init__.py`
+- implement the Python wrapper in `python/sglang/kernels/aot/python/sgl_kernel/elementwise.py`
+- then re-export it from `python/sglang/kernels/aot/python/sgl_kernel/__init__.py`
 
-For example, in `sgl-kernel/python/sgl_kernel/elementwise.py`, add:
+For example, in `python/sglang/kernels/aot/python/sgl_kernel/elementwise.py`, add:
 
 ```python
 import torch
@@ -190,13 +190,13 @@ def scale(
     return out
 ```
 
-Then re-export it from `sgl-kernel/python/sgl_kernel/__init__.py` following the existing import style used by other kernels.
+Then re-export it from `python/sglang/kernels/aot/python/sgl_kernel/__init__.py` following the existing import style used by other kernels.
 
 ---
 
 ## Step 6: Write tests (required)
 
-Create `sgl-kernel/tests/test_scale.py`:
+Create `python/sglang/kernels/aot/tests/test_scale.py`:
 ```python
 import pytest
 
@@ -241,7 +241,7 @@ if __name__ == "__main__":
 
 ## Step 7: Add a benchmark (required)
 
-Create `sgl-kernel/benchmark/bench_scale.py`:
+Create `python/sglang/kernels/aot/benchmark/bench_scale.py`:
 
 ```python
 import itertools
@@ -306,14 +306,14 @@ if __name__ == "__main__":
 Build:
 
 ```bash
-cd sgl-kernel
+cd python/sglang/kernels/aot
 make build -j16
 ```
 
 If you need to limit host resource usage:
 
 ```bash
-cd sgl-kernel
+cd python/sglang/kernels/aot
 make build -j1 MAX_JOBS=2 CMAKE_ARGS="-DSGL_KERNEL_COMPILE_THREADS=1"
 ```
 
@@ -324,8 +324,8 @@ make build -j1 MAX_JOBS=2 CMAKE_ARGS="-DSGL_KERNEL_COMPILE_THREADS=1"
 After building successfully, run the test and benchmark:
 
 ```bash
-pytest sgl-kernel/tests/test_scale.py -q
-python sgl-kernel/benchmark/bench_scale.py
+pytest python/sglang/kernels/aot/tests/test_scale.py -q
+python python/sglang/kernels/aot/benchmark/bench_scale.py
 ```
 
 PR CI also runs `pr-test-sgl-kernel.yml`, including the B200 job
@@ -339,29 +339,29 @@ Blackwell coverage signal for AOT `sgl-kernel` changes.
 - **Async CUDA errors**: `CUDA_LAUNCH_BLOCKING=1`
 - **Memory errors**: `compute-sanitizer --tool memcheck python ...`
 - **Build is too slow / OOM**: reduce `MAX_JOBS` and `SGL_KERNEL_COMPILE_THREADS`
-- **Binary bloat**: use `sgl-kernel/analyze_whl_kernel_sizes.py`
+- **Binary bloat**: use `python/sglang/kernels/aot/analyze_whl_kernel_sizes.py`
 - **CMake sources list**: if your `.cu` file is missing from `SOURCES`, the symbol will be undefined at link time
 
 ---
 
 ## References
 
-- `sgl-kernel/README.md`
-- `sgl-kernel/include/sgl_kernel_ops.h`
-- `sgl-kernel/csrc/common_extension.cc`
-- `sgl-kernel/CMakeLists.txt`
-- `sgl-kernel/include/utils.h` — `DISPATCH_PYTORCH_DTYPE_TO_CTYPE_FLOAT_FP16` macro and friends
-- `sgl-kernel/csrc/elementwise/activation.cu` — reference for the FP16/BF16/FP32 dispatch pattern
+- `python/sglang/kernels/aot/README.md`
+- `python/sglang/kernels/aot/include/sgl_kernel_ops.h`
+- `python/sglang/kernels/aot/csrc/common_extension.cc`
+- `python/sglang/kernels/aot/CMakeLists.txt`
+- `python/sglang/kernels/aot/include/utils.h` — `DISPATCH_PYTORCH_DTYPE_TO_CTYPE_FLOAT_FP16` macro and friends
+- `python/sglang/kernels/aot/csrc/elementwise/activation.cu` — reference for the FP16/BF16/FP32 dispatch pattern
 
 ## Summary of Files Created/Modified
 
 ```
-sgl-kernel/csrc/elementwise/scale.cu          # NEW: CUDA kernel + launcher
-sgl-kernel/include/sgl_kernel_ops.h           # MODIFIED: C++ declaration
-sgl-kernel/csrc/common_extension.cc           # MODIFIED: schema + dispatch registration
-sgl-kernel/CMakeLists.txt                     # MODIFIED: add source file (alphabetical)
-sgl-kernel/python/sgl_kernel/elementwise.py   # MODIFIED: Python wrapper
-sgl-kernel/python/sgl_kernel/__init__.py      # MODIFIED: re-export Python API
-sgl-kernel/tests/test_scale.py                # NEW: tests
-sgl-kernel/benchmark/bench_scale.py           # NEW: benchmark
+python/sglang/kernels/aot/csrc/elementwise/scale.cu          # NEW: CUDA kernel + launcher
+python/sglang/kernels/aot/include/sgl_kernel_ops.h           # MODIFIED: C++ declaration
+python/sglang/kernels/aot/csrc/common_extension.cc           # MODIFIED: schema + dispatch registration
+python/sglang/kernels/aot/CMakeLists.txt                     # MODIFIED: add source file (alphabetical)
+python/sglang/kernels/aot/python/sgl_kernel/elementwise.py   # MODIFIED: Python wrapper
+python/sglang/kernels/aot/python/sgl_kernel/__init__.py      # MODIFIED: re-export Python API
+python/sglang/kernels/aot/tests/test_scale.py                # NEW: tests
+python/sglang/kernels/aot/benchmark/bench_scale.py           # NEW: benchmark
 ```

.claude/skills/llm-torch-profiler-analysis/references/fuse-overlap-catalog.md

@@ -145,7 +145,7 @@ Stable entries should be folded into the mainline family rows above.
 | PR `#21491` FlashInfer TRTLLM FP8 MoE with fused shared experts | `num_fused_shared_experts`<br>`trtllm_fp8_block_scale_moe` | `PR #21491`<br>`python/sglang/srt/layers/moe/fused_moe_triton/fused_moe.py`<br>`python/sglang/srt/models/deepseek_v2.py` | FlashInfer TRTLLM FP8 MoE path can fuse shared experts inside the routed MoE kernel | On FP8 TRTLLM MoE discussions, treat fused shared experts as an upstream pattern that already has a concrete PR. |
 | PR `#22005` fused add + RMSNorm + per-token FP8 quant | `fused_add_rmsnorm_per_token_quant`<br>`per_token_quant_fp8` | `PR #22005`<br>`python/sglang/kernels/jit/csrc/elementwise/fused_add_rmsnorm_per_token_quant.cuh`<br>`python/sglang/kernels/jit/fused_add_rmsnorm_per_token_quant.py` | CUDA JIT kernel keeps normed values in registers and emits BF16 + FP8 outputs plus per-token scales | If FP8 online-quant traces show add+norm followed by per-token quant, treat this as an in-flight upstream CUDA fuse family. |
 | PR `#20667` Qwen3.5 fused QK norm + RoPE + KV cache write | `fused_qk_norm_rope_cache_pts_quant_shuffle`<br>`fused_qk_norm_mrope_3d_cache_pts_quant_shuffle`<br>`rotary_dim` | `PR #20667`<br>`python/sglang/srt/models/qwen3_5.py`<br>`python/sglang/srt/models/utils.py` | ROCm / AITER path fuses Q / K RMSNorm, partial or 3D RoPE, and direct KV cache write for Qwen3.5 attention | Treat split QK-norm + RoPE + cache-store on Qwen3.5 as a concrete in-flight upstream family, not a novel idea. |
-| PR `#22392` CUTLASS FP8 GEMM replacing nvjet | `cutlass_scaled_mm`<br>`fp8_scaled_mm`<br>`nvjet`<br>`cudaMemsetAsync` | `PR #22392`<br>`sgl-kernel/python/sgl_kernel/gemm.py`<br>`python/sglang/srt/layers/quantization/fp8_utils.py` | Runtime replacement swaps nvjet FP8 GEMMs for CUTLASS kernels, removing per-launch memset bubbles and extra output-copy kernels | Treat nvjet GEMM + memset bubble ladders as an in-flight SGLang linear-kernel family before calling them novel. |
+| PR `#22392` CUTLASS FP8 GEMM replacing nvjet | `cutlass_scaled_mm`<br>`fp8_scaled_mm`<br>`nvjet`<br>`cudaMemsetAsync` | `PR #22392`<br>`python/sglang/kernels/aot/python/sgl_kernel/gemm.py`<br>`python/sglang/srt/layers/quantization/fp8_utils.py` | Runtime replacement swaps nvjet FP8 GEMMs for CUTLASS kernels, removing per-launch memset bubbles and extra output-copy kernels | Treat nvjet GEMM + memset bubble ladders as an in-flight SGLang linear-kernel family before calling them novel. |
 | PR `#18612` NVFP4 CUTLASS MoE fused SiLU+Mul+quant | `silu_and_mul_scaled_nvfp4`<br>`nvfp4 expert quant`<br>`cutlass moe` | `PR #18612`<br>`python/sglang/srt/layers/moe/cutlass_w4a8_moe.py`<br>`python/sglang/kernels/ops/quantization/nvfp4_gemm_swiglu_nvfp4_quant.py` | Fuses MoE activation epilogue and NVFP4 expert quantization before the CUTLASS MoE second GEMM | Treat split SiLU+Mul then NVFP4 expert quant in CUTLASS MoE traces as an in-flight upstream SGLang family. |
 | PR `#22918` FlashInfer per-token NVFP4 MoE | `per_token_nvfp4`<br>`trtllm_fp4_block_scale_moe`<br>`FlashInfer MoE` | `PR #22918`<br>`python/sglang/srt/layers/moe/fused_moe_triton/fused_moe.py` | Adds FlashInfer-backed per-token NVFP4 MoE execution so expert quant/dequant work can move into the fused MoE backend | Treat standalone per-token NVFP4 MoE support kernels as a candidate missing backend-selection path, not an automatically novel kernel idea. |
 | PR `#22851` NSA top-k backend and FlashInfer / PyTorch top-k split | `nsa topk`<br>`flashinfer_topk`<br>`pytorch_topk`<br>`fast_topk_transform` | `PR #22851`<br>`python/sglang/srt/layers/attention/nsa_backend.py` | Makes NSA top-k backend selection explicit and aligns fused top-k transform with FlashInfer / PyTorch fallbacks | When NSA top-k dominates decode, first classify it as backend selection or fused-transform eligibility work. |

.claude/skills/llm-torch-profiler-analysis/scripts/triage_kernel_helpers.py

@@ -840,7 +840,7 @@ FUSION_PATTERN_REGISTRY: Tuple[FusionPatternSpec, ...] = (
         pattern="PR #22392 CUTLASS FP8 scaled MM replacing nvjet",
         candidate_path=(
             "PR #22392"
-            "<br>sgl-kernel/python/sgl_kernel/gemm.py"
+            "<br>python/sglang/kernels/aot/python/sgl_kernel/gemm.py"
             "<br>python/sglang/srt/layers/quantization/fp8_utils.py"
         ),
         active_keywords=("cutlass_scaled_mm", "fp8_scaled_mm"),
@@ -2632,7 +2632,7 @@ def fusion_framework_hints(spec: FusionPatternSpec) -> set[str]:
         hints.add("tokenspeed")
     if "tensorrt_llm/" in text:
         hints.add("trtllm")
-    if any(token in text for token in ("python/sglang/", "sgl-kernel/", "sgl_kernel/")):
+    if any(token in text for token in ("python/sglang/", "sgl_kernel/")):
         hints.add("sglang")
     return hints
 

.github/CODEOWNERS

@@ -67,8 +67,8 @@
 /python/sglang/srt/speculative @Ying1123 @merrymercy @hnyls2002 @Qiaolin-Yu
 /python/sglang/srt/utils/hf_transformers @JustinTong0323
 /python/sglang/srt/weight_cache @liusy58 @QiuMike @alexnails
-/sgl-kernel @ispobock @BBuf @yizhang2077 @merrymercy @FlamingoPg @HaiShaw
-/sgl-kernel/csrc/musa @yeahdongcn
+/python/sglang/kernels/aot @ispobock @BBuf @yizhang2077 @merrymercy @FlamingoPg @HaiShaw
+/python/sglang/kernels/aot/csrc/musa @yeahdongcn
 /sgl-model-gateway @slin1237 @CatherineSue
 /sgl-model-gateway/benches @slin1237
 /sgl-model-gateway/bindings/python @CatherineSue @key4ng @slin1237

.github/labeler.yml

@@ -9,12 +9,14 @@ model-gateway:
 # Kernel specific
 sgl-kernel:
   - changed-files:
-    - any-glob-to-any-file: 'sgl-kernel/**/*'
+    - any-glob-to-any-file: 'python/sglang/kernels/aot/**/*'
 
 # JIT kernel specific
 jit-kernel:
   - changed-files:
-    - any-glob-to-any-file: 'python/sglang/kernels/**/*'
+    - any-glob-to-any-file:
+      - 'python/sglang/kernels/!(*.md)'
+      - 'python/sglang/kernels/!(aot)/**/*'
 
 # Documentation
 documentation:
@@ -90,7 +92,7 @@ blackwell:
   - changed-files:
     - any-glob-to-any-file:
       - '**/*nvfp4*'
-      - 'sgl-kernel/csrc/attention/cutlass_sm100_mla/**/*'
+      - 'python/sglang/kernels/aot/csrc/attention/cutlass_sm100_mla/**/*'
       - 'python/sglang/srt/layers/attention/trtllm_mla_backend.py'
       - 'python/sglang/srt/layers/attention/trtllm_mha_backend.py'
 

.github/workflows/_pr-test-check-changes.yml

@@ -82,7 +82,9 @@ jobs:
               - ".github/workflows/pr-gate.yml"
               - ".github/actions/**"
               - "python/pyproject.toml"
-              - "python/sglang/!(multimodal_gen)/**/!(*.md)"
+              - "python/sglang/!(multimodal_gen|kernels)/**/!(*.md)"
+              - "python/sglang/kernels/!(*.md)"
+              - "python/sglang/kernels/!(aot)/**/!(*.md)"
               - "scripts/ci/cuda/*"
               - "scripts/ci/utils/*"
               - "test/**/!(*.md)"
@@ -104,11 +106,12 @@ jobs:
               - "test/registered/kernels/**"
               # sglang.kernels is the migrated kernel namespace (RFC #29630 / #30044); the
               # base-b-kernel suites import it directly, so kernel edits must run them.
-              - "python/sglang/kernels/**"
+              - "python/sglang/kernels/!(*.md)"
+              - "python/sglang/kernels/!(aot)/**"
             sgl_kernel:
               # Intentionally excludes ".github/workflows/pr-test-sgl-kernel.yml" —
               # see API-side detector below for rationale.
-              - "sgl-kernel/**/!(*.md|THIRDPARTYNOTICES.txt|LICENSE)"
+              - "python/sglang/kernels/aot/**/!(*.md|THIRDPARTYNOTICES.txt|LICENSE)"
 
       - name: Determine full-parallel mode
         id: parallel-mode

.github/workflows/_pr-test-sgl-kernel-build.yml

@@ -100,19 +100,19 @@ jobs:
 
       - name: Build wheel for Python ${{ matrix.python-version }} and CUDA ${{ matrix.cuda-version }}
         run: |
-          cd sgl-kernel
+          cd python/sglang/kernels/aot
           ./build.sh "${{ matrix.python-version }}" "${{ matrix.cuda-version }}"
         env:
           USE_CCACHE: 1
 
       - name: Verify wheel artifacts
         run: |
-          ls -alh sgl-kernel/dist
-          ls -alh sgl-kernel/dist/*.whl
+          ls -alh python/sglang/kernels/aot/dist
+          ls -alh python/sglang/kernels/aot/dist/*.whl
 
       - name: Upload artifacts
         uses: actions/upload-artifact@v4
         with:
           name: wheel-python${{ matrix.python-version }}-cuda${{ matrix.cuda-version }}${{ inputs.arch_suffix }}
-          path: sgl-kernel/dist/*
+          path: python/sglang/kernels/aot/dist/*
           if-no-files-found: error

.github/workflows/_pr-test-stage.yml

@@ -108,7 +108,7 @@ jobs:
         if: ${{ fromJson(inputs.check_changes).sgl_kernel == 'true' && steps.rc.outputs.artifact_version == 'v4' }}
         uses: actions/download-artifact@v4
         with:
-          path: sgl-kernel/dist/
+          path: python/sglang/kernels/aot/dist/
           merge-multiple: true
           pattern: wheel-python3.10-cuda*
 
@@ -116,7 +116,7 @@ jobs:
         if: ${{ fromJson(inputs.check_changes).sgl_kernel == 'true' && steps.rc.outputs.artifact_version == 'v6' }}
         uses: actions/download-artifact@v6
         with:
-          path: sgl-kernel/dist/
+          path: python/sglang/kernels/aot/dist/
           merge-multiple: true
           pattern: wheel-python3.10-cuda*
 

.github/workflows/diffusion-ci-gt-gen.yml

@@ -186,17 +186,17 @@ jobs:
             diffusion-ci/consistency_gt/official_generated/case_map.json
           sparse-checkout-cone-mode: false
 
-      - name: Prepare sgl-kernel/dist for prebuilt wheel
+      - name: Prepare AOT dist for prebuilt wheel
         if: inputs.kernel_artifact_run_id != ''
         run: |
-          ls -alh sgl-kernel/dist || true
-          rm -rf sgl-kernel/dist/* || true
+          ls -alh python/sglang/kernels/aot/dist || true
+          rm -rf python/sglang/kernels/aot/dist/* || true
 
       - name: Download prebuilt sgl-kernel wheel
         if: inputs.kernel_artifact_run_id != ''
         uses: actions/download-artifact@v4
         with:
-          path: sgl-kernel/dist/
+          path: python/sglang/kernels/aot/dist/
           merge-multiple: true
           name: wheel-python3.10-cuda13.0
           run-id: ${{ inputs.kernel_artifact_run_id }}
@@ -382,17 +382,17 @@ jobs:
         with:
           ref: ${{ inputs.ref || github.ref }}
 
-      - name: Prepare sgl-kernel/dist for prebuilt wheel
+      - name: Prepare AOT dist for prebuilt wheel
         if: inputs.kernel_artifact_run_id != ''
         run: |
-          ls -alh sgl-kernel/dist || true
-          rm -rf sgl-kernel/dist/* || true
+          ls -alh python/sglang/kernels/aot/dist || true
+          rm -rf python/sglang/kernels/aot/dist/* || true
 
       - name: Download prebuilt sgl-kernel wheel
         if: inputs.kernel_artifact_run_id != ''
         uses: actions/download-artifact@v4
         with:
-          path: sgl-kernel/dist/
+          path: python/sglang/kernels/aot/dist/
           merge-multiple: true
           name: wheel-python3.10-cuda13.0
           run-id: ${{ inputs.kernel_artifact_run_id }}
@@ -457,17 +457,17 @@ jobs:
         with:
           ref: ${{ inputs.ref || github.ref }}
 
-      - name: Prepare sgl-kernel/dist for prebuilt wheel
+      - name: Prepare AOT dist for prebuilt wheel
         if: inputs.kernel_artifact_run_id != ''
         run: |
-          ls -alh sgl-kernel/dist || true
-          rm -rf sgl-kernel/dist/* || true
+          ls -alh python/sglang/kernels/aot/dist || true
+          rm -rf python/sglang/kernels/aot/dist/* || true
 
       - name: Download prebuilt sgl-kernel wheel
         if: inputs.kernel_artifact_run_id != ''
         uses: actions/download-artifact@v4
         with:
-          path: sgl-kernel/dist/
+          path: python/sglang/kernels/aot/dist/
           merge-multiple: true
           name: wheel-python3.10-cuda13.0
           run-id: ${{ inputs.kernel_artifact_run_id }}
@@ -532,17 +532,17 @@ jobs:
         with:
           ref: ${{ inputs.ref || github.ref }}
 
-      - name: Prepare sgl-kernel/dist for prebuilt wheel
+      - name: Prepare AOT dist for prebuilt wheel
         if: inputs.kernel_artifact_run_id != ''
         run: |
-          ls -alh sgl-kernel/dist || true
-          rm -rf sgl-kernel/dist/* || true
+          ls -alh python/sglang/kernels/aot/dist || true
+          rm -rf python/sglang/kernels/aot/dist/* || true
 
       - name: Download prebuilt sgl-kernel wheel
         if: inputs.kernel_artifact_run_id != ''
         uses: actions/download-artifact@v4
         with:
-          path: sgl-kernel/dist/
+          path: python/sglang/kernels/aot/dist/
           merge-multiple: true
           name: wheel-python3.10-cuda13.0
           run-id: ${{ inputs.kernel_artifact_run_id }}

.github/workflows/lint.yml

@@ -59,7 +59,7 @@ jobs:
       - name: Run sgl-kernel clang-format checks
         uses: DoozyX/clang-format-lint-action@v0.20
         with:
-          source: sgl-kernel
+          source: python/sglang/kernels/aot
           extensions: h,c,cpp,hpp,cu,cuh,cc
           clangFormatVersion: 20
           style: file

.github/workflows/nightly-72-gpu-gb200.yml

@@ -132,7 +132,7 @@ jobs:
       image_tag: ${{ steps.build.outputs.image_tag }}
     steps:
       # Self-hosted runners retain the workspace across jobs. Prior `docker buildx`
-      # runs on this node leave root-owned build artifacts (e.g. sgl-kernel/build/)
+      # runs on this node leave root-owned build artifacts (e.g. python/sglang/kernels/aot/build/)
       # that actions/checkout cannot remove, causing EACCES on rmdir. Wipe them
       # via a throwaway root container before checkout recreates the workspace.
       - name: Clean workspace (remove root-owned files from prior runs)

.github/workflows/nightly-test-musa.yml

@@ -74,11 +74,11 @@ jobs:
         if: steps.gate.outputs.run_job == 'true'
         timeout-minutes: 30
         run: |
-          pytest sgl-kernel/tests/test_per_token_quant_fp8.py
-          pytest sgl-kernel/tests/speculative/test_eagle_utils.py
-          pytest sgl-kernel/tests/speculative/test_ngram_utils.py
-          pytest sgl-kernel/tests/speculative/test_speculative_sampling.py
-          pytest sgl-kernel/tests/test_torch_defaults_reset.py
+          pytest python/sglang/kernels/aot/tests/test_per_token_quant_fp8.py
+          pytest python/sglang/kernels/aot/tests/speculative/test_eagle_utils.py
+          pytest python/sglang/kernels/aot/tests/speculative/test_ngram_utils.py
+          pytest python/sglang/kernels/aot/tests/speculative/test_speculative_sampling.py
+          pytest python/sglang/kernels/aot/tests/test_torch_defaults_reset.py
 
   # ==================== General: multimodal layer ====================
   nightly-test-musa-general-multimodal-layer:

.github/workflows/pr-test-amd-rocm720.yml

@@ -15,7 +15,7 @@ on:
   #     - "python/**"
   #     - "scripts/ci/**"
   #     - "test/**"
-  #     - "sgl-kernel/**"
+  #     - "python/sglang/kernels/aot/**"
   #     - ".github/workflows/pr-test-amd-rocm720.yml"
   #     - "docker/rocm.Dockerfile"
   # pull_request:
@@ -23,7 +23,7 @@ on:
   #     - "python/**"
   #     - "scripts/ci/**"
   #     - "test/**"
-  #     - "sgl-kernel/**"
+  #     - "python/sglang/kernels/aot/**"
   #     - ".github/workflows/pr-test-amd-rocm720.yml"
   #     - "docker/rocm.Dockerfile"
   workflow_dispatch:
@@ -187,7 +187,9 @@ jobs:
         with:
           filters: |
             main_package:
-              - "python/sglang/!(multimodal_gen)/**/!(*.md)"
+              - "python/sglang/!(multimodal_gen|kernels)/**/!(*.md)"
+              - "python/sglang/kernels/!(*.md)"
+              - "python/sglang/kernels/!(aot)/**/!(*.md)"
               - "python/pyproject_rocm.toml"
               - "python/pyproject_other.toml"
               - "scripts/ci/amd/*"
@@ -195,10 +197,11 @@ jobs:
               - "test/**/!(*.md)"
               - ".github/workflows/pr-test-amd-rocm720.yml"
             sgl_kernel:
-              - "sgl-kernel/**/!(*.md|THIRDPARTYNOTICES.txt|LICENSE)"
+              - "python/sglang/kernels/aot/**/!(*.md|THIRDPARTYNOTICES.txt|LICENSE)"
               - ".github/workflows/pr-test-amd-rocm720.yml"
             jit_kernel:
-              - "python/sglang/kernels/**"
+              - "python/sglang/kernels/!(*.md)"
+              - "python/sglang/kernels/!(aot)/**"
               - "test/registered/kernels/**"
               - ".github/workflows/pr-test-amd-rocm720.yml"
             multimodal_gen:
@@ -268,15 +271,15 @@ jobs:
       - name: Run test
         timeout-minutes: 30
         run: |
-          docker exec -w /sglang-checkout/sgl-kernel/tests ci_sglang python3 -m pytest test_moe_align.py
-          docker exec -w /sglang-checkout/sgl-kernel/tests ci_sglang python3 -m pytest test_moe_topk_softmax.py
-          docker exec -w /sglang-checkout/sgl-kernel/tests/speculative ci_sglang python3 -m pytest test_eagle_utils.py
-          docker exec -w /sglang-checkout/sgl-kernel/tests ci_sglang python3 -m pytest test_apply_token_bitmask_inplace.py
-          docker exec -w /sglang-checkout/sgl-kernel/tests ci_sglang python3 -m pytest test_activation.py
-          docker exec -w /sglang-checkout/sgl-kernel/tests ci_sglang python3 -m pytest test_topk.py
-          docker exec -w /sglang-checkout/sgl-kernel/tests ci_sglang python3 -m pytest test_kvcacheio.py
-          docker exec -w /sglang-checkout/sgl-kernel/tests ci_sglang python3 -m pytest test_moe_topk_sigmoid.py
-          docker exec -w /sglang-checkout/sgl-kernel/tests ci_sglang python3 -m pytest test_torch_defaults_reset.py
+          docker exec -w /sglang-checkout/python/sglang/kernels/aot/tests ci_sglang python3 -m pytest test_moe_align.py
+          docker exec -w /sglang-checkout/python/sglang/kernels/aot/tests ci_sglang python3 -m pytest test_moe_topk_softmax.py
+          docker exec -w /sglang-checkout/python/sglang/kernels/aot/tests/speculative ci_sglang python3 -m pytest test_eagle_utils.py
+          docker exec -w /sglang-checkout/python/sglang/kernels/aot/tests ci_sglang python3 -m pytest test_apply_token_bitmask_inplace.py
+          docker exec -w /sglang-checkout/python/sglang/kernels/aot/tests ci_sglang python3 -m pytest test_activation.py
+          docker exec -w /sglang-checkout/python/sglang/kernels/aot/tests ci_sglang python3 -m pytest test_topk.py
+          docker exec -w /sglang-checkout/python/sglang/kernels/aot/tests ci_sglang python3 -m pytest test_kvcacheio.py
+          docker exec -w /sglang-checkout/python/sglang/kernels/aot/tests ci_sglang python3 -m pytest test_moe_topk_sigmoid.py
+          docker exec -w /sglang-checkout/python/sglang/kernels/aot/tests ci_sglang python3 -m pytest test_torch_defaults_reset.py
 
   sgl-kernel-unit-test-2-gpu-amd-rocm720:
     needs: [check-changes]
@@ -657,7 +660,7 @@ jobs:
         if: needs.check-changes.outputs.sgl_kernel == 'true'
         uses: actions/download-artifact@v4
         with:
-          path: sgl-kernel/dist/
+          path: python/sglang/kernels/aot/dist/
           merge-multiple: true
           pattern: wheel-python3.10-cuda12.9
 
@@ -797,7 +800,7 @@ jobs:
         if: needs.check-changes.outputs.sgl_kernel == 'true'
         uses: actions/download-artifact@v4
         with:
-          path: sgl-kernel/dist/
+          path: python/sglang/kernels/aot/dist/
           merge-multiple: true
           pattern: wheel-python3.10-cuda12.9
 
@@ -934,7 +937,7 @@ jobs:
         if: needs.check-changes.outputs.sgl_kernel == 'true'
         uses: actions/download-artifact@v4
         with:
-          path: sgl-kernel/dist/
+          path: python/sglang/kernels/aot/dist/
           merge-multiple: true
           pattern: wheel-python3.10-cuda12.9
 

.github/workflows/pr-test-amd.yml

@@ -11,7 +11,7 @@ on:
       - "python/**"
       - "scripts/ci/**"
       - "test/**"
-      - "sgl-kernel/**"
+      - "python/sglang/kernels/aot/**"
       - ".github/workflows/pr-test-amd.yml"
       - "docker/rocm.Dockerfile"
   workflow_dispatch:
@@ -175,7 +175,9 @@ jobs:
         with:
           filters: |
             main_package:
-              - "python/sglang/!(multimodal_gen)/**/!(*.md)"
+              - "python/sglang/!(multimodal_gen|kernels)/**/!(*.md)"
+              - "python/sglang/kernels/!(*.md)"
+              - "python/sglang/kernels/!(aot)/**/!(*.md)"
               - "python/pyproject_rocm.toml"
               - "python/pyproject_other.toml"
               - "scripts/ci/amd/*"
@@ -183,10 +185,11 @@ jobs:
               - "test/**/!(*.md)"
               - ".github/workflows/pr-test-amd.yml"
             sgl_kernel:
-              - "sgl-kernel/**/!(*.md|THIRDPARTYNOTICES.txt|LICENSE)"
+              - "python/sglang/kernels/aot/**/!(*.md|THIRDPARTYNOTICES.txt|LICENSE)"
               - ".github/workflows/pr-test-amd.yml"
             jit_kernel:
-              - "python/sglang/kernels/**"
+              - "python/sglang/kernels/!(*.md)"
+              - "python/sglang/kernels/!(aot)/**"
               - "test/registered/kernels/**"
               - ".github/workflows/pr-test-amd.yml"
             multimodal_gen:
@@ -266,15 +269,15 @@ jobs:
               "$@"
             fi
           }
-          run_pytest docker exec -w /sglang-checkout/sgl-kernel/tests ci_sglang python3 -m pytest test_moe_align.py
-          run_pytest docker exec -w /sglang-checkout/sgl-kernel/tests ci_sglang python3 -m pytest test_moe_topk_softmax.py
-          run_pytest docker exec -w /sglang-checkout/sgl-kernel/tests/speculative ci_sglang python3 -m pytest test_eagle_utils.py
-          run_pytest docker exec -w /sglang-checkout/sgl-kernel/tests ci_sglang python3 -m pytest test_apply_token_bitmask_inplace.py
-          run_pytest docker exec -w /sglang-checkout/sgl-kernel/tests ci_sglang python3 -m pytest test_activation.py
-          run_pytest docker exec -w /sglang-checkout/sgl-kernel/tests ci_sglang python3 -m pytest test_topk.py
-          run_pytest docker exec -w /sglang-checkout/sgl-kernel/tests ci_sglang python3 -m pytest test_kvcacheio.py
-          run_pytest docker exec -w /sglang-checkout/sgl-kernel/tests ci_sglang python3 -m pytest test_moe_topk_sigmoid.py
-          run_pytest docker exec -w /sglang-checkout/sgl-kernel/tests ci_sglang python3 -m pytest test_torch_defaults_reset.py
+          run_pytest docker exec -w /sglang-checkout/python/sglang/kernels/aot/tests ci_sglang python3 -m pytest test_moe_align.py
+          run_pytest docker exec -w /sglang-checkout/python/sglang/kernels/aot/tests ci_sglang python3 -m pytest test_moe_topk_softmax.py
+          run_pytest docker exec -w /sglang-checkout/python/sglang/kernels/aot/tests/speculative ci_sglang python3 -m pytest test_eagle_utils.py
+          run_pytest docker exec -w /sglang-checkout/python/sglang/kernels/aot/tests ci_sglang python3 -m pytest test_apply_token_bitmask_inplace.py
+          run_pytest docker exec -w /sglang-checkout/python/sglang/kernels/aot/tests ci_sglang python3 -m pytest test_activation.py
+          run_pytest docker exec -w /sglang-checkout/python/sglang/kernels/aot/tests ci_sglang python3 -m pytest test_topk.py
+          run_pytest docker exec -w /sglang-checkout/python/sglang/kernels/aot/tests ci_sglang python3 -m pytest test_kvcacheio.py
+          run_pytest docker exec -w /sglang-checkout/python/sglang/kernels/aot/tests ci_sglang python3 -m pytest test_moe_topk_sigmoid.py
+          run_pytest docker exec -w /sglang-checkout/python/sglang/kernels/aot/tests ci_sglang python3 -m pytest test_torch_defaults_reset.py
           exit $failures
 
   sgl-kernel-unit-test-2-gpu-amd:
@@ -681,7 +684,7 @@ jobs:
         if: needs.check-changes.outputs.sgl_kernel == 'true'
         uses: actions/download-artifact@v4
         with:
-          path: sgl-kernel/dist/
+          path: python/sglang/kernels/aot/dist/
           merge-multiple: true
           pattern: wheel-python3.10-cuda12.9
 
@@ -821,7 +824,7 @@ jobs:
         if: needs.check-changes.outputs.sgl_kernel == 'true'
         uses: actions/download-artifact@v4
         with:
-          path: sgl-kernel/dist/
+          path: python/sglang/kernels/aot/dist/
           merge-multiple: true
           pattern: wheel-python3.10-cuda12.9
 
@@ -960,7 +963,7 @@ jobs:
         if: needs.check-changes.outputs.sgl_kernel == 'true'
         uses: actions/download-artifact@v4
         with:
-          path: sgl-kernel/dist/
+          path: python/sglang/kernels/aot/dist/
           merge-multiple: true
           pattern: wheel-python3.10-cuda12.9
 

.github/workflows/pr-test-arm64.yml

@@ -55,7 +55,7 @@ jobs:
               - "python/sglang/!(multimodal_gen)/**/!(*.md)"
               - "python/pyproject_cpu.toml"
               - "test/**/!(*.md)"
-              - "sgl-kernel/**/*.!(md|txt)"
+              - "python/sglang/kernels/aot/**/*.!(md|txt)"
               - ".github/workflows/pr-test-arm64.yml"
               - "docker/arm64.Dockerfile"
 

.github/workflows/pr-test-jit-kernel.yml

@@ -56,14 +56,14 @@ jobs:
       - name: Cleanup
         if: inputs.sgl_kernel == 'true'
         run: |
-          ls -alh sgl-kernel/dist || true
-          rm -rf sgl-kernel/dist/* || true
+          ls -alh python/sglang/kernels/aot/dist || true
+          rm -rf python/sglang/kernels/aot/dist/* || true
 
       - name: Download artifacts
         if: inputs.sgl_kernel == 'true'
         uses: actions/download-artifact@v4
         with:
-          path: sgl-kernel/dist/
+          path: python/sglang/kernels/aot/dist/
           merge-multiple: true
           pattern: wheel-python3.10-cuda13.0
 
@@ -94,14 +94,14 @@ jobs:
       - name: Cleanup
         if: inputs.sgl_kernel == 'true'
         run: |
-          ls -alh sgl-kernel/dist || true
-          rm -rf sgl-kernel/dist/* || true
+          ls -alh python/sglang/kernels/aot/dist || true
+          rm -rf python/sglang/kernels/aot/dist/* || true
 
       - name: Download artifacts
         if: inputs.sgl_kernel == 'true'
         uses: actions/download-artifact@v4
         with:
-          path: sgl-kernel/dist/
+          path: python/sglang/kernels/aot/dist/
           merge-multiple: true
           pattern: wheel-python3.10-cuda13.0
 
@@ -134,14 +134,14 @@ jobs:
       - name: Cleanup
         if: inputs.sgl_kernel == 'true'
         run: |
-          ls -alh sgl-kernel/dist || true
-          rm -rf sgl-kernel/dist/* || true
+          ls -alh python/sglang/kernels/aot/dist || true
+          rm -rf python/sglang/kernels/aot/dist/* || true
 
       - name: Download artifacts
         if: inputs.sgl_kernel == 'true'
         uses: actions/download-artifact@v4
         with:
-          path: sgl-kernel/dist/
+          path: python/sglang/kernels/aot/dist/
           merge-multiple: true
           pattern: wheel-python3.10-cuda13.0
 
@@ -174,14 +174,14 @@ jobs:
       - name: Cleanup
         if: inputs.sgl_kernel == 'true'
         run: |
-          ls -alh sgl-kernel/dist || true
-          rm -rf sgl-kernel/dist/* || true
+          ls -alh python/sglang/kernels/aot/dist || true
+          rm -rf python/sglang/kernels/aot/dist/* || true
 
       - name: Download artifacts
         if: inputs.sgl_kernel == 'true'
         uses: actions/download-artifact@v4
         with:
-          path: sgl-kernel/dist/
+          path: python/sglang/kernels/aot/dist/
           merge-multiple: true
           pattern: wheel-python3.10-cuda13.0
 

.github/workflows/pr-test-mlx.yml

@@ -43,12 +43,14 @@ jobs:
           base: main
           filters: |
             main_package:
-              - "python/sglang/!(multimodal_gen)/**/!(*.md)"
+              - "python/sglang/!(multimodal_gen|kernels)/**/!(*.md)"
+              - "python/sglang/kernels/!(*.md)"
+              - "python/sglang/kernels/!(aot)/**/!(*.md)"
               - "python/pyproject_other.toml"
               - "test/**/!(*.md)"
               - ".github/workflows/pr-test-mlx.yml"
             sgl_kernel:
-              - "sgl-kernel/**/!(*.md|THIRDPARTYNOTICES.txt|LICENSE)"
+              - "python/sglang/kernels/aot/**/!(*.md|THIRDPARTYNOTICES.txt|LICENSE)"
               - ".github/workflows/pr-test-mlx.yml"
 
   # ==================== PR Gate ==================== #

.github/workflows/pr-test-multimodal-gen.yml

@@ -98,7 +98,7 @@ jobs:
         if: inputs.sgl_kernel == 'true'
         uses: actions/download-artifact@v4
         with:
-          path: sgl-kernel/dist/
+          path: python/sglang/kernels/aot/dist/
           merge-multiple: true
           pattern: wheel-python3.10-cuda*
 
@@ -164,7 +164,7 @@ jobs:
         if: inputs.sgl_kernel == 'true'
         uses: actions/download-artifact@v4
         with:
-          path: sgl-kernel/dist/
+          path: python/sglang/kernels/aot/dist/
           merge-multiple: true
           pattern: wheel-python3.10-cuda*
 
@@ -227,7 +227,7 @@ jobs:
         if: inputs.sgl_kernel == 'true'
         uses: actions/download-artifact@v4
         with:
-          path: sgl-kernel/dist/
+          path: python/sglang/kernels/aot/dist/
           merge-multiple: true
           pattern: wheel-python3.10-cuda*
 
@@ -289,7 +289,7 @@ jobs:
         if: inputs.sgl_kernel == 'true'
         uses: actions/download-artifact@v4
         with:
-          path: sgl-kernel/dist/
+          path: python/sglang/kernels/aot/dist/
           merge-multiple: true
           pattern: wheel-python3.10-cuda*
 
@@ -356,7 +356,7 @@ jobs:
         if: inputs.sgl_kernel == 'true'
         uses: actions/download-artifact@v4
         with:
-          path: sgl-kernel/dist/
+          path: python/sglang/kernels/aot/dist/
           merge-multiple: true
           pattern: wheel-python3.10-cuda*
 
@@ -401,7 +401,7 @@ jobs:
         if: inputs.sgl_kernel == 'true'
         uses: actions/download-artifact@v4
         with:
-          path: sgl-kernel/dist/
+          path: python/sglang/kernels/aot/dist/
           merge-multiple: true
           pattern: wheel-python3.10-cuda*
 
@@ -455,7 +455,7 @@ jobs:
         if: inputs.sgl_kernel == 'true'
         uses: actions/download-artifact@v4
         with:
-          path: sgl-kernel/dist/
+          path: python/sglang/kernels/aot/dist/
           merge-multiple: true
           pattern: wheel-python3.10-cuda*
 

.github/workflows/pr-test-musa.yml

@@ -83,7 +83,7 @@ jobs:
               - "python/sglang/multimodal_gen/test/server/musa/**"
             sgl_kernel:
               - ".github/workflows/pr-test-musa.yml"
-              - "sgl-kernel/csrc/musa/**"
+              - "python/sglang/kernels/aot/csrc/musa/**"
 
   # ==================== PR Gate ==================== #
   pr-gate:
@@ -217,11 +217,11 @@ jobs:
       - name: Run sgl-kernel test
         timeout-minutes: 20
         run: |
-          pytest sgl-kernel/tests/test_per_token_quant_fp8.py
-          pytest sgl-kernel/tests/speculative/test_eagle_utils.py
-          pytest sgl-kernel/tests/speculative/test_ngram_utils.py
-          pytest sgl-kernel/tests/speculative/test_speculative_sampling.py
-          pytest sgl-kernel/tests/test_torch_defaults_reset.py
+          pytest python/sglang/kernels/aot/tests/test_per_token_quant_fp8.py
+          pytest python/sglang/kernels/aot/tests/speculative/test_eagle_utils.py
+          pytest python/sglang/kernels/aot/tests/speculative/test_ngram_utils.py
+          pytest python/sglang/kernels/aot/tests/speculative/test_speculative_sampling.py
+          pytest python/sglang/kernels/aot/tests/test_torch_defaults_reset.py
 
 
   pr-test-musa-finish:

.github/workflows/pr-test-sgl-kernel.yml

@@ -44,13 +44,13 @@ jobs:
 
       - name: Cleanup
         run: |
-          ls -alh sgl-kernel/dist || true
-          rm -rf sgl-kernel/dist/* || true
+          ls -alh python/sglang/kernels/aot/dist || true
+          rm -rf python/sglang/kernels/aot/dist/* || true
 
       - name: Download artifacts
         uses: actions/download-artifact@v4
         with:
-          path: sgl-kernel/dist/
+          path: python/sglang/kernels/aot/dist/
           merge-multiple: true
           pattern: wheel-python3.10-cuda*
 
@@ -62,7 +62,7 @@ jobs:
       - name: Run test
         timeout-minutes: 30
         run: |
-          cd sgl-kernel
+          cd python/sglang/kernels/aot
           pytest tests/
 
   sgl-kernel-benchmark-test:
@@ -79,13 +79,13 @@ jobs:
 
       - name: Cleanup
         run: |
-          ls -alh sgl-kernel/dist || true
-          rm -rf sgl-kernel/dist/* || true
+          ls -alh python/sglang/kernels/aot/dist || true
+          rm -rf python/sglang/kernels/aot/dist/* || true
 
       - name: Download artifacts
         uses: actions/download-artifact@v4
         with:
-          path: sgl-kernel/dist/
+          path: python/sglang/kernels/aot/dist/
           merge-multiple: true
           pattern: wheel-python3.10-cuda*
 
@@ -97,7 +97,7 @@ jobs:
       - name: Run benchmark tests
         timeout-minutes: 45
         run: |
-          cd sgl-kernel/benchmark
+          cd python/sglang/kernels/aot/benchmark
           echo "Running sgl-kernel benchmark tests in CI mode..."
 
           echo "CI environment variable: $CI"
@@ -126,13 +126,13 @@ jobs:
 
       - name: Cleanup
         run: |
-          ls -alh sgl-kernel/dist || true
-          rm -rf sgl-kernel/dist/* || true
+          ls -alh python/sglang/kernels/aot/dist || true
+          rm -rf python/sglang/kernels/aot/dist/* || true
 
       - name: Download artifacts
         uses: actions/download-artifact@v4
         with:
-          path: sgl-kernel/dist/
+          path: python/sglang/kernels/aot/dist/
           merge-multiple: true
           pattern: wheel-python3.10-cuda*
 
@@ -144,7 +144,7 @@ jobs:
       - name: Run sgl-kernel unit tests on B200
         timeout-minutes: 30
         run: |
-          cd sgl-kernel
+          cd python/sglang/kernels/aot
           pytest tests/
 
   # Adding a single CUDA13 build-and-run check for the kernel
@@ -157,13 +157,13 @@ jobs:
 
   #     - name: Cleanup
   #       run: |
-  #         ls -alh sgl-kernel/dist || true
-  #         rm -rf sgl-kernel/dist/* || true
+  #         ls -alh python/sglang/kernels/aot/dist || true
+  #         rm -rf python/sglang/kernels/aot/dist/* || true
 
   #     - name: Download CUDA 13.0 artifacts
   #       uses: actions/download-artifact@v4
   #       with:
-  #         path: sgl-kernel/dist/
+  #         path: python/sglang/kernels/aot/dist/
   #         merge-multiple: true
   #         pattern: wheel-python3.10-cuda*
 
@@ -174,5 +174,5 @@ jobs:
   #     - name: Run kernel unit tests
   #       timeout-minutes: 30
   #       run: |
-  #         cd sgl-kernel
+  #         cd python/sglang/kernels/aot
   #         pytest tests/

.github/workflows/pr-test-xeon.yml

@@ -66,7 +66,7 @@ jobs:
               - "python/sglang/!(multimodal_gen)/**/!(*.md)"
               - "python/pyproject_cpu.toml"
               - "test/**/!(*.md)"
-              - "sgl-kernel/**/!(*.md|THIRDPARTYNOTICES.txt|LICENSE)"
+              - "python/sglang/kernels/aot/**/!(*.md|THIRDPARTYNOTICES.txt|LICENSE)"
               - ".github/workflows/pr-test-xeon.yml"
               - "docker/xeon.Dockerfile"
 

.github/workflows/pr-test-xpu.yml

@@ -57,7 +57,7 @@ jobs:
               - "python/sglang/!(multimodal_gen)/**/!(*.md)"
               - "python/pyproject_xpu.toml"
               - "test/**/!(*.md)"
-              - "sgl-kernel/**/!(*.md|THIRDPARTYNOTICES.txt|LICENSE)"
+              - "python/sglang/kernels/aot/**/!(*.md|THIRDPARTYNOTICES.txt|LICENSE)"
               - ".github/workflows/pr-test-xpu.yml"
               - "docker/xpu.Dockerfile"
 

.github/workflows/release-whl-kernel.yml

@@ -5,7 +5,7 @@ on:
     branches:
       - main
     paths:
-      - sgl-kernel/python/sgl_kernel/version.py
+      - python/sglang/kernels/aot/python/sgl_kernel/version.py
   workflow_dispatch:
     inputs:
       target:
@@ -54,7 +54,7 @@ jobs:
     runs-on: ${{ matrix.runner }}
     steps:
       # Self-hosted build nodes retain the workspace across jobs. Prior builds
-      # leave root-owned artifacts under sgl-kernel/build/ that actions/checkout
+      # leave root-owned artifacts under python/sglang/kernels/aot/build/ that actions/checkout
       # cannot remove, causing EACCES on rmdir. Wipe them via a throwaway root
       # container before checkout recreates the workspace.
       - name: Clean workspace (remove root-owned files from prior runs)
@@ -74,7 +74,7 @@ jobs:
 
       - name: Build wheels
         run: |
-          cd sgl-kernel
+          cd python/sglang/kernels/aot
           chmod +x ./build.sh
           ./build.sh "${{ matrix.python-version }}" "${{ matrix.cuda-version }}" ${{ matrix.arch == 'aarch64' && 'aarch64' || '' }}
         env:
@@ -85,7 +85,7 @@ jobs:
         uses: actions/upload-artifact@v4
         with:
           name: wheel-python${{ matrix.python-version }}-cuda${{ matrix.cuda-version }}${{ matrix.arch == 'aarch64' && '-aarch64' || '' }}
-          path: sgl-kernel/dist/*
+          path: python/sglang/kernels/aot/dist/*
 
   release-cu129:
     needs: build-cu129-matrix
@@ -98,7 +98,7 @@ jobs:
       - name: Download artifacts
         uses: actions/download-artifact@v4
         with:
-          path: sgl-kernel/dist/
+          path: python/sglang/kernels/aot/dist/
           merge-multiple: true
           pattern: wheel-*
 
@@ -106,7 +106,7 @@ jobs:
         id: set_tag_name
         run: |
           if [ -z "${{ inputs.tag_name }}" ]; then
-            TAG_NAME="v$(cat sgl-kernel/python/sgl_kernel/version.py | cut -d'"' -f2)"
+            TAG_NAME="v$(cat python/sglang/kernels/aot/python/sgl_kernel/version.py | cut -d'"' -f2)"
             echo "tag_name=$TAG_NAME" >> $GITHUB_OUTPUT
           else
             echo "tag_name=${{ inputs.tag_name }}" >> $GITHUB_OUTPUT
@@ -119,7 +119,7 @@ jobs:
           repository: sgl-project/whl
           token: ${{ secrets.GH_PAT_FOR_WHL_RELEASE }}
           files: |
-            sgl-kernel/dist/*
+            python/sglang/kernels/aot/dist/*
 
       - name: Clone wheel index
         run: git clone https://oauth2:${WHL_TOKEN}@github.com/sgl-project/whl.git sgl-whl
@@ -155,7 +155,7 @@ jobs:
     runs-on: ${{ matrix.runner }}
     steps:
       # Self-hosted build nodes retain the workspace across jobs. Prior builds
-      # leave root-owned artifacts under sgl-kernel/build/ that actions/checkout
+      # leave root-owned artifacts under python/sglang/kernels/aot/build/ that actions/checkout
       # cannot remove, causing EACCES on rmdir. Wipe them via a throwaway root
       # container before checkout recreates the workspace.
       - name: Clean workspace (remove root-owned files from prior runs)
@@ -175,7 +175,7 @@ jobs:
 
       - name: Build wheels
         run: |
-          cd sgl-kernel
+          cd python/sglang/kernels/aot
           chmod +x ./build.sh
           ./build.sh "${{ matrix.python-version }}" "${{ matrix.cuda-version }}" ${{ matrix.arch == 'aarch64' && 'aarch64' || '' }}
         env:
@@ -183,7 +183,7 @@ jobs:
           NVCC_THREADS: 8
 
       - name: Strip +cu130 local version for PyPI upload
-        working-directory: sgl-kernel
+        working-directory: python/sglang/kernels/aot
         run: |
           set -eux
           pip install wheel
@@ -208,7 +208,7 @@ jobs:
           ls -lh dist-pypi/
 
       - name: Upload to PyPI
-        working-directory: sgl-kernel
+        working-directory: python/sglang/kernels/aot
         run: |
           pip install twine
           python3 -m twine upload --skip-existing dist-pypi/* -u __token__ -p ${{ secrets.PYPI_TOKEN_SGLANG_KERNEL }}
@@ -217,7 +217,7 @@ jobs:
         uses: actions/upload-artifact@v4
         with:
           name: wheel-python${{ matrix.python-version }}-cuda${{ matrix.cuda-version }}${{ matrix.arch == 'aarch64' && '-aarch64' || '' }}
-          path: sgl-kernel/dist/*
+          path: python/sglang/kernels/aot/dist/*
 
   release-cu130:
     needs: build-cu130-matrix
@@ -230,7 +230,7 @@ jobs:
       - name: Download artifacts
         uses: actions/download-artifact@v4
         with:
-          path: sgl-kernel/dist/
+          path: python/sglang/kernels/aot/dist/
           merge-multiple: true
           pattern: wheel-*
 
@@ -238,7 +238,7 @@ jobs:
         id: set_tag_name
         run: |
           if [ -z "${{ inputs.tag_name }}" ]; then
-            TAG_NAME="v$(cat sgl-kernel/python/sgl_kernel/version.py | cut -d'"' -f2)"
+            TAG_NAME="v$(cat python/sglang/kernels/aot/python/sgl_kernel/version.py | cut -d'"' -f2)"
             echo "tag_name=$TAG_NAME" >> $GITHUB_OUTPUT
           else
             echo "tag_name=${{ inputs.tag_name }}" >> $GITHUB_OUTPUT
@@ -251,7 +251,7 @@ jobs:
           repository: sgl-project/whl
           token: ${{ secrets.GH_PAT_FOR_WHL_RELEASE }}
           files: |
-            sgl-kernel/dist/*
+            python/sglang/kernels/aot/dist/*
 
       - name: Clone wheel index
         run: git clone https://oauth2:${WHL_TOKEN}@github.com/sgl-project/whl.git sgl-whl
@@ -281,7 +281,7 @@ jobs:
         rocm-version: ["700", "720"]
     steps:
       # Self-hosted build nodes retain the workspace across jobs. Prior builds
-      # leave root-owned artifacts under sgl-kernel/build/ that actions/checkout
+      # leave root-owned artifacts under python/sglang/kernels/aot/build/ that actions/checkout
       # cannot remove, causing EACCES on rmdir. Wipe them via a throwaway root
       # container before checkout recreates the workspace.
       - name: Clean workspace (remove root-owned files from prior runs)
@@ -301,8 +301,8 @@ jobs:
 
       - name: Build wheels
         run: |
-          cp 3rdparty/amd/wheel/sgl-kernel/* sgl-kernel/
-          cd sgl-kernel
+          cp 3rdparty/amd/wheel/sgl-kernel/* python/sglang/kernels/aot/
+          cd python/sglang/kernels/aot
           chmod +x ./build_rocm.sh
           ./build_rocm.sh "${{ matrix.rocm-version }}"
 
@@ -310,7 +310,7 @@ jobs:
         uses: actions/upload-artifact@v4
         with:
           name: wheel-python${{ matrix.python-version }}-rocm${{ matrix.rocm-version }}
-          path: sgl-kernel/dist/*
+          path: python/sglang/kernels/aot/dist/*
 
   release-rocm700:
     needs: build-rocm-matrix
@@ -323,7 +323,7 @@ jobs:
       - name: Download artifacts
         uses: actions/download-artifact@v4
         with:
-          path: sgl-kernel/dist/
+          path: python/sglang/kernels/aot/dist/
           merge-multiple: true
           pattern: wheel-*-rocm700
 
@@ -331,7 +331,7 @@ jobs:
         id: set_tag_name
         run: |
           if [ -z "${{ inputs.tag_name }}" ]; then
-            TAG_NAME="v$(cat sgl-kernel/python/sgl_kernel/version.py | cut -d'"' -f2)"
+            TAG_NAME="v$(cat python/sglang/kernels/aot/python/sgl_kernel/version.py | cut -d'"' -f2)"
             echo "tag_name=$TAG_NAME" >> $GITHUB_OUTPUT
           else
             echo "tag_name=${{ inputs.tag_name }}" >> $GITHUB_OUTPUT
@@ -344,7 +344,7 @@ jobs:
           repository: sgl-project/whl
           token: ${{ secrets.GH_PAT_FOR_WHL_RELEASE }}
           files: |
-            sgl-kernel/dist/*
+            python/sglang/kernels/aot/dist/*
 
       - name: Clone wheel index
         run: git clone https://oauth2:${WHL_TOKEN}@github.com/sgl-project/whl.git sgl-whl
@@ -374,7 +374,7 @@ jobs:
       - name: Download artifacts
         uses: actions/download-artifact@v4
         with:
-          path: sgl-kernel/dist/
+          path: python/sglang/kernels/aot/dist/
           merge-multiple: true
           pattern: wheel-*-rocm720
 
@@ -382,7 +382,7 @@ jobs:
         id: set_tag_name
         run: |
           if [ -z "${{ inputs.tag_name }}" ]; then
-            TAG_NAME="v$(cat sgl-kernel/python/sgl_kernel/version.py | cut -d'"' -f2)"
+            TAG_NAME="v$(cat python/sglang/kernels/aot/python/sgl_kernel/version.py | cut -d'"' -f2)"
             echo "tag_name=$TAG_NAME" >> $GITHUB_OUTPUT
           else
             echo "tag_name=${{ inputs.tag_name }}" >> $GITHUB_OUTPUT
@@ -395,7 +395,7 @@ jobs:
           repository: sgl-project/whl
           token: ${{ secrets.GH_PAT_FOR_WHL_RELEASE }}
           files: |
-            sgl-kernel/dist/*
+            python/sglang/kernels/aot/dist/*
 
       - name: Clone wheel index
         run: git clone https://oauth2:${WHL_TOKEN}@github.com/sgl-project/whl.git sgl-whl
@@ -425,7 +425,7 @@ jobs:
         musa-version: ["43"]
     steps:
       # Self-hosted build nodes retain the workspace across jobs. Prior builds
-      # leave root-owned artifacts under sgl-kernel/build/ that actions/checkout
+      # leave root-owned artifacts under python/sglang/kernels/aot/build/ that actions/checkout
       # cannot remove, causing EACCES on rmdir. Wipe them via a throwaway root
       # container before checkout recreates the workspace.
       - name: Clean workspace (remove root-owned files from prior runs)
@@ -451,19 +451,19 @@ jobs:
 
       - name: Build wheels
         run: |
-          cd sgl-kernel
+          cd python/sglang/kernels/aot
           mv pyproject_musa.toml pyproject.toml
           python setup_musa.py sdist bdist_wheel
 
       - name: Rename MUSA wheels
         run: |
-          bash scripts/ci/musa/rename_wheels_musa.sh ${{ matrix.musa-version }} sgl-kernel/dist
+          bash scripts/ci/musa/rename_wheels_musa.sh ${{ matrix.musa-version }} python/sglang/kernels/aot/dist
 
       - name: Upload artifacts
         uses: actions/upload-artifact@v4
         with:
           name: wheel-python${{ matrix.python-version }}-musa${{ matrix.musa-version }}
-          path: sgl-kernel/dist/*
+          path: python/sglang/kernels/aot/dist/*
 
   release-musa43:
     needs: build-musa43
@@ -474,7 +474,7 @@ jobs:
       - name: Download artifacts
         uses: actions/download-artifact@v4
         with:
-          path: sgl-kernel/dist/
+          path: python/sglang/kernels/aot/dist/
           merge-multiple: true
           pattern: wheel-*
 
@@ -482,7 +482,7 @@ jobs:
         id: set_tag_name
         run: |
           if [ -z "${{ inputs.tag_name }}" ]; then
-            TAG_NAME="v$(cat sgl-kernel/python/sgl_kernel/version.py | cut -d'"' -f2)"
+            TAG_NAME="v$(cat python/sglang/kernels/aot/python/sgl_kernel/version.py | cut -d'"' -f2)"
             echo "tag_name=$TAG_NAME" >> $GITHUB_OUTPUT
           else
             echo "tag_name=${{ inputs.tag_name }}" >> $GITHUB_OUTPUT
@@ -495,7 +495,7 @@ jobs:
           repository: sgl-project/whl
           token: ${{ secrets.GH_PAT_FOR_WHL_RELEASE }}
           files: |
-            sgl-kernel/dist/*
+            python/sglang/kernels/aot/dist/*
 
       - name: Clone wheel index
         run: git clone https://oauth2:${WHL_TOKEN}@github.com/sgl-project/whl.git sgl-whl

.gitignore

@@ -263,9 +263,9 @@ python/kernel.lock
 
 # MUSA section
 # Generated source files by torchada
-sgl-kernel/csrc_musa/
-sgl-kernel/include_musa/
-sgl-kernel/csrc/**/*_musa/
+python/sglang/kernels/aot/csrc_musa/
+python/sglang/kernels/aot/include_musa/
+python/sglang/kernels/aot/csrc/**/*_musa/
 
 # MUSA core dump files
 *.mudmp

.pre-commit-config.yaml

@@ -42,6 +42,7 @@ repos:
           (?x)^(
           .*/__init__\.py$|
           .*\.ipynb$|
+          python/sglang/kernels/aot/.*|
           python/sglang/srt/grpc/.*_pb2\.py$|
           python/sglang/srt/grpc/.*_pb2_grpc\.py$|
           python/sglang/srt/grpc/.*_pb2\.pyi$|

docker/arm64.Dockerfile

@@ -42,7 +42,7 @@ RUN source $HOME/.local/bin/env && \
     cd python && \
     cp pyproject_cpu.toml pyproject.toml && \
     uv pip install . && \
-    cd ../sgl-kernel && \
+    cd sglang/kernels/aot && \
     cp pyproject_cpu.toml pyproject.toml && \
     uv pip install .
 

docker/rocm.Dockerfile

@@ -317,11 +317,11 @@ RUN if [ "$BRANCH_TYPE" = "local" ]; then \
        fi \
     && rm -rf /tmp/local_src \
     && cd sglang \
-    && cd sgl-kernel \
+    && cd python/sglang/kernels/aot \
     && rm -f pyproject.toml \
     && mv pyproject_rocm.toml pyproject.toml \
     && AMDGPU_TARGET=$GPU_ARCH_LIST python setup_rocm.py install \
-    && cd .. \
+    && cd ../../../.. \
     && rm -rf python/pyproject.toml && mv python/pyproject_other.toml python/pyproject.toml \
     && if [ "$BUILD_TYPE" = "srt" ]; then \
          export SETUPTOOLS_SCM_PRETEND_VERSION="${SETUPTOOLS_SCM_PRETEND_VERSION}" && python -m pip --no-cache-dir install -e "python[srt_hip,diffusion_hip]"; \

docker/xeon.Dockerfile

@@ -40,7 +40,7 @@ RUN source /opt/.venv/bin/activate && \
     cd python && \
     cp pyproject_cpu.toml pyproject.toml && \
     uv pip install . && \
-    cd ../sgl-kernel && \
+    cd sglang/kernels/aot && \
     cp pyproject_cpu.toml pyproject.toml && \
     uv pip install .
 

docs_new/docs/developer_guide/contribution_guide.mdx

@@ -164,7 +164,7 @@ Users listed in [CI_PERMISSIONS.json](https://github.com/sgl-project/sglang/blob
 
 ## How to update sgl-kernel
 Since sglang and the `sglang-kernel` (prior `sgl-kernel`) distribution are separate Python packages, our current GitHub CI infrastructure does not support updating a kernel and using it immediately within the same pull request (PR).
-To add a new kernel or modify an existing one in the `sgl-kernel/` source tree, you must use multiple PRs.
+To add a new kernel or modify an existing one in the `python/sglang/kernels/aot/` source tree, you must use multiple PRs.
 
 Follow these steps:
 

docs_new/docs/hardware-platforms/amd_gpu.mdx

@@ -50,11 +50,11 @@ cd sglang
 
 # Compile sgl-kernel
 pip install --upgrade pip
-cd sgl-kernel
+cd python/sglang/kernels/aot
 python setup_rocm.py install
 
 # Install sglang python package along with diffusion support
-cd ..
+cd ../../../..
 rm -rf python/pyproject.toml && mv python/pyproject_other.toml python/pyproject.toml
 pip install -e "python[all_hip]"
 ```

docs_new/docs/hardware-platforms/apple_metal.mdx

@@ -36,7 +36,7 @@ source sglang-metal/bin/activate
 
 # (Optional) Compile sgl-kernel
 uv pip install --upgrade pip
-uv run sgl-kernel/setup_metal.py install
+uv run python/sglang/kernels/aot/setup_metal.py install
 
 # Install sglang python package along with diffusion support
 rm -f python/pyproject.toml && mv python/pyproject_other.toml python/pyproject.toml

docs_new/docs/hardware-platforms/cpu_server.mdx

@@ -126,7 +126,7 @@ uv pip install --upgrade pip setuptools
 uv pip install .
 
 # Build the CPU backend kernels
-cd ../sgl-kernel
+cd sglang/kernels/aot
 cp pyproject_cpu.toml pyproject.toml
 uv pip install .
 ```

docs_new/docs/hardware-platforms/mthreads_gpu.mdx

@@ -19,11 +19,11 @@ cd sglang
 
 # Compile sgl-kernel
 pip install --upgrade pip
-cd sgl-kernel
+cd python/sglang/kernels/aot
 python setup_musa.py install
 
 # Install sglang python package along with diffusion support
-cd ..
+cd ../../../..
 rm -f python/pyproject.toml && mv python/pyproject_other.toml python/pyproject.toml
 pip install -e "python[all_musa]"
 ```

python/MANIFEST.in

@@ -0,0 +1 @@
+prune sglang/kernels/aot

python/pyproject.toml

@@ -199,6 +199,12 @@ killall_sglang = "sglang.cli.killall:main"
   "multimodal_gen/apps/realtime_webui/**/*"
 ]
 
+[tool.setuptools.exclude-package-data]
+"sglang" = [
+  "kernels/aot/*",
+  "kernels/aot/**/*",
+]
+
 [tool.setuptools.packages.find]
 exclude = [
   "assets*",
@@ -207,6 +213,7 @@ exclude = [
   "dist*",
   "playground*",
   "scripts*",
+  "sglang.kernels.aot*",
   "tests*",
 ]
 
@@ -218,6 +225,7 @@ exclude = [
   "dist*",
   "playground*",
   "scripts*",
+  "sglang/kernels/aot*",
   "tests*",
 ]
 

python/sglang/kernels/aot/README.md

@@ -1,6 +1,6 @@
 # sglang-kernel (prior sgl-kernel)
 
-[Kernel Library](https://github.com/sgl-project/sglang/tree/main/sgl-kernel) for LLM inference engines
+[Kernel Library](https://github.com/sgl-project/sglang/tree/main/python/sglang/kernels/aot) for LLM inference engines
 
 <div align="center">
 
@@ -9,7 +9,7 @@
 
 </div>
 
-`sglang-kernel` provides optimized compute primitives for LLM inference engines, enabling efficient inference for large language models and vision-language models through custom kernel operations. The source tree remains under the `sgl-kernel/` directory and the Python import path remains `sgl_kernel`.
+`sglang-kernel` provides optimized compute primitives for LLM inference engines, enabling efficient inference for large language models and vision-language models through custom kernel operations. The source tree lives under the `python/sglang/kernels/aot/` directory and the Python import path remains `sgl_kernel`.
 
 ## Installation
 Requires torch == 2.11.0
@@ -48,11 +48,11 @@ make build MAX_JOBS=2 CMAKE_ARGS="-DSGL_KERNEL_COMPILE_THREADS=1"
 
 ### Steps to add a new kernel:
 
-1. Implement the kernel in [csrc](https://github.com/sgl-project/sglang/tree/main/sgl-kernel/csrc)
-2. Expose the interface in [include/sgl_kernel_ops.h](https://github.com/sgl-project/sglang/blob/main/sgl-kernel/include/sgl_kernel_ops.h)
-3. Create torch extension in [csrc/common_extension.cc](https://github.com/sgl-project/sglang/blob/main/sgl-kernel/csrc/common_extension.cc)
-4. Update [CMakeLists.txt](https://github.com/sgl-project/sglang/blob/main/sgl-kernel/CMakeLists.txt) to include new CUDA source
-5. Expose Python interface in [python](https://github.com/sgl-project/sglang/blob/main/sgl-kernel/python/sgl_kernel)
+1. Implement the kernel in [csrc](https://github.com/sgl-project/sglang/tree/main/python/sglang/kernels/aot/csrc)
+2. Expose the interface in [include/sgl_kernel_ops.h](https://github.com/sgl-project/sglang/blob/main/python/sglang/kernels/aot/include/sgl_kernel_ops.h)
+3. Create torch extension in [csrc/common_extension.cc](https://github.com/sgl-project/sglang/blob/main/python/sglang/kernels/aot/csrc/common_extension.cc)
+4. Update [CMakeLists.txt](https://github.com/sgl-project/sglang/blob/main/python/sglang/kernels/aot/CMakeLists.txt) to include new CUDA source
+5. Expose Python interface in [python](https://github.com/sgl-project/sglang/blob/main/python/sglang/kernels/aot/python/sgl_kernel)
 6. Add test and benchmark
 
 ### Development Tips
@@ -95,7 +95,7 @@ m.impl("fwd", torch::kCUDA, make_pytorch_shim(&mha_fwd));
 
 ### Testing & Benchmarking
 
-1. Add pytest tests in [tests/](https://github.com/sgl-project/sglang/tree/main/sgl-kernel/tests), if you need to skip some test, please use `@pytest.mark.skipif`
+1. Add pytest tests in [tests/](https://github.com/sgl-project/sglang/tree/main/python/sglang/kernels/aot/tests), if you need to skip some test, please use `@pytest.mark.skipif`
 
 ```python
 @pytest.mark.skipif(
@@ -103,7 +103,7 @@ m.impl("fwd", torch::kCUDA, make_pytorch_shim(&mha_fwd));
 )
 ```
 
-2. Add benchmarks using [triton benchmark](https://triton-lang.org/main/python-api/generated/triton.testing.Benchmark.html) in [benchmark/](https://github.com/sgl-project/sglang/tree/main/sgl-kernel/benchmark)
+2. Add benchmarks using [triton benchmark](https://triton-lang.org/main/python-api/generated/triton.testing.Benchmark.html) in [benchmark/](https://github.com/sgl-project/sglang/tree/main/python/sglang/kernels/aot/benchmark)
 
    **We recommend using `triton.testing.do_bench_cudagraph` for kernel benchmarking**: