sglang

benchmark

[JIT Kernel] Multi-GPU test/bench framework for custom all-reduce + TP QKNorm (#26706 )

2026-06-14 17:20:36 +08:00

csrc

[AMD] Enable JIT staged HiCache write-back and fix CPU-index crash (#28534 )

2026-07-09 01:22:37 -07:00

diffusion

[diffusion] fix: fix z-Image accuracy (#29742 )

2026-07-08 09:08:32 +08:00

dsa

[AMD] Fix DeepSeekV4 server cutlass error (#30374 )

2026-07-07 14:43:05 -07:00

dsv4

[DSV4] perf: Make FP8 quant output tensor contiguous (#27926 )

2026-07-07 17:40:59 -07:00

dsv32

[DSA] Use cos_sin_cache for DSA indexer fusion (#29613 )

2026-06-29 23:49:00 -07:00

include/sgl_kernel

[Tiny] Fix Lint in #30645 (#30699 )

2026-07-09 15:30:48 -07:00

kv_canary

[lint] Enable Ruff UP037 to drop redundant quoted annotations (#27984 )

2026-06-11 17:38:05 -07:00

lplb

LPLB: linear-programming load balancer for MoE expert parallelism (#24515 )

2026-06-16 10:19:42 -07:00

minimax_m3

[minimax-m3] Split 1/4: sparse attention ops + JIT kernels + config foundation (#28712 )

2026-06-22 13:10:43 -07:00

tests

ci: make multi-GPU jit test hangs attributable from the CI log (#29925 )

2026-07-07 19:56:32 -07:00

triton

[AMD] Fuse sigmoid + mul into single Triton kernel for shared expert gating (#27636 )

2026-06-16 01:16:58 -07:00

trtllm_lora_temp

[LoRA] BF16 support + EP cuda-graph crash fix for experimental_sgl_trtllm MoE-LoRA (#28953 )

2026-06-24 21:16:50 -07:00

__init__.py

Move fa4 from sgl-kernel to jit kernel (#17353 )

2026-01-24 15:25:03 +08:00

__main__.py

[Chore] Clean up JIT compilation flags (#21022 )

2026-03-25 18:08:40 +08:00

.clang-format

[JIT Kernel][Feature] Support JIT custom all reduce (rewrite as v2) (#19880 )

2026-03-20 18:24:07 +08:00

activation.py

Nemotron perf changes (#26733 )

2026-06-05 22:31:46 -07:00

add_constant.py

[1/n jit_kernel restruct] unify cache usage and clean up naming in ngram_embedding (#20244 )

2026-03-10 15:53:43 +08:00

all_reduce.py

[CAR] Let custom allreduce support VMM based allocation (#27593 )

2026-06-16 04:49:05 -07:00

awq_dequantize.py

[Kernel Slimming] Migrate AWQ marlin repack kernel to JIT (#18949 )

2026-02-23 22:05:27 +08:00

awq_marlin_repack.py

[misc] clean up kernel API (#21325 )

2026-03-25 09:10:23 +08:00

clamp_position.py

Replace clamp_position with JIT kernel + platform dispatch (#20999 )

2026-03-21 21:26:26 -07:00

concat_mla.py

[Move sgl-kernel Kernel to JIT] Add JIT concat MLA kernels (#17889 )

2026-02-03 10:49:17 +08:00

cutedsl_bf16_gemm.py

Support Cutedsl BF16 GEMM JIT kernel (#30117 )

2026-07-06 19:16:16 -07:00

cutedsl_dsv3_fused_a_gemm.py

Support JIT fused A GEMM (MLA down projection) and support GLM-5 hidden size, SM120 (#27397 )

2026-06-27 13:32:06 -07:00

cutedsl_fp8_paged_mqa_logits.py

Cute-DSL FP8 MQA logits (#25220 )

2026-07-06 23:34:07 -07:00

cutedsl_gdn.py

[jit-kernel] Add CuTe DSL GDN Decode Kernel (#15631 )

2026-01-18 12:54:36 -08:00

cutedsl_kda.py

[KDA] Support CuTeDSL KDA decode kernel (#21203 )

2026-03-25 09:47:09 +08:00

dsv3_fused_a_gemm.py

Support JIT fused A GEMM (MLA down projection) and support GLM-5 hidden size, SM120 (#27397 )

2026-06-27 13:32:06 -07:00

dsv3_router_gemm.py

[JIT Kernel] Migrate dsv3_router_gemm from AOT sgl-kernel to JIT kernel (#21531 )

2026-06-26 11:52:18 -07:00

fixup_zero_kv.py

Fix trtllm mla chunked-prefill zero-length bug (#22291 ) (#22688 )

2026-04-20 22:10:13 -07:00

flash_attention_v3.py

Upgrade fa3 hash (#28394 )

2026-06-17 13:32:11 -07:00

flash_attention_v4.py

Kernels community fa3 (#20796 )

2026-04-07 12:48:44 -07:00

flash_attention.py

[FA]Add lost params in fa varlen func (#28642 )

2026-06-18 13:30:20 -07:00

fp8_quantize.py

[perf] prepare_prefill_qkv hook + fp8 quantize jit kernel (#25460 )

2026-05-20 14:20:49 -07:00

fused_a_gemm.py

Support JIT fused A GEMM (MLA down projection) and support GLM-5 hidden size, SM120 (#27397 )

2026-06-27 13:32:06 -07:00

fused_eh_norm.py

Add fused EH norm for DeepSeek NextN (#29667 )

2026-07-01 02:46:21 -07:00

fused_metadata_copy.py

[Refactor] Rename NSA → DSA: user-facing aliases, file/class/import rename (#25821 )

2026-05-20 00:18:04 -07:00

fused_qknorm_rope.py

[jit_kernel] Optimize fused_qknorm_rope: deduplicate sincosf for interleave RoPE (#21654 )

2026-04-01 09:04:13 +08:00

fused_store_index_cache.py

[Refactor] Rename NSA → DSA: user-facing aliases, file/class/import rename (#25821 )

2026-05-20 00:18:04 -07:00

gptq_marlin_repack.py

[misc] clean up kernel API (#21325 )

2026-03-25 09:10:23 +08:00

gptq_marlin.py

[misc] clean up kernel API (#21325 )

2026-03-25 09:10:23 +08:00

hadamard.py

Support piecewise CUDA graph with NSA (#23351 )

2026-05-22 14:39:50 -07:00

hicache.py

[HiCache]Support hybrid pool staged H2D kernel (#28434 )

2026-06-19 09:48:03 +08:00

hisparse.py

[PD & HiSparse] Add DeepSeek V4 support for HiSparse direct Prefill-to-Decode DRAM (#24880 )

2026-06-05 15:39:48 +08:00

kpool_topk_transform.py

[JIT] Add kpool_topk_transform JIT kernel (#28670 )

2026-06-22 01:04:21 -07:00

kvcache.py

[spec] Misc defensive guards for EAGLE draft KV indexing (#27486 )

2026-06-07 21:27:46 -07:00

minimax_decode_topk.py

[minimax-m3] Split 1/4: sparse attention ops + JIT kernels + config foundation (#28712 )

2026-06-22 13:10:43 -07:00

minimax_qknorm_rope.py

[minimax-m3] Split 1/4: sparse attention ops + JIT kernels + config foundation (#28712 )

2026-06-22 13:10:43 -07:00

minimax_quant_ue8m0.py

[minimax-m3] Split 1/4: sparse attention ops + JIT kernels + config foundation (#28712 )

2026-06-22 13:10:43 -07:00

minimax_store_kv_index.py

[minimax-m3] Split 1/4: sparse attention ops + JIT kernels + config foundation (#28712 )

2026-06-22 13:10:43 -07:00

mla_kv_pack_quantize_fp8.py

perf(mla): hybrid Triton fused cat+FP8-quantize for MLA chunked-prefill K/V (#25333 )

2026-05-15 10:51:00 -07:00

moe_align.py

[sgl-kernel] support > 1024 experts in moe_align_block_size kernel (#21610 )

2026-04-08 11:45:13 -07:00

moe_finalize_fuse_shared.py

[DeepSeek V3] Defer moe finalize and fused it with main stream add (#27720 )

2026-06-12 19:00:51 -07:00

moe_fused_gate.py

[MoE] Fix moe_fused_gate out-of-range expert id on all-NaN rows (fixes eagle_dp_attention crash) (#30079 )

2026-07-03 18:48:51 -07:00

moe_lora_align.py

[5/n] Lora support cuda graph (#21647 )

2026-04-04 00:31:46 -07:00

moe_topk_sigmoid.py

[minimax-m3] Split 1/4: sparse attention ops + JIT kernels + config foundation (#28712 )

2026-06-22 13:10:43 -07:00

moe_wna16_marlin.py

Revert "[Feature] NVFP4 Marlin fallback for non-Blackwell GPUs (SM75+… (#22047 )

2026-04-03 13:12:30 -07:00

mp.py

ci: make multi-GPU jit test hangs attributable from the CI log (#29925 )

2026-07-07 19:56:32 -07:00

mxfp8.py

[JIT Kernel][1/2]Migrate MXFP8 Group GEMM & Quant into JIT (#23833 )

2026-04-29 22:50:09 +08:00

ngram_corpus.py

[Spec][Ngram] Clean up unused stateless batchMatch (#22487 )

2026-04-10 21:52:56 -07:00

ngram_embedding.py

[Model] Support LongCat 2.0 FP8 (#30275 )

2026-07-07 19:51:12 +08:00

norm.py

[bugfix] Honor cast_x_before_out_mul in RMSNorm.forward_cuda residual path (#25920 )

2026-05-28 01:22:46 -07:00

nvfp4.py

[Perf][Kernel] Fuse SiLU+Mul into NVFP4 Expert Quantization for CUTLASS MoE (#18612 )

2026-06-29 16:51:01 -07:00

per_tensor_quant_fp8.py

[misc] clean up kernel API (#21325 )

2026-03-25 09:10:23 +08:00

per_token_group_quant_8bit_v2.py

[jit-kernel] Support per token group quant 8bit v2 jit kernel (#27449 )

2026-06-13 12:15:09 +08:00

per_token_group_quant_8bit.py

[misc] clean up kernel API (#21325 )

2026-03-25 09:10:23 +08:00

resolve_future_token_ids.py

Replace _resolve_future_token_ids with JIT kernel + platform dispatch (#20976 )

2026-03-20 01:47:03 -07:00

rmsnorm_hf.py

[Fix/Kernel] Add JIT rmsnorm_hf kernel to fix transformers backend MMLU accuracy regression (#22931 )

2026-04-23 12:00:31 +08:00

rope.py

[misc] clean up kernel API (#21325 )

2026-03-25 09:10:23 +08:00

set_mla_kv_buffer.py

perf(mla): TMA bulk-store set_mla_kv_buffer (up to 12× over baseline) (#25311 )

2026-05-14 18:23:41 -07:00

sparse_mla_q8kv8_prefill_sm90.py

[Kernel] Add SM90 Q8KV8 FP8 Sparse MLA Prefill JIT Kernel with Tests and Benchmark (#25751 )

2026-06-30 09:00:11 +08:00

timestep_embedding.py

[Diffusion] Reuse shared AlignedVector and tidy jit_kernel/diffusion (#29664 )

2026-06-30 11:38:22 +08:00

triton_store_cache.py

[AMD] Dsv4/pr1 fix run time issue (#25898 )

2026-05-23 16:04:14 -07:00

utils.py

LPLB: linear-programming load balancer for MoE expert parallelism (#24515 )

2026-06-16 10:19:42 -07:00