minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
12,520 Commits 3 Branches 0 Tags
10375a103774df29ec221a6b59cd7ade052d09ea
Commit Graph
333 Commits
Author SHA1 Message Date
R0CKSTAR 74d70af09a [Apple Silicon] Add Metal kernel support in sgl-kernel (#23449) 
Signed-off-by: Xiaodong Ye <yeahdongcn@gmail.com>
 2026-05-11 17:54:27 -07:00
Brayden Zhongandb8zhong 05d1ab51e8 Enable PDL for various kernels in DSV32/GLM5 (#23965) 
Co-authored-by: b8zhong <b8zhong@users.noreply.github.com>
 2026-05-09 03:42:56 -07:00
Yuxuan Zhang d49fc092cb [Bug Fix] GLM-5.1: drop constexpr on page_indice_batch_offset, skip offloader post_init on draft worker, support N=32 in copy_to_gpu_no_ce (#23550) 2026-05-09 15:43:45 +08:00
Yibo Cai 55d8223c2b [sgl-kernel/cpu] support w8a8 int8 model for arm cpu (#16045) 
skip gpu test as this one is not related to gpu backend.
 2026-05-08 14:47:06 +08:00
JoeyandR0CKSTAR 15e6572f21 [MUSA][18/N] Add MUSA-optimized kernel implementations for hot ops (#23255) 
Signed-off-by: Joey-gvwal <joey_gvwal@yeah.net>
Co-authored-by: R0CKSTAR <yeahdongcn@gmail.com>
 2026-05-07 20:38:33 -07:00
Mandepudi Rani ChowdaryandMa Mingfei 55224fff08 Add Arm64 CPU Phase 1A CI bootstrap (#22123) 
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
 2026-05-08 09:28:23 +08:00
jianan-guZheng, BeileiMa MingfeiXinyuan Tong
 		10fd0faccd [CPU] Add Qwen3.5 model optimization for CPU (#19484) 
Co-authored-by: Zheng, Beilei <beilei.zheng@intel.com>
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2026-04-26 10:12:36 -07:00
Jia GuoandClaude Opus 4.6 587fd15bd2 perf: eliminate attention DtoD copy by passing pre-allocated output to FA (#21985) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-04-24 12:05:16 -07:00
Ma Mingfei 23e4d381f0 [CPU] remove RECORD_FUNCTION (#23528) 2026-04-24 09:18:29 +08:00
MARATRIXandAlex Nails 74c2e5bacd [MUSA][8/N] Port CUDA kernels that are compatible with MUSA (#17946) 
Signed-off-by: yafeng.li <yafeng.li@mthreads.com>
Co-authored-by: Alex Nails <alex.nails@radixark.ai>
 2026-04-23 18:04:58 -07:00
jianan-guandMa Mingfei ad0fc88810 [CPU] [Quantization] Add GPTQ/AWQ 4bits quantization support for CPU (#22685) 
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
 2026-04-22 13:34:02 -07:00
Tarushii Goel 3ebf066d13 [sgl] update specdec sampling kernel to return valid token ID (#22643) 2026-04-21 20:28:19 -07:00
Ma Mingfei 929e00eeab [CPU] expand the interface of shared_expert without scaling factor (#22933) 
merge since this is CPU only change on sgl-kernel.
 2026-04-21 20:03:39 +08:00
Yuhao YangClaude Opus 4.6Kangyan-Zhou
 		fe9b9b254b Fix segfault in cudaMemcpyBatchAsync on CUDA 13.0 (#23136) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Kangyan-Zhou <zky314343421@gmail.com>
 2026-04-20 12:20:22 -07:00
blzhengCopilotMa Mingfei
 		0dcfae5553 [CPU] Add gemma4_rmsnorm_cpu kernel (#22842) 
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
 2026-04-17 13:03:16 +08:00
Chunyuan WUandMa Mingfei 6c89214584 [CPU][sgl-kernel] extend_attention_cpu and flash_attn_varlen_func: fix nan for large seq (#22434) 
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
 2026-04-17 13:01:01 +08:00
jianan-guandMa Mingfei 2ab141547d [CPU] Add apply_routed_scaling_factor_on_output support for biased_grouped_topk fusion (#22413) 
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
 2026-04-10 15:16:05 +08:00
Yibo CaiandMa Mingfei 4644d28213 [sgl-kernel/cpu] fix build error on non-x86 platform (#22245) 
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
 2026-04-10 09:58:07 +08:00
Brayden Zhong 6aafe756b9 Revert "[Feature] NVFP4 Marlin fallback for non-Blackwell GPUs (SM75+… (#22047) 2026-04-03 13:12:30 -07:00
Mook 991f3aa5b3 [Feature] NVFP4 Marlin fallback for non-Blackwell GPUs (SM75+) (#19652) 2026-04-03 10:48:15 +08:00
Xiaoyu Zhang cdd7d6a227 Remove obsolete sgl-kernel legacy paths (#21528) 2026-04-01 09:00:20 +08:00
Ma Mingfei af62bd9486 [CPU] Implement MXFP4 Gemm kernels for intel AMX to support GPT OSS series. (#14385) 2026-03-29 23:44:12 -07:00
blzhengandMa Mingfei ed01e1d5d6 [CPU] add kernel apply_rotary_pos_emb_cpu for Qwen3-VL and Qwen3-Omni (#13121) 
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
 2026-03-29 23:43:46 -07:00
Ma Mingfei 6da8f5f69e fix topk softmax performance issue (#14702) 2026-03-29 23:43:16 -07:00
Johnsonms 8a56a7b04d [jit_kernel] Migrate cast (downcast_fp8) from sgl-kernel AOT to JIT (#19103) 2026-03-27 13:21:44 +08:00
Minglei Zhu a12fea21ed perf(sgl-kernel): expose get_scheduler_metadata for FA3 decode optimization (#21103) 2026-03-25 13:17:27 -07:00
Brayden Zhong 009eee85a0 CUTLASS FP8 Blockwise GEMM improvement of SM120 (#20887) 2026-03-22 17:55:54 +08:00
Cao E 274581fb77 Add support for more batch sizes in cpu_graph_runner (#13881) 2026-03-19 09:50:56 -07:00
Qi Yuhang cb8105fe28 [sgl-kernel][6/7]Support Expert Specialization Grouped GEMM (#15471) 2026-03-19 15:39:52 +08:00
blzhengandFan Yin cd22aa27a9 [CPU] Add FP8 Bmm support (#9744) 
Co-authored-by: Fan Yin <1106310035@qq.com>
 2026-03-18 22:19:48 -07:00
blzhengandWu, Chunyuan c2b01bd2fc [CPU] fix bug in AVX512 implementation of flash_attn_softmax (#20220) 
Co-authored-by: Wu, Chunyuan <chunyuan.wu@intel.com>
 2026-03-18 22:18:47 -07:00
Ma Mingfei 687d9eb66f [CPU] Optimize image preprocessor performance for Qwen2VLImageProcessorFast (#15168) 2026-03-18 22:18:15 -07:00
Ma Mingfei 62d7454976 optimize conv3d used in patch embedding (#16040) 2026-03-18 22:17:53 -07:00
blzhengandgemini-code-assist[bot] cbea9f6909 [CPU] improve numa memory binding (#19666) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-03-18 22:15:50 -07:00
Zaili WangandMa Mingfei 2f4babe32b [CPU] support LayerNorm with 3D shape (#15075) 
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
 2026-03-18 22:15:24 -07:00
blzhengandMa Mingfei dc6aa26ce9 [CPU] Add mrope kernel for Qwen3-vl (#12531) 
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
 2026-03-18 22:12:48 -07:00
Xiaoyu Zhang 25e38216b6 [kernel slimming] Clean many useless sgl-kernel deprecated kernels (#20277) 2026-03-14 16:45:54 +08:00
Mohammad Miadh Angkad f88acf8780 [JIT Kernel] Reland NVFP4 kernels to JIT (#20012) 2026-03-07 10:31:08 +08:00
Baizhou Zhang 51e5dc845a Revert "[Kernel Slimming] Migrate NVFP4 kernels to JIT" (#20005) 2026-03-05 19:40:00 -08:00
Mohammad Miadh Angkad 2bdd89a6cd [Kernel Slimming] Migrate NVFP4 kernels to JIT (#19437) 2026-03-05 15:22:28 +08:00
huangtingweiandhzh0425 36dc973cbf [HiCache] refactor page_first_direct io kernel (#18113) 
Co-authored-by: hzh0425 <hzh0425@apache.org>
 2026-02-27 11:43:14 -08:00
Cao E 4f0f6cd9d0 Add torch.compile support for qwen3-next on CPU (#12444) 2026-02-26 23:28:03 -08:00
Xiaoyu Zhang 054bd71086 [sgl-kernel slimming] remove sgl-kernel moe-wna16-marlin (#19379) 2026-02-27 09:17:46 +08:00
Qi YuhangandXiaoyu Zhang 88ad3b894a [sgl-kernel][Feat][B200][2/N] Support MXFP8 Grouped GEMM in Blackwell (#14640) 
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
 2026-02-26 11:23:37 +08:00
pansicheng 2ad475b4ed use flashinfer.sampling (#18696) 2026-02-26 10:02:38 +08:00
Xiaoyu Zhang 9dff933164 [Kernel Slimming] Remove sgl-kernel AOT marlin kernels (#19241) 2026-02-25 10:08:22 +08:00
blake-sncandClaude Opus 4.6 0d30896015 fix(sgl-kernel): use >= 120 for SM12x CUDA kernel dispatch (#18750) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-16 00:44:47 +08:00
SoluMilken 07a24f1a38 update pre-commit config (#18860) 2026-02-16 00:18:31 +08:00
Xiaoyu Zhang c29394e3c8 [kernel slimming] Move fast_hadamard_transform to jit_kernel (#18475) 2026-02-14 23:06:21 +08:00
Xiaoyu Zhang 9e9e949261 speed up sgl-kernel build (#18586) 2026-02-12 23:43:22 +08:00