minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
7,455 Commits 3 Branches 0 Tags
842807843671e1f469fe272a3f69a5582925004b
Commit Graph
52 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Yuan Luoandluoyuan.luo b2b09f5f24 [VLM] Introduce Cache for positional embedding ids for Qwen-VL family (#14292) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-12-04 12:32:00 +08:00
Yuan Luoandluoyuan.luo 21ec99beff [VLM][Doc] Document for VLM DP Encoder (#14279) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-12-02 15:08:37 +08:00
Yuan Luoandluoyuan.luo 26aebf83d3 [VLM] Support Piecewise CUDA Graph for Qwen3-Omni-MOE (#14222) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-12-02 10:12:10 +08:00
Yuan Luoandluoyuan.luo 630a693081 [VLM] Boost Memory Pool based CUDA IPC (#14123) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-12-01 17:17:46 +08:00
Yuan Luoandluoyuan.luo e12c78aab6 [sgl-kernel][1/2] Fused qk_norm_rope for Qwen3-MoE (#14036) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-11-28 12:25:15 +08:00
Yuan Luoandluoyuan.luo ca5c8b16f6 [VLM] Support InternVL Vision Encoder Data Parallelism (#13925) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-11-26 11:43:05 +08:00
Yuan Luoandluoyuan.luo 8ef11569a2 [VLM] Revise InternVL Piecewise CUDA Graph Supporting (#13846) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-11-24 22:15:10 +08:00
Yuan Luoluoyuan.luoXu Yongfei
 		f56b9b42e6 [Bugfix] Add jit kernel files in packaging (#13829) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: Xu Yongfei <xuyongfei.xyf@antgroup.com>
 2025-11-24 12:32:16 +08:00
Yuan Luoandluoyuan.luo 5625e32cae [VLM] Replace torch.repeat_interleave with faster np.repeat for Qwen-VL series (#13736) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-11-22 22:45:32 +08:00
Yuan Luoandluoyuan.luo 475962a139 [VLM] Support Piecewise CUDA Graph for InternVL (#13640) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-11-21 13:21:51 +08:00
Yuan Luoluoyuan.luoYuhao Yang
 		af6bcadcf7 [VLM] Support Piecewise CUDA Graph for Qwen2.5-VL (#13055) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: Yuhao Yang <yhyang201@gmail.com>
 2025-11-20 10:23:44 +08:00
Yuan Luoandluoyuan.luo f0021c0dc8 Add feature flag for mm inputs processing optimization (#13278) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-11-15 14:43:44 +08:00
Yuan Luoluoyuan.luoTianyu Guo
 		706502ff6c [VLM] Support PP for Qwen2.5-VL (#13075) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
 2025-11-12 23:18:44 +08:00
Yuan Luoluoyuan.luo羽癫
 		fd3034da75 [VLM] Optimize qwen_vl preprocess_video (#12240) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: 羽癫 <yudian.zy@antgroup.com>
 2025-11-06 14:55:01 +08:00
Yuan Luoandluoyuan.luo bb517fe393 [HotFix] Disable torch dynamo for mrope_triton kernel (#12593) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-11-04 23:26:56 +08:00
Yuan Luoandluoyuan.luo 819fc59123 Add prefix for torch symm mem (#12506) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-11-02 11:23:05 -08:00
Yuan Luoandluoyuan.luo c30ebb9300 [VLM] Optimize async mm data process mechanism (#12066) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-11-01 01:24:53 +08:00
Yuan Luoandluoyuan.luo 17a57fd862 [Perf] Optimize multimodal mm_inputs process in scheduler (#11910) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-10-30 10:48:10 +08:00
Yuan Luoluoyuan.luohuangtingwei
 		813bd6f85c [2/2] Use moe_sum_reduce cuda kernel (#10654) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: huangtingwei <141888744+huangtingwei9988@users.noreply.github.com>
 2025-10-28 12:01:57 +08:00
Yuan Luoandluoyuan.luo f389f01714 Optimize triton_mrope with torch compile (#12112) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-10-27 23:49:22 +08:00
Yuan Luoandluoyuan.luo 7ef5d8afd4 Revise POINTSV15Chat model (#12049) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-10-24 17:09:45 +08:00
Yuan Luoluoyuan.luo羽癫b8zhong
 		74de76c685 Revise MRotaryEmbedding's forward (#11859) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: 羽癫 <yudian.zy@antgroup.com>
Co-authored-by: b8zhong <b8zhong@uwaterloo.ca>
 2025-10-21 10:38:29 +08:00
Yuan Luoluoyuan.luob8zhong
 		271d3d0d50 Support mrope triton kernel and add unit test (#11722) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: b8zhong <b8zhong@uwaterloo.ca>
 2025-10-20 11:51:07 +08:00
Yuan Luoandluoyuan.luo 0b6f535f66 [Reland] perf: optimize qwen-vl with symm mem allreduce (#11457) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-10-13 17:51:25 +08:00
Yuan Luoandluoyuan.luo 3b9d97f335 perf: optimize qwen-vl with symm mem allreduce (#11381) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-10-10 22:24:45 +08:00
Yuan Luoandluoyuan.luo b5044fbf12 Replace pad with cat for better performance (#11388) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-10-10 12:03:17 +08:00
Yuan Luoandluoyuan.luo 4f42c8cd3e [sgl-kernel] Support float64 moe_sum_reduce cuda kernel (#11068) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-10-07 14:31:11 +00:00
Yuan Luoandluoyuan.luo 590f2da052 [Feat] Support Torch Symm Mem AllReduce (#10571) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-10-05 13:55:19 -07:00
Yuan Luoluoyuan.luoPGFLMG
 		42245551ef [sgl-kernel] Optimize concat_mla_k kernel (#10543) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: PGFLMG <1106310035@qq.com>
 2025-09-28 23:04:22 +08:00
Yuan Luoandluoyuan.luo a5095d6262 Fuse write kv buffer into rope for qwen3 moe & bailing moe (#10749) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-09-26 15:18:41 +08:00
Yuan Luoluoyuan.luoXiaoyu Zhang
 		616a3e20df [sgl-kernel] Support moe_sum_reduce cuda kernel (#10321) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
 2025-09-19 14:12:09 +08:00
Yuan Luoluoyuan.luo羽癫
 		24dc2bee97 Fix Bailing MoE model bugs (#10362) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: 羽癫 <yudian.zy@antgroup.com>
 2025-09-12 00:36:02 -07:00
Yuan Luoluoyuan.luoXiaoyu Zhang
 		cb3918a091 Optimize moe_sum_reduce_kernel (#9477) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
 2025-09-07 09:16:18 +08:00
Yuan Luoandluoyuan.luo ec15c8360e Optimize Qwen3-moe model by using flashinfer fused allreduce (#9973) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-09-04 20:48:53 +08:00
Yuan Luoandluoyuan.luo 968e181826 Fix triton_fused_moe unit test and benchmark (#9276) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-08-18 00:54:33 -07:00
Yuan Luoandluoyuan.luo 53dcc750b6 [sgl-kernel] Support FlashInfer top_k_top_p_sampling_from_logits (#9060) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-08-14 10:56:36 -07:00
Yuan Luoandluoyuan.luo 432f2053dd [sgl-kernel] 1/N Refactor sglang cutlass 3x - gemm fp8 blockwise sm90 (#8913) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-08-14 10:55:54 -07:00
Yuan Luo 1bd5316873 fix benchmark fp8 blockwise group gemm (#8815) 2025-08-06 21:02:21 +08:00
Yuan Luoandluoyuan.luo 3b87a9e8ae Fix bug of refactoring TopKOutput in w4afp8 (#8745) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-08-03 20:05:02 -07:00
Yuan Luoluoyuan.luoXiaoyu ZhangJieXin Liang
 		3bdcdd134b [Hot-Fix] moe_aligned_block_size CI failed in AMD (#8461) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
Co-authored-by: JieXin Liang <Alcanderian@users.noreply.github.com>
 2025-07-31 00:28:32 +08:00
Yuan Luoluoyuan.luoCheng WanCheng Wan
 		b3eac168e7 Support triton kernels v3.4.0 for fused_moe (#8258) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: Cheng Wan <cwan@x.ai>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
 2025-07-27 02:28:49 -07:00
Yuan Luoandluoyuan.luo 0c8dab9e67 [sgl-kernel] Opt per_token_quant_fp8 with warp reduce (#8130) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-07-23 21:22:59 +08:00
Yuan Luo af1cc8fe2d [kernel] opt moe align block kernel by block/warp scan algorithm (#7884) 2025-07-17 19:33:02 +08:00
Yuan Luoandluoyuan.luo 253454de9b Integrate triton moe kernel (#7689) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-07-06 20:05:49 -07:00
Yuan Luoandluoyuan.luo 84727a5139 [sgl-kernel] Add cuda kernel for moe_ep_silu_and_mul (#6919) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-06-11 20:43:08 -07:00
Yuan Luoandluoyuan.luo 43baba649e [EP] Add cuda kernel for moe_ep_post_reorder (#6837) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-06-05 00:33:47 -07:00
Yuan Luoandluoyuan.luo 55444ed667 [EP] Add cuda kernel for moe_ep_pre_reorder (#6699) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-06-01 20:49:01 -07:00
Yuan Luoandluoyuan.luo c087ddd686 Refine pre_reorder_triton_kernel slightly to improve performance (#6627) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-05-28 00:15:23 -07:00
Yuan Luoandluoyuan.luo 121f92c583 Add main for merge state tests (#6492) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-05-21 21:56:25 -07:00
Yuan Luoandluoyuan.luo 30ca18f423 Refactor group_concurrent_contiguous in NIXL (#6214) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-05-21 11:55:04 +08:00
Yuan Luoandluoyuan.luo 67b7d5b1df [PD] Vectorise group_concurrent_contiguous in NumPy (#5834) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-05-01 22:42:37 +08:00
Yuan Luoandluoyuan.luo d06a83fb01 Support dynamic connection and TP 16 (#5351) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-04-15 17:08:07 +08:00