minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
12,775 Commits 3 Branches 0 Tags
062f6f7ae8ea24e06b5e4d035e8abe27c4aa7360
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Yuan Luoandluoyuan.luo 28758d37dd Add FlashInfer SM90 cutlass MXFP4 MoE backend (W4A16) for GPT-OSS + DeepSeek-V4 (#24816) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-05-13 14:53:18 -07:00
Yuan Luoandluoyuan.luo d3fd91ed97 [Gemma4] Optimize Gemm4 with fused Q/K/V RMSNorm + per-expert FP8 ckpt loader (#24696) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-05-10 00:24:12 -07:00
Yuan Luoandluoyuan.luo a61a14f416 [KDA] Optimize prefill kernels with diagonal and recompute fuse (#24271) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-05-09 08:52:51 +08:00
Yuan Luoandluoyuan.luo e5c58eb9d6 [VLM] Optimize Gemma4 VLM with PCG and fuse RMSNorm + residual add + scalar (#24048) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-05-04 09:36:26 -07:00
Yuan Luoandluoyuan.luo 48daa831ea [KDA] Fuse gate+cumsum and reuse chunk index for KDA (#23038) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-04-21 17:54:20 +08:00
Yuan Luoandluoyuan.luo 03a87068ea [KDA] Fuse scaled_dot_kkt + solve_tril + recompute_w_u for KDA (#21604) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-03-31 20:57:27 -07:00
Yuan Luoandluoyuan.luo c7adca9992 Fix kimi-linear launch server error (#21752) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-03-31 21:07:08 +08:00
Yuan Luoandluoyuan.luo 343a7ac652 [GDN] Fuse GDN kkt + solve_tril into one kernel (#21411) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-03-29 12:02:07 +08:00
Yuan Luo ee15c104ef [CI] hot-fix ci lint (#21608) 2026-03-28 21:32:39 +08:00
Yuan Luoandluoyuan.luo f273ba1ccc [KDA] Support CuTeDSL KDA decode kernel (#21203) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-03-25 09:47:09 +08:00
Yuan Luoandluoyuan.luo 5bdc07d974 [Qwen3.5] Fuse split/reshape/cat ops in GDN projection with Triton kernel (#21019) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-03-23 23:17:01 +08:00
Yuan Luoandluoyuan.luo d9794ef9f7 [Qwen3-Next] Fuse Qwen3-Next GDN's qkvz_proj and ba_proj (#19321) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-03-20 09:25:29 +08:00
Yuan Luo 9c87e137ee [GDN] Support GDN packed decode (#20627) 2026-03-18 13:20:07 +08:00
Yuan Luoandluoyuan.luo 22e67876d6 [Omni] Optimize AudioEncoder for Qwen3_Omni_Thinker (#18185) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-03-14 23:00:17 +08:00
Yuan Luoluoyuan.luoKaixi Hou
 		e29305c120 [GDN] Add benchmark for sglang gdn prefill (#20428) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: Kaixi Hou <kaixih@nvidia.com>
 2026-03-12 22:25:02 +08:00
Yuan Luo 649d6f2bc8 [GDN] Change Attention State Layout from [N, HV, K, V] to [N, HV, V, K] (#20283) 2026-03-12 10:53:12 +08:00
Yuan Luoandluoyuan.luo 751c454099 Add DeepSeek3.2 and GlmMoeDsa into moe tune (#18876) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-03-10 17:12:58 +08:00
Yuan Luoandluoyuan.luo 97a2a9be0f [VLM] Replace conv3d proj with linear for GLM4V (#20033) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-03-07 22:50:47 -08:00
Yuan Luoandluoyuan.luo 7da590d4d0 [Qwen3.5] Support Qwen3.5 Pipeline Parallelism (#19670) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-03-07 23:34:08 +08:00
Yuan Luo f7de9375ac [GDN][Qwen3-Next][Qwen3.5] Fuse fused_gdn_gating and fused_recurrent_gated_delta_rule_update in verify_target (#19775) 2026-03-06 21:42:44 +08:00
Yuan Luoandluoyuan.luo 82e7139c06 [VLM] Support cos sin cache for Ernie4.5-VL (#19743) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-03-04 10:54:23 +08:00
Yuan Luoandluoyuan.luo f6ee6dc8c3 [JIT-kernel] Add unit test for nsa indexer fused_store_k_cache (#19389) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-03-02 12:18:11 +08:00
Yuan Luoandluoyuan.luo d2885a9094 [Qwen3-Next] Support gdn fused_rms_norm_gated (#19434) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-02-27 23:08:08 +08:00
Yuan Luoluoyuan.luoDarkSharpness
 		4e843f1216 [DeepSeek-V3.2][JIT-kernel] Support nsa fuse store indexer k cache (#19148) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: DarkSharpness <76582120+darksharpness@users.noreply.github.com>
 2026-02-26 10:23:10 +08:00
Yuan Luoluoyuan.luoMohammad Miadh Angkad
 		7c9e8e2def [Re-land][jit kernel] Support per_token_group_quant_8bit jit kernel (#19140) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: Mohammad Miadh Angkad <mangkad.bsdsba2027@aim.edu>
 2026-02-26 09:53:57 +08:00
Yuan Luoandluoyuan.luo 31c7dc9d99 [VLM] Introduce FlashInfer CUDNN Prefill as ViT Backend (#19003) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-02-24 19:49:22 +08:00
Yuan Luoandluoyuan.luo 7d953440ec [jit kernel] Support per_token_group_quant_8bit jit kernel (#18905) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-02-20 21:01:05 +08:00
Yuan Luoandluoyuan.luo 8a82c70297 [VLM] Optimize Ernie4.5-VL rotary embedding with fused triton kernel (#18856) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-02-16 11:19:44 +08:00
Yuan Luoandluoyuan.luo fa0ef6e4f7 [VLM][LLM] Optimize fused_moe triton kernel tma (#18782) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-02-14 14:35:26 +08:00
Yuan Luoandluoyuan.luo 9227d4f748 [Fix] Remove no use code in MiMo-V2-Flash (#18051) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-02-01 15:34:09 -08:00
Yuan Luoandluoyuan.luo afebb7ab78 Optimize custom-all-reduce (#17674) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-02-01 18:59:31 +08:00
Yuan Luoandluoyuan.luo 4ea4f2a20c [VLM] Optimize get_rope_index for GLM4v (#17420) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-02-01 18:59:15 +08:00
Yuan Luoandluoyuan.luo 7bb41989fa [1/N] Optimize All Reduce - Benchmark different AR operations (#13797) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-01-26 22:44:13 +08:00
Yuan Luo 1e8db18290 [Kimi-Linear] Remove duplicated code in kimi-linear (#17731) 2026-01-26 14:20:24 +08:00
Yuan Luoandluoyuan.luo 0c8165ffbd [Kimi-Linear] Refactor Kimi-Linear to support RadixLinearAttention (#17506) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-01-24 21:27:13 +08:00
Yuan Luoandluoyuan.luo e6b7c04947 [Kimi-Linear] Refactor kimi-linear gate calculation to avoid duplicated code (#17160) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-01-20 14:29:24 +08:00
Yuan Luoandluoyuan.luo 6d29d8ab16 [VLM][Reland] Refactor load_mm_data to improve performance (#16152) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-01-18 14:11:17 +08:00
Yuan Luoandluoyuan.luo 969faaa410 [diffusion] fix: revise fa4 backend to support blackwell (#17077) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-01-14 23:31:46 +08:00
Yuan Luo feae615b11 [VLM] Support ViT CUDA Graph for InternVL (#16732) 2026-01-14 17:29:23 +08:00
Yuan Luoandluoyuan.luo d1ec93e3ac Optimize layernorm_gated for Qwen3-Next (#16397) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-01-10 20:55:31 +08:00
Yuan Luoandluoyuan.luo ad20127359 [CI] Remove duplicate code in test_mamba_ut (#16854) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-01-10 16:16:56 +08:00
Yuan Luoandluoyuan.luo 08636f72b5 [Fix CI] Fix test_mamba_unittest.py (#16810) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-01-09 15:03:47 -08:00
Yuan Luoandluoyuan.luo 53846746bf [VLM] Fix CUDA IPC OOM (#16118) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-01-07 11:30:35 +08:00
Yuan Luoandluoyuan.luo 5f3eb377e0 [VLM] Support request level max_dynamic_patch for OpenAI request (#16268) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-01-04 13:04:43 +08:00
Yuan Luoandluoyuan.luo 9bd64d739b [VLM] Add doc for ViT CUDA Graph (#16343) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-01-04 10:05:23 +08:00
Yuan Luoandluoyuan.luo 3a42c5e341 [VLM] Adopt jit qk_norm kernel in VLM (#16171) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-01-01 10:10:36 +08:00
Yuan Luoandluoyuan.luo 94bcc19bce [VLM] Support Video for InternVL3_5 (#15942) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-12-30 17:07:49 +08:00
Yuan Luoandluoyuan.luo 086813ae8a [VLM] refactor: refactor load_mm_data to improve performance (#14644) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-12-26 13:17:53 +08:00
Yuan Luoandluoyuan.luo b9af8d2eb9 [VLM] Support apply qk norm in multi cuda streams (#15720) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-12-25 14:35:07 +08:00
Yuan Luoandluoyuan.luo 828dec1c7c [VLM] Tiny: Unify VLM environment variables (#15572) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-12-22 17:32:37 +08:00
Yuan Luoandluoyuan.luo 050f108c29 Optimize Bailing-MoE with FlashInfer Fused All-Reduce (#15526) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-12-20 17:34:04 -08:00
Yuan Luoandluoyuan.luo 019517a356 [VLM] Support ViT Piecewise CUDA Graph for Qwen3-VL (#15320) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-12-20 21:00:07 +08:00
Yuan Luoandluoyuan.luo 165f5c04cb Optimize MiMo-V2-Flash by flashinfer fused allreduce (#15464) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-12-20 17:45:18 +08:00
Yuan Luoandluoyuan.luo 8fa3dc36c5 [VLM] Support cos sin cache for Qwen3-VL & GLM-4.1V (#15205) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-12-18 08:39:50 +08:00
Yuan Luoandluoyuan.luo 3912ee4991 [VLM] feat: support chunked vit attention (#14907) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-12-15 12:11:02 +08:00
Yuan Luoluoyuan.luokousakawang
 		3b8a824b8b [VLM] Support VLM ViT Piecewise CUDA Graph (#14422) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: kousakawang <wanghanpei@bytedance.com>
 2025-12-13 20:49:40 +08:00
Yuan Luoandluoyuan.luo 03836d85d2 [GLM-4.6V] Support Pipeline Parallelism for GLM-4.6V & GLM-4.1V (#14720) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-12-10 16:40:12 +08:00
Yuan Luoandluoyuan.luo 26d95008b6 [apply][2/2] Fused qk_norm_rope for Qwen3-MoE (#13998) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-12-07 20:25:18 +08:00
Yuan Luoandluoyuan.luo b2b09f5f24 [VLM] Introduce Cache for positional embedding ids for Qwen-VL family (#14292) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-12-04 12:32:00 +08:00
Yuan Luoandluoyuan.luo 21ec99beff [VLM][Doc] Document for VLM DP Encoder (#14279) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-12-02 15:08:37 +08:00
Yuan Luoandluoyuan.luo 26aebf83d3 [VLM] Support Piecewise CUDA Graph for Qwen3-Omni-MOE (#14222) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-12-02 10:12:10 +08:00
Yuan Luoandluoyuan.luo 630a693081 [VLM] Boost Memory Pool based CUDA IPC (#14123) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-12-01 17:17:46 +08:00
Yuan Luoandluoyuan.luo e12c78aab6 [sgl-kernel][1/2] Fused qk_norm_rope for Qwen3-MoE (#14036) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-11-28 12:25:15 +08:00
Yuan Luoandluoyuan.luo ca5c8b16f6 [VLM] Support InternVL Vision Encoder Data Parallelism (#13925) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-11-26 11:43:05 +08:00
Yuan Luoandluoyuan.luo 8ef11569a2 [VLM] Revise InternVL Piecewise CUDA Graph Supporting (#13846) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-11-24 22:15:10 +08:00
Yuan Luoluoyuan.luoXu Yongfei
 		f56b9b42e6 [Bugfix] Add jit kernel files in packaging (#13829) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: Xu Yongfei <xuyongfei.xyf@antgroup.com>
 2025-11-24 12:32:16 +08:00
Yuan Luoandluoyuan.luo 5625e32cae [VLM] Replace torch.repeat_interleave with faster np.repeat for Qwen-VL series (#13736) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-11-22 22:45:32 +08:00
Yuan Luoandluoyuan.luo 475962a139 [VLM] Support Piecewise CUDA Graph for InternVL (#13640) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-11-21 13:21:51 +08:00
Yuan Luoluoyuan.luoYuhao Yang
 		af6bcadcf7 [VLM] Support Piecewise CUDA Graph for Qwen2.5-VL (#13055) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: Yuhao Yang <yhyang201@gmail.com>
 2025-11-20 10:23:44 +08:00
Yuan Luoandluoyuan.luo f0021c0dc8 Add feature flag for mm inputs processing optimization (#13278) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-11-15 14:43:44 +08:00
Yuan Luoluoyuan.luoTianyu Guo
 		706502ff6c [VLM] Support PP for Qwen2.5-VL (#13075) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
 2025-11-12 23:18:44 +08:00
Yuan Luoluoyuan.luo羽癫
 		fd3034da75 [VLM] Optimize qwen_vl preprocess_video (#12240) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: 羽癫 <yudian.zy@antgroup.com>
 2025-11-06 14:55:01 +08:00
Yuan Luoandluoyuan.luo bb517fe393 [HotFix] Disable torch dynamo for mrope_triton kernel (#12593) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-11-04 23:26:56 +08:00
Yuan Luoandluoyuan.luo 819fc59123 Add prefix for torch symm mem (#12506) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-11-02 11:23:05 -08:00
Yuan Luoandluoyuan.luo c30ebb9300 [VLM] Optimize async mm data process mechanism (#12066) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-11-01 01:24:53 +08:00
Yuan Luoandluoyuan.luo 17a57fd862 [Perf] Optimize multimodal mm_inputs process in scheduler (#11910) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-10-30 10:48:10 +08:00
Yuan Luoluoyuan.luohuangtingwei
 		813bd6f85c [2/2] Use moe_sum_reduce cuda kernel (#10654) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: huangtingwei <141888744+huangtingwei9988@users.noreply.github.com>
 2025-10-28 12:01:57 +08:00
Yuan Luoandluoyuan.luo f389f01714 Optimize triton_mrope with torch compile (#12112) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-10-27 23:49:22 +08:00
Yuan Luoandluoyuan.luo 7ef5d8afd4 Revise POINTSV15Chat model (#12049) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-10-24 17:09:45 +08:00
Yuan Luoluoyuan.luo羽癫b8zhong
 		74de76c685 Revise MRotaryEmbedding's forward (#11859) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: 羽癫 <yudian.zy@antgroup.com>
Co-authored-by: b8zhong <b8zhong@uwaterloo.ca>
 2025-10-21 10:38:29 +08:00
Yuan Luoluoyuan.luob8zhong
 		271d3d0d50 Support mrope triton kernel and add unit test (#11722) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: b8zhong <b8zhong@uwaterloo.ca>
 2025-10-20 11:51:07 +08:00
Yuan Luoandluoyuan.luo 0b6f535f66 [Reland] perf: optimize qwen-vl with symm mem allreduce (#11457) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-10-13 17:51:25 +08:00
Yuan Luoandluoyuan.luo 3b9d97f335 perf: optimize qwen-vl with symm mem allreduce (#11381) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-10-10 22:24:45 +08:00
Yuan Luoandluoyuan.luo b5044fbf12 Replace pad with cat for better performance (#11388) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-10-10 12:03:17 +08:00
Yuan Luoandluoyuan.luo 4f42c8cd3e [sgl-kernel] Support float64 moe_sum_reduce cuda kernel (#11068) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-10-07 14:31:11 +00:00
Yuan Luoandluoyuan.luo 590f2da052 [Feat] Support Torch Symm Mem AllReduce (#10571) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-10-05 13:55:19 -07:00
Yuan Luoluoyuan.luoPGFLMG
 		42245551ef [sgl-kernel] Optimize concat_mla_k kernel (#10543) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: PGFLMG <1106310035@qq.com>
 2025-09-28 23:04:22 +08:00
Yuan Luoandluoyuan.luo a5095d6262 Fuse write kv buffer into rope for qwen3 moe & bailing moe (#10749) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-09-26 15:18:41 +08:00
Yuan Luoluoyuan.luoXiaoyu Zhang
 		616a3e20df [sgl-kernel] Support moe_sum_reduce cuda kernel (#10321) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
 2025-09-19 14:12:09 +08:00
Yuan Luoluoyuan.luo羽癫
 		24dc2bee97 Fix Bailing MoE model bugs (#10362) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: 羽癫 <yudian.zy@antgroup.com>
 2025-09-12 00:36:02 -07:00
Yuan Luoluoyuan.luoXiaoyu Zhang
 		cb3918a091 Optimize moe_sum_reduce_kernel (#9477) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
 2025-09-07 09:16:18 +08:00
Yuan Luoandluoyuan.luo ec15c8360e Optimize Qwen3-moe model by using flashinfer fused allreduce (#9973) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-09-04 20:48:53 +08:00
Yuan Luoandluoyuan.luo 968e181826 Fix triton_fused_moe unit test and benchmark (#9276) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-08-18 00:54:33 -07:00
Yuan Luoandluoyuan.luo 53dcc750b6 [sgl-kernel] Support FlashInfer top_k_top_p_sampling_from_logits (#9060) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-08-14 10:56:36 -07:00
Yuan Luoandluoyuan.luo 432f2053dd [sgl-kernel] 1/N Refactor sglang cutlass 3x - gemm fp8 blockwise sm90 (#8913) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-08-14 10:55:54 -07:00
Yuan Luo 1bd5316873 fix benchmark fp8 blockwise group gemm (#8815) 2025-08-06 21:02:21 +08:00
Yuan Luoandluoyuan.luo 3b87a9e8ae Fix bug of refactoring TopKOutput in w4afp8 (#8745) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-08-03 20:05:02 -07:00
Yuan Luoluoyuan.luoXiaoyu ZhangJieXin Liang
 		3bdcdd134b [Hot-Fix] moe_aligned_block_size CI failed in AMD (#8461) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
Co-authored-by: JieXin Liang <Alcanderian@users.noreply.github.com>
 2025-07-31 00:28:32 +08:00
Yuan Luoluoyuan.luoCheng WanCheng Wan
 		b3eac168e7 Support triton kernels v3.4.0 for fused_moe (#8258) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: Cheng Wan <cwan@x.ai>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
 2025-07-27 02:28:49 -07:00
Yuan Luoandluoyuan.luo 0c8dab9e67 [sgl-kernel] Opt per_token_quant_fp8 with warp reduce (#8130) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2025-07-23 21:22:59 +08:00