 Xiaoyu ZhangandKe Bao
|
7a4309cc8a
|
[sgl-kernel performace] fix fp8 quant kernels dispatch __nv_fp8_e4m3 bug to improve performance 10%-20% (#8499)
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
|
2025-07-29 23:31:54 +08:00 |
|
|
Xiaoyu Zhang
|
2262369905
|
Revert "[kernel] opt moe align block kernel by block/warp scan algorithm" (#8457)
|
2025-07-28 01:35:43 -07:00 |
|
|
Xiaoyu Zhang
|
9045cc1eb8
|
[torch.compile bug] avoid biased_grouped_topk_impl func repeatedly triggering torch.compile in forward pass (#8353)
|
2025-07-25 21:17:47 +08:00 |
|
|
Xiaoyu Zhang
|
a167fd0bcb
|
[code style] Clean dead triton kernel code in fused_moe and useless vllm_ops import (#8310)
|
2025-07-24 14:38:30 +08:00 |
|
|
Xiaoyu Zhang
|
aa2056091a
|
delete uselese code caused by fuse allreduce+add_rmsnorm pr (#7970)
|
2025-07-11 19:43:38 -07:00 |
|
|
Xiaoyu Zhang
|
49a5915f53
|
[ready b200] fuse allreduce+add_rmsnorm in prepare_attention + mlp module (#7775)
|
2025-07-10 15:12:39 -07:00 |
|
|
Xiaoyu Zhang
|
2e7ab862e3
|
Fix illegal memory in trtllm allreduce fusion (#7864)
|
2025-07-08 11:47:17 -07:00 |
|
|
Xiaoyu Zhang
|
8e64140e35
|
[b200] support trt-llm allreduce fuse rms_norm_add kernel (#7621)
|
2025-07-02 19:36:20 -07:00 |
|
|
Xiaoyu Zhang
|
ff2e9c9479
|
Add small requirements for benchmark/parse_result tools (#7671)
|
2025-06-30 21:52:20 -07:00 |
|
|
Xiaoyu Zhang
|
8ecad0b16f
|
[benchmark] fbgemm benchmark support bandwidth report and support fbgemm_cutlass_gmm (#7422)
|
2025-06-24 09:44:55 -07:00 |
|
|
Xiaoyu Zhang
|
0ae1e9a755
|
refine fused_moe benchmark (#7221)
|
2025-06-15 21:21:32 -07:00 |
|
|
Xiaoyu Zhang
|
3712abfaf9
|
Fuse routed scaling factor in deepseek (#6970)
|
2025-06-08 15:24:24 -07:00 |
|
|
Xiaoyu Zhang
|
fa3592cfeb
|
rebase h20 fused_moe config (#6966)
|
2025-06-08 05:01:34 -07:00 |
|
|
Xiaoyu Zhang
|
515ef4facb
|
Fuse routed scaling factor in topk_reduce kernel (#6220)
|
2025-06-07 11:06:50 -07:00 |
|
|
Xiaoyu Zhang
|
bae4fdc7ab
|
add fbgemm moe grouped gemm kernel benchmark (#6924)
|
2025-06-07 02:57:30 -07:00 |
|
|
Xiaoyu Zhang
|
8b5f83ed3b
|
reduce torch.zeros overhead in moe align block size kernel (#6369)
|
2025-06-07 02:47:36 -07:00 |
|
|
Xiaoyu Zhang
|
2a413829f4
|
Add triton version as a fused_moe_triton config search key to avoid performace decrease in different Triton version (#5955)
|
2025-06-07 02:43:50 -07:00 |
|
|
Xiaoyu ZhangandJieXin Liang
|
bd75690f4e
|
fix ep_moe_reorder kernel bugs (#6858)
Co-authored-by: JieXin Liang <Alcanderian@users.noreply.github.com>
|
2025-06-04 19:13:59 +08:00 |
|
|
Xiaoyu Zhang
|
076103535c
|
fix log_info_on_rank0 error when run benchmark (#6260)
|
2025-05-28 00:20:01 -07:00 |
|
|
Xiaoyu Zhang
|
9f2c9568f0
|
[doc] add a note for --n-share-experts-fusion args (#6154)
|
2025-05-11 23:18:38 -07:00 |
|
|
Xiaoyu Zhang
|
d25398cbc8
|
fix custom_allreduce namespace (#6039)
|
2025-05-06 19:13:06 -07:00 |
|
|
Xiaoyu Zhang
|
5bb0accbcf
|
cutlass 3.9 supported to improve fp8_blockwise_gemm (#5820)
|
2025-04-28 21:52:36 -07:00 |
|
|
Xiaoyu Zhang
|
1cc326032d
|
simplify fused_moe config logging (#5801)
|
2025-04-28 17:04:54 -07:00 |
|
|
Xiaoyu Zhang
|
e132cba2a8
|
fused moe triton tuning script support qwen3 (#5842)
|
2025-04-28 09:13:04 -07:00 |
|
|
Xiaoyu Zhang
|
ef15dcda26
|
Add a doc to fix sgl-kernel build link error in py39 with ccache (#5809)
|
2025-04-27 21:34:27 -07:00 |
|
|
Xiaoyu Zhang
|
ad4df30741
|
Dockerfile.dev pip scikit_build_core (#5807)
|
2025-04-27 21:14:20 -07:00 |
|
|
Xiaoyu Zhang
|
18ce468d56
|
update triton 3.2.0 h200 fused moe triton config and add warning about triton fused_moe_kernel performance degradation due to different Triton versions. (#5740)
|
2025-04-25 16:24:59 -07:00 |
|
|
Xiaoyu Zhang
|
d9dd529854
|
enable DeepSeek V3 shared_experts_fusion in sm90 (#5571)
|
2025-04-20 12:46:42 -07:00 |
|
|
Xiaoyu Zhang
|
d58e354472
|
simplify the control logic for using shared experts fusion (#5504)
|
2025-04-19 13:17:35 -07:00 |
|
|
Xiaoyu Zhang
|
bf86c5e990
|
restruct compressed_tensors_w8a8_fp8 (#5475)
|
2025-04-19 04:52:15 -07:00 |
|
|
Xiaoyu Zhang
|
bed05878f6
|
fix kimi vl running bug after rebase main (#5461)
|
2025-04-18 00:17:34 -07:00 |
|
|
Xiaoyu Zhang
|
8e09b37077
|
Sgl kernel fused_moe_gate support n_shared_experts (#5440)
|
2025-04-17 23:05:15 -07:00 |
|
|
Xiaoyu Zhang
|
06a1656e02
|
[doc] Update benchmark_and_profiling.md (#5449)
|
2025-04-15 23:27:34 -07:00 |
|
|
Xiaoyu ZhangandYineng Zhang
|
38076dea84
|
apply fused moe gate in ds v3/r1 (#5371)
Co-authored-by: Yineng Zhang <me@zhyncs.com>
|
2025-04-14 16:24:26 -07:00 |
|
|
Xiaoyu Zhang
|
3e4794aad8
|
refine fused_moe tuning docs (#5294)
|
2025-04-12 10:01:13 -07:00 |
|
|
Xiaoyu Zhang
|
690ec20587
|
Delete python/sglang/srt/layers/moe/fused_moe_triton/configs/E=257,N=… (#5321)
|
2025-04-12 10:00:03 -07:00 |
|
|
Xiaoyu Zhang
|
60bcbf2a35
|
remove moe_align_block_size torch.zeros in small batch/expert mode (#5298)
|
2025-04-11 12:13:55 -07:00 |
|
|
Xiaoyu Zhang
|
f730362ee2
|
reduce moe_align_block_size_kernel small batch mode overhead (#5086)
|
2025-04-09 17:59:35 -07:00 |
|
|
Xiaoyu Zhang
|
87eddedfa2
|
[ci] fix ci test fused_moe op (#5102)
|
2025-04-09 08:52:46 -07:00 |
|
|
Xiaoyu Zhang
|
db452760e5
|
[ci] fix llama4 ci error (#5126)
|
2025-04-07 21:15:46 +08:00 |
|
|
Xiaoyu Zhang
|
924ca7c92c
|
Add DeepSeek V3/R1 shared experts fusion (#4918)
|
2025-04-04 01:59:29 -07:00 |
|
|
Xiaoyu Zhang
|
772d2a191d
|
try to fix ci oserror (#5024)
|
2025-04-03 02:45:05 -07:00 |
|
|
Xiaoyu Zhang
|
2c8fd99363
|
[sgl-kernel] per token group quant support COLUMN MAJOR (#4817)
|
2025-04-02 18:29:59 -07:00 |
|
|
Xiaoyu Zhang
|
e9c6ce461d
|
sgl scaled_fp8_quant support output padding (#4861)
|
2025-04-02 23:53:57 +08:00 |
|
|
Xiaoyu Zhang
|
04e3ff6975
|
Support compressed tensors fp8w8a8 (#4743)
|
2025-03-26 13:21:25 -07:00 |
|
|
Xiaoyu Zhang
|
804d250a0d
|
remove useless backend forward in rotary_embedding (#4500)
|
2025-03-17 23:54:00 -07:00 |
|
|
Xiaoyu Zhang
|
dd865befde
|
[Hotfix] solve fp8 w8a8 ci test fail (#4531)
|
2025-03-17 23:17:04 -07:00 |
|
|
Xiaoyu Zhang
|
9b81f9bd34
|
sglang quant module remove vllm dependency (#4507)
|
2025-03-17 15:51:59 -07:00 |
|
|
Xiaoyu Zhang
|
7130a7cea9
|
refine sgl_moe_align_block_size_benchmark (#4327)
|
2025-03-11 22:48:38 -07:00 |
|
|
Xiaoyu Zhang
|
23308a9032
|
fix per_token_group_quant_fp8 illegal memory when num_groups % 16 != 0 (#4231)
|
2025-03-10 01:42:58 -07:00 |
|
|
Xiaoyu Zhang
|
79a321af55
|
revert pr 3628 to pass test_mla ci (#4219)
|
2025-03-08 21:15:14 -08:00 |
|
|
Xiaoyu Zhang
|
b3251e9f40
|
refine quant kernel code style (#4211)
|
2025-03-08 05:47:35 -08:00 |
|
|
Xiaoyu Zhang
|
ad55f17182
|
[quant kernel] sgl-kernel support per_tensor_quant fp8 (#3786)
|
2025-03-06 18:05:43 -08:00 |
|
|
Xiaoyu Zhang
|
50f28f65a0
|
fix typo in deep gemm benchmarking(#3991)
|
2025-03-02 00:34:00 -08:00 |
|
|
Xiaoyu Zhang
|
90a55e2566
|
add deepgemm and sglang fp8 block-wise gemm benchmark (#3893)
|
2025-03-01 23:01:58 -08:00 |
|
|
Xiaoyu Zhang
|
55a7ec388f
|
use warp shuffle style reduce and flashinfer vectorize (#3628)
|
2025-02-19 20:53:51 +08:00 |
|
|
Xiaoyu Zhang
|
c38f3aed24
|
support multi-gpu block-gemm tuning (#3639)
|
2025-02-18 00:00:35 +08:00 |
|
|
Xiaoyu Zhang
|
3efbdf68b9
|
fix sgl-kernel codestyle (#3563)
|
2025-02-14 18:05:52 +08:00 |
|
|
Xiaoyu Zhang
|
f076328bb7
|
fix moe_align_kernel shm init not sync bug (#3534)
|
2025-02-13 16:47:00 +08:00 |
|
|
Xiaoyu Zhang
|
45e3a7bc41
|
use sgl_per_token_group_quant_fp8 kernel (#3493)
|
2025-02-12 18:40:42 +08:00 |
|
|
Xiaoyu Zhang
|
693c2600e0
|
refine deepseek_v3 launch server doc (#3522)
|
2025-02-12 17:27:07 +08:00 |
|
|
Xiaoyu Zhang
|
bb418ced80
|
optimize per token group quant fp8 (#3490)
|
2025-02-11 22:19:05 +08:00 |
|
|
Xiaoyu Zhang
|
2f47d710ae
|
refine some typo (#3473)
|
2025-02-10 23:35:44 +08:00 |
|
|
Xiaoyu Zhang
|
cdae77b03d
|
optimize moe_align_kernel cuda (#3347)
|
2025-02-07 00:53:46 +08:00 |
|
|
Xiaoyu Zhang
|
ad3499858e
|
clean moe align block kernel code and add acc test (#3332)
|
2025-02-06 16:42:36 +08:00 |
|
|
Xiaoyu Zhang
|
3c8ac78dc1
|
optimize test_fused_moe style (#3268)
|
2025-02-03 18:56:18 +08:00 |
|
|
Xiaoyu Zhang
|
81262c7b72
|
clean up useless file (#3192)
|
2025-01-28 14:29:30 +08:00 |
|
|
Xiaoyu Zhang
|
5d9d15e70f
|
support fp32 in sampling_scaling_penalties kernel (#3121)
|
2025-01-25 16:52:17 +08:00 |
|
|
Xiaoyu Zhang
|
e0cd65c2b6
|
[hotfix] fix test_sampling_scaling_penalties.py ci test (#3084)
|
2025-01-24 00:33:59 +08:00 |
|
|
Xiaoyu Zhang
|
f1b6861828
|
use flashinfer vec_dtypes in sgl_kernel (#3083)
|
2025-01-23 22:19:04 +08:00 |
|
|
Xiaoyu Zhang
|
ac2dc35d0e
|
support lightning_attention_decode in sgl-kernel for MiniMax-Text-01 (#3030)
|
2025-01-23 15:29:20 +08:00 |
|
|
Xiaoyu Zhang
|
83452dbb4a
|
fix file name spelling mistake and useless variable in minmax-text-01-lightning_attention (#2971)
|
2025-01-18 18:56:13 -08:00 |
|
|
Xiaoyu Zhang
|
c2f212d672
|
optimize MiniMax-Text-01 lightning_attn_decode triton (#2966)
|
2025-01-18 23:41:01 +08:00 |
|
|
Xiaoyu Zhang
|
78e974b2a5
|
[kernel] MiniMax-Text-01 decode lightning_attn with triton (#2920)
|
2025-01-16 12:51:38 -08:00 |
|
|
Xiaoyu Zhang
|
ab31793661
|
[kernel] MiniMax-Text-01 prefill lightning_attn with triton (#2911)
|
2025-01-16 14:18:29 +08:00 |
|
|
Xiaoyu Zhang
|
f005758f2b
|
introduce CUB in sgl-kernel (#2887)
|
2025-01-14 19:48:59 +08:00 |
|
|
Xiaoyu Zhang
|
d08c77c434
|
Sampling penalties memory interface (#2870)
|
2025-01-13 23:09:00 +08:00 |
|
|
Xiaoyu Zhang
|
e2b16c4716
|
add sampling_scaling_penalties kernel (#2846)
|
2025-01-12 19:38:17 -08:00 |
|
|
Xiaoyu Zhang
|
f0e15dc6ab
|
[HotFix] fix fp8 scale load failed in tp>1 (#2837)
|
2025-01-11 14:34:26 +08:00 |
|
|
Xiaoyu Zhang
|
380930a959
|
add benchmark_moe_align_blocks (#2767)
|
2025-01-07 14:20:50 +08:00 |
|
|
Xiaoyu Zhang
|
ded9fcd09a
|
improve moe_align_kernel for deepseek v3 (#2735)
|
2025-01-06 00:28:22 +08:00 |
|
|
Xiaoyu Zhang
|
286cad3ee3
|
h200 tuning fused_moe_triton config for Mixtral 8x7B/8x22B and Qwen2 57BA14B (#2689)
|
2024-12-31 23:17:36 +08:00 |
|
|
Xiaoyu Zhang
|
9254a33ad4
|
avoid fused_moe_triton padding circular import (#2624)
|
2024-12-28 14:01:35 +08:00 |
|
|
Xiaoyu Zhang
|
9a23c48456
|
h100 tuning fused_moe_triton for qwen2 moe (#2560)
|
2024-12-26 03:13:31 -08:00 |
|
|
Xiaoyu Zhang
|
7d672d277b
|
[kernel optimize] benchmark write_req_to_token_pool_triton and optimize kernel (#2509)
|
2024-12-22 02:31:02 -08:00 |
|
|
Xiaoyu Zhang
|
a0592c059f
|
[Benchmark] add a benchmark for hf/vllm/sglang rmsnorm (#2486)
|
2024-12-15 13:52:08 +08:00 |
|
|
Xiaoyu Zhang
|
3844feb9bb
|
Add a unittest for fused_moe (#2416)
|
2024-12-08 22:46:10 -08:00 |
|
|
Xiaoyu Zhang
|
34b364e073
|
optimize cuda graph max_bs_settings on low-end gpus (#2360)
|
2024-12-06 01:13:04 -08:00 |
|
|
|
262e370f78
|
[benchmark] Add fused_moe_triton benchmark and tuning tools (#2225)
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: HAI <hixiao@gmail.com>
|
2024-11-29 13:36:45 -08:00 |
|
|
Xiaoyu Zhang
|
dd44173dad
|
[Fused moe] add tuning fused configs for qwen2 57b and mixtral 8x7b (#2167)
|
2024-11-25 10:37:50 +08:00 |
|
|
Xiaoyu Zhang
|
023d0a73df
|
fix small typos in docs (#2047)
|
2024-11-15 11:09:10 -08:00 |
|
|
Xiaoyu Zhang
|
eff468dd5a
|
fix test_embedding_models prompt length too long's bug (#2015)
|
2024-11-12 23:21:16 +08:00 |
|
|
Xiaoyu Zhang
|
a1bd719031
|
fix a bug in v1_embeeding_request (#2014)
|
2024-11-12 16:49:45 +08:00 |
|
|
Xiaoyu Zhang
|
027e65248f
|
support echo=true and logprobs in openai api when logprobs=1 in lm-evaluation-harness (#1998)
|
2024-11-11 23:21:20 -08:00 |
|
|
Xiaoyu Zhang
|
605972195b
|
check user-specified model_max_len with hf derived max_model_len (#1778)
|
2024-10-24 12:40:36 -07:00 |
|