 Yineng Zhang
|
df84ab2a5b
|
update sgl-kernel 3rdparty (#4228)
|
2025-03-09 01:16:05 -08:00 |
|
|
Yineng Zhang
|
89ccb533ad
|
use sgl-kernel 0.0.4 (#4224)
|
2025-03-08 23:43:09 -08:00 |
|
|
Yineng Zhang
|
5c7dd14ba1
|
chore: bump v0.0.4 for sgl-kernel (#4223)
|
2025-03-08 23:01:59 -08:00 |
|
|
Yineng Zhang
|
ee132a4515
|
use latest sgl-kernel for mla test (#4222)
|
2025-03-08 22:27:47 -08:00 |
|
|
Yineng Zhang
|
96d0e37fa7
|
Revert "Minor improvement to per_tensor_quant_fp8 (#4197)" (#4198)
|
2025-03-07 22:57:09 -08:00 |
|
|
Yineng Zhang
|
70866b6f4f
|
use same version for ci and pyproject (#4187)
|
2025-03-07 10:39:55 -08:00 |
|
|
Yineng Zhang
|
eb61f5c9af
|
Revert "ROCm: Flex Attention Enablement with custom backends (#4178)" (#4186)
|
2025-03-07 10:27:52 -08:00 |
|
|
Yineng Zhang
|
b55a621ffb
|
fix int8 doc link (#4179)
|
2025-03-07 02:49:19 -08:00 |
|
|
Yineng Zhang
|
7e3bb52705
|
update release-pypi-kernel
|
2025-03-07 01:48:47 -08:00 |
|
|
Yineng Zhang
|
96263f275c
|
chore: bump v0.0.3.post7 for sgl-kernel (#4176)
|
2025-03-07 01:15:34 -08:00 |
|
|
Yineng ZhangandStefan He
|
94a2b9d33e
|
Put utils in ifndef USE_ROCM to fix CI (#4167) (#4168)
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
|
2025-03-07 00:01:17 -08:00 |
|
|
Yineng Zhang
|
fc671f66c1
|
chore: bump v0.4.3.post3 (#4114)
|
2025-03-05 17:26:10 -08:00 |
|
|
Yineng Zhang
|
0aaccbbfec
|
revert deepseek docs (#4109)
|
2025-03-05 13:23:11 -08:00 |
|
|
Yineng Zhang
|
1baa9e6cf9
|
docs: update README (#4044)
|
2025-03-03 17:09:18 -08:00 |
|
|
Yineng Zhang
|
f3b99f73b3
|
update flashinfer-python version
|
2025-02-28 16:31:59 -08:00 |
|
|
Yineng Zhang
|
564bdf29f7
|
upgrade flashinfer v0.2.2.post1 (#3934)
|
2025-02-27 09:53:48 -08:00 |
|
|
Yineng Zhang
|
5d86016855
|
revert "Docs: Reorngaize dpsk links #3900" (#3933)
|
2025-02-27 08:57:13 -08:00 |
|
|
Yineng Zhang
|
fe0673f1cc
|
set NCCL_IB_GID_INDEX=3 for multi node NVIDIA InfiniBand if needed (#3698)
|
2025-02-19 20:50:22 +08:00 |
|
|
Yineng Zhang
|
f983213a1f
|
update pr-test (#3663)
|
2025-02-18 17:23:43 +08:00 |
|
|
Yineng Zhang
|
07ab4d4a2d
|
fix #3654
|
2025-02-18 15:16:16 +08:00 |
|
|
Yineng Zhang
|
058d199d4e
|
use transformers 4.48.3 (#3650)
|
2025-02-18 04:40:47 +08:00 |
|
|
Yineng Zhangandpankajroark
|
a5375adc3a
|
chore: bump v0.4.3.post2 (#3645)
Co-authored-by: pankajroark <pankajroark@users.noreply.github.com>
|
2025-02-18 02:48:30 +08:00 |
|
|
Yineng Zhang
|
75d171a9c5
|
chore: update flashinfer v0.2.1.post2 (#3644)
|
2025-02-18 02:47:42 +08:00 |
|
|
Yineng Zhang
|
714f3e6362
|
feat: support flashinfer mla with prefix cache (#3643)
|
2025-02-18 02:06:43 +08:00 |
|
|
Yineng Zhang
|
2e6be53e7d
|
fix Dockerfile.rocm
|
2025-02-17 22:13:03 +08:00 |
|
|
Yineng Zhang
|
e782eb7e6a
|
chore: bump v0.4.3.post1 (#3638)
|
2025-02-17 21:58:19 +08:00 |
|
|
Yineng Zhang
|
e319153be8
|
update unit test (#3636)
|
2025-02-17 21:06:10 +08:00 |
|
|
Yineng Zhang
|
32b44d2fca
|
add mtp unit test (#3634)
|
2025-02-17 19:04:07 +08:00 |
|
|
Yineng Zhang
|
5f1a485d9e
|
Revert "[ROCm] Use tl.range() in block GEMM kernels with `num_stage… (#3632)
|
2025-02-17 18:01:21 +08:00 |
|
|
Yineng Zhang
|
bbc47c348f
|
fix apply_token_bitmask_inplace_cuda (#3594)
|
2025-02-15 23:55:08 +08:00 |
|
|
Yineng Zhangandispobock
|
dfce926921
|
fix high qps crash when enable mtp (#3592)
Co-authored-by: ispobock <ispobaoke@hotmail.com>
|
2025-02-15 23:11:28 +08:00 |
|
|
Yineng Zhang
|
6718b10996
|
fix eagle unit test (#3591)
|
2025-02-15 23:10:48 +08:00 |
|
|
Yineng Zhang
|
4e23c961e8
|
docs: update install (#3581)
|
2025-02-14 18:54:50 +08:00 |
|
|
Yineng Zhang
|
31eec35ba8
|
fix doc (#3558)
|
2025-02-14 10:11:31 +08:00 |
|
|
Yineng Zhang
|
ac963be234
|
update flashinfer-python (#3557)
|
2025-02-14 09:52:56 +08:00 |
|
|
Yineng Zhang
|
e0b9a423c8
|
chore: bump v0.4.3 (#3556)
|
2025-02-14 09:43:14 +08:00 |
|
|
Yineng Zhang
|
e082142519
|
chore: bump 0.0.3.post6 sgl-kernel (#3555)
|
2025-02-14 08:55:15 +08:00 |
|
|
Yineng Zhang
|
70f894b810
|
feat: support flashinfer mla attention for deepseek v3 (#3550)
|
2025-02-14 08:50:14 +08:00 |
|
|
Yineng Zhang
|
20de05a753
|
update README (#3543)
|
2025-02-13 17:22:11 +08:00 |
|
|
Yineng Zhang
|
4430c0a513
|
chore: bump 0.0.3.post5 sgl-kernel (#3530)
|
2025-02-13 01:51:46 +08:00 |
|
|
Yineng Zhang
|
b96e92e6e6
|
chore: bump 0.0.3.post4 sgl-kernel (#3523)
|
2025-02-12 17:28:36 +08:00 |
|
|
Yineng Zhang
|
2f48221033
|
docs: update install
|
2025-02-12 03:13:31 +08:00 |
|
|
Yineng Zhang
|
4fe92bfca5
|
fix mla test (#3469)
|
2025-02-10 21:12:00 +08:00 |
|
|
Yineng Zhang
|
cddb1cdf8f
|
chore: bump v0.4.2.post4 (#3459)
|
2025-02-10 14:12:16 +08:00 |
|
|
Yineng Zhang
|
fa1b40e00d
|
use nvcr.io/nvidia/tritonserver:24.04-py3-min as base image (#3457)
|
2025-02-10 13:52:33 +08:00 |
|
|
Yineng Zhang
|
27c4c9cf52
|
remove _grouped_size_compiled_for_decode_kernels (#3453)
|
2025-02-10 13:01:21 +08:00 |
|
|
Yineng Zhang
|
36f6fc5093
|
feat: enable ragged fa3 by default on hopper 12.4+ (#3442)
|
2025-02-10 07:43:01 +08:00 |
|
|
Yineng Zhang
|
d87272750b
|
fix ci (#3441)
|
2025-02-10 04:22:28 +08:00 |
|
|
Yineng Zhang
|
6239d0b2e7
|
chore: bump sgl-kernel v0.0.3.post3 (#3440)
|
2025-02-10 04:00:52 +08:00 |
|
|
Yineng Zhang
|
4cfd3add6d
|
support version in sgl-kernel (#3439)
|
2025-02-10 03:49:52 +08:00 |
|
|
Yineng Zhang
|
85986bb978
|
compatible with new outlines (#3435)
|
2025-02-10 01:51:30 +08:00 |
|
|
Yineng Zhang
|
64c8713573
|
remove activation dependency in fused_moe (#3433)
|
2025-02-10 01:18:57 +08:00 |
|
|
Yineng Zhang
|
1646149a83
|
fix draft cuda graph capture failure (#3431)
|
2025-02-09 23:16:20 +08:00 |
|
|
Yineng Zhang
|
bc72e5bd32
|
add cuda graph capture failure possible solution (#3430)
|
2025-02-09 22:57:11 +08:00 |
|
|
Yineng Zhang
|
014cab4dd2
|
update forward_return_lse (#3425)
|
2025-02-09 20:18:44 +08:00 |
|
|
Yineng Zhang
|
4d2dbeaca7
|
remove cutex dependency (#3422)
|
2025-02-09 18:33:20 +08:00 |
|
|
Yineng Zhang
|
29daf498cd
|
fix cu118 link issue (#3421)
|
2025-02-09 18:16:44 +08:00 |
|
|
Yineng Zhang
|
60abdb3e7c
|
minor: cleanup test_eagle_infer (#3415)
|
2025-02-09 09:34:30 +08:00 |
|
|
Yineng Zhang
|
6222e1c228
|
add disable cuda graph unit test for eagle 2 (#3412)
|
2025-02-09 08:02:56 +08:00 |
|
|
Yineng ZhangandYing Sheng
|
fad315cb8e
|
fix EAGLE 2 non greedy case (#3407)
Co-authored-by: Ying Sheng <sqy1415@gmail.com>
|
2025-02-09 07:28:34 +08:00 |
|
|
Yineng Zhang
|
f90db8bc07
|
fix typo
|
2025-02-08 22:16:42 +08:00 |
|
|
Yineng Zhang
|
c1f5f99f60
|
chore: bump v0.4.2.post3 (#3369)
|
2025-02-07 08:20:03 -08:00 |
|
|
Yineng Zhang
|
fa82dfccdd
|
fix EagleVerifyInput (#3378)
|
2025-02-07 22:30:43 +08:00 |
|
|
Yineng Zhang
|
5da3d21c8b
|
update pr-test ci (#3376)
|
2025-02-07 21:08:35 +08:00 |
|
|
Yineng Zhang
|
f287037673
|
update sgl-kernel version (#3374)
|
2025-02-07 20:51:06 +08:00 |
|
|
Yineng ZhangandYing Sheng
|
f9905d59a8
|
support speculative decoding kernel in sgl-kernel (#3373)
Co-authored-by: Ying Sheng <sqy1415@gmail.com>
|
2025-02-07 20:29:51 +08:00 |
|
|
Yineng Zhang
|
45c87e083f
|
fix undefined symbol cudaGetDriverEntryPointByVersion (#3372)
|
2025-02-07 19:32:45 +08:00 |
|
|
Yineng Zhang
|
2b1808cec4
|
update unit test in AMD CI (#3366)
|
2025-02-07 17:25:16 +08:00 |
|
|
Yineng Zhang
|
adeee15204
|
fix sgl-kernel build failure on AMD (#3352)
|
2025-02-07 00:35:59 +08:00 |
|
|
Yineng Zhang
|
7348d9627e
|
add AMD guide for DeepSeek-R1 (#3338)
|
2025-02-06 16:54:40 +08:00 |
|
|
Yineng Zhang
|
25ed22b685
|
update pull request template (#3337)
|
2025-02-06 16:48:02 +08:00 |
|
|
Yineng Zhang
|
07e58a2dcb
|
update README (#3324)
|
2025-02-06 07:13:05 +08:00 |
|
|
Yineng Zhang
|
7aad8d1854
|
chore: bump v0.4.2.post2 (#3313)
|
2025-02-05 17:35:02 +08:00 |
|
|
Yineng Zhang
|
6186a8f889
|
update flashinfer install index url (#3293)
|
2025-02-05 00:44:35 +08:00 |
|
|
Yineng Zhangandpankajroark
|
d39899e85c
|
upgrade flashinfer v0.2.0.post2 (#3288)
Co-authored-by: pankajroark <pankajroark@users.noreply.github.com>
|
2025-02-04 21:41:40 +08:00 |
|
|
Yineng Zhang
|
4b6f62e2bc
|
add Atlas Cloud for Adoption and Sponsorship (#3276)
|
2025-02-04 05:31:30 +08:00 |
|
|
Yineng Zhang
|
897e2e253a
|
add Nebius for Adoption and Sponsorship (#3274)
|
2025-02-04 04:41:26 +08:00 |
|
|
Yineng Zhang
|
00fa7d0417
|
add copyright for sgl-kernel (#3270)
|
2025-02-03 21:34:44 +08:00 |
|
|
|
013021b6a1
|
refactor EAGLE 2 (#3269)
Co-authored-by: Ying Sheng <sqy1415@gmail.com>
Co-authored-by: merrymercy <lianminzheng@gmail.com>
Co-authored-by: Ying1123 <sqy1415@gmail.com>
|
2025-02-03 20:52:30 +08:00 |
|
|
Yineng Zhang
|
959dca4fc7
|
use srt VocabParallelEmbedding (#3252)
|
2025-02-01 22:23:09 +08:00 |
|
|
Yineng Zhang
|
f2b3a3188e
|
Update README
|
2025-02-01 21:19:15 +08:00 |
|
|
Yineng Zhang
|
ad6740977b
|
add contact us in README (#3251)
|
2025-02-01 19:47:44 +08:00 |
|
|
Yineng Zhang
|
8db776f049
|
support QuickGELU (#3250)
|
2025-02-01 19:31:47 +08:00 |
|
|
Yineng Zhang
|
4eb4b401cc
|
update and simplify CustomOp (#3249)
|
2025-02-01 18:56:44 +08:00 |
|
|
Yineng ZhangandHandH1998
|
7b020cca2d
|
add tuning block wise fp8 (#3242)
Co-authored-by: HandH1998 <007aabbcc411@gmail.com>
|
2025-02-01 03:58:18 +08:00 |
|
|
Yineng Zhang
|
7876279ea7
|
update cutlass dependency (#3240)
|
2025-02-01 03:13:44 +08:00 |
|
|
Yineng Zhang
|
34e405e01f
|
update sgl-kernel version for sglang (#3238)
|
2025-02-01 02:14:41 +08:00 |
|
|
Yineng Zhang
|
7811bfdaa7
|
compatible with flashinfer v0.2 (#3235)
|
2025-02-01 01:32:18 +08:00 |
|
|
Yineng Zhang
|
cf0f7eafe6
|
chore: bump v0.4.2.post1 (#3233)
|
2025-01-31 20:35:55 +08:00 |
|
|
Yineng Zhang
|
b49d6d0fee
|
support 12.5 CUDA runtime (#3231)
|
2025-01-31 20:31:38 +08:00 |
|
|
Yineng Zhang
|
3ee62235c6
|
revert the MoE dependence (#3230)
|
2025-01-31 16:51:41 +08:00 |
|
|
Yineng Zhang
|
9602c2aac7
|
keep the parts needed for moe_kernels (#3218)
|
2025-01-31 00:39:47 +08:00 |
|
|
Yineng Zhang
|
e81d7f11de
|
add tensorrt_llm moe_gemm as 3rdparty (#3217)
|
2025-01-30 23:49:14 +08:00 |
|
|
Yineng ZhangandBBuf
|
222ce6f1da
|
add tensorrt_llm common and cutlass_extensions as 3rdparty (#3216)
Co-authored-by: BBuf <35585791+BBuf@users.noreply.github.com>
|
2025-01-30 23:04:41 +08:00 |
|
|
Yineng Zhang
|
468d23cff9
|
update setup for sgl-kernel (#3214)
|
2025-01-30 19:47:50 +08:00 |
|
|
Yineng Zhang
|
c38b5fb4f4
|
update 3rdparty and rms norm for sgl-kernel (#3213)
|
2025-01-30 19:32:21 +08:00 |
|
|
Yineng Zhang
|
cf142b6eb8
|
fix: update Dockerfile for cu118 (#3181)
|
2025-01-27 23:46:44 +08:00 |
|
|
Yineng Zhang
|
4ab43cfb3e
|
chore: bump v0.4.2 (#3180)
|
2025-01-27 21:42:05 +08:00 |
|
|
Yineng Zhang
|
2f79f58873
|
feat: use sgl-kernel 0.0.3 in sglang (#3179)
|
2025-01-27 21:39:52 +08:00 |
|
|
|
8a96f74988
|
chore: bump 0.0.3 for sgl-kernel (#3178)
Co-authored-by: ispobock <ispobaoke@hotmail.com>
Co-authored-by: BBuf <35585791+BBuf@users.noreply.github.com>
Co-authored-by: HandH1998 <007aabbcc411@gmail.com>
Co-authored-by: yizhang2077 <1109276519@qq.com>
Co-authored-by: ByronHsu <byronhsu1230@gmail.com>
|
2025-01-27 20:29:28 +08:00 |
|