 Ke Bao
|
699384cb01
|
Set schedule policy more conservative for DP attention (#2096)
|
2024-11-19 20:57:18 -08:00 |
|
|
Ke Bao
|
62832bb272
|
Support cuda graph for DP attention (#2061)
|
2024-11-17 16:29:20 -08:00 |
|
|
Ke Bao
|
976bc302e5
|
Support DP MLA (#1970)
|
2024-11-16 09:01:43 +00:00 |
|
|
Ke Bao
|
b808a38365
|
Filter empty prompt in random bench serving (#2011)
|
2024-11-12 14:53:41 +08:00 |
|
|
Ke Bao
|
16eb33ffe2
|
Update vocab embedding deps and add TP switch (#1856)
|
2024-10-31 20:13:07 -07:00 |
|
|
Ke Bao
|
c77762d57f
|
Fix Triton decode kernel & ut (#1819)
|
2024-10-27 10:54:38 -07:00 |
|
|
Ke Bao
|
d10b933a36
|
Fix srt dependency (#1685)
|
2024-10-16 08:21:20 -07:00 |
|
|
Ke Bao
|
68f8b60d22
|
Fix chunked prefill condition (#1594)
|
2024-10-07 06:34:14 +00:00 |
|
|
Ke Bao
|
8d4ed42ad5
|
MoE torch compile (#1497)
|
2024-09-24 01:46:59 -07:00 |
|
|
Ke Bao
|
b8ccaf4d73
|
Add MLA gsm8k eval (#1484)
|
2024-09-21 11:16:13 +08:00 |
|
|
Ke Bao
|
a68cb201dd
|
Fix triton head num (#1482)
|
2024-09-21 10:25:20 +08:00 |
|
|
Ke Bao
|
b3710d2c93
|
Fix attention backend (#1448)
|
2024-09-17 14:07:53 +00:00 |
|
|
Ke Bao
|
c6b6d2e71b
|
Enable MLA by default (#1447)
|
2024-09-17 11:42:48 +00:00 |
|
|
Ke Bao
|
76524b70d1
|
Fix torch compile for deepseek-v2 (#1442)
|
2024-09-17 00:52:08 -07:00 |
|
|
Ke Bao
|
33b54e7c40
|
Add pytorch sampling backend ut (#1425)
|
2024-09-15 01:15:30 +10:00 |
|
|
Ke BaoandYineng Zhang
|
6cb32ef92c
|
Support Triton fp8 e5m2 kv cache (#1286)
Co-authored-by: Yineng Zhang <me@zhyncs.com>
|
2024-09-01 19:46:40 +10:00 |
|
|
Ke BaoandYineng Zhang
|
f414352ae6
|
Transpose mla weight offline (#1261)
Co-authored-by: Yineng Zhang <me@zhyncs.com>
|
2024-08-30 16:45:40 +10:00 |
|
|
Ke BaoandYineng Zhang
|
2c615d120f
|
[Feature] Support fp8 e5m2 kv cache with flashinfer (#1204)
Co-authored-by: Yineng Zhang <me@zhyncs.com>
|
2024-08-25 17:38:11 -07:00 |
|
|
Ke BaoandYineng Zhang
|
df191254ab
|
Optimize MLA/GQA/MQA Triton decoding (#1138)
Co-authored-by: Yineng Zhang <me@zhyncs.com>
|
2024-08-19 20:23:07 +10:00 |
|
|
Ke BaoandYineng Zhang
|
162f3ccb01
|
Fix layernorm input shape (#1066)
Co-authored-by: Yineng Zhang <me@zhyncs.com>
|
2024-08-13 15:48:07 +10:00 |
|
|
Ke Bao
|
a9c833d580
|
Fix union operator (#940)
|
2024-08-06 00:46:34 +08:00 |
|
|
Ke Bao
|
e1eae1fd15
|
Support MLA for DeepSeek-V2 with Triton - step 1 (#905)
|
2024-08-05 03:40:33 +10:00 |
|
|
Ke Bao
|
3fdab91912
|
Fix TransformerTokenizer init for chatglm2 & 3 (#761)
|
2024-07-27 02:44:46 -07:00 |
|
|
Ke Bao
|
824a77d04d
|
Fix hf config loading (#702)
|
2024-07-23 11:39:08 -07:00 |
|
|
Ke Bao
|
5303c1ed22
|
Support Mistral-Nemo (#691)
|
2024-07-22 03:36:53 +10:00 |
|
|
Ke Bao
|
0ac94c36cb
|
Fallback when sampling failed (#678)
|
2024-07-20 10:44:54 -07:00 |
|
|
Ke Bao
|
ec2150b294
|
Fix kill process util (#666)
|
2024-07-19 21:43:11 -07:00 |
|
|
Ke Bao
|
e822e5900b
|
Optimize radix tree matching (#364)
|
2024-04-17 09:47:37 -07:00 |
|