 Byron Hsu
|
17536e7e3d
|
Fix edge case for truncated (#1747)
|
2024-10-23 00:00:25 -04:00 |
|
|
Byron Hsu
|
cd0be7489f
|
[doc] improve engine doc and add to readme (#1670)
|
2024-10-14 19:56:21 -07:00 |
|
|
Byron Hsu
|
56503d9bc9
|
[1/N] Remove CacheConfig import in all model files (#1658)
|
2024-10-14 09:06:34 -07:00 |
|
|
Byron Hsu
|
c3f2fc5a7a
|
[doc] Add engine section in backend.md (#1656)
|
2024-10-13 00:33:58 -07:00 |
|
|
Byron Hsu
|
862cd265e5
|
[engine] support async and streaming (#1614)
|
2024-10-11 15:26:25 -07:00 |
|
|
Byron Hsu
|
01fdb2f377
|
Fix test_vision_openai_server on CI (#1620)
|
2024-10-10 16:34:13 -07:00 |
|
|
Byron Hsu
|
e8613df071
|
[Engine] Fix generate hanging issue after the first call (#1606)
|
2024-10-08 04:26:56 +00:00 |
|
|
Byron Hsu
|
565b05f02f
|
Use atexit hook to implicitly shutdown Runtime (#1595)
|
2024-10-07 05:18:45 +00:00 |
|
|
Byron Hsu
|
551a3a9d38
|
Provide an offline engine API (#1567)
|
2024-10-06 20:27:03 -07:00 |
|
|
Byron Hsu
|
17e998f1a8
|
Test consistency for single and batch seperately (#1590)
|
2024-10-06 22:02:27 +00:00 |
|
|
Byron Hsu
|
2422de5193
|
Support min_tokens in sgl.gen (#1573)
|
2024-10-05 21:51:12 -07:00 |
|
|
Byron Hsu
|
521f862d90
|
Fix runtime.generate when sampling param is not passed (#1582)
|
2024-10-05 17:59:05 -07:00 |
|
|
Byron Hsu
|
34c32d2820
|
Fix styling (#1583)
|
2024-10-05 17:52:14 -07:00 |
|
|
Byron Hsu
|
dde8bb16fe
|
default sampling param should be deepcopied (#1581)
|
2024-10-05 17:27:43 -07:00 |
|
|
Byron Hsu
|
8ac3ccc060
|
Backend method not found when SRT Runtime is used (#1576)
|
2024-10-05 11:47:35 -07:00 |
|
|
Byron Hsu
|
6bfdb4031d
|
[Easy] use .text() instead of .text (#1577)
|
2024-10-05 11:07:41 -07:00 |
|
|
Byron Hsu
|
8c0efa514d
|
remove assertion in triton attention and add an unit test (#1385)
|
2024-09-11 03:22:07 -07:00 |
|
|
Byron Hsu
|
8e6bdf851c
|
[triton] Support head_dim not 2^n in triton extend and decode attention (#1281)
|
2024-09-09 01:30:24 -07:00 |
|
|
Byron Hsu
|
4a9f8ea43b
|
[doc] Fix more broken links (#1294)
|
2024-09-01 14:46:36 -07:00 |
|
|
Byron Hsu
|
00b19f198f
|
[triton] Remove the zero initialization of qk_acc by directly writing the result (#1288)
|
2024-09-01 03:12:06 -07:00 |
|
|
Byron Hsu
|
6cc9c52521
|
[doc] fix quick start link (#1282)
|
2024-08-31 22:54:34 -07:00 |
|