Commit Graph

Select branches

Hide Pull Requests

dsv41-pd

dsv41-pd-visioncp

main

Mono Color

• dff2860a69 Fix CORS compatibility with OpenAI, vLLM, TGI, LMDeploy (#1373) josephroccaandYineng Zhang 2024-09-11 00:35:03 +08:00
• e72275cf7f Support MiniCPM3 (#1371) William 2024-09-10 17:57:52 +08:00
• fec2d1223c [Fix] fix bug of undefined is_single in meth create_abort_task (#1370) wangchao 2024-09-10 16:17:37 +08:00
• 8d1095dbf0 [Docs] Improve documentations (#1368) Lianmin Zheng 2024-09-09 20:48:28 -07:00
• 743007e1ce Adding Documentation for installation (#1300) Chayenneandzhaochen20 2024-09-10 10:09:13 +08:00
• 9144ed1067 Support OpenAI API json_schema response format (#1363) zifeitong 2024-09-09 19:08:25 -07:00
• 69b3bb9ae1 Unify forward mode (#1360) Liangsheng Yin 2024-09-09 13:49:29 -07:00
• 689ff588ec [CI] Return output logprobs in unit test (#1361) Ying Sheng 2024-09-09 13:05:13 -07:00
• a7c47e0f02 Add torchao quant (int4/int8/fp8) to llama models (#1341) Jerry ZhangandLianmin Zheng 2024-09-09 05:32:41 -07:00
• e4d68afcf0 [Minor] Many cleanup (#1357) Lianmin Zheng 2024-09-09 04:14:11 -07:00
• c9b75917d5 [server] Passing model_override_args to launch_server via the CLI. (#1298) Kai-Hsun Chen 2024-09-09 02:14:25 -07:00
• 662ecd9368 [Feat] Add modalities for vision server when handling pixel values for llava (#1346) Kaichen Zhang - NTU 2024-09-09 17:07:34 +08:00
• 8e6bdf851c [triton] Support head_dim not 2^n in triton extend and decode attention (#1281) Byron Hsu 2024-09-09 01:30:24 -07:00
• 05bea6883c Fix some online scheduling delay (#1345) Liangsheng Yin 2024-09-07 20:46:27 -07:00
• ab4a83b259 Optimize schedule (#1339) Liangsheng Yin 2024-09-05 14:30:26 -07:00
• 62f15eea5a docs: add conclusion (#1340) Yineng Zhang 2024-09-06 04:25:14 +10:00
• 79794af52d docs: highlight ttft itl and throughput (#1337) Yineng Zhang 2024-09-06 00:00:06 +10:00
• 3494b32c3a docs: update README (#1336) Yineng Zhang 2024-09-05 23:39:44 +10:00
• eda7c09048 Remove useless fields in global_config.py (#1328) Lianmin Zheng 2024-09-04 05:37:32 -07:00
• 5ab9418f5b [Doc] update news (#1327) Yineng Zhang 2024-09-04 21:21:21 +10:00
• 843e63d809 Fix the flaky test test_moe_eval_accuracy_large.py (#1326) Lianmin Zheng 2024-09-04 04:15:11 -07:00
• a63c8275c6 chore: bump v0.3.0 (#1320) Yineng Zhang 2024-09-04 06:32:18 +10:00
• dc67d97693 misc: speedup load safetensors (#1319) Yineng Zhangandispobock 2024-09-04 04:29:53 +10:00
• 1e495e0847 [Fix] Fix select by ensuring each request has at least one token (#1318) Lianmin Zheng 2024-09-03 06:31:45 -07:00
• 12cb115d38 Fix llama2 weight loader (#1317) Lianmin Zheng 2024-09-03 05:32:14 -07:00
• c500f96bb1 Update README.md for llava-onevision instructions (#1313) Lianmin Zheng 2024-09-03 01:43:08 -07:00
• 474317f2b6 Support Phi3 mini and medium (#1299) Jani Monoses 2024-09-03 07:49:40 +03:00
• f64eae3a29 [Fix] Reduce memory usage for loading llava model & Remove EntryClassRemapping (#1308) Lianmin Zheng 2024-09-02 21:44:45 -07:00
• a5a134f39f Fix bugs in sampler with CUDA graph / torch.compile (#1306) Liangsheng Yin 2024-09-02 16:18:48 -07:00
• 2561ed012c feat: update nightly gsm8k eval (#1304) Yineng Zhang 2024-09-03 01:18:41 +10:00
• 9999442756 Release v0.2.15 (#1295) Lianmin Zheng 2024-09-01 22:22:38 -07:00
• 6def9b018c Fix hang when doing s += None. (#1297) Max Shawabkehandmax99x 2024-09-01 21:56:33 -07:00
• 47f20da223 Fix regex mask (#1296) Liangsheng Yin 2024-09-01 21:50:58 -07:00
• 4a9f8ea43b [doc] Fix more broken links (#1294) Byron Hsu 2024-09-01 14:46:36 -07:00
• 58fa607622 Fix the flaky tests in test_moe_eval_accuracy_large.py (#1293) Lianmin Zheng 2024-09-01 12:20:46 -07:00
• 6487ef64c6 ci: add nightly eval (#1291) Yineng Zhang 2024-09-02 03:19:49 +10:00
• 9b0805242e fix: resolve fp8 for mixtral (#1290) Yineng Zhang 2024-09-02 00:29:06 +10:00
• 32a4141d5a Allow new lines during JSON generation (#1277) Enrique Shockwave 2024-09-01 11:42:29 +01:00
• 0836055324 [Chore] Rename model_overide_args to model_override_args (#1284) Kai-Hsun ChenandYineng Zhang 2024-09-01 03:14:56 -07:00
• 00b19f198f [triton] Remove the zero initialization of qk_acc by directly writing the result (#1288) Byron Hsu 2024-09-01 03:12:06 -07:00
• 6cb32ef92c Support Triton fp8 e5m2 kv cache (#1286) Ke BaoandYineng Zhang 2024-09-01 17:46:40 +08:00
• 761b2cebd6 [CI] merge all ci tests into one file (#1289) Lianmin Zheng 2024-09-01 02:36:56 -07:00
• 54772f784a feat: fix fp8 for MLA and support bmm fp8 for DeepSeek V2 (#1285) Yineng Zhangandispobock 2024-09-01 17:28:06 +10:00
• 1b5d56f7f8 [CI] Add more multi-gpu tests (#1280) Lianmin Zheng 2024-09-01 00:27:25 -07:00
• d134c139a1 Optimize the update flashinfer indices (#1262) xiaobochen 2024-08-31 23:40:28 -07:00
• 6cc9c52521 [doc] fix quick start link (#1282) Byron Hsu 2024-08-31 22:54:34 -07:00
• 52cefdbf57 fix: resolve the fp8 bug introduced by vLLM 0.5.5 (#1276) Yineng Zhang 2024-09-01 00:44:29 +10:00
• 51c554d812 Allow more flexible assistant and system response (#1256) Christopher Chou 2024-08-30 11:51:44 -07:00
• 79ece2c51f Report median instead of mean in bench_latency.py (#1269) Lianmin Zheng 2024-08-30 06:05:01 -07:00
• 55f5976b42 Update README.md - Supported Models add Exaone 3.0 (#1267) 김종곤andYineng Zhang 2024-08-30 17:49:07 +09:00
• b7f8341014 EXAONE 3.0 Model Support (#1258) 김종곤andYineng Zhang 2024-08-30 17:08:28 +09:00
• f414352ae6 Transpose mla weight offline (#1261) Ke BaoandYineng Zhang 2024-08-30 14:45:40 +08:00
• a362340b33 fix: multimodal_config in monkey_patch_vllm_dummy_weight_loader (#1260) lxww302 2024-08-29 23:43:41 -07:00
• 381dd57bd6 Sampler cudagraph (#1253) Liangsheng Yin 2024-08-28 18:58:52 -07:00
• 8153168c96 fix data racing due to mutable reference using deepcopy (#1255) Zhiqiang Xie 2024-08-28 18:57:54 -07:00
• 6c34d6339c make json_schema usable from gen (#1254) Enrique Shockwave 2024-08-29 02:57:10 +01:00
• 13ac95b894 chore: bump v0.2.14.post2 (#1250) Yineng Zhang 2024-08-29 04:46:33 +10:00
• 492143bf32 fix: resolve qwen2 moe weight loader (#1252) Yineng Zhang 2024-08-29 04:25:46 +10:00
• 0a97d7962d [Fix] Fix OOM in llava base class (#1249) Lianmin Zheng 2024-08-28 08:38:50 -07:00
• c411f32e1c feat: replace GeluAndMul (#1234) Yineng Zhang 2024-08-29 00:07:02 +10:00
• bf53bf5142 [Fix] Fix llava on multi images (#1247) Lianmin Zheng 2024-08-28 06:33:05 -07:00
• b1a540ec42 feat: update GemmaRMSNorm (#1232) Yineng Zhang 2024-08-28 22:47:34 +10:00
• 66975360e7 fix: increase max_new_tokens when testing generation models (#1244) Yineng Zhang 2024-08-28 22:12:36 +10:00
• 6c49831394 Add sglang.bench_latency to CI (#1243) Lianmin Zheng 2024-08-28 04:20:54 -07:00
• f25f4dfde5 hotfix: revert sampler CUDA Graph (#1242) Yineng Zhang 2024-08-28 21:16:47 +10:00
• 184ae1c683 Update README.md (#1239) Lianmin Zheng 2024-08-28 02:15:52 -07:00
• 198974cd1a feat: support sm75 with FlashInfer v0.1.6 (#1233) Yineng Zhang 2024-08-28 18:39:12 +10:00
• 6cc38b2bf3 [Minor] Add more type annotations (#1237) Lianmin Zheng 2024-08-28 00:54:26 -07:00
• 1ece2cda3d Fix bench latency benchmark (#1225) Liangsheng Yin 2024-08-28 00:37:32 -07:00
• c8a9e79186 Fix readme (#1236) Dr. Artificial曾小健 2024-08-28 14:51:41 +08:00
• 3602692c7c feat: replace get_act_fn for gpt_bigcode (#1231) Yineng Zhang 2024-08-27 21:15:31 +10:00
• 909f34363b [FIX] Wrong logger (#1230) havetc 2024-08-27 12:10:46 +02:00
• 5ff25cdf5b [Minor] add delete test and delete tmp file on ci server (#1227) yichuan~ 2024-08-26 22:04:52 -07:00
• 2f1d92834f [FEAT] Support batches cancel (#1222) caiyueliangandYineng Zhang 2024-08-27 07:28:26 +08:00
• c61a1b6f97 Torch compile CI throughput test (#1223) Liangsheng Yin 2024-08-26 13:52:58 -07:00
• 9935f97b3e [FEAT] JSON constrained support (#1125) havetcandYineng Zhang 2024-08-26 18:37:26 +02:00
• c5fe11a8e1 chore: bump v0.2.14 (#1155) Yineng Zhang 2024-08-27 00:28:24 +10:00
• 75ce37f401 Move sampler into CUDA graph (#1201) Liangsheng YinandYineng Zhang 2024-08-26 07:02:50 -07:00
• 97589a60a2 [CI] Parallelize unit tests in CI (#1219) Mingyi 2024-08-25 21:54:02 -07:00
• 632d506d0b minor: improve CI and dependencies (#1212) Liangsheng Yin 2024-08-25 21:26:31 -07:00
• 3579162ab1 [Fix] Multi-images loading error (#1218) Kaichen Zhang - NTU 2024-08-26 11:58:51 +08:00
• 7514b9f8d3 [CI] Fix CI (#1217) Mingyi 2024-08-25 19:56:42 -07:00
• 158e8f1e2d improve the threshold and ports in tests (#1215) Mingyi 2024-08-25 19:02:08 -07:00
• d3efcb3930 Update workflow files (#1214) Lianmin Zheng 2024-08-25 17:45:35 -07:00
• 2c615d120f [Feature] Support fp8 e5m2 kv cache with flashinfer (#1204) Ke BaoandYineng Zhang 2024-08-26 08:38:11 +08:00
• 61bb223e0f Update CI runner docs (#1213) Lianmin Zheng 2024-08-25 17:31:52 -07:00
• 15f1a49d2d Update CI workflows (#1210) Lianmin Zheng 2024-08-25 16:43:07 -07:00
• 308d024092 [CI] Fix the issue of unit test hanging (#1211) Ying Sheng 2024-08-25 16:21:37 -07:00
• ab4990e4bf [Minor] Temporarily skip flaky test (#1209) Ying Sheng 2024-08-25 14:49:23 -07:00
• 902278008a [Minor] Improve the function organization in TokenizerManager & improve loggers (#1208) Lianmin Zheng 2024-08-25 14:46:34 -07:00
• 30b4f771b0 Support Alibaba-NLP/gte-Qwen2-7B-instruct embedding Model (#1186) ChayenneandYing Sheng 2024-08-26 01:29:12 +08:00
• 66e7dcaf70 [Fix] Fixing the multi-images error for llava-onevision (#1205) Kaichen Zhang - NTU 2024-08-26 01:28:23 +08:00
• bc4c7a3545 Relax the assert in moe throughput test to fix the flaky CI (#1207) Lianmin Zheng 2024-08-25 10:27:02 -07:00
• 1cb4da5c5f [Fix] the issue of random order when input is a list (#1199) Ying Sheng 2024-08-24 21:43:03 -07:00
• e61d13acdf [CI] Fix the problem of hf runner too slow (#1202) Ying Sheng 2024-08-24 18:35:55 -07:00
• b20daf982a Update README.md (#1198) Lianmin Zheng 2024-08-24 14:50:05 -07:00
• f6af3a6561 Cleanup readme, llava examples, usage examples and nccl init (#1194) Lianmin Zheng 2024-08-24 08:02:23 -07:00
• c9064e6fd9 feat: use gelu_tanh_and_mul (#1193) Yineng Zhang 2024-08-24 18:58:16 +10:00
• a5b14ad043 [Feat/WIP] add llava-onevision, with support for (1) siglip encoder, (2) qwen2 decoder (3) openai api compatible server. (#1123) Kaichen Zhang - NTUandBo Li 2024-08-24 05:11:16 +08:00
• 5fafcac008 Fix benchmark script (#1185) Ying Sheng 2024-08-22 02:03:25 -07:00