Commit Graph

Select branches

Hide Pull Requests

dsv41-pd

main

Mono Color

• da19434c2f Benchmark Updates (#382) Liangsheng Yin 2024-04-24 02:23:01 +08:00
• 150d7020ed Revert removing the unused imports (#385) Liangsheng Yin 2024-04-23 22:36:33 +08:00
• 9acc6e3504 add .isort.cfg (#378) Liangsheng Yin 2024-04-22 22:38:09 +08:00
• cf9d8efdd3 llama3 instruct template (#372) Enrique Shockwave 2024-04-21 17:40:12 +01:00
• 1bf1cf1953 Reduce overhead when fork(1) (#375) Liangsheng Yin 2024-04-21 17:25:14 +08:00
• e822e5900b Optimize radix tree matching (#364) Ke Bao 2024-04-18 00:47:37 +08:00
• ca4f1ab89c Update model support in readme (#370) Ying Sheng 2024-04-17 00:16:32 -07:00
• 2b6d999191 Fix issue #367 – System message not supported for Anthropic (anthropic.BadRequestError) (#368) FronxandYing Sheng 2024-04-16 20:18:24 +02:00
• 65501a9cf1 Fix commandr import; format code Lianmin Zheng 2024-04-16 18:10:12 +00:00
• db611066ad support command-r (#369) ZhouXingg 2024-04-17 01:36:51 +08:00
• c93293c57e Update README.md (#358) Ikko Eltociear Ashimine 2024-04-10 00:39:30 +09:00
• 62b3812b69 Time cost utils (#355) Liangsheng Yin 2024-04-09 23:27:31 +08:00
• 550a4f78f3 Fix typos in infer_batch.py (#354) Tom Dörr 2024-04-09 09:10:05 +02:00
• ff99c38a07 Add timeout to get_meta_info (#346) SimoneRaponiandsimone 2024-04-03 16:22:06 +02:00
• c9de3e169c Eliminate 2 gpu ops during sampling when logit_bias is zero (#338) Qubitiumandhnyls2002 2024-04-03 13:56:06 +08:00
• ed27a6b992 Revert "Eliminate 2 gpu ops during sampling when logit_bias is zero" (#345) Liangsheng Yin 2024-04-03 12:45:01 +08:00
• 463c6632a8 Eliminate 2 gpu ops during sampling when logit_bias is zero (#343) Liangsheng YinandQubitium 2024-04-02 19:14:55 +08:00
• b0890631a0 fix gemma import error Ying Sheng 2024-04-01 07:35:58 +00:00
• cb389c91bc Fix llava parallelism/fork bug (#315) Junlong Li 2024-03-29 10:24:54 +08:00
• eddaa2b599 Add support for new autogptq quant_config.checkpoint_format (#332) Qubitium 2024-03-29 10:24:16 +08:00
• 2af565b3bb [model] DBRX-instruct support (#337) Liangsheng Yin 2024-03-29 01:05:19 +08:00
• 3842eba5fa Logprobs Refractor (#331) Liangsheng Yin 2024-03-28 14:34:49 +08:00
• 24e59f5350 model_runner simplify (#329) Liangsheng Yin 2024-03-24 19:48:37 +08:00
• 7523541962 model_rpc style improvement (#293) Liangsheng Yin 2024-03-24 15:41:24 +08:00
• 64ee9c030e Openrouter usage example (#327) Jani Monoses 2024-03-23 19:16:24 +02:00
• 30d17840fc Update dependencies (#326) Jani Monoses 2024-03-23 19:15:58 +02:00
• ce216c80dc Cleanup codebase: removed unnecessary code/logic (#298) Qubitium 2024-03-24 01:15:16 +08:00
• 51104cd405 Update version to v0.1.14 (#324) Lianmin Zheng 2024-03-22 13:42:22 -07:00
• e2b2f0a213 Support oai in benchmark/mmlu (#323) Lianmin Zheng 2024-03-22 13:37:57 -07:00
• b57abe1663 Add StableLM model. (#301) Jani Monoses 2024-03-22 22:24:08 +02:00
• e57f079275 Use Anthropic messages API (#304) Jani Monoses 2024-03-22 22:23:31 +02:00
• 08df63a6f8 [Fix/Potential Bugs] Can not correctly import models in python/sglang/srt/models (#311) Li Bo 2024-03-23 03:19:58 +08:00
• 77835756a7 Fix outlines-0.0.35 incompatibility (#291) ZhouGongZaiShiandZX 2024-03-23 03:19:11 +08:00
• ed31579971 Fix marlin model loading compat with autogptq (#290) LiurlandLRL 2024-03-13 13:15:43 +08:00
• 92e2d74fd0 Fix env (docker) compat due to __file__ usage (#288) Qubitium 2024-03-13 13:02:48 +08:00
• d9b3b01883 enable marlin kernels (#286) Enrique Shockwave 2024-03-13 02:10:12 +00:00
• 745ea007ac Fix Incorrect CURL Request Example in README (#287) Arsalan 2024-03-12 22:09:38 -04:00
• ad1dd74673 Fix flashinfer >= 0.0.3 compat (#282) Qubitium 2024-03-12 21:45:58 +08:00
• eb4308c4c9 adding the triton docker build minimal example (#242) Arsalan 2024-03-12 03:16:06 -04:00
• b2eb080501 Fix Runtime missing some ServerArgs options (#281) Qubitium 2024-03-11 22:32:15 +08:00
• 4aa5dd2c5f Update version to v0.1.13 (#280) Lianmin Zheng 2024-03-11 05:49:27 -07:00
• 13662fd533 Fix RuntimeEndpoint (#279) Lianmin Zheng 2024-03-11 05:24:24 -07:00
• d5ae2ebaa2 Add Support for API Key Authentication (#230) Alessio Dalla Piazza 2024-03-11 13:16:10 +01:00
• 1b35547927 Organize server_args (#277) Liangsheng Yin 2024-03-11 20:06:52 +08:00
• faba293a0d Improve gemma and documentations (#278) Lianmin Zheng 2024-03-11 04:43:39 -07:00
• 89885b31ef Gemma Support (#256) Liangsheng Yin 2024-03-11 12:14:27 +08:00
• 64fe311593 replace skip_embed with input_embeds (#222) Geary.Z 2024-03-11 10:04:52 +08:00
• a7ace9c88d Fix qwen config (#261) Liangsheng Yin 2024-03-11 09:54:18 +08:00
• a833de05d3 Add logo (#275) Lianmin Zheng 2024-03-10 18:51:47 -07:00
• 30d67b2bca Add set_var to interpreter.py (#263) Lin Tianchuan 2024-03-07 23:20:11 +08:00
• b0b722ee8e Refactor ChatTemplate for Enhanced Clarity and Efficiency (#201) Xinwei Xiong 2024-03-03 17:52:36 +08:00
• 01b07ea3ac Add SSL Cert Functionality (#224) Srinivas Billa 2024-03-03 09:41:41 +00:00
• dfb13ac455 Fix addr reuse in check_port (#253) Liangsheng Yin 2024-03-03 17:09:16 +08:00
• ec90b9c054 Upload agent_calls.jsonl download link (#226) Liangsheng Yin 2024-02-24 19:03:46 +08:00
• 9759d927cf fix chatml template (#195) Enrique Shockwave 2024-02-24 08:34:22 +00:00
• 8d0a7fae3b Fix interpreter.py get_var(var_name) in text iter when stream is not enabled (#198) Zhang Wenbin 2024-02-24 16:27:34 +08:00
• c4e9ebe3a4 Fix stop str merging (#225) Liangsheng YinandEnrique Shockwave 2024-02-24 16:05:21 +08:00
• 3c2c5869ad Support outlines > 0.0.31 (#219) Cody Yu 2024-02-23 23:06:17 -08:00
• 4cb9aaedf3 Fix logprobs with logprob_start_len (#193) Cody Yu 2024-02-22 10:33:03 -08:00
• 9de9a46815 Added the ability to Modify the Context Length (#210) psych0v0yager 2024-02-20 18:22:56 -06:00
• ce3b261053 Update README.md (#207) Ikko Eltociear Ashimine 2024-02-20 02:09:03 +09:00
• 91e036334f Adjust outlines version. (#200) Liangsheng Yinandcomaniac 2024-02-17 13:40:39 +08:00
• 2a74748b2f Pin outlines version (#196) Cody Yu 2024-02-16 13:01:40 -08:00
• 63ba630bbb Refactor decoding logprob and add completion_tokens_wo_jump_forward (#189) Cody Yu 2024-02-15 10:54:20 -08:00
• 6493256b7d improve print Lianmin Zheng 2024-02-12 12:43:48 +00:00
• 06008bc295 Fix server launch for jupyter notebook (#186) Lianmin Zheng 2024-02-12 04:43:14 -08:00
• bb824da41a Add Together and AzureOpenAI examples (#184) Lianmin Zheng 2024-02-12 01:06:38 -08:00
• 931213245c correct reference dtype openai.py (#181) Yaya Sy 2024-02-11 22:26:20 +01:00
• c97fdae4aa correct a mistake on the README.md (#182) Yaya Sy 2024-02-11 22:25:57 +01:00
• 624b21e742 Update version to 0.1.12 (#178) Lianmin Zheng 2024-02-11 06:43:45 -08:00
• c51020cf0c Fix the chat template for llava-v1.6-34b & format code (#177) Lianmin Zheng 2024-02-11 05:50:13 -08:00
• 50afed4eaa Support extra field regex in OpenAI API (#172) Cody Yu 2024-02-10 17:21:33 -08:00
• 4d303c4fa3 Fix token usage with jump forward (#174) Cody Yu 2024-02-09 20:06:15 -08:00
• 37b42297f8 import outlines (#168) Liangsheng Yin 2024-02-09 10:13:02 +08:00
• cba5027332 Fix BaseCache metric (#170) Cody Yu 2024-02-08 17:23:09 -08:00
• a6aa46dd3f minor Ying Sheng 2024-02-08 04:35:25 +00:00
• 405f26b00b Add Auth Token to RuntimeEndpoint (#162) Srinivas Billa 2024-02-08 04:07:31 +00:00
• b1a3a454ee add --disable-disk-cache (#160) Liangsheng YinandJa1Zhou 2024-02-08 00:50:12 +08:00
• 79e6b84bec Update README.md Ying Sheng 2024-02-06 23:14:59 -08:00
• 26c3494152 [Submodule] Change FlashInfer to import (#156) Cody Yu 2024-02-06 19:28:29 -08:00
• cb8e1982f8 Update README.md Ying Sheng 2024-02-06 18:44:37 -08:00
• 23f05005fd Format code & move functions (#155) Lianmin Zheng 2024-02-06 13:27:46 -08:00
• a7334aeea1 Support decode token logprobs (#130) Cody Yu 2024-02-06 12:24:55 -08:00
• ee1df26a77 Update README.md Lianmin Zheng 2024-02-06 11:35:42 -08:00
• 3ae78a09b3 Add gptq quantization model support (#141) Arcmoon 2024-02-07 03:35:04 +08:00
• ccbe1e67d8 Temporary fix OpenAI API for Pydantic v1/v2 (#153) Cody Yu 2024-02-06 11:34:15 -08:00
• e2bf732bc3 add openai error handler with retry and logger (#148) LiviaSun 2024-02-06 12:38:41 +08:00
• 322421fae3 Add warmup to SRT server (#146) Cody Yu 2024-02-05 14:21:16 -08:00
• 8ff870bf3e improve docs Lianmin Zheng 2024-02-05 11:22:06 +00:00
• 26f0bedc8f jump-forward rename (#144) Liangsheng Yin 2024-02-05 16:50:37 +08:00
• 82fa69b3cc fix undfined variable (#142) Yaya Sy 2024-02-04 23:27:52 +01:00
• 8fb7459e08 update json decoding docs Lianmin Zheng 2024-02-03 17:42:01 -08:00
• bb3a3b6675 Support Faster JSON decoding for llava (#137) Liangsheng Yin 2024-02-03 23:32:05 +08:00
• 45d6592d40 Fix no-cache mode (#136) Ying Sheng 2024-02-03 04:59:06 -08:00
• f6bfe3aaff Release 0.1.11 (#134) Ying Sheng 2024-02-03 02:50:13 -08:00
• e095b16236 Add max_prefill_num_token into server arguments (#133) Ying Sheng 2024-02-03 02:35:54 -08:00
• 67be11c790 fix bug of race condition in copy() Ying Sheng 2024-02-03 01:38:00 -08:00
• cd8c3ccd95 Fix is_multimodal_model judge (#132) Liangsheng Yin 2024-02-03 11:48:01 +08:00
• 9c121f2a45 minor fix: result dump format hnyls2002 2024-02-02 09:58:24 +00:00
• 03e04b2331 update docs for Yi-VL Lianmin Zheng 2024-02-01 22:44:05 +00:00