Lianmin Zheng b01df48cf2 [Fix] Adjust default chunked prefill size and cuda graph max bs according to GPU memory capacity (#2044)

2024-11-15 06:21:57 -08:00

..

configs

support internlm2-reward (#1994)

2024-11-11 15:09:58 -08:00

constrained

Fix dependency and error message for xgrammar (#2024)

2024-11-13 14:04:25 -08:00

layers

Fix torch.compile for MoE (#2033)

2024-11-14 01:30:24 -08:00

lora

Update vllm to 0.6.3 (#1711) (#1720)

2024-10-19 20:45:41 -07:00

managers

Fix the default arguments of bench_offline_throughput.py & simplify detokenizer manager (#2042)

2024-11-15 05:02:44 -08:00

mem_cache

Update docs (#1839)

2024-10-30 02:49:08 -07:00

metrics

Simplify prometheus metrics (#1981)

2024-11-10 04:39:32 -08:00

model_executor

Add download_dir ServerArgs property (#2027)

2024-11-13 23:26:56 -08:00

models

Fix weight loading for tied word embedding when TP > 1 (#2009)

2024-11-11 17:52:42 -08:00

openai_api

Expose no_stop_trim and skip_special_tokens in openai api (#2039)

2024-11-14 19:09:21 -08:00

sampling

Expose no_stop_trim and skip_special_tokens in openai api (#2039)

2024-11-14 19:09:21 -08:00

conversation.py

Llama3.2 vision model support (#1551)

2024-10-21 15:01:21 -07:00

hf_transformers_utils.py

Let reward model take text inputs instead of message lists (#1907)

2024-11-03 13:27:12 -08:00

mm_utils.py

minor: Add basic editorconfig and pre-commit hooks to enforce style for whitespaces (#1926)

2024-11-06 13:46:04 +00:00

server_args.py

[Fix] Adjust default chunked prefill size and cuda graph max bs according to GPU memory capacity (#2044)

2024-11-15 06:21:57 -08:00

server.py

Fix json benchmark (#2043)

2024-11-15 05:33:43 -08:00

utils.py

[Fix] Adjust default chunked prefill size and cuda graph max bs according to GPU memory capacity (#2044)

2024-11-15 06:21:57 -08:00