Lianmin Zheng 2854a5ea9f Fix the overhead due to penalizer in bench_latency (#1496)

2024-09-23 07:38:14 -07:00

..

configs

[Minor] move triton attention kernels into a separate folder (#1379)

2024-09-10 15:15:08 -07:00

constrained

Add constrained_json_whitespace_pattern to ServerArgs (#1438)

2024-09-16 13:29:18 -07:00

layers

Fix triton head num (#1482)

2024-09-21 10:25:20 +08:00

lora

[Feature, Hardware] Enable SGLang on AMD GPUs via PyTorch for ROCm (#1420)

2024-09-17 07:43:52 +00:00

managers

Fix the overhead due to penalizer in bench_latency (#1496)

2024-09-23 07:38:14 -07:00

mem_cache

[Feature] Support fp8 e5m2 kv cache with flashinfer (#1204)

2024-08-25 17:38:11 -07:00

model_executor

Fix the overhead due to penalizer in bench_latency (#1496)

2024-09-23 07:38:14 -07:00

models

feat: update linear deps 1/N (#1305)

2024-09-19 20:53:11 +08:00

openai_api

[API, Feature] Support response prefill for openai API (#1490)

2024-09-22 06:46:17 -07:00

sampling

Simplify sampler and its error handling (#1441)

2024-09-16 21:23:31 -07:00

conversation.py

[Minor] move triton attention kernels into a separate folder (#1379)

2024-09-10 15:15:08 -07:00

hf_transformers_utils.py

[Minor] move triton attention kernels into a separate folder (#1379)

2024-09-10 15:15:08 -07:00

mm_utils.py

[Feat/WIP] add llava-onevision, with support for (1) siglip encoder, (2) qwen2 decoder (3) openai api compatible server. (#1123)

2024-08-23 14:11:16 -07:00

server_args.py

Add bench_server_latency.py (#1452)

2024-09-18 00:56:06 -07:00

server.py

[Feature, Hardware] Enable SGLang on AMD GPUs via PyTorch for ROCm (#1420)

2024-09-17 07:43:52 +00:00

utils.py

feat: update linear deps 1/N (#1305)

2024-09-19 20:53:11 +08:00