Commit Graph

Select branches

Hide Pull Requests

dsv41-pd

dsv41-pd-visioncp

main

Mono Color

• b808a38365 Filter empty prompt in random bench serving (#2011) Ke Bao 2024-11-12 14:53:41 +08:00
• 602ebc661d remove sglang folder in rust (#2010) Byron Hsu 2024-11-11 20:45:52 -08:00
• 530ae1bdc8 Fix weight loading for tied word embedding when TP > 1 (#2009) Lianmin Zheng 2024-11-11 17:52:42 -08:00
• befc6beb86 Fix a typo in io_struct.py (#2008) Lianmin Zheng 2024-11-11 16:34:10 -08:00
• 59a5ba9be0 [Minor] Remove unused imports (#2006) Lianmin Zheng 2024-11-11 15:36:14 -08:00
• 86c37d010a fix sglang_router not found (#2005) Byron Hsu 2024-11-11 15:20:14 -08:00
• f18b9c7252 support internlm2-reward (#1994) RangiLyu 2024-11-12 07:09:58 +08:00
• 3e33574374 run rust test on ubuntu instead of 1-gpu-runner (#2003) Byron Hsu 2024-11-11 14:46:08 -08:00
• 0d94f1dd03 Bump router to 0.0.3 (#2004) Byron Hsu 2024-11-11 14:42:22 -08:00
• e728258d34 release router from py38 to py312 (#2002) Byron Hsu 2024-11-11 14:30:25 -08:00
• 239eafbd2e Fix rust unit test and pypi token (#2001) Byron Hsu 2024-11-11 14:18:21 -08:00
• 9d427265fd Add Engine::encode example (#2000) James Xu 2024-11-11 16:43:35 -05:00
• 00ffde206f setup router python binding ci (#1999) Byron Hsu 2024-11-11 12:19:32 -08:00
• ddeb9d42de Add engine encode (#1995) James XuandByron Hsu 2024-11-11 14:48:17 -05:00
• aaf0a3156e docs: add slides link in README (#1997) Yineng Zhang 2024-11-11 21:03:16 +08:00
• f9633fa9b9 [rust] cache-aware DP - approx tree (#1934) Byron Hsu 2024-11-10 21:57:32 -08:00
• 087ab83223 [Performance, Triton] Optimize over mask compute to tl.load in fused_moe_kernel (#1980) HAI 2024-11-10 18:54:43 -08:00
• 8169c6f4ef Add gen-shared-prefix dataset in bench_serving (#1990) Byron Hsu 2024-11-10 16:39:56 -08:00
• 3d043319aa [CI] Balance unit tests (#1988) Lianmin Zheng 2024-11-10 11:45:01 -08:00
• a8aad9357d qwen2vl fix bug for #1971 #1897 (#1984) yizhang2077 2024-11-11 00:10:45 +08:00
• 47ffe7af81 docs: add shm size for docker run (#1986) Yineng Zhang 2024-11-10 22:14:48 +08:00
• b3523af8eb fix: update pyzmq version (#1983) Yineng Zhang 2024-11-10 21:33:23 +08:00
• 1929c06762 Simplify prometheus metrics (#1981) Lianmin ZhengandMohit Reddy 2024-11-10 04:39:32 -08:00
• ed53ac84b4 Specify zmq Version Requirement (#1982) Huanzhi (Hans) Mao 2024-11-10 01:32:07 -08:00
• 520f0094e4 [CI] balance unit tests (#1977) Lianmin Zheng 2024-11-09 16:46:14 -08:00
• 9c939a3d8b Clean up metrics code (#1972) Lianmin Zheng 2024-11-09 15:43:20 -08:00
• 549e8b8366 [Minor] Fix a typo in test_torchao.py (#1976) Lianmin Zheng 2024-11-09 15:07:27 -08:00
• a1f32867ca Update pr-test-rust.yml to add a "finish" step (#1975) Lianmin Zheng 2024-11-09 13:53:35 -08:00
• 760552e068 Update README.md (#1974) Lianmin Zheng 2024-11-09 11:32:13 -08:00
• d9aada9db1 Introducing SGLang Guru on Gurubase.io (#1745) Kursat Aktas 2024-11-09 22:29:26 +03:00
• f11eb90fe4 Initialize model_worker_batch variable (#1973) Enrique Shockwave 2024-11-09 19:28:02 +00:00
• 95a4ed129a Fix metrics (#1963) Yudi Xue 2024-11-08 23:21:11 -08:00
• d1150e9a00 Updated Instructions on Profiling SGLang Infer System with AMD GPUs (#1966) leishaoSCandwunhuang 2024-11-08 23:19:03 -08:00
• e3126e3c5f Update README.md's Slack invitation link (#1962) Chayenne 2024-11-08 11:46:25 -08:00
• a509552087 [minor] Improve code style and compatibility (#1961) Lianmin Zheng 2024-11-08 02:19:41 -08:00
• 7ef0084b0d Add sentence_transformers to CI dependency (#1958) Lianmin Zheng 2024-11-08 01:21:29 -08:00
• f9a377f650 [Release, ROCm] release ROCm docker build for AMD MI GPUs (#1957) HAI 2024-11-08 00:14:15 -08:00
• 4ade15dd32 Adjust reward model's score module and pooler module order for reducing computation (#1956) aqweteddy 2024-11-08 16:10:54 +08:00
• 8dc84da084 Remove the useless to_srt_kwargs (#1955) Lianmin Zheng 2024-11-07 23:15:08 -08:00
• f16eb15d0d Gemma2 reward model support (#1954) aqweteddy 2024-11-08 14:42:27 +08:00
• 5bc2508b80 Monitoring documentation (#1933) Yudi Xue 2024-11-07 22:14:16 -08:00
• a71a44f203 Update setup_github_runner.md (#1952) Lianmin Zheng 2024-11-07 19:20:47 -08:00
• 691808d587 Add a timeout for execute-notebook.yml (#1951) Lianmin Zheng 2024-11-07 18:28:29 -08:00
• d32fba2a4d [ENV, ROCm] update environment settings (#1939) HAI 2024-11-07 18:24:36 -08:00
• 67c424cce3 [Performance, Triton Kernel Args] extend_attention, optimize kern args to _fwd_kernel (#1941) HAI 2024-11-07 18:24:02 -08:00
• 1ae270c5d0 [Doc] fix docs (#1949) Lianmin Zheng 2024-11-07 18:20:41 -08:00
• c77c1e05ba fix black in pre-commit (#1940) Chayenne 2024-11-07 15:42:47 -08:00
• dca87ec348 [Docs] fix 404 - Contributor Guide (#1942) HAI 2024-11-07 00:50:45 -08:00
• 4b1d7a2583 Add Rust Router Python Binding (#1891) Austin LiuandByronHsu 2024-11-07 10:08:30 +08:00
• a5e0defb5a minor: Add basic editorconfig and pre-commit hooks to enforce style for whitespaces (#1926) Xuehai Pan 2024-11-06 21:46:04 +08:00
• 96766101b4 [rust] refactor server and router (#1922) Byron Hsu 2024-11-06 00:02:02 -08:00
• a146d9990e support prometheus metrics (#1853) 3 people 2024-11-06 12:42:53 +08:00
• f5113e50ae [Doc] improve relative links and structure (#1924) Lianmin Zheng 2024-11-05 01:12:10 -08:00
• 02755768d3 Change judge to classify & Modify make file (#1920) Chayenne 2024-11-04 23:53:44 -08:00
• 463d56bf44 Update CODEOWNERS (#1916) Byron Hsu 2024-11-04 17:13:41 -08:00
• 530ff541cf [router] Impl radix tree and set up CI (#1893) Byron HsuandLianmin Zheng 2024-11-04 10:56:52 -08:00
• 3cd2809277 [Docs, ROCm] update install to cover ROCm with MI GPUs (#1915) HAI 2024-11-04 01:40:57 -08:00
• 704f8e8ed1 Add Reward API Docs etc (#1910) ChayenneandChayenne 2024-11-03 22:33:03 -08:00
• 1853c3523b Fix regex docs (#1909) Lianmin Zheng 2024-11-03 14:18:16 -08:00
• 65859754f1 Release v0.3.5 (#1908) Lianmin Zheng 2024-11-03 13:48:11 -08:00
• 2ce32db6fb Let reward model take text inputs instead of message lists (#1907) Lianmin ZhengandKyle Corbitt 2024-11-03 13:27:12 -08:00
• 793b79dbe9 feat: support truss endpoint for benchmark serving (#1906) Yineng Zhang 2024-11-03 12:56:10 -08:00
• 1363b51983 Escape backwards slash (#1902) Iñaki Arango 2024-11-03 12:27:11 -08:00
• 0abbf289a8 Unify the model type checking (#1905) Lianmin Zheng 2024-11-03 12:25:39 -08:00
• c17c578108 Simplify tokenizer manager (#1904) Lianmin Zheng 2024-11-03 08:38:26 -08:00
• 916b3cdddc Allow passing dtype and max_new_tokens to HF reference script (#1903) Jani Monoses 2024-11-03 18:24:37 +02:00
• 838dcda162 Simplify tokenizer manager (#1899) Lianmin Zheng 2024-11-03 03:52:38 -08:00
• efbc116a0f Do not use longest prefix matching when #queue-req is large (#1896) Lianmin Zheng 2024-11-03 01:45:20 -07:00
• 6aed0445ed turn off log (#1895) Chayenne 2024-11-03 00:19:12 -07:00
• 908dd7f9aa Add engine api (#1894) Chayenne 2024-11-02 22:03:38 -07:00
• f4cd804073 Fix ci and link error (#1892) ChayenneandChayenne 2024-11-02 19:08:49 -07:00
• be7986e005 Fix docs (#1890) Lianmin Zheng 2024-11-02 13:26:32 -07:00
• 5a5f18432f Fix docs ci (#1888) Chayenne 2024-11-02 11:57:22 -07:00
• 7b394e5f2b Fix docs (#1889) Lianmin Zheng 2024-11-02 11:46:00 -07:00
• 3b60558dd7 Native api (#1886) ChayenneandChayenne 2024-11-02 01:02:17 -07:00
• 5a9a4f41c6 Update index.rst (#1885) Lianmin Zheng 2024-11-02 00:20:33 -07:00
• 72e979bfb5 add native api docs (#1883) ChayenneandChayenne 2024-11-02 00:17:30 -07:00
• 146f613405 Fix incorrect context length for llama3.2-11b (#1873) Ran Chen 2024-11-02 00:04:50 -07:00
• 660ecb731f Fix doc links (#1882) Lianmin Zheng 2024-11-01 20:42:30 -07:00
• 2565cb0f40 Update docs and workflow (#1881) Lianmin Zheng 2024-11-01 20:29:41 -07:00
• 066e8a4ef0 Update docs title (#1879) Lianmin Zheng 2024-11-01 20:00:41 -07:00
• 2134f0898c Fix links in the docs (#1878) Lianmin Zheng 2024-11-01 18:21:14 -07:00
• a54f278d44 Add a FAQ documentation (#1877) Lianmin Zheng 2024-11-01 18:16:29 -07:00
• d1b31b0684 Improve docs and fix the broken links (#1875) Lianmin Zheng 2024-11-01 17:47:44 -07:00
• d59a47828c [3rdparty, document] Updated Documentation that covers performance tuning techniques for AMD Instinct GPUs. (#1871) jacky.chengandroot 2024-11-02 03:12:59 +08:00
• 104bf2609b minor: update nightly eval (#1867) Yineng Zhang 2024-11-01 21:38:29 +08:00
• 3bf3d011ed Add vlm document (#1866) ChayenneandChayenne 2024-11-01 00:51:15 -07:00
• d86a2d6562 minor: add human eval (#1754) Yineng Zhang 2024-11-01 14:29:20 +08:00
• 16eb33ffe2 Update vocab embedding deps and add TP switch (#1856) Ke Bao 2024-11-01 11:13:07 +08:00
• 61cf00e112 change file tree (#1859) ChayenneandChayenne 2024-10-31 20:10:16 -07:00
• b9fd178f1b Fix retraction + overlap (#1860) Liangsheng YinandLianmin Zheng 2024-10-31 18:27:42 -07:00
• d8e9d61f86 [Build, ROCm] Dockerfile.rocm for Instinct GPUs, with package updates (#1861) HAI 2024-10-31 16:38:16 -07:00
• a2e0424abf Fix memory leak for chunked prefill 2 (#1858) Lianmin ZhengandLiangsheng Yin 2024-10-31 14:51:51 -07:00
• 8ce202a493 delete unused character (#1855) geeker-smallwhite 2024-10-31 19:33:55 +08:00
• d913d52c9a Fix warnings in doc build (#1852) Lianmin Zheng 2024-10-30 22:28:00 -07:00
• 0ab7bcaf66 Simplify documentation in README.md (#1851) Lianmin Zheng 2024-10-30 21:57:49 -07:00
• 438526a814 Refactor tokenizer manager (#1846) Byron Hsu 2024-10-30 21:32:18 -07:00
• f7102fbd2b Fix mixed chunked prefill (#1850) Lianmin Zheng 2024-10-30 21:20:41 -07:00
• a7a0a6886b Make decode log interval configurable (#1847) Byron Hsu 2024-10-30 19:59:20 -07:00
• 2d4ce1b792 [Performance, Triton Kernel Args] _decode_grouped_softmax_reducev_fwd… (#1845) HAI 2024-10-30 17:33:36 -07:00