Commit Graph

Select branches

Hide Pull Requests

dsv41-pd

dsv41-pd-visioncp

main

Mono Color

• c1e097ca66 Revert "Dump requests to a folder" (#2869) Lianmin Zheng 2025-01-13 06:21:25 -08:00
• 6ec75e626d add qwen2 eagle model (#2863) Lzhang-hub 2025-01-13 21:29:33 +08:00
• d855653bd4 minor: fix release docs (#2868) Yineng Zhang 2025-01-13 21:18:39 +08:00
• 336ff5b9f5 Fix typos in io_struct.py (#2867) Lianmin Zheng 2025-01-13 05:13:02 -08:00
• 3b141e1509 Dump requests (#2862) Lianmin Zheng 2025-01-13 04:51:56 -08:00
• 6249e4a19e Revert "Integration of TurboMind AWQ" (#2866) Lianmin Zheng 2025-01-13 04:44:39 -08:00
• f3516c2894 Fix quant kernel accuracy issue (#2865) Ke Bao 2025-01-13 20:32:17 +08:00
• 17de02f98d Integration of TurboMind AWQ (#2828) bjmsongandroot 2025-01-13 20:14:16 +08:00
• 51ab3ccf47 Collect more metrics: num_requests_total (#2859) Lianmin Zheng 2025-01-13 03:57:39 -08:00
• 67008f4b32 Use only one GPU for MLA CI tests (#2858) Lianmin Zheng 2025-01-13 03:55:33 -08:00
• 4536d72446 minor: use ubuntu-latest instead of self-hosted runner for amd build (#2861) Yineng Zhang 2025-01-13 18:58:56 +08:00
• 41d7e5b7e6 docs: update link (#2857) Yineng Zhang 2025-01-13 18:40:48 +08:00
• 20a9f5dfe0 fix: not delete CNAME (#2860) Yineng Zhang 2025-01-13 18:36:40 +08:00
• 42f3909963 Unify sglang coding style (#2856) kkandLin, Soga 2025-01-13 18:12:44 +08:00
• 72c7776355 Fix linear.py and improve weight loading (#2851) Lianmin ZhengandSangBin Cho 2025-01-13 01:39:14 -08:00
• 4093aa4660 [Fix]eagle2 health_generate is first request,apiserver will core (#2853) justdoit 2025-01-13 17:01:21 +08:00
• e808c1df3e Integrate ROCm ater package for ck moe function feasibility (#2854) 3 people 2025-01-13 16:23:07 +08:00
• a18ab81ddd Update base image for ROCm (#2852) sogalinandHAI 2025-01-13 14:39:44 +08:00
• 0bb0f76311 Support FP8 E4M3 KV Cache (#2786) bjmsongandroot 2025-01-13 13:17:11 +08:00
• 85b2e05770 Add int8 quant kernel (#2848) Ke Bao 2025-01-13 13:16:58 +08:00
• a879c2fb4c fix sgl-kernel build (#2850) Yineng Zhang 2025-01-13 12:27:17 +08:00
• e2b16c4716 add sampling_scaling_penalties kernel (#2846) Xiaoyu Zhang 2025-01-13 11:38:17 +08:00
• c4f9707e16 Improve: Token-In Token-Out Usage for RLHF (#2843) Shi Shuai 2025-01-11 23:14:26 +00:00
• 197cbf9bab docs: update README (#2841) Yineng Zhang 2025-01-11 23:11:38 +08:00
• f624901cdd chore: bump v0.4.1.post5 (#2840) Yineng Zhang 2025-01-11 23:10:02 +08:00
• f0e15dc6ab [HotFix] fix fp8 scale load failed in tp>1 (#2837) Xiaoyu Zhang 2025-01-11 14:34:26 +08:00
• f1769586d6 Update threshold in test_nightly_gsm8k_eval.py (#2836) Lianmin Zheng 2025-01-10 20:37:34 -08:00
• 5d6e9467d4 Cache controller for hierarchical caching (#2804) Zhiqiang Xie 2025-01-10 20:22:01 -08:00
• a47bf39123 [Eagle2] Fix multiple concurrent request crashes (#2730) justdoit 2025-01-11 06:00:43 +08:00
• b170646991 Fix port number overflow (#2826) TianYu GUO 2025-01-11 05:44:32 +08:00
• 5413ec2bbe [Bugfix] Fix bug in fork logic caused by null text_ (#2835) Muqi Li 2025-01-11 05:37:00 +08:00
• f290bd4332 [Bugfix] Fix embedding model hangs with --enable-metrics (#2822) Chang Su 2025-01-10 13:14:51 -08:00
• 8f15789314 Add more metrics to serving benchmark. (#2819) Pratyush Patel 2025-01-10 07:30:44 -08:00
• 2db03a04ca Update README.md (#2833) Lianmin ZhengandHeiner 2025-01-10 03:49:04 -08:00
• 5cc1170552 Doc: add block-wise FP8 in dpsk model reference (#2830) Chayenne 2025-01-10 00:26:59 -08:00
• 11fffbc95a [Doc]: Deepseek reference docs (#2787) Xiaotong Jiang 2025-01-09 13:43:12 -08:00
• 4f077c01b8 minor: support specifying local dataset path for gsm8k and hellaswag (#2816) sleepcoo 2025-01-09 22:24:42 +08:00
• 679c3bcacf Fix typo in cuda_graph_bs (#2813) Lianmin Zheng 2025-01-09 03:03:24 -08:00
• 656aed58c6 Remove vllm dependency in model config (#2809) Yunmeng 2025-01-09 17:51:56 +08:00
• b5fb4ef58a Update modelopt config and fix running issue (#2792) Ke Bao 2025-01-08 18:04:30 +08:00
• 2e6346fc2e Docs：Update the style of llma 3.1 405B docs (#2789) Chayenne 2025-01-08 01:07:54 -08:00
• 977f785dad Docs: Rewrite docs for LLama 405B and ModelSpace (#2773) mlmzandChayenne 2025-01-08 16:02:59 +08:00
• 8a6906127a Improve linear.py to load sharded weights & remove the dependency of Parameters from vllm (#2784) Lianmin ZhengandSangBin Cho rkooo567@gmail.com 2025-01-07 23:29:10 -08:00
• 694e41925e [eagle2] fix end check when target model verify (#2723) JJJJOHNSON 2025-01-08 13:46:02 +08:00
• b22f3f6475 Fix nightly accuracy tests (#2780) Lianmin Zheng 2025-01-07 21:02:35 -08:00
• 6fb5768372 Disable math eval on nightly CI temporarily (#2779) Lianmin Zheng 2025-01-07 18:17:34 -08:00
• 51caee740f Host memory pool for hierarchical caching (#2771) Zhiqiang Xie 2025-01-07 13:38:37 -08:00
• 58f9060efe Update int8 gemm config (#2774) Ke Bao 2025-01-07 19:47:37 +08:00
• bdc1acf6cd Misc fix for min_p_sampling, --cuda-graph-bs (#2761) Lianmin Zheng 2025-01-07 02:52:53 -08:00
• 6d08ce2aa9 Use Optional with None default (#2770) HAI 2025-01-07 01:35:08 -08:00
• 380930a959 add benchmark_moe_align_blocks (#2767) Xiaoyu Zhang 2025-01-07 14:20:50 +08:00
• 9dec582dab Remove --modelopt-config in server_args (#2758) Lianmin Zheng 2025-01-06 16:35:45 -08:00
• b01febdca0 Update README.md (#2757) 3 people 2025-01-06 15:36:23 -08:00
• 1acbaf1b5a Add generator-style run_batch function (#2513) Xingyao Wangandopenhands 2025-01-06 18:04:55 -05:00
• 287427e2e6 Enable Nvidia's ModelOpt fp8 quantized models (#2535) Zhiyu 2025-01-06 14:54:52 -08:00
• b8574f6953 Clean up eagle code (#2756) Lianmin Zheng 2025-01-06 14:54:18 -08:00
• 2855caa481 feat: add devcontainer.json for VSCode development (#2745) 王博伟 2025-01-07 06:00:55 +08:00
• 2329e1ddd0 Support llamafy/Qwen-Qwen2.5-7B-Instruct-llamafied (#2748) Xu-Chenandchenxu02 2025-01-07 05:56:28 +08:00
• 0f3eb1d294 Support cutlass Int8 gemm (#2752) Ke Bao 2025-01-06 22:51:22 +08:00
• 06dd2eab84 Remove unused var in moe_align_kernel (#2751) Ke Bao 2025-01-06 22:13:28 +08:00
• 439f65809f Fix sgl-kernel cu118 compile issue (#2750) Ke Bao 2025-01-06 21:59:31 +08:00
• 2f0d386496 chore: bump v0.4.1.post4 (#2713) Yineng Zhang 2025-01-06 01:29:54 +08:00
• 3900a94afe Support twoshot kernel (#2688) yizhang2077 2025-01-06 00:47:16 +08:00
• ded9fcd09a improve moe_align_kernel for deepseek v3 (#2735) Xiaoyu Zhang 2025-01-06 00:28:22 +08:00
• bc6ad367c2 fix lint (#2733) Yineng Zhang 2025-01-05 14:45:42 +08:00
• 3a22a303d1 Revert the GLOO_SOCKET_IFNAME change (#2731) Lianmin Zheng 2025-01-04 20:13:16 -08:00
• bdb3929dbb Refactor SchedulePolicy to improve code organization (#2571) libra 2025-01-04 00:05:16 +08:00
• f5d0865b25 feat: Support VLM in reference_hf (#2726) Ce Gao 2025-01-03 22:32:30 +08:00
• afdee7b1a9 [Docs] fix 404 - Contributor Guide, again (#2727) Ce Gao 2025-01-03 22:21:38 +08:00
• cb34d848ac Update README.md (#2722) 4 people 2025-01-03 00:32:20 -08:00
• 0f9cc6d8d3 Fix package loss for small models (#2717) Lianmin Zhengandsdli1995 2025-01-02 18:25:26 -08:00
• c7ae474a49 [Feature, Hardware] Enable DeepseekV3 on AMD GPUs (#2601) 5 people 2025-01-03 08:23:19 +08:00
• bdf946bf81 Support loading pre-sharded moe weights (#2716) Lianmin Zheng 2025-01-02 15:07:37 -08:00
• 8c8779cd05 [Fix] fix retract error in eagle speculative decoding (#2711) yukavioandkavioyu 2025-01-03 02:28:39 +08:00
• 1775b963db [Fix] fix incorrectly overwriting the port specified in ServerArgs (#2714) Mick 2025-01-03 02:28:22 +08:00
• dd2e2d275f Docs: Update documentation workflow and contribution guide (#2704) Shi ShuaiandChayenne 2025-01-02 17:18:31 +00:00
• a990daff9c Included multi-node DeepSeekv3 example (#2707) Rodrigo Garcia 2025-01-02 15:17:03 +01:00
• ba5112ff69 feat: support moe_align_block_size_triton (#2712) Yineng ZhangandWANDY666 2025-01-02 21:47:44 +08:00
• 815dce0554 Eagle speculative decoding part 4: Add EAGLE2 worker (#2150) 3 people 2025-01-02 19:22:34 +08:00
• ad20b7957e Eagle speculative decoding part 3: small modifications to the general scheduler (#2709) Lianmin Zhengandkavioyu 2025-01-02 02:09:08 -08:00
• 9183c23eca Speed up update_weights_from_tensor (#2695) fzyzcjy 2025-01-02 18:05:19 +08:00
• 148254d4db Improve moe reduce sum kernel performance (#2705) kkandwunhuang 2025-01-02 17:11:06 +08:00
• a4d6d6f1dd [feat]: Add math eval to CI nightly run (#2663) Xiaotong JiangandChayenne 2025-01-01 15:29:35 -08:00
• 062c48d2bd [Docs] Add Support for Pydantic Structured Output Format (#2697) Shi Shuai 2025-01-01 23:08:43 +00:00
• b6e0cfb5e1 ROCm base image update (#2692) kkandwunhuang 2025-01-01 12:12:19 +08:00
• 0d8d97b8e6 Doc: Rename contribution_guide.md (#2691) Chayenne 2024-12-31 14:35:48 -08:00
• 0a765bbccc Docs: Refactor Contribution Guide (#2690) Shi Shuai 2024-12-31 22:11:00 +00:00
• 286cad3ee3 h200 tuning fused_moe_triton config for Mixtral 8x7B/8x22B and Qwen2 57BA14B (#2689) Xiaoyu Zhang 2024-12-31 23:17:36 +08:00
• dc7eb01f19 [Fix] fix openai adapter (#2685) Ying Sheng 2024-12-31 02:48:19 -08:00
• b0524c3789 Eagle speculative decoding part 2: Fix cuda graph + DP attention hanging (#2684) Lianmin Zhengandyukavio 2024-12-31 02:25:05 -08:00
• 6c42fa229d Update README.md (#2683) Lianmin Zheng 2024-12-31 00:13:10 -08:00
• d49b13c6f8 feat: use CUDA 12.4 by default (for FA3) (#2682) Yineng Zhang 2024-12-31 15:52:09 +08:00
• bedc4c7a50 misc: update CODEOWNERS (#2680) Yineng Zhang 2024-12-31 15:04:50 +08:00
• f44d143949 Support target model verification in the attention backend (#2678) Lianmin Zhengandyukavio 2024-12-30 22:58:55 -08:00
• b6b57fc200 minor: cleanup sgl-kernel (#2679) Yineng Zhang 2024-12-31 14:52:00 +08:00
• b4403985d0 Add cutlass submodule for sgl-kernel (#2676) Ke Bao 2024-12-31 14:28:29 +08:00
• 339c69a243 Improve the computation for time_per_output_token Prometheus metrics (#2674) Lianmin Zheng 2024-12-30 21:40:14 -08:00
• f707470019 CI: Update scripts to fail fast (#2672) fzyzcjy 2024-12-31 11:04:01 +08:00
• 21ec66e59e Minor follow-up fixes for the logprob refactor (#2670) Lianmin Zheng 2024-12-30 05:42:08 -08:00
• c5210dfa38 AMD DeepSeek_V3 FP8 Numerical fix (#2667) HAI 2024-12-30 05:31:12 -08:00