Commit Graph

Select branches

Hide Pull Requests

dsv41-pd

dsv41-pd-visioncp

main

Mono Color

• 155890e4d1 [Minor] fix documentations (#5756) Lianmin Zheng 2025-04-26 17:48:43 -07:00
• 1f963d7f64 Bugfix for minicpmo vision test (#5760) Yi Zhang 2025-04-26 23:18:02 +08:00
• 04d0123fd9 [Fix]: support deepseek-vl2-tiny model (#5552) ZXNandbppps 2025-04-26 17:52:53 +08:00
• feda9b11b3 fix: fix one more bug from merging mm_inputs (#5718) 3 people 2025-04-26 09:28:33 +09:00
• c3948ba67e Reorder loop in shared expert weight loading (#5719) Ke Bao 2025-04-26 08:27:42 +08:00
• 269c457e05 [Docs] Update runtime/engine/readme.md (#5737) Michael Yao 2025-04-26 07:39:29 +08:00
• 18ce468d56 update triton 3.2.0 h200 fused moe triton config and add warning about triton fused_moe_kernel performance degradation due to different Triton versions. (#5740) Xiaoyu Zhang 2025-04-26 07:24:59 +08:00
• 21514ff5bd Disable flaky eagle tests (#5753) Lianmin Zheng 2025-04-25 15:54:39 -07:00
• 5641a09458 Revert "[Model] Support ArcticForCausalLM architecture (Snowflake/snowflake-arctic-instruct)" (#5754) Lianmin Zheng 2025-04-25 15:50:28 -07:00
• 3dd3538c18 Pin torch audio to 2.6.0 (#5750) Lianmin Zheng 2025-04-25 15:06:28 -07:00
• 93c6fb12c7 Fix: deepseek forward absorb (#5723) michael-amdandispobock 2025-04-25 13:48:55 -07:00
• 11e27d0926 [PD]: Support Muti Prefill in one node (#5704) IANandshuaills 2025-04-26 00:30:47 +08:00
• 50eda8398e [PD] Add kvargs table and thread pool for kvcache sender of mooncake (#5738) shangmingc 2025-04-25 18:15:01 +08:00
• c55550cbf0 [PD] Better logs (#5715) Liangsheng Yin 2025-04-25 17:25:45 +08:00
• 43fb95c2fa [Model] Support ArcticForCausalLM architecture (Snowflake/snowflake-arctic-instruct) (#5078) Brayden Zhongandvincent-4 2025-04-25 03:24:09 -04:00
• 7d9679b74d Add MMMU benchmark results (#4491) Ravi ThejaandRavi Theja Desetty 2025-04-25 12:53:53 +05:30
• b5be56944b [Doc] Fix a link to Weilin Zhao (#5706) Michael Yao 2025-04-25 02:02:27 +08:00
• d2b8d0b8d8 Add example to use sgl engine with fastapi (#5648) Ravi ThejaandRavi Theja Desetty 2025-04-24 21:27:05 +05:30
• a14654dd68 Fix weight loading bug for Deepseek v3+nextn (#5684) Baizhou Zhang 2025-04-24 06:29:56 -07:00
• 5d93a950ee [BugFix] Fix combination of MTP and --n-share-experts-fusionwith R1 (#5707) Yuhong Guo 2025-04-24 21:13:51 +08:00
• c998d04b46 vlm: enable radix cache for qwen-vl models (#5349) MickandXinyuan Tong 2025-04-24 12:35:05 +09:00
• 7d0edf3cae chore: bump sgl-kernel 0.1.0 (#5688) Yineng Zhang 2025-04-23 14:23:59 -07:00
• ce4ecba477 fix: only compile ApplyTokenBitmaskInplace cu124+ (#5686) Yineng Zhang 2025-04-23 14:17:42 -07:00
• b1f6d89b5f fix: update truss bench_serving (#5683) Yineng Zhang 2025-04-23 13:28:35 -07:00
• 7c99103f4c [Doc] Fix two 404 links caused by sglang typo (#5667) Michael Yao 2025-04-23 23:21:55 +08:00
• de071366cd tune the threshold of gemma-2-27b-it in test_nightly_gsm8k_eval.py (#5677) Lianmin Zheng 2025-04-23 05:31:17 -07:00
• e0673969b9 [PD] Add support for dp attention with mooncake (#5530) shangmingc 2025-04-23 17:20:27 +08:00
• 127ff8982e fix torchvision::nms not exist (#5671) Yineng Zhang 2025-04-23 02:17:21 -07:00
• 8777a1d24b fix gemma3 unit test (#5670) Yineng Zhang 2025-04-23 02:14:01 -07:00
• 711efe7814 Integrating PD disaggregation with DP attention and DeepEP (#5435) Cheng WanandByron Hsu 2025-04-23 01:46:01 -07:00
• fbb5f229d4 fix awq_dequantize import (#5669) Yineng Zhang 2025-04-23 01:36:26 -07:00
• 15fabcc07f fix sgl-kernel unit tests (#5666) Yineng Zhang 2025-04-23 01:18:30 -07:00
• e62c49557d [1/2] Add FP8 Blockscale MoE CUTLASS kernel for Blackwell (#5281) Elfie Guo 2025-04-22 22:28:20 -07:00
• 71d1785f2d Remove unnecessary torch.full in DeepSeek (#5601) fzyzcjy 2025-04-23 12:24:29 +08:00
• 3f87f83116 Fuse q_a_proj and kv_a_proj (#5619) Baizhou Zhang 2025-04-22 20:35:08 -07:00
• ce5412b62e Turn on DeepGemm By Default and Update Doc (#5628) Baizhou Zhang 2025-04-22 16:10:08 -07:00
• 7282ab741a fix: update bench_speculative (#5649) Yineng Zhang 2025-04-22 16:08:15 -07:00
• b0feda090c Revert "Support aiter RMSNorm in AMD" (#5646) HAI 2025-04-22 15:20:24 -07:00
• 6b6e748775 Remove q concat in FA3 backend for DeepSeek decode (#5638) Ke Bao 2025-04-23 02:43:12 +08:00
• 917324862e [fix] reduce dp capture bs (#5634) JieXin Liangandalcanerian 2025-04-23 02:08:45 +08:00
• 2ed96c7a8a fix flashmla bug (#5272) lukec 2025-04-23 01:36:23 +08:00
• 2aa3f5e2d0 [feature] Add H20 fp8_w8a8 FusedMoE config for --n-share-experts-fusion=16 (#5641) saltyfish66andyuethe 2025-04-23 00:33:13 +08:00
• 76d17c7ecb Fix shared experts fusion error without quantization (#5632) lambert0312 2025-04-23 00:22:26 +08:00
• 70d040f904 [NFC] Remove duplicate compressed-tensors (#5640) Connector Switch 2025-04-23 00:10:25 +08:00
• 4418f599a5 Fix FA3 DeepSeek prefill performance regression (#5624) JieXin Liangandispobock 2025-04-22 16:41:41 +08:00
• 04f2abcb34 fix: gemma 3 not use softcap (#5622) Yineng Zhang 2025-04-22 01:16:08 -07:00
• 506be6b892 [fix] fix compile_deep_gemm missing kv_b_proj (#5620) JieXin Liang 2025-04-22 15:06:36 +08:00
• 2343d8df7d [fix] force use deepgemm in compile_deep_gemm (#5618) JieXin Liang 2025-04-22 12:36:02 +08:00
• 92bb64bc86 [Doc] Fix a 404 link to llama-405b (#5615) Michael Yao 2025-04-22 11:39:37 +08:00
• 11b23ae97b Remove extra copy in deepseek forward absorb (#5578) Ke Baoandsaienduri 2025-04-22 10:33:21 +08:00
• b9c87e781d chore: bump v0.4.5.post3 (#5611) Yineng Zhang 2025-04-21 18:16:20 -07:00
• 968ef51562 Support aiter RMSNorm in AMD (#5510) michael-amdandJieXin Liang 2025-04-21 17:40:39 -07:00
• 1343200299 Clean up mem settings (#5610) Lianmin Zheng 2025-04-21 17:19:00 -07:00
• c2942907d5 [feature] enable pre compile jit deep_gemm (#5580) JieXin Liang 2025-04-22 07:52:53 +08:00
• e69a219074 Enhance GPU memory settings (#5604) Liangsheng Yin 2025-04-22 06:15:00 +08:00
• bf98d2e377 [PD] Support prefill overlap + Ensure no race condition (#5609) Byron Hsu 2025-04-21 12:12:56 -07:00
• e65b9f21e3 [PD] Support decode overlap schedule (#5608) Byron Hsu 2025-04-21 12:06:16 -07:00
• 4dce1cc608 [PD] Add NIXL transfer backend (#5477) Trevor Morris 2025-04-21 10:36:12 -07:00
• deded17f38 [PD] Fix edge case and simplify large page size + chunked prefill (#5589) Byron Hsu 2025-04-21 10:27:02 -07:00
• f29a718f63 [PD] Fix generate endpoint of min_lb for PD (#5598) shangmingc 2025-04-21 21:39:18 +08:00
• 3f57b00a59 Support PD bootstrap fields on /v1/chat/completions endpoint (#5488) Yongtong Wu 2025-04-21 16:10:58 +08:00
• 453d412cdb Tiny update error hint (#5037) fzyzcjy 2025-04-21 15:47:47 +08:00
• dc86f25a57 Tiny remove duplicated code (#5021) fzyzcjy 2025-04-21 15:47:32 +08:00
• 08289eaa3e Support o1 model on Azure (#4980) Chuyue SunandShan Yu 2025-04-21 00:46:09 -07:00
• 3b6d539f63 [Fix] Enhance DP Attention for IPv6 Compatibility (#4937) Lucius 2025-04-21 15:44:11 +08:00
• 57131dd955 [Feat.] Enable grafana to show metrics (#4718) Huapeng Zhouandzhaochenyang20 2025-04-21 15:43:42 +08:00
• a7591ecf69 Update protocol.py to fix #4589 (#4590) moontidef 2025-04-21 15:42:50 +08:00
• c44f2869c9 Modify metrics service endpoint (#3443) lambert0312 2025-04-21 15:35:38 +08:00
• 685d8980c3 Tiny add warning when cannot recognize bool env var (#5348) fzyzcjy 2025-04-21 14:11:29 +08:00
• 70645f4d7d upstream hicache fixes (#5570) Zhiqiang Xie 2025-04-20 23:08:30 -07:00
• 188f0955fa Add Speculative Decoding Eagle3 topk > 1 (#5318) 3 people 2025-04-20 22:58:28 -07:00
• eef9433b46 Fix flush cache (#5590) Lianmin Zheng 2025-04-20 22:56:40 -07:00
• 97cb762bb6 [misc] remove is_cuda_available (#5319) JieXin Liang 2025-04-21 09:16:51 +08:00
• 1195182040 Tiny add Engine.flush_cache API (#5241) fzyzcjy 2025-04-21 09:15:03 +08:00
• 5239d79568 Speedup shared expert weight construction by avoid cloning (#5188) fzyzcjy 2025-04-21 09:12:01 +08:00
• f08154193c Perform Batch Tokenization. (#5141) Sundara Raman Ramachandran 2025-04-20 18:10:37 -07:00
• 2b3bdc938e Correct grafana heatmap. (#5019) mac0ne 2025-04-21 08:58:56 +08:00
• 5fc4b6004e Add sanity check for max_running_requests (#5016) fzyzcjy 2025-04-21 08:56:49 +08:00
• b868526d94 Fix one more issue reported by torchfix (#4859) Brayden Zhong 2025-04-20 20:49:27 -04:00
• 502524e2da compressed_tensors: port w8a16 fp8 from vllm (#4852) Juwan Yoo 2025-04-21 09:48:31 +09:00
• 4c7640079c check marlin format before attempting conversion (#4675) Enrique Shockwave 2025-04-21 01:47:09 +01:00
• 9f3bd2ad39 Feat: Implement JSON Mode (response_format.type="json_object") (#4733) kyle-pena-kuzcoandKyle Pena 2025-04-20 20:41:22 -04:00
• 8de53da989 smaller and non gated models for docs (#5378) simveit 2025-04-21 02:38:25 +02:00
• fac17acf08 add function call parser for DeepSeek V3 (#5224) Yi Zhou 2025-04-21 08:38:08 +08:00
• 8b39274e34 [Feature] Prefill assistant response - add continue_final_message parameter (#4226) Adarsh ShirawalmathandChayenne 2025-04-21 06:07:18 +05:30
• 5156d5a413 Add test config yamls for Deepseek v3 (#5433) Baizhou Zhang 2025-04-20 17:28:52 -07:00
• c951d312ed [PD] Fix large page size + chunk prefill (#5588) Byron Hsu 2025-04-20 17:21:54 -07:00
• dcb8232596 Fix ChatCompletionMessageGenericParam to allow for None content (#5452) AmadeusW 2025-04-21 08:15:38 +08:00
• 66c0ff9e31 fix: use fa3 for gemma2 (#5586) Yineng Zhang 2025-04-20 17:02:09 -07:00
• 9a7e83e899 Fix enable chunked prefill for Llama4 (#5575) tarinkk 2025-04-20 20:01:30 -04:00
• 417b44eba8 [Feat] upgrade pytorch2.6 (#5417) lukec 2025-04-21 07:06:34 +08:00
• 475e2e378a [PD] Fix server crash when using batch requests (#5531) fzyzcjy 2025-04-21 07:02:23 +08:00
• fba86b6b54 Tiny improve error message (#5526) fzyzcjy 2025-04-21 07:00:15 +08:00
• 072b4d0398 Add document for LoRA serving (#5521) Baizhou Zhang 2025-04-20 14:37:57 -07:00
• 9c43477710 Super tiny fix typo (#5559) fzyzcjy 2025-04-21 05:21:18 +08:00
• fa2f677e18 Fix torch memory saver not enabled in DP scenario (#5560) fzyzcjy 2025-04-21 05:20:52 +08:00
• 463d4b7400 Fix DeepEP cannot run on latest master (#5567) fzyzcjy 2025-04-21 05:19:42 +08:00
• 9924bbe153 Fix bench_serving fail when zero warmup requests (#5574) fzyzcjy 2025-04-21 05:16:03 +08:00
• fbdc94ba59 Release v0.4.5.post2 (#5582) Lianmin Zheng 2025-04-20 14:12:37 -07:00
• b54b5a96e4 [Doc]Add instruction for profiling with bench_one_batch (#5581) Baizhou Zhang 2025-04-20 14:05:36 -07:00