Commit Graph

Select branches

Hide Pull Requests

dsv41-pd

dsv41-pd-visioncp

main

Mono Color

• c7c7dbebbe [PD] Release initial code (#4654) 6 people 2025-03-21 14:47:47 -07:00
• 417fc72f6f Align completion and chat_completion response to OpenAI API (#4637) Yuhong Guo 2025-03-21 13:59:04 +08:00
• c6ec70290f [docs] Add links and fix grammars in deploy_on_k8s.md (#4641) Michael Yao 2025-03-21 13:55:23 +08:00
• 4c56e5dbee Set deepgemm to the default value in the hopper architecture. (#4613) lukec 2025-03-21 13:03:00 +08:00
• 7b5fc71972 fix SUPPORT_CUTLASS_BLOCK_FP8 flag (#4640) Cheng Wan 2025-03-21 00:45:07 -04:00
• ad4e58bf67 Support fp8 gemm for blackwell (#4558) Shu Wang 2025-03-20 14:40:28 -05:00
• bfb03c6182 Update doc for MTP and DP attention (#4622) Ke Bao 2025-03-21 02:31:48 +08:00
• b36ab493b3 Enable setting sglang logger from Env Variable SGLANG_LOGGING_CONFIG_PATH (#4592) Yuhong Guo 2025-03-20 17:10:32 +08:00
• 9e93ef3f8e [fix] fix illegal mem access and clean up triton attention backend (#4571) JieXin Liang 2025-03-20 17:01:52 +08:00
• fad86a6863 Support n in OpenAI API completions (#3446) 4 people 2025-03-19 22:46:46 -07:00
• df7014a8d2 avoid cudaStreamSynchronize in DeepSeekV2AttentionMLA (#4577) strgrbandZhang Kaihong 2025-03-20 01:02:26 +08:00
• 4942074174 [fix] fix initialization of _ENABLE_TORCH_INFERENCE_MODE (#4549) JieXin Liang 2025-03-20 00:57:59 +08:00
• ba52fd1868 Add clang-format to pre-commit config (#4583) Hongbo Xu 2025-03-20 00:50:19 +08:00
• b6944f97a6 Support FlashMLA backend cuda graph (#4514) 4 people 2025-03-19 23:25:34 +08:00
• f44db16c8e [Feature] Integrate DeepEP into SGLang (#4232) 3 people 2025-03-19 23:16:31 +08:00
• f9c53cbb42 Create col-major and tma-aligned x_scale for deep_gemm.gemm_fp8_fp8_bf16_nt (#4515) strgrbandZhang Kaihong 2025-03-19 15:02:43 +08:00
• 90532b7627 [Fix] Fix raw_bs bug when using flashinfer mla and eagle (#4557) Baizhou Zhang 2025-03-18 21:26:53 -07:00
• c0e9a36c5f Optimize Triton decoding kernel for dynamic workload (#4553) JieXin Liang 2025-03-19 12:25:38 +08:00
• 588865f0e0 [Feature] Support Tensor Parallelism and Weight Slicing for Lora (#4274) 3 people 2025-03-18 23:33:07 -04:00
• 3196999f63 Reduce computation and communication in DP attention (#4521) Cheng Wan 2025-03-18 16:41:36 -04:00
• 9e0186f352 [Feature] Support EAGLE 3 (#4247) James Liu 2025-03-18 10:35:23 -04:00
• 8baf9a0c18 [Fix] Type annotation correction for UpdateWeightsFromTensorReqInput (#4532) Wei WuandZhiqiang Xie 2025-03-18 15:52:47 +08:00
• c787298547 use sgl custom all reduce (#4441) Yineng Zhang 2025-03-18 00:46:41 -07:00
• 45212ce18b Add deepseek v2 torch compile pr test (#4538) Ke Bao 2025-03-18 15:29:24 +08:00
• c16b33ccac cleanup deps 3/n (#4541) Yineng Zhang 2025-03-18 00:11:36 -07:00
• 2d0045125f Fix the incorrect args in benchmark_and_profiling.md (#4542) Albert 2025-03-18 15:07:06 +08:00
• 804d250a0d remove useless backend forward in rotary_embedding (#4500) Xiaoyu Zhang 2025-03-18 14:54:00 +08:00
• dd865befde [Hotfix] solve fp8 w8a8 ci test fail (#4531) Xiaoyu Zhang 2025-03-18 14:17:04 +08:00
• d373a48c98 fix: second_per_grid_ts should be used to get mrope position (#3682) Mick 2025-03-18 09:12:38 +08:00
• 98be3bd306 refactor: rewrite bench-mmmu-sglang (#4458) Mick 2025-03-18 09:11:47 +08:00
• a98290aea3 Unit test for Hierarchical Caching (#4486) Zhiqiang Xie 2025-03-17 17:45:00 -07:00
• 9b81f9bd34 sglang quant module remove vllm dependency (#4507) Xiaoyu Zhang 2025-03-18 06:51:59 +08:00
• f81a27f65e upgrade sgl-kernel 0.0.5.post3 (#4522) Yineng Zhang 2025-03-17 14:49:56 -07:00
• 988ab646ec bump v0.0.5.post3 (#4520) Yineng Zhang 2025-03-17 13:05:59 -07:00
• 3ded4b215d Revert "feat: update grouped_topk to support softmax and sigmoid" (#4505) Ke Bao 2025-03-18 02:30:26 +08:00
• f4d7ab7a63 [sgl-router] improvement to avoid hang (#4482) 3 people 2025-03-17 10:37:50 -07:00
• c38ca4fc8e Update readme (#4517) Lianmin Zheng 2025-03-17 08:22:42 -07:00
• 82dec1f70b Remove redundant type conversion (#4513) Lianmin Zheng 2025-03-17 05:57:35 -07:00
• 5f9b2c62ff [ROCm] fix dtype (#4510) yiakwy-xpu-ml-framework-team 2025-03-17 20:20:50 +08:00
• 5493c3343e Fix data parallel + tensor parallel (#4499) Lianmin Zheng 2025-03-17 05:13:16 -07:00
• f2ab37e500 [Doc] add doc for quantization w8a8_fp8 or w8a8_int8 (#4495) HandH1998 2025-03-17 17:25:00 +08:00
• 91ba98fe50 [Fix] Resolve GPU Memory Leak in update_weights_from_tensor (#4446) Wei Wu 2025-03-17 16:54:30 +08:00
• c614dbdf95 Nicer standalone engine inferface (#4480) Yinghai Lu 2025-03-17 01:42:04 -07:00
• 927ca935a7 Constraint Decoding: Tool call with text (#4067) Xihuai Wang 2025-03-17 16:06:46 +08:00
• ef3c2dd08e Support Online Quantization for W8A8 (#4485) Stefan He 2025-03-17 00:28:56 -07:00
• 75b656488a Support serving DeepSeek-R1-Channel-INT8 with 32 L40S. (#4418) Wenbo Yang 2025-03-17 15:03:43 +08:00
• 0f52fb55ec config: Update fused moe config (#4493) Mick 2025-03-17 14:51:58 +08:00
• d6d21640d3 [Feature] Support Deepseek-VL2 (#2798) 4 people 2025-03-17 14:07:59 +08:00
• 0212d2e288 [Fix] use torch.inference_mode() instead of torch.no_grad() (#4372) JieXin Liang 2025-03-17 13:54:16 +08:00
• 8cc300f536 Fix router test (#4483) Byron Hsu 2025-03-16 22:49:47 -07:00
• 452db50808 Constraint Decoding: Set xgrammar as the default grammar backend (#4386) mlmz 2025-03-17 09:53:43 +08:00
• d1112d8548 Add endpoint for file support, purely to speed up processing of input_embeds. (#2797) Rin Intachuen 2025-03-17 08:30:37 +07:00
• 48efec7b05 Feature: support code completion (#3612) woodx 2025-03-17 09:26:19 +08:00
• 9b8333d992 [ROCm] enable moe topk softmax in amd (#4448) yiakwy-xpu-ml-framework-team 2025-03-17 09:16:55 +08:00
• f5bbf6037d Fix: Complete int32 to int64 conversion (#4465) Zhiqiang Xie 2025-03-16 18:14:27 -07:00
• 5cbd709ea1 Fix: modelscope env comment (#4474) huiwq1990 2025-03-17 09:11:33 +08:00
• 2e4a1e2d05 Initialize image processor for skip-tokenizer-init codepath (#4479) Yinghai LuandAlex Kirillov 2025-03-16 18:10:09 -07:00
• 9d02bb3e2a Urgent model support: support gemma-3-it (#4424) Mick 2025-03-17 08:37:32 +08:00
• 402db5c58c Benchmark: Statistical Analysis of the Output Stability of the Deepseek Model (#4202) ZelinTanandChayenne 2025-03-17 08:32:57 +08:00
• 754a0e8278 Update CODEOWNERS (#4484) Lianmin Zheng 2025-03-16 17:10:15 -07:00
• 799fb5f455 400 on empty input_ids (#4481) Yinghai Lu 2025-03-16 14:01:23 -07:00
• 25e1816eff fix custom allreduce performance/accuracy problem (#4477) Yi Zhang 2025-03-17 03:16:30 +08:00
• a53fe428f9 Support FlashMLA backend (#4472) lukecandyinfan98 2025-03-17 00:07:06 +08:00
• 1b859295f4 [Eagle] Remove the greedy branch and some redundant code (#4363) Ying ShengandSehoon Kim 2025-03-16 02:48:55 -07:00
• 9971dc2283 Revert "feat: Add FlashMLA submodule (#4449)" (#4470) Yineng Zhang 2025-03-16 01:30:05 -07:00
• 3db35c1af4 Release sgl-kernel v0.0.5.post2 (#4469) Lianmin Zheng 2025-03-16 01:01:53 -07:00
• 52a34d7448 Add greedy verification kernel (#4383) Ying Sheng 2025-03-16 00:58:26 -07:00
• 06d12b39d3 Remove filter for pr-tests (#4468) Lianmin Zheng 2025-03-16 00:57:26 -07:00
• c30976fb41 Fix finish step for pr tests and notebook tests (#4467) Lianmin Zheng 2025-03-16 00:52:06 -07:00
• 1a3fa75f2f [Fix] use torch.cat instead of torch.concat to prevent entering the Autograd backends. (#4466) JieXin Liang 2025-03-16 15:02:47 +08:00
• 81f431eded feat: Add FlashMLA submodule (#4449) Shi Shuaiandsleepcoo 2025-03-16 14:30:25 +08:00
• 65b7c9b78f cleanup deps 2/n (#4464) Yineng Zhang 2025-03-15 23:06:17 -07:00
• 2c4f5ccac1 Fix minor style (#4460) Lianmin Zheng 2025-03-15 21:51:12 -07:00
• 158430473e Fix typos (#4368) Wang Ran (汪然) 2025-03-16 12:27:58 +08:00
• 8ec2ce0726 perf: update fused moe config (#4459) Mick 2025-03-16 12:23:57 +08:00
• 1fd0cf8a7b Update comment in qwen2.py (#4447) Michael Feil 2025-03-15 21:14:29 -07:00
• bf63ee54ed Auto-detect device if not specified in server arguments. (#4423) vikram singh shekhawat 2025-03-16 09:43:51 +05:30
• 22c96f78a6 typos: Update sampling_params.md (#4391) Wang Ran (汪然)andLianmin Zheng 2025-03-16 07:40:18 +08:00
• 2892b9bb97 bugfix: Update sampling_params.py (#4413) Wang Ran (汪然) 2025-03-16 07:39:19 +08:00
• 470b474075 Update bench_serving.py (#4454) Xu Song 2025-03-16 07:33:58 +08:00
• 26c372c13c docs: Add Llama 3.3 to supported models (#4453) 江家瑋 2025-03-16 07:33:43 +08:00
• 86d9baedc2 [Fix] Fix errors when using the device except cuda. (#4455) Chen Shengzhi 2025-03-16 07:33:00 +08:00
• 21d485f835 Fix test_create_kvindices unit test (#4452) lukec 2025-03-16 07:01:04 +08:00
• 035ac2ab74 ci: update transformers==4.48.3 (#4451) Mick 2025-03-16 04:27:26 +08:00
• e1a5e7e47d docs: hot fix torch compile cache (#4442) Chayenne 2025-03-14 19:05:59 -07:00
• ad1ae7f7cd use topk_softmax with sgl-kernel (#4439) Yineng Zhang 2025-03-14 15:59:06 -07:00
• e73167ade3 Fix maximum recursion depth triggered on exception exit (#4438) Lianmin Zheng 2025-03-14 15:12:26 -07:00
• 862fe52241 bump v0.0.5.post1 (#4437) Yineng Zhang 2025-03-14 15:00:26 -07:00
• 61e4433caf Add moe topk softmax templated from vllm (#4302) Qingquan Song 2025-03-14 12:03:33 -07:00
• 660305c38a [Doc] fix wrong flag in deepseek documentation (#4427) Zhan Lu 2025-03-14 18:30:55 +00:00
• 642ab418f3 [bug] fix duplicate variable MAX_PIXELS in qwen_vl.py (#4419) Baoyuan Qi 2025-03-14 16:28:25 +08:00
• 1ce4878d31 feat(remote_model): support variable remote backend for model loader (#3964) wangyu 2025-03-14 15:40:44 +08:00
• 977d7cd26a cleanup deps 1/n (#4400) Yineng Zhangandsleepcoo 2025-03-14 00:00:33 -07:00
• 0e0ec70200 Hierarchical Caching supports MLA (#4009) Lu ChangqiandZhiqiang Xie 2025-03-14 11:42:14 +08:00
• bb37855653 Update CODEOWNERS (#4403) Lianmin Zheng 2025-03-13 17:54:40 -07:00
• ba80c102f9 bump v0.4.4.post1 (#4402) Yineng Zhang 2025-03-13 17:53:46 -07:00
• fbdb50501f Hot fix for hicache with new page aligned radixtree (#4397) Zhiqiang Xie 2025-03-13 15:50:49 -07:00
• f0afaf5289 Add a dummy grok test case (#4399) Lianmin Zheng 2025-03-13 15:29:48 -07:00
• 85d2365d33 Fix the output of hidden states after HTTP requests (#4269) Qiaolin Yu 2025-03-13 17:54:06 -04:00
• 5fe79605a8 Fix Llama3.3 tool call support (#4320) Chang Su 2025-03-13 14:01:41 -07:00