Commit Graph

Select branches

Hide Pull Requests

dsv41-pd

dsv41-pd-visioncp

main

Mono Color

• 6b45a21d16 Reorganize c++ source files in sgl-kernel with multiple folders (#4025) Lianmin Zheng 2025-03-03 05:32:30 -08:00
• a7000a7650 Update metrics documentation (#3264) Yudi Xue 2025-03-03 05:03:58 -08:00
• 1a8f995c46 remove cache configs in model definitions (#4031) Lianmin Zheng 2025-03-03 05:00:50 -08:00
• a3ab768a2b Clean up custom allreduce (#4029) Lianmin Zheng 2025-03-03 04:59:53 -08:00
• 66301e124f Improve code styles (#4021) Lianmin Zheng 2025-03-03 03:20:23 -08:00
• ac2387279e Support penalty in overlap mode; return logprob with chunked prefill; improve benchmark scripts (#3988) 4 people 2025-03-03 00:12:04 -08:00
• 0194948fd9 Optimize Triton Kernel of Group GEMM in DeepGEMM Benchmark (#4014) Stefan He 2025-03-02 23:29:55 -08:00
• b4d34cd35d Fix nightly-test CI (#3826) yinfan98 2025-03-03 15:14:45 +08:00
• 728e175fc4 Add examples to token-in-token-out for LLM (#4010) Chayenne 2025-03-02 21:03:49 -08:00
• 9e1014cf99 Revert "Add fast decode plan for flashinfer mla" (#4008) Lianmin Zheng 2025-03-02 19:29:10 -08:00
• fa56106731 Add fast decode plan for flashinfer mla (#3987) Baizhou Zhang 2025-03-02 19:16:37 -08:00
• 7fbab730bd [feat] add small vocab table for eagle's draft model[1]. (#3822) 3 people 2025-03-03 10:58:45 +08:00
• b7e274f2d9 Add Benchmark for DeepGEMM Group GEMM (#3993) Stefan He 2025-03-02 17:47:21 -08:00
• 9cf4077294 Enable custom AR for AMD GPUs and maintain it in sgl-kernel (#3406) Hubert Lu 2025-03-02 15:19:06 -08:00
• d3fe9bae56 Add accuracy test for TP torch compile (#3994) Ke Bao 2025-03-03 05:18:18 +08:00
• 00ce7e311c Fix all gather torch compile (#3992) Ke Baoandyizhang2077 2025-03-02 16:41:38 +08:00
• 50f28f65a0 fix typo in deep gemm benchmarking(#3991) Xiaoyu Zhang 2025-03-02 16:34:00 +08:00
• 90a55e2566 add deepgemm and sglang fp8 block-wise gemm benchmark (#3893) Xiaoyu Zhang 2025-03-02 15:01:58 +08:00
• 407e2b923d Update CODEOWNERS (#3989) Lianmin Zheng 2025-03-01 21:47:30 -08:00
• 40782f05d7 Refactor: Move return_hidden_states to the generate input (#3985) Qiaolin YuandBeichen-Ma 2025-03-01 20:51:29 -05:00
• 18bb216c28 Revert "[MOE] enable efficient moe_alignment multi-blocks execution (3x~6x)" (#3982) Chayenne 2025-02-28 23:57:17 -08:00
• 6b859e7ddd Docs: add special warning to engine docs (#3979) Chayenne 2025-02-28 21:59:20 -08:00
• 930da877c4 rename FunctionCallReqInput to ParseFunctionCallReq (#3976) Chayenne 2025-02-28 18:46:25 -08:00
• 3f8a441437 Docs: Add redline to highlight main process (#3977) Chayenne 2025-02-28 18:37:15 -08:00
• aceb420179 Docs: add type hint to smapling parameters (#3975) Chayenne 2025-02-28 18:21:20 -08:00
• 90a4b7d98a [Feature]Support ragged prefill in flashinfer mla backend (#3967) 3 people 2025-02-28 18:13:56 -08:00
• f3b99f73b3 update flashinfer-python version Yineng Zhang 2025-02-28 16:31:59 -08:00
• 9e74ee91da Update cutlass dependency (#3966) Elfie Guo 2025-02-28 16:16:31 -08:00
• 77a6c9d229 Remove unused imports from rocm mla kernel. (#3963) Chaitanya Sri Krishna Lolla 2025-02-28 23:31:08 +05:30
• e3e0bc50a9 [Feature] SPMD for SGLang + Verl (#3852) fzyzcjy 2025-03-01 01:53:10 +08:00
• bac414ab53 [Feature] integrate Structural Tag in xgrammar backend for function calling (#3566) mlmzandshuaills 2025-02-28 15:33:41 +08:00
• eec3f6d1eb [Bugfix] Fix tokenizer_manager not getting 400 when req is too long (#3678) Chang Suandvoidxb 2025-02-27 22:59:43 -08:00
• 90bc26a813 set a strict sgl-kernel version (#3950) Chayenne 2025-02-27 22:44:57 -08:00
• ec0a72c2d9 Fix bench_serving not recognizing OPENAI_API_KEY (#3870) Kebe 2025-02-28 12:18:53 +08:00
• 1c96fa86cf [MOE] enable efficient moe_alignment multi-blocks execution (3x~6x) (#3613) yiakwy-xpu-ml-framework-team 2025-02-28 11:42:48 +08:00
• bc20e93f2d [feat] Add Vertex AI compatible prediction route for /generate (#3866) KCFindstr 2025-02-27 19:42:15 -08:00
• d38878523d Fix the doc link for sampling params (#3861) Qiaolin Yu 2025-02-27 16:31:43 -05:00
• 564bdf29f7 upgrade flashinfer v0.2.2.post1 (#3934) Yineng Zhang 2025-02-27 09:53:48 -08:00
• 5d86016855 revert "Docs: Reorngaize dpsk links #3900" (#3933) Yineng Zhang 2025-02-27 08:57:13 -08:00
• d281587989 Improve: Support xgrammar 0.1.14 (#3593) Enrique Shockwave 2025-02-27 16:42:54 +00:00
• b0df5d240b Tuning Script for Feature DeepSeek V3/R1 INT8 Quantization (block-wise) (#3922) laixinandsleepcoo 2025-02-27 18:59:46 +08:00
• 3e02526b1f [Doc] Add experimental tag for flashinfer mla (#3925) Baizhou Zhang 2025-02-27 01:55:36 -08:00
• d8a98a2cad [Docs] Improve DPSK docs in dark mode (#3914) Stefan He 2025-02-27 00:13:04 -08:00
• 0519269d20 [Docs] Disable notebook CI when merge to main (#3905) Qing 2025-02-26 22:13:33 -08:00
• d6898dd253 Add return hidden state in the native API (#3897) 3 people 2025-02-27 01:06:54 -05:00
• 71ed01833d [doc] Update document for flashinfer mla (#3907) Baizhou Zhang 2025-02-26 20:40:45 -08:00
• 8b681d7724 [Rocm] Fix to the rocm_mla_decode_rope.py returning random result (#3898) Tianxing Wu 2025-02-27 03:05:30 +02:00
• 194eea1774 [doc] update sponsorship (#3903) ybyang 2025-02-27 08:28:15 +08:00
• acd1a15921 Docs: Implemented frontend docs (#3791) simveitandChayenne 2025-02-27 00:30:05 +01:00
• 7c1692aa90 Docs: Reorngaize dpsk links (#3900) Chayenne 2025-02-26 15:16:31 -08:00
• 8f019c7d1a Docs: Move dpsk docs forward a step (#3894) Chayenne 2025-02-26 11:43:20 -08:00
• 7551498a69 [Feature] Support llguidance for constrained decoding (#3298) JC1DA 2025-02-26 10:41:49 -08:00
• 44a2c4bd56 Docs: improve link to docs (#3860) simveitandChayenne 2025-02-26 19:29:25 +01:00
• c9fc4a9d26 Docs: delete sgl-kernel install in docs (#3845) simveit 2025-02-26 18:25:43 +01:00
• 21463e321a Expert Parallelism (EP) Support for DeepSeek V3/R1 (#3602) 4 people 2025-02-26 18:29:37 +08:00
• 3dc9ff3ce8 [doc] fixed dpsk quant faq (#3865) Shenggui Li 2025-02-26 11:40:47 +08:00
• 06427dfab1 [doc] added quantization doc for dpsk (#3843) Shenggui Li 2025-02-26 01:43:28 +08:00
• 60524920ba [Bug]: Fix maximum recursion depth triggered on exception exit (#3519) Kebe 2025-02-26 01:39:38 +08:00
• 107710268a [BugFix] Fix crash when receive a req with structed output in DP attention mode. (#3841) IAN 2025-02-26 01:32:05 +08:00
• 4606e2a3fe Bug: fix capture_bs (#3857) who who who 2025-02-26 00:40:35 +08:00
• 127998cc41 Fix allgather ops inside cuda graphs (#3709) Nicolas Castet 2025-02-25 10:39:10 -06:00
• c0bb9eb3b3 [improve] made timeout configurable (#3803) Shenggui Li 2025-02-25 16:26:08 +08:00
• 7036d6fc67 [Bug]: Add missing clamp to llavavid (#3787) Yueyang Pan 2025-02-25 04:10:15 +01:00
• 6ce9dbe828 [ROCm] Enable Fused MLA Triton kernel for DeepSeekV3 (#3237) Chaitanya Sri Krishna LollaandHAI 2025-02-25 07:44:31 +05:30
• 3758d209a0 [Doc] Fix typo in server-argument description (#3641) Yuanheng Zhao 2025-02-25 08:57:13 +08:00
• faf29e0b23 Docs: fix doc site copyright to current year (#3741) Wilson Wu 2025-02-25 08:56:04 +08:00
• b0743ea059 Docs: fix dead link in router.md (#3799) He1pa 2025-02-25 08:53:57 +08:00
• 60b771c815 Improve: fix typos (#3801) Wang Ran (汪然)andChayenne 2025-02-25 08:51:23 +08:00
• d7934cde45 Fix CI and install docs (#3821) Lianmin Zheng 2025-02-24 16:17:38 -08:00
• 62bbd34393 Revert "Extract generation_manager from tokenizer_manager" (#3829) Lianmin Zheng 2025-02-24 14:49:16 -08:00
• f2388f6b95 Revert "Rename TokenizerManager to StdOrchestrator" (#3828) Lianmin Zheng 2025-02-24 14:47:59 -08:00
• c9745ee082 Fix pandas dependency in CI (#3818) Lianmin Zheng 2025-02-24 05:56:57 -08:00
• 1a6e97577a Feature DeepSeek V3/R1 INT8 Quantization (block-wise) (#3730) laixinandHandH1998 2025-02-24 21:43:35 +08:00
• b110084654 Refactor flashinfer logic for deepseek v3 and fix accuracy bug (#3785) Baizhou Zhang 2025-02-24 04:07:25 -08:00
• 27a46317b6 Fix dependency (#3813) Lianmin Zheng 2025-02-24 03:50:58 -08:00
• c979580817 Update readme (#3809) Lianmin Zheng 2025-02-24 00:31:08 -08:00
• 6c7a152c5a Hierarchical Caching for SGLang (#2693) 3 people 2025-02-23 21:56:30 -08:00
• 4d2a88bdff [Docs]Add instruction for manually stopping nsys profiler (#3795) Baizhou Zhangandzhaochenyang20 2025-02-23 13:21:48 -08:00
• 45360b2fa9 Improve: Rename TokenizerManager to StdOrchestrator (#3116) fzyzcjy 2025-02-23 16:30:58 +08:00
• 3f41b18455 Improve: Extract generation_manager from tokenizer_manager (#3115) fzyzcjy 2025-02-23 15:25:45 +08:00
• 45205d88a0 bench: Add MMMU benchmark for vLM (#3562) Mick 2025-02-23 00:10:59 +08:00
• 9087694006 Improve: Use TypeBasedDispatcher in DetokenizerManager (#3117) fzyzcjy 2025-02-22 11:50:46 +08:00
• a3339d8cac Bug: Fix weight loader error when LM head weights are tied (#3766) fzyzcjy 2025-02-22 09:53:12 +08:00
• 14d90617b0 Bug: fix lm head weights in Qwen models (#3777) Chayenne 2025-02-21 16:49:31 -08:00
• d37f95511d Improve: Tiny fix Olmo2 (#3348) fzyzcjy 2025-02-22 08:09:35 +08:00
• c66b2c9cf1 Add support for nvidia modelopt fp8 kv cache (#3223) Zhiyu 2025-02-21 15:04:58 -08:00
• 20b765a26e Model: Support Qwen 72B RM model. (#3772) simveit 2025-02-21 23:38:21 +01:00
• e310722266 Docs: Update offline_engine_api and add links (#3773) Chayenne 2025-02-21 14:15:52 -08:00
• e074e76b31 docs: Add offline engine launch example and documentation (#3771) Shi Shuai 2025-02-21 19:25:52 +00:00
• 4592afc27d Docs: Fix layout to docs (#3733) simveit 2025-02-21 20:24:13 +01:00
• 9af0e21ef5 [bug] fixed batch api for DeepSeek V3/R1 (#3754) Shenggui Li 2025-02-22 02:28:16 +08:00
• c7c79b16cd [Fix] OpenAI API adapter tokenizer encoding (#3432) Shi Shuai 2025-02-21 17:24:15 +00:00
• d8d75d256a Change description of nvidia jetson docs (#3761) Shakhizat Nurgaliyev 2025-02-21 17:44:22 +05:00
• 1df6eabd5d feat: Add SageMaker support (#3740) Andrew Smith 2025-02-21 22:31:09 +11:00
• 0c227ee373 feat: update grouped_topk to support softmax and sigmoid (#3680) zixuanzhang226 2025-02-21 00:30:15 -08:00
• 5c54ef0352 AMD/ROCm: update AITER repo to ROCm/aiter (#3747) HAI 2025-02-21 00:18:08 -08:00
• c6a4852136 [docs] added torch.compile cache to dpsk manual (#3737) Shenggui Li 2025-02-21 16:11:40 +08:00
• 4f678c87d8 docs: fix 404 link (#3588) Trayvon Panandyqtianust 2025-02-21 07:15:57 +08:00
• e79f7420be [Fix] Fix bugs and refactor codes in lora for better scalability. (#3652) 3 people 2025-02-20 14:51:57 -05:00
• ac05310098 [Docs] Modify ep related server args and remove cublas part of deepseek (#3732) Baizhou Zhang 2025-02-20 11:37:56 -08:00