Commit Graph

Select branches

Hide Pull Requests

dsv41-pd

dsv41-pd-visioncp

main

Mono Color

• 9d24c3ffb0 chore: tiny remove duplicated code (#6392) doujiang24 2025-05-18 17:17:32 +08:00
• 24161c5913 The Gemma template is missing a newline after the user role. (#6331) Yury SulskyandYury Sulsky 2025-05-18 01:57:27 -07:00
• eabcf82acb feat: add long context example (#6391) Yineng Zhang 2025-05-18 01:45:17 -07:00
• c47a51db7e Clean up AMD CI (#6365) Sai Enduri 2025-05-18 01:17:28 -07:00
• 11553c1a37 Add pipeline parallelism for Qwen2 and Qwen3 Model (#6250) libra 2025-05-18 15:42:55 +08:00
• 01dd39bac1 refactor: minor refactors regarding multimodal processing (#6187) Mick 2025-05-18 13:53:20 +08:00
• b3f3d610fd Do not use FA3 for mistral (#6379) Lianmin Zheng 2025-05-17 19:47:34 -07:00
• f07c6a009b chore: upgrade sgl-kernel v0.1.3 (#6377) Yineng Zhang 2025-05-17 19:47:05 -07:00
• 4bb816d444 Fix CI tests (#6362) Lianmin Zheng 2025-05-17 19:16:45 -07:00
• c250939ecb [Fix Chat API] add request id for chat/completion for tracing (#6364) ybyang 2025-05-18 09:58:22 +08:00
• b6909aa223 fix: allow launch_dummy_health_check_server to start inside of running asyncio loop (#6330) ishandhanani 2025-05-17 18:32:41 -07:00
• f87283573e Add expert distribution APIs for engine (#6290) fzyzcjy 2025-05-18 09:31:51 +08:00
• 73187152a4 Reland tiny refactor DefaultModelLoader.Source (#6041) fzyzcjy 2025-05-18 08:11:20 +08:00
• 4086566516 Fix expert distribution recorder and profiler command stuck forever (#6284) fzyzcjy 2025-05-18 08:10:44 +08:00
• fd08c04821 Support custom DeepEP tuning config (#6257) fzyzcjy 2025-05-18 08:09:42 +08:00
• 26ebb849eb Tiny refactor bench_serving to extract RequestFuncOutput.init_new (#6108) fzyzcjy 2025-05-18 08:08:52 +08:00
• 02973cd9a4 Tiny refactor bench_serving to improve extensibility (#6134) fzyzcjy 2025-05-18 08:07:58 +08:00
• 6d95a35abf Support outputing details for bench_serving (#6107) fzyzcjy 2025-05-18 08:06:52 +08:00
• 01d2838c0f Fix stop_profile does not wait for finishing (#4741) fzyzcjy 2025-05-18 08:06:15 +08:00
• e3b8a72291 [fix] illegal memory in _fwd_kernel_ep_scatter_2 and _fwd_kernel_ep_gather (#6348) xutizhou 2025-05-18 08:01:42 +08:00
• 3cf1473a09 Use monotonic clock for interval measurement (#6211) Lifu Huang 2025-05-17 16:49:18 -07:00
• 2716830802 Speed up when having padding tokens in DeepEP (#6175) fzyzcjy 2025-05-18 07:44:05 +08:00
• e3bed74afb [router] Add /list_workers endpoint to router (#6366) Zilin Zhu 2025-05-18 00:49:02 +08:00
• e9ef39d2e9 docs: Update the MD files (#6373) Vincent ZhongandBrayden Zhong 2025-05-17 12:23:16 -04:00
• 205d5cb407 perf: Optimize local attention memory allocation in FlashAttentionBackend (#6356) Chang Su 2025-05-17 01:45:46 -07:00
• 3d7f7a43c8 chore: bump sgl-kernel v0.1.3 (#6368) Yineng Zhang 2025-05-17 00:15:55 -07:00
• 2df9d40aa6 Minor code cleanup refactor for DeepSeek models (#6324) fzyzcjy 2025-05-17 10:06:03 +08:00
• 8dc191f237 Fix one wasted kernel in DeepSeek and minor refactor (#6316) fzyzcjy 2025-05-17 10:05:33 +08:00
• 64825b8395 model(vlm): mistral 3.1 (#5099) Kiv ChenandKivenChen 2025-05-16 18:36:18 -07:00
• 69748d088d docs: update readme (#6361) Yineng Zhang 2025-05-16 16:16:23 -07:00
• dcc0a45618 Fix amd ci (#6360) Lianmin Zheng 2025-05-16 15:33:10 -07:00
• c2b7ddca49 [Minor] cleanup unused imports (#6358) Lianmin Zheng 2025-05-16 14:52:38 -07:00
• abebd9399c Update CODEOWNERS (#6359) Lianmin Zheng 2025-05-16 14:51:36 -07:00
• 4bd2952a37 feat: add dp attention support for Qwen 2/3 MoE models, fixes #6088 (#6121) 3 people 2025-05-17 05:44:10 +08:00
• 6fc9357503 [2/2] Add python wrapper for CUTLASS FP8 Blockscale MoE Kernel. (#5694) Elfie Guo 2025-05-16 13:14:07 -07:00
• 839fb31e5f [Fix] Improve dependencies for Blackwell image (#6334) Baizhou Zhang 2025-05-16 12:38:22 -07:00
• f19a9204cd Support precomputed multimodal features for Qwen-VL and Gemma3 models. (#6136) Yury SulskyandYury Sulsky 2025-05-16 12:26:15 -07:00
• c23a7072b6 Upgrade CUTLASS 4.0 (#6336) Elfie Guoandzhyncs 2025-05-15 17:42:23 -07:00
• e07a6977e7 Minor improvements of TokenizerManager / health check (#6327) Lianmin Zheng 2025-05-15 15:29:25 -07:00
• cd8d4b9dfc Fix lora bench (#6302) Qiaolin Yu 2025-05-15 13:09:55 -04:00
• f194e14fb7 Reduce MoE memory usage (#6147) fzyzcjy 2025-05-16 00:38:28 +08:00
• cfc9f9ab8d Fix gpu mem check on CPU (#6317) Yi Liu 2025-05-16 00:37:45 +08:00
• fb4959b2c5 Add fp8 gemm kernel for CPU in sgl-kernel and add gemm UT (#6216) 3 people 2025-05-16 00:10:40 +08:00
• 9a405274e2 [misc] remove redundant platform codes (#6298) JieXin Liang 2025-05-15 15:51:30 +08:00
• 2e4babdb0a [Feat] Support FlashMLA backend with MTP and FP8 KV cache (#6109) 7 people 2025-05-15 15:48:09 +08:00
• 44a3783d13 [fix][RL] Remove the incorrect barrier in init_weights_update_group (#5914) Zilin Zhu 2025-05-15 10:15:21 +08:00
• f3bf611054 feat: add flush cache to EngineBase and HttpServerEngineAdapter (#6009) Junrong Lin 2025-05-15 10:15:02 +08:00
• 198b9056d1 [AMD] Fix Llama 4 Scout and Maverick accuracy issues on MI300X (#6274) Hubert Lu 2025-05-14 15:07:29 -07:00
• 73eb67c087 Enable unit tests for AMD CI. (#6283) Sai Enduri 2025-05-14 12:55:36 -07:00
• 9a91fa0ed1 docs: fix a bad redirect (#6300) Brayden Zhong 2025-05-14 13:27:19 -04:00
• cd7c8a8de6 doc: update developer guide regarding mllms (#6138) 3 people 2025-05-14 23:13:13 +08:00
• 3e350a931e [Bug] Fix accidental logger override caused by internVL. (#6282) Lifu Huang 2025-05-13 23:29:25 -07:00
• fb71725c98 Fix a bug in schedule_policy (#6276) Ying Sheng 2025-05-13 18:04:00 -07:00
• 912788c095 perf: optimize local_block_table memory allocation (#6273) Chang Su 2025-05-13 17:18:38 -07:00
• 0f75b907c6 [CPU] Add CMakeLists.txt for sgl-kernel (#6115) blzheng 2025-05-14 06:30:37 +08:00
• 16267d4fa7 chore: bump v0.4.6.post4 (#6245) Yineng Zhang 2025-05-13 01:57:51 -07:00
• 0f5cb8cae1 Enable MI325X AMD CI. (#6259) Sai Enduri 2025-05-13 01:49:33 -07:00
• 17299f088a [misc] deep_gemm fallback to NVRTC when NVCC not found (#6252) JieXin Liang 2025-05-13 16:41:35 +08:00
• 5380cd7ea3 model(vlm): pixtral (#5084) Kiv Chen 2025-05-13 00:16:10 -07:00
• b2e95f62b4 Fix two issues related to --moe-dense-tp-size=1 (#5657) 3 people 2025-05-13 02:51:39 -04:00
• 1ab14c4c5c [VERL Use Case] Add torch_memory_saver into deps (#6247) Stefan He 2025-05-12 19:09:03 -07:00
• 3c32895cbe [Llama4] Add docs note about enable multimodal (#6235) Brayden Zhong 2025-05-12 22:05:47 -04:00
• ac2324c177 Skip the flaky test_stateful_custom_logit_processor (#6251) Lianmin Zheng 2025-05-12 18:29:41 -07:00
• ef8ec07b2c Support tuning moe for llama 4 model (#6042) fzyzcjy 2025-05-13 06:47:01 +08:00
• f24fc5b86d fix typo (#6248) Yineng Zhang 2025-05-12 15:45:12 -07:00
• d18c6b3358 Support incremental streaming of logprob/token_ids between scheduler and detokenizer (#6225) Lianmin ZhengandSangBin Cho 2025-05-12 14:33:38 -07:00
• f1c896007a [PD] Add support for different TP sizes per DP rank (#5922) shangmingc 2025-05-13 04:55:42 +08:00
• 983c663de6 Update AMD nightly deps. (#6241) Sai Enduri 2025-05-12 13:39:20 -07:00
• f94543d22b chore: add hf_xet dep (#6243) Yineng Zhang 2025-05-12 13:08:40 -07:00
• e8e18dcdcc Revert "fix some typos" (#6244) Lianmin Zheng 2025-05-12 12:53:26 -07:00
• bad7c26fdc [PP] Fix init_memory_pool desync & add PP for mixtral (#6223) Ying Sheng 2025-05-12 12:38:09 -07:00
• 12319a6787 [Docs] Add docs for SGLANG_ and SGL_ environment variables (#6206) Brayden Zhong 2025-05-12 13:45:41 -04:00
• d738ab52f8 fix some typos (#6209) applesaucethebunandBrayden Zhong 2025-05-12 13:42:38 -04:00
• 3ee40ff919 [CI] Re-enable pd disaggregation test (#6231) shangmingc 2025-05-13 01:09:12 +08:00
• 0f334945c6 [CI] Fix PD mooncake dependency error (#6212) shangmingc 2025-05-13 01:08:49 +08:00
• fba8eccd7e Log if cuda graph is used & extend cuda graph capture to cuda-graph-max-bs (#6201) Lianmin ZhengandSangBin Cho 2025-05-12 00:17:33 -07:00
• 7d3a3d4510 Update AMD CI docker to v0.4.6.post3-rocm630. (#6213) Sai Enduri 2025-05-12 00:00:46 -07:00
• 25c83fff6a Performing Vocabulary Parallelism for LM Head across Attention TP Groups (#5558) Cheng Wanandliusy58 2025-05-12 02:36:29 -04:00
• 9f2c9568f0 [doc] add a note for --n-share-experts-fusion args (#6154) Xiaoyu Zhang 2025-05-12 14:18:38 +08:00
• 3f2702ae51 Fix start_profile does not support with_stack and record_shapes (#6043) fzyzcjy 2025-05-12 14:11:32 +08:00
• 6ea05950b1 Fix release-docs.yml to not use python 3.9 (#6204) Lianmin Zheng 2025-05-11 16:04:55 -07:00
• e7dd906c5c Update README.md (#6202) Lianmin Zheng 2025-05-11 14:34:12 -07:00
• 6e2da51561 Replace time.time() to time.perf_counter() for benchmarking. (#6178) Lifu Huang 2025-05-11 14:32:49 -07:00
• e9a47f4cb5 Add dev-deepep docker image (#6198) fzyzcjy 2025-05-12 04:17:55 +08:00
• 03227c5fa6 [CI] Reorganize the 8 gpu tests (#6192) Lianmin Zheng 2025-05-11 10:55:06 -07:00
• 01bdbf7f80 Improve structured outputs: fix race condition, server crash, metrics and style (#6188) Lianmin Zheng 2025-05-11 08:36:16 -07:00
• 94d42b6794 [Docs] minor Qwen3 and reasoning parser docs fix (#6032) Adarsh Shirawalmath 2025-05-11 20:52:46 +05:30
• 69276f619a doc: fix the erroneous documents and example codes about Alibaba-NLP/gme-Qwen2-VL-2B-Instruct (#6199) mlmz 2025-05-11 23:22:11 +08:00
• 41a645f556 Handle empty input string for embedding models (#5621) Ravi ThejaandRavi Theja Desetty 2025-05-11 20:47:15 +05:30
• 230106304d chore: upgrade sgl-kernel v0.1.2.post1 (#6196) Yineng Zhangandalcanderian 2025-05-11 07:41:37 -07:00
• 45b4dcf037 chore: bump sgl-kernel v0.1.2.post1 (#6195) Yineng Zhang 2025-05-11 02:24:10 -07:00
• 213e8c7dd5 chore: upgrade deepgemm (#6073) Yineng Zhang 2025-05-11 02:17:24 -07:00
• 41273fd71f fix: handle None multimodal_inputs during merging and filtering batches in disaggregation decode mode (#6169) Yusong Gao 2025-05-11 15:28:21 +08:00
• e9bebafb19 [fix] remove mixtral from is_fa3_default_architecture (#6191) JieXin Liang 2025-05-11 15:15:54 +08:00
• 4d1c9db66c feat: support loogle eval (#6190) Yineng Zhang 2025-05-10 23:52:44 -07:00
• 17c36c5511 [CI] Disabled deepep tests temporarily because it takes too much time. (#6186) Lianmin Zheng 2025-05-10 23:40:50 -07:00
• 31d1f6e7f4 [PD] Add simple unit test for disaggregation feature (#5654) shangmingc 2025-05-11 13:35:27 +08:00
• a823c6e834 Remove duplicate IO Struct test (#6180) Emmanuel Ferdman 2025-05-11 07:57:26 +03:00
• 2ce8793519 Add typo checker in pre-commit (#6179) applesaucethebunandBrayden Zhong 2025-05-11 00:55:00 -04:00
• de167cf5fa Fix request abortion (#6184) Lianmin Zheng 2025-05-10 21:54:46 -07:00