Commit Graph

Select branches

Hide Pull Requests

dsv41-pd

dsv41-pd-visioncp

main

Mono Color

• b116b21a46 [AMD] Temporarily disable test_no_overlap_scheduler and test_vision_chunked_prefill (#7717) Hubert Lu 2025-07-02 12:39:18 -07:00
• 88f484ce4c Apply dsv3 router gemm kernel for deepseek-r1 fp4 (#7677) Baizhou Zhang 2025-07-02 12:30:18 -07:00
• 8e03b641ba [1/n] apply wna16marlin kernel in moe weight only quantization (#7683) 6 people 2025-07-02 14:21:25 +08:00
• b3fa5dc3c8 Fix GPTQMarlinMoE (#7697) Kyungmin Lee 2025-07-02 14:34:43 +09:00
• 00aec6ad6c Apply dsv3_fused_a_gemm kernel (#7635) Ke Bao 2025-07-02 13:32:05 +08:00
• 1a08358aed Improve error handling for requests with unloaded LoRA path(s) (#7642) Lifu Huang 2025-07-01 20:05:34 -07:00
• f18a8fddd4 chore: upgrade flashinfer v0.2.7.post1 (#7698) Yineng Zhang 2025-07-01 14:05:57 -07:00
• a7efbb2757 fix(model loader): use safe_open to prevent file handle leaks. (#7684) Simon_CQK 2025-07-02 04:18:35 +08:00
• 93b6785d78 add description for llama4 eagle3 (#7688) Yi Zhang 2025-07-01 16:19:19 +08:00
• f9eb04ddb2 upgrade sgl kernel to 0.2.1 for main (#7676) Zhiqiang Xie 2025-07-01 00:00:13 -07:00
• 3a911b854d Refactor mm processors and Enable mixed modality processing (#7629) Xinyuan Tong 2025-06-30 23:14:48 -07:00
• 886d344964 support llama4 eagle3 (#6985) 5 people 2025-07-01 13:34:10 +08:00
• 637bfee448 chore: bump sgl-kernel v0.2.1 (#7675) Yineng Zhang 2025-06-30 22:12:33 -07:00
• 6005eceee3 [CPU] remove process_group from inputs of shm_allreduce and shm_allgather (#7486) Chunyuan WU 2025-07-01 12:54:11 +08:00
• ff2e9c9479 Add small requirements for benchmark/parse_result tools (#7671) Xiaoyu Zhang 2025-07-01 12:52:20 +08:00
• 3e34e9004f Fix: sync prepare_fp8_layer_for_marlin with latest vllm changes (#7648) narutolhy 2025-06-30 21:51:01 -07:00
• 7349717e4b [doc] update lws doc for pd (#7318) ybyang 2025-07-01 10:39:04 +08:00
• 392e441ad1 chore: upgrade flashinfer v0.2.7 jit (#7663) Yineng Zhang 2025-06-30 13:26:26 -07:00
• 7248272ccc Add dsv3 router gemm kernel (#7627) Baizhou Zhang 2025-06-29 23:31:55 -07:00
• 22352d47a9 Improve streaming, log_level, memory report, weight loading, and benchmark script (#7632) Lianmin ZhengandKan Wu 2025-06-29 23:16:19 -07:00
• c5131f7a2f [CPU] add c++ kernel to bind CPU cores and memory node (#7524) Chunyuan WU 2025-06-30 10:45:25 +08:00
• 78700893ee [EAGLE] remove a wrong adjustment for page_size > 1 & topk > 1 in server_args.py (#7643) Lianmin Zheng 2025-06-29 19:25:28 -07:00
• 663c04f76e Update CODEOWNERS (#7640) Lianmin Zheng 2025-06-29 16:58:43 -07:00
• 3b3f1e3aeb [AMD] Add unit-test-sgl-kernel-amd to AMD CI (#7539) Hubert Lu 2025-06-29 15:50:09 -07:00
• b691dcc490 [misc] reduce weird rope_scaling_factor warning (#7176) JieXin Liang 2025-06-30 06:42:45 +08:00
• 0c9c6c75a8 Move files related to EPLB (#7580) fzyzcjy 2025-06-30 06:39:38 +08:00
• e3f9b54819 [bugfix] fix runtime dropping panic in editable (#7628) Simo Lin 2025-06-29 15:38:28 -07:00
• b3cff3651e Fix sgl-router startup crash (#7619) finetune 2025-06-29 23:41:34 +02:00
• 8f335b5bd6 Fix stream reasoning parser and Adds Kimi reasoning parser (#7432) Xinyuan Tong 2025-06-29 14:39:05 -07:00
• b2264076dc Add @mickqian as the CODEOWNERS of multimodal (#7636) Lianmin Zheng 2025-06-29 09:27:33 -07:00
• 04b35190e2 Add dsv3 fused a gemm to sgl-kernel (#7630) Ke Bao 2025-06-29 17:52:24 +08:00
• 071a1f51ae [Minor] clean up multimodal processor and tokenizer manager (#7624) Lianmin Zheng 2025-06-29 02:50:14 -07:00
• 7c0db3a6c5 [bugfix] Remove PR comment posting from Rust benchmark workflow (#7625) Simo Lin 2025-06-28 22:10:01 -07:00
• c45e49d817 oai: Adds support for OpenAI chat completions API in bench_serving (#7036) 3 people 2025-06-28 15:59:20 -07:00
• d80539291b docs: add gb200 nvl72 and a16z grant (#7620) Yineng Zhang 2025-06-28 02:08:09 -07:00
• 00c7b1ad07 Let EP prefill support new DeepGEMM (#7310) fzyzcjy 2025-06-28 16:45:30 +08:00
• 82eccae44e Let ep_scatter support arbitrary strides / ue8m0 format (#7309) fzyzcjy 2025-06-28 16:38:33 +08:00
• a8c10aeeee fix unit tests (#7618) Yineng Zhang 2025-06-28 00:32:41 -07:00
• eb429b88a4 [PD] Respect sampling_params.max_new_tokens when PD disaggregation is activated (#7598) Shangming Cai 2025-06-28 13:22:01 +08:00
• 49538d111b Support dynamic LoRA loading / unloading in engine/server API (#7446) Lifu Huang 2025-06-27 21:00:27 -07:00
• cfe2edac38 [BUG] fix local_rank in initialize_dp_attention (#7584) Sheng Qi 2025-06-28 11:01:01 +08:00
• 2373faa317 Fix flakiness in LoRA batch test. (#7552) Lifu Huang 2025-06-27 19:51:43 -07:00
• 9efb2993da Tiny add logs for expert location updater (#7308) fzyzcjy 2025-06-28 10:12:33 +08:00
• a5317b2fd3 [CPU] add optimizations for INT8 and FP8 DeepSeek (#6769) Chunyuan WUandZheng, Beilei 2025-06-28 10:04:29 +08:00
• eb6c2c1663 Hybrid kv cache for LLaMA4 (#6563) 5 people 2025-06-27 21:58:55 -04:00
• 357921aa51 Fix: Minicpm (#7612) Xinyuan Tong 2025-06-27 17:32:29 -07:00
• c071198c1d [router] add centralized configuration module for sgl-router (#7588) Simo Lin 2025-06-27 15:42:02 -07:00
• d7374d7467 Fix broken CI TestVILAServer (#7610) Lifu Huang 2025-06-27 15:01:03 -07:00
• ce3a3e8783 Move multimodal processors into a separate folder (#7581) Lianmin Zheng 2025-06-27 11:58:24 -07:00
• 41650b0d70 feat: support compatibility between MTP and two-batch-overlap (#7225) Qiaolin YuandCheng Wan 2025-06-27 01:10:27 -07:00
• 1b95162008 Updates transformers and timm dependencies (#7577) Xinyuan Tong 2025-06-27 00:30:17 -07:00
• 29bd4c8135 [CI] Add CI Testing for Prefill-Decode Disaggregation with Router (#7540) Keyang Ru 2025-06-27 00:18:56 -07:00
• 031f64aa1b Add e2e test for multi instance multi stage memory release/resume occupuation (#7208) Ata Fatahi 2025-06-26 17:40:38 -07:00
• 3d7cdb2ebd Fix MTP error when enabling two-batch overlap (#7569) fzyzcjy 2025-06-27 06:40:54 +08:00
• 604efe07e1 Updates Gemma3n MLP layer to adapt latest transformers version (#7573) Xinyuan Tong 2025-06-26 15:07:22 -07:00
• 1b8cf77b01 [Fix] incorrect assert in EPLB (#7575) Cheng Wan 2025-06-26 14:59:20 -07:00
• bb9b608c86 [PD][NIXL] Set is_sorted=False to fix NIXL_ERR_NOT_FOUND (#7330) Trevor Morris 2025-06-26 10:39:39 -07:00
• 69183f8808 chore: bump v0.4.8.post1 (#7559) Yineng Zhang 2025-06-26 02:21:12 -07:00
• 9b00990bea chore: remove vlm unnecessary import (#7541) 3 people 2025-06-26 01:38:15 -07:00
• 4d67025a1d chore: improve ci bug reporting (#7542) Mick 2025-06-26 16:32:44 +08:00
• 0e05fe8cf4 Update seed in CPU UTs to avoid flaky failure with single test (#7544) YanbingJiang 2025-06-26 12:25:50 +08:00
• 2390a2bc8d Add Tencent HunYuanMoEV1 model support (#7549) Meng, Peng 2025-06-26 11:59:53 +08:00
• 16d76b9f23 [CMake] Fix sgl-kernel CMakeLists for Blackwell (#7543) Ruihang Lai 2025-06-25 22:00:46 -04:00
• 5c2142579a [PD] Raise error for incompatible mooncake version and some minor fixes (#7527) Shangming Cai 2025-06-26 09:55:24 +08:00
• b8df43ab9c Fix gathered_buffer issues in tbo (#7531) Qiaolin Yu 2025-06-25 14:42:21 -07:00
• a1c1ebe935 Fix FP8 KV Cache Support in FA3 Backend (#7148) Yuhong Guo 2025-06-25 17:14:40 +08:00
• fe2a0f962f minor: 'role' must be system/assistant/tool, but case insensitive for now (#7499) mlmz 2025-06-25 17:11:03 +08:00
• 20beb3702b feat: add return hidden_states at async generation (#7507) eigen 2025-06-25 05:10:09 -04:00
• 00fbd8a484 Fix typo of flash_cache (#7513) Stefan He 2025-06-25 02:04:41 -07:00
• 802815e40b take aiter get_rope back (#7521) valarLip 2025-06-25 17:03:33 +08:00
• 4c6675c4fc enable aiter fp8 blockscale quant (#7520) valarLip 2025-06-25 17:02:31 +08:00
• e21aa1df67 [PD] Add different TP sizes support for no-MLA models (#6793) 3 people 2025-06-25 17:00:22 +08:00
• f3cbd24541 feat: send kvmetrics from sglang scheduler (#6721) zixuanzhang226 2025-06-25 01:57:49 -07:00
• 506a2d5934 npu fused op (#7386) ll819214andLi Junwen 2025-06-25 16:54:20 +08:00
• a07f8ae4b7 [CI] Upgrade mooncake to v0.3.4.post2 to fix potential slice failed bug (#7522) Shangming Cai 2025-06-25 16:49:22 +08:00
• 7eb47b0f3d [CPU] [BF16] Call fused_experts_cpu, weight_packed_linear and bmm_cpu kernel in DeepSeek model (#6641) Chunyuan WUandThien Tran 2025-06-25 16:43:33 +08:00
• bc2e5645c4 fix: force synchronization between TP workers when update_weights (#6626) DangKaianddangkai.dk 2025-06-25 16:35:59 +08:00
• 3abc30364d [ci] add router benchmark script and CI (#7498) Simo Lin 2025-06-25 01:28:25 -07:00
• afeed46530 clean duplicate code (#7512) linzhuo 2025-06-25 16:22:20 +08:00
• 587b4c6e92 EPLB support for MTP (#7510) yilian49 2025-06-25 01:16:56 -07:00
• 7b9a174a7a [PD][Spec] Fix hidden state transfer for spec decode (#7516) Shangming Cai 2025-06-25 15:42:07 +08:00
• 03c039c48e [OAI] patch origin request_id logic (#7508) ybyang 2025-06-25 11:09:38 +08:00
• 57ab776910 Fuse sorted_token_ids padding to moe_align_block_size kernel (#7437) Ke Bao 2025-06-25 08:44:27 +08:00
• 112b496a6c misc: Improvement to serving_chat.py and add more ut (#7489) Chang Su 2025-06-24 17:19:51 -07:00
• 3562256bb2 fix: Add --model as an alias for --model-path in server_args (#7505) Chang Su 2025-06-24 12:08:08 -07:00
• 5f527834a8 [PD] NIXL: Register kv args in advance and cleanup finished requests (#6717) Trevor Morris 2025-06-24 11:26:09 -07:00
• 9f1787fa60 Support multi-thread model weight loading (#7277) xianzhiT 2025-06-25 01:39:10 +08:00
• 8ecad0b16f [benchmark] fbgemm benchmark support bandwidth report and support fbgemm_cutlass_gmm (#7422) Xiaoyu Zhang 2025-06-25 00:44:55 +08:00
• 7151194bb2 Remove cumsum_buffer initilization (#7439) Ke Bao 2025-06-24 17:22:16 +08:00
• 2ed68d7a6c [PD Disaggregation] replace transfer with batch transfer for better performance (#7236) Francis 2025-06-24 17:12:04 +08:00
• e984d5073b enable aiter_biased_grouped_topk kernel (#7423) valarLip 2025-06-24 17:09:42 +08:00
• 755f314785 [AMD] add aiter fused moe in DeepEP path (#7268) Alex Sun 2025-06-24 17:05:47 +08:00
• 7c3a12c000 chore: bump v0.4.8 (#7493) Yineng Zhang 2025-06-23 23:14:22 -07:00
• e846d95ef6 chore: bump sgl-kernel v0.2.0 (#7490) Yineng Zhang 2025-06-23 22:29:50 -07:00
• 15f3401343 Fix MTP with Deepseek R1 Fp4 (#7376) Charles Chen 2025-06-23 21:38:07 -07:00
• d04163b3fa Fix RequestValidationError response format (#7487) Chang Su 2025-06-23 20:35:11 -07:00
• 7732bbe458 bugfix: Prevent global mutation of conv.stop_str across requests (#7347) huangtingweiandChang Su 2025-06-24 10:36:23 +08:00
• ed0a0b692c Perormance: Enable cuda graph for dp idle batch (#7269) 4 people 2025-06-24 08:34:13 +08:00
• fa42e41962 ci: Revert openai_server related tests in AMD suites (#7449) Chang Su 2025-06-23 15:28:22 -07:00
• e5afb88b1c Support weight loading without mmap (#7469) Yuhong Guo 2025-06-24 06:13:59 +08:00