Commit Graph

Select branches

Hide Pull Requests

dsv41-pd

dsv41-pd-visioncp

main

Mono Color

• 7059ae16fb chore: update pr test xeon (#7008) Yineng Zhang 2025-06-09 10:08:44 -07:00
• 51d9a597f9 cleanup tmp dir (#7007) Yineng Zhang 2025-06-09 09:26:04 -07:00
• 56ccd3c22c chore: upgrade flashinfer v0.2.6.post1 jit (#6958) 6 people 2025-06-09 09:22:39 -07:00
• 98c00a2df1 Fix torch profiler bugs for bench_offline_throughput.py (#6557) Yueyang Pan 2025-06-09 20:33:41 +08:00
• 451ffe74d9 support qwen3 emebedding (#6990) Pan Lyu 2025-06-09 16:32:49 +08:00
• b1e5a33ae3 Eliminate stream sync to speed up LoRA batch init (#6960) Lifu Huang 2025-06-09 00:22:45 -07:00
• 9d5fa68b90 Use torch.compile to fuse flash attention decode metadata preparation (#6973) Lianmin Zheng 2025-06-08 23:05:40 -07:00
• 2c18642502 Enable more unit tests for AMD CI. (#6983) Sai Enduri 2025-06-08 19:41:55 -07:00
• 18efb5e8e0 [perf][sgl-kernel] extend cutlass_mla_decode to support num_head < 128 (#6929) JieXin Liang 2025-06-09 10:37:34 +08:00
• de1350ea20 Minor remove one kernel for DeepSeek (#6977) fzyzcjy 2025-06-09 08:41:35 +08:00
• 86fe943bc3 Fix expert distribution dumping causes OOM (#6967) fzyzcjy 2025-06-09 08:41:14 +08:00
• 9ecb18568b Fix triton sliding window test case (#6981) Lianmin Zheng 2025-06-08 17:20:46 -07:00
• cc74499d51 Fix draft extend ut stability with flush cache (#6979) Ke Bao 2025-06-09 08:09:32 +08:00
• 0c1f03a23d Sync cuda graph runners (#6976) Lianmin Zheng 2025-06-08 16:12:25 -07:00
• 3712abfaf9 Fuse routed scaling factor in deepseek (#6970) Xiaoyu Zhang 2025-06-09 06:24:24 +08:00
• 971a0dfa32 Extend cuda graph capture bs for B200 (#6937) Baizhou Zhang 2025-06-08 05:13:22 -07:00
• 2fc1299562 Remove unnecessary kernels of num_token_non_padded (#6965) fzyzcjy 2025-06-08 20:09:17 +08:00
• 20d3ad3b58 Fix CI and triton moe Configs (#6974) Lianmin Zheng 2025-06-08 05:06:46 -07:00
• fa3592cfeb rebase h20 fused_moe config (#6966) Xiaoyu Zhang 2025-06-08 20:01:34 +08:00
• 608668e143 Slightly improve the sampler to skip unnecessary steps (#6956) Lianmin Zheng 2025-06-08 03:18:54 -07:00
• 6c0a48282a chore: bump sgl-kernel v0.1.7 (#6963) Yineng Zhang 2025-06-08 02:43:15 -07:00
• 4740288303 [AMD] Add more tests to per-commit-amd (#6926) Hubert Lu 2025-06-08 01:08:37 -07:00
• 1fb76ebb93 Revert "Fuse routed scaling factor in topk_reduce kernel (#6220)" (#6968) Yineng Zhang 2025-06-07 21:02:49 -07:00
• c2c4f57f63 [DeepseekR1-FP4] Add Support for nvidia/DeepSeekR1-FP4 model (#6853) Pavani Majety 2025-06-07 17:24:35 -07:00
• 23881fa60c chore: upgrade sgl-kernel v0.1.6.post1 (#6957) Yineng Zhang 2025-06-07 17:18:55 -07:00
• 8db3ac55a9 chore: bump sgl-kernel v0.1.6.post1 (#6955) Yineng Zhang 2025-06-07 15:25:46 -07:00
• 3e56f557fd Add a CUDA kernel for fusing mapping and weighted sum for MoE. (#6916) Elfie GuoandElfie Guo 2025-06-07 15:24:39 -07:00
• 62fec60d81 Add H20 fused MoE kernel tuning configs for DeepSeek-R1/V3 (#6885) Xu Wenqing 2025-06-08 06:17:34 +08:00
• e7759778e5 [misc] add is_cpu() (#6950) JieXin Liang 2025-06-08 06:13:45 +08:00
• 77e928d00e Update server timeout time in AMD CI. (#6953) Sai Enduri 2025-06-07 15:10:27 -07:00
• 515ef4facb Fuse routed scaling factor in topk_reduce kernel (#6220) Xiaoyu Zhang 2025-06-08 02:06:50 +08:00
• f5599ef124 Refactor global_server_args_dict (#6866) fzyzcjy 2025-06-07 18:10:35 +08:00
• c499591ac8 Add canary for EPLB rebalancing (#6895) fzyzcjy 2025-06-07 18:09:33 +08:00
• e1ce44cdb1 Disabling mixed chunked prefill when eagle is enabled (#6874) Swipe4057 2025-06-07 13:06:58 +03:00
• f1114e7ff3 [Docker] Add docker file for SGL Router (#6915) Arthur Cheng 2025-06-07 02:58:43 -07:00
• bae4fdc7ab add fbgemm moe grouped gemm kernel benchmark (#6924) Xiaoyu Zhang 2025-06-07 17:57:30 +08:00
• 6153f2ff6e chore: upgrade sgl-kernel v0.1.6 (#6945) JieXin Liang 2025-06-07 17:53:26 +08:00
• 8b5f83ed3b reduce torch.zeros overhead in moe align block size kernel (#6369) Xiaoyu Zhang 2025-06-07 17:47:36 +08:00
• 2a413829f4 Add triton version as a fused_moe_triton config search key to avoid performace decrease in different Triton version (#5955) Xiaoyu Zhang 2025-06-07 17:43:50 +08:00
• d5c097a2f9 Tiny re-introduce profile id logging (#6912) fzyzcjy 2025-06-07 17:32:50 +08:00
• 9736cd3b7d [Bugfix] pipeline parallelism and Eagle Qwen2 (#6910) Swipe4057 2025-06-07 11:58:50 +03:00
• 2f715f51cc Minor compile fused topk (#6944) fzyzcjy 2025-06-07 16:40:38 +08:00
• d664ca18f2 chore: bump sgl-kernel v0.1.6 (#6943) Yineng Zhang 2025-06-07 00:25:22 -07:00
• 22fe787852 [sgl-kernel] update deepgemm (#6942) JieXin Liang 2025-06-07 14:24:41 +08:00
• c4ffbeca19 Add triton fused moe kernel config for E=257 on B200 (#6939) Baizhou Zhang 2025-06-06 23:15:01 -07:00
• f8eaaab817 [fix] logical_to_all_physical_map index 256 is out of bounds in EP parallel. (#6767) miter 2025-06-07 12:32:33 +08:00
• 697b0f71f0 [Refactor] image data process in bench_serving (#6879) Xinyuan Tong 2025-06-06 21:11:17 -07:00
• 132dad874d [PD] Optimize transfer queue forward logic for dummy rank (#6922) shangmingc 2025-06-07 09:26:14 +08:00
• 60fdad7cf3 Sync the changes on cuda graph runners (#6932) Lianmin Zheng 2025-06-06 18:23:52 -07:00
• 61ce91ed28 Tiny support customize DeepEP max dispatch tokens per rank (#6934) fzyzcjy 2025-06-07 08:18:35 +08:00
• e6b7053b60 Fix a bug in abort & Improve docstrings for abort (#6931) Lianmin Zheng 2025-06-06 14:35:45 -07:00
• 5f91c82526 [Feature] Support Flashinfer fmha on Blackwell (#6930) Jianan Ji 2025-06-06 15:57:50 -04:00
• b819381fec AITER backend extension and workload optimizations (#6838) 3 people 2025-06-05 23:00:18 -07:00
• 562f279a2d [CPU] enable CI for PRs, add Dockerfile and auto build task (#6458) 3 people 2025-06-06 04:43:54 +08:00
• 8b2474898b bugfix(OAI): Fix image_data processing for jinja chat templates (#6877) Chang Su 2025-06-05 13:37:01 -07:00
• 0df6765c83 [CUTLASS-FP4-MOE] Introduce CutlassMoEParams class for easy initialization of Cutlass Grouped Gems Metadata (#6887) Pavani Majety 2025-06-05 13:13:14 -07:00
• 35b65cf0ca Use deepgemm instead of triton for fused_qkv_a_proj_with_mqa (#6890) fzyzcjy 2025-06-06 02:37:05 +08:00
• dd1012fcbe [PD] Fix potential perf spike caused by tracker gc and optimize doc (#6764) shangmingc 2025-06-06 01:56:02 +08:00
• 44aab7f91c oai: fix openAI client error with single request via batch api (#6170) Ravi ThejaandRavi Theja Desetty 2025-06-05 15:51:47 +05:30
• 43baba649e [EP] Add cuda kernel for moe_ep_post_reorder (#6837) Yuan Luoandluoyuan.luo 2025-06-05 15:33:47 +08:00
• 0166403c20 Support Blackwell DeepEP docker images (#6868) fzyzcjy 2025-06-05 15:07:53 +08:00
• bcf66ef3e1 Tiny allow profiler API to auto create directory (#6865) fzyzcjy 2025-06-05 15:07:03 +08:00
• 0de5e7d40f Support layerwise rebalancing experts (#6851) fzyzcjy 2025-06-05 15:05:52 +08:00
• 72a110f664 Tiny update error hints (#6846) fzyzcjy 2025-06-05 15:05:28 +08:00
• 5aff1e9392 Fix Qwen3MoE missing token padding optimization (#6820) fzyzcjy 2025-06-05 15:04:59 +08:00
• 8e3797be1c support 1 shot allreduce in 1-node and 2-node using mscclpp (#6277) zyksir 2025-06-05 13:11:24 +08:00
• 4474eaf552 Support LoRA in TestOpenAIVisionServer and fix fused kv_proj loading bug. (#6861) Lifu Huang 2025-06-04 22:08:30 -07:00
• 499f5e620c Fix one missing arg in DeepEP (#6878) Cheng Wan 2025-06-04 19:14:47 -07:00
• 81964328b7 Set num_fused_shared_experts as num_shared_experts when shared_experts fusion is not disabled (#6736) Cheng Wan 2025-06-04 15:53:22 -07:00
• f0f84975f4 feat: add dp-rank to KV events (#6852) ishandhanani 2025-06-04 15:29:34 -07:00
• 3f1e433903 Decoder-only Scoring API (#6460) Chanh NguyenandChanh Nguyen 2025-06-04 14:14:54 -07:00
• cf9815ba69 [Refactor] Multimodal data processing for VLM (#6659) Xinyuan Tong 2025-06-04 11:22:33 -07:00
• bd75690f4e fix ep_moe_reorder kernel bugs (#6858) Xiaoyu ZhangandJieXin Liang 2025-06-04 19:13:59 +08:00
• 180ff5eecc [fix] recover auto-dispatch for rmsnorm and rope (#6745) JieXin Liang 2025-06-04 12:44:20 +08:00
• 37f1547587 [FEAT] Add transformers backend support (#5929) Marc Sun 2025-06-04 06:05:29 +02:00
• 8a5480528d [Refactor] Rename n_share_experts_fusion as num_fused_shared_experts (#6735) Cheng Wan 2025-06-03 17:48:24 -07:00
• b6d0ce9f78 Minor add metrics to expert location updater (#6816) fzyzcjy 2025-06-03 14:59:11 +08:00
• 0ea330ca34 Fix wrong weight reference in dynamic EPLB (#6818) fzyzcjy 2025-06-03 14:26:04 +08:00
• 27e327b415 fix new_page_count_next_decode (#6671) pansicheng 2025-06-03 13:48:52 +08:00
• ff00895c46 Add CPU optimized kernels for topk and rope fusions (#6456) jianan-gu 2025-06-03 08:37:34 +08:00
• ff91474825 [Router] Fix k8s Service Discovery (#6766) Arthur ChengandSimo Lin 2025-06-02 16:57:23 -07:00
• eb38c7d1ca [1/2] Add Kernel support for Cutlass based Fused FP4 MoE (#6093) Pavani Majety 2025-06-02 13:48:03 -07:00
• df7f61ee7d Speed up rebalancing when using non-static dispatch algorithms (#6812) fzyzcjy 2025-06-03 02:18:17 +08:00
• ef21729c1d Fix profiles do not have consistent names (#6811) fzyzcjy 2025-06-03 02:17:22 +08:00
• f5159315b2 Add simple utility to dump tensors for debugging (#6815) fzyzcjy 2025-06-03 02:15:31 +08:00
• 6d7b6696d4 Tiny fix EPLB assertion about rebalancing period and recorder window size (#6813) fzyzcjy 2025-06-03 02:13:33 +08:00
• 6376b632eb Tiny log prefill time (#6780) fzyzcjy 2025-06-03 01:28:27 +08:00
• e05e29d178 Refactor CustomOp to avoid confusing bugs (#5382) fzyzcjy 2025-06-03 01:27:36 +08:00
• a2cb5913a0 Add draft extend CUDA graph for flashinfer backend (#6805) Ke Bao 2025-06-02 16:51:26 +08:00
• 55444ed667 [EP] Add cuda kernel for moe_ep_pre_reorder (#6699) Yuan Luoandluoyuan.luo 2025-06-02 11:49:01 +08:00
• 20fd53b8f6 Correctly abort the failed grammar requests & Improve the handling of abort (#6803) Lianmin Zheng 2025-06-01 19:00:07 -07:00
• 6a47b73024 Remove contiguous before Flashinfer groupwise fp8 gemm (#6804) Baizhou Zhang 2025-06-01 18:30:54 -07:00
• c429919def misc: cache is_hopper_arch (#6799) Wenxuan Tan 2025-06-01 17:28:31 -05:00
• 1da8d23051 chore: update blackwell docker (#6800) Yineng Zhang 2025-06-01 13:37:40 -07:00
• 2f7420bc84 [Feat] Enable PDL automatically on Hopper architecture (#5981) Huapeng Zhou 2025-06-01 12:30:17 -07:00
• c6a0cacc35 Update CI tests for Llama4 models (#6421) Ravi Theja 2025-06-01 09:22:15 +05:30
• 0a9bfc20ab [Minor] Always append newline after image token when parsing chat message (#6797) Lifu Huang 2025-05-31 20:50:33 -07:00
• 34c63731fc chore: upgrade sgl-kernel v0.1.5 (#6795) Yineng Zhang 2025-05-31 18:32:00 -07:00
• 2d72fc47cf Improve profiler and integrate profiler in bench_one_batch_server (#6787) Lianmin Zheng 2025-05-31 15:53:55 -07:00
• b520d02888 chore: bump sgl-kernel v0.1.5 (#6794) Yineng Zhang 2025-05-31 14:54:00 -07:00