Commit Graph

Select branches

Hide Pull Requests

dsv41-pd

dsv41-pd-visioncp

main

Mono Color

• 5c705b1dce Add perf tests for LoRA (#8314) Lifu Huang 2025-07-26 14:55:22 -07:00
• b7094a5ef1 model: support intern-s1 (#8350) 5 people 2025-07-27 04:48:51 +08:00
• da0c026084 Tiny assert EPLB is used together with expert parallel (#8381) fzyzcjy 2025-07-26 18:20:39 +08:00
• 3212c2ad3f vlm: optimize tensor transport (#6003) MickandXinyuan Tong 2025-07-26 17:41:01 +08:00
• 534756749a chore: improvements on mm_utils (#7737) Mick 2025-07-26 17:38:56 +08:00
• ce32bc2ba9 Extract update_weights from RL Engine to SGLang to keep simplicity and fix torch reduce (#8267) 4 people 2025-07-26 02:00:59 -07:00
• e236d8fee8 Save peak memory in logits processor (#8343) Cheng Wan 2025-07-26 01:46:42 -07:00
• 4fa44d63c6 chore: improve mmmu benchmark (#7000) MickandXinyuan Tong 2025-07-26 16:19:45 +08:00
• e6312d271d Uodate Dockerfile.gb200 to latest sglang (#8356) kyleliang-nv 2025-07-26 00:22:06 -07:00
• 8af145b7dc Fix test_moe_fused_gate_combined sgl-kernel ci test (#8374) Ke Bao 2025-07-26 09:30:12 +08:00
• 2272c2a5b5 chore: bump v0.4.9.post4 (#8305) Yineng Zhang 2025-07-25 17:12:47 -07:00
• 3ec0b21229 [CI] Fix flaky threshold (#8370) Lianmin Zheng 2025-07-25 16:41:56 -07:00
• 58c468f404 Fix FP4 MoE accuracy from missing routed_scaling_factor (#8333) Trevor Morris 2025-07-25 16:40:23 -07:00
• f8ca2368b2 fix: kimi k2 xgrammar crash (#8367) 3 people 2025-07-25 15:44:01 -07:00
• d8ee15643b [Feat] Add reasoning parser for Qwen/Qwen3-235B-A22B-Thinking-2507 (#8363) Chang Su 2025-07-25 14:59:42 -07:00
• 7181ec8cfc fix: upgrade nccl version (#8359) Yineng Zhang 2025-07-25 14:59:02 -07:00
• ed2e313eb6 Clean up server_args, triton cache manager (#8332) Lianmin Zheng 2025-07-25 14:14:51 -07:00
• f8260f2539 [Bugfix][Feat] Add XML-ish grammar in EBNFComposer and fix misc bugs in Qwen3 detector (#8357) Chang Su 2025-07-25 12:03:16 -07:00
• 12cb760a37 Add H20-3e fused MoE kernel tuning configs for Qwen3-Coder-480B-A35B-Instruct (#8344) Xu Wenqing 2025-07-26 01:58:12 +08:00
• 1b9cea5ade [P/D] Support ipv6 in P/D scenario (#7858) Stepan KargaltsevandShangming Cai 2025-07-25 18:53:30 +03:00
• 9045cc1eb8 [torch.compile bug] avoid biased_grouped_topk_impl func repeatedly triggering torch.compile in forward pass (#8353) Xiaoyu Zhang 2025-07-25 21:17:47 +08:00
• 70e37b97bf chore: upgrade mooncake 0.3.5 (#8341) Shangming Cai 2025-07-25 16:17:26 +08:00
• 15d2759174 [CPU] Add tutorial docs for SGL on CPU (#8000) Zaili Wang 2025-07-25 15:03:16 +08:00
• af4b9bae95 [AMD] Add silu_and_mul, gelu_and_mul, gelu_tanh_and_mul, and gelu_quick kernels for AMD GPUs (#7135) 3 people 2025-07-24 23:44:28 -07:00
• 7ad6b766c5 fix: Fix failed functional tests https://github.com/meta-llama/llama-stack-evals (#8266) Ying Wang 2025-07-24 23:11:32 -07:00
• c0fb25e949 DP Enhancement (#8280) Cheng Wan 2025-07-24 21:36:21 -07:00
• 28d4d47280 [Feature] Integrate quick allreduce and select the best allreduce implementation (#6619) li haoyangandilmarkov 2025-07-25 11:48:42 +08:00
• f4674df646 support idle batch for TBO (#8233) ZhichenJiang 2025-07-25 11:43:52 +08:00
• d40846d456 breakdown kernel update (#8334) Zhiqiang Xie 2025-07-24 17:33:17 -07:00
• 145482f422 HiCache Storage TP Refinement (#8307) Zhiqiang Xieandpansicheng 2025-07-24 17:31:47 -07:00
• 39fe1e880d [router] add request format unit test (#8300) Simo Lin 2025-07-24 14:30:37 -07:00
• 33c4b4d04e [router] add streaming unit test (#8299) Simo Lin 2025-07-24 14:30:27 -07:00
• 8d1c5b948e chore: upgrade flashinfer v0.2.9rc1 (#8301) Swipe4057andYineng Zhang 2025-07-25 01:29:56 +04:00
• a167fd0bcb [code style] Clean dead triton kernel code in fused_moe and useless vllm_ops import (#8310) Xiaoyu Zhang 2025-07-24 14:38:30 +08:00
• 2f86f3ad62 [router] add endpoint unit test (#8298) Simo Lin 2025-07-23 23:26:44 -07:00
• bfb118c01e fix bug when eos_ids==0 (#8315) Minho Ryu 2025-07-24 15:18:47 +09:00
• f6e07f2796 [router] fix pd model completion request (#8303) Simo Lin 2025-07-23 23:18:29 -07:00
• 5dd0f870ab [bug] fix pd completion protocol for batching support (#8317) Simo Lin 2025-07-23 23:18:17 -07:00
• f7e102d56a Pin the version of petit kernel to fix the APIs (#8235) Haohui Mai 2025-07-23 17:57:20 -07:00
• 0e5fa67773 [AMD] Pull latest image for AMD CI (#8070) michael-amd 2025-07-23 17:56:14 -07:00
• 624a3b8d1f Fix incomplete tool call capture issue in streaming response of DeepSeek-V3 when enable MTP (#7562) xianzhiT 2025-07-24 08:40:23 +08:00
• 01079e174f feat(function call): complete utility method for KimiK2Detector and enhance documentation (#8043) Chang Su 2025-07-23 17:37:31 -07:00
• 0e7a5b2694 fix: prevent crashes due to logit bias dimension mismatch (#7685) J 2025-07-23 15:30:55 -07:00
• 4953f4ca9a chore: upgrade sgl-kernel 0.2.7 (#8304) Yineng Zhang 2025-07-23 15:07:27 -07:00
• 38000a5f44 Fix gemma3n with hybrid swa (#8240) Xinyuan TongandXinyuan Tong 2025-07-23 13:29:18 -07:00
• 70251e935e fix: match chat-template for internvl3 (#8262) Xinyuan TongandXinyuan Tong 2025-07-23 13:29:03 -07:00
• c87d4fec99 Fix the issue of incorrect finish reason in final stream response chunk returned during tool call (#7708) xianzhiTandXinyuan Tong 2025-07-24 04:28:53 +08:00
• a99801e075 [Performance][PD Disaggregation] optimize TokenToKVPoolAllocator by sorting free pages (#8133) YiXRandXingrui Yi 2025-07-24 04:28:12 +08:00
• 4c605235aa fix: workaround for deepgemm warmup issue (#8302) Yineng Zhang 2025-07-23 12:01:51 -07:00
• 6f8f4aeea4 [router] add common ut infra to mock worker and app (#8295) Simo Lin 2025-07-23 10:07:51 -07:00
• 0c8dab9e67 [sgl-kernel] Opt per_token_quant_fp8 with warp reduce (#8130) Yuan Luoandluoyuan.luo 2025-07-23 21:22:59 +08:00
• f39037fffb HiCache Fix (#8288) Zhiqiang Xieandpansicheng 2025-07-23 01:51:32 -07:00
• ce86e201df bug fix and tag (#8282) Zhiqiang Xie 2025-07-23 01:50:31 -07:00
• b43263307f Hicache IO kernel refactoring (#8264) Zhiqiang Xie 2025-07-23 01:49:03 -07:00
• 8abd3e77fe Introduce Stable LoRA ID System for Overlapped Updates and Prefix Caching (#8261) Lifu Huang 2025-07-23 00:32:16 -07:00
• e885bfdc6a Fix sgl-kernel ci test (#8284) Ke Bao 2025-07-23 14:01:47 +08:00
• e2d66f60c8 Skip llama4 vision module loading when multimodal disabled (#8272) Ke BaoandMick 2025-07-23 12:41:25 +08:00
• 01c000043c chore: bump v0.4.9.post3 (#8265) Yineng Zhang 2025-07-22 15:55:48 -07:00
• 0dfe2491ac Preliminary Support for Qwen3XMLDetector (#8260) yhyang201andChayenne 2025-07-23 06:49:38 +08:00
• ff45ab7a5f [Benchmark] add disable-auto-run param for hicache/bench_multiturn (#7822) 3 people 2025-07-23 05:02:40 +08:00
• 0f8b538614 [fix] benchmark : routed_scaling_factor is None (#8059) Peter PanandXiaoyu Zhang 2025-07-22 23:55:35 +08:00
• c33499a67b fix: sgl-router remove dead code (#8257) Rui Chen 2025-07-22 23:41:23 +08:00
• e50109f2ed [AMD] Remove vllm's scaled_fp8_quant and moe_sum when SGLANG_USE_AITER=1 (#7484) Hubert Lu 2025-07-21 17:33:19 -07:00
• 69adc4f81c fix: retrieve mm token by modality, raise error if none (#8221) Xinyuan TongandXinyuan Tong 2025-07-21 17:06:35 -07:00
• 114837854f docs: update 2025 h2 roadmap (#8237) Yineng Zhang 2025-07-21 14:02:48 -07:00
• 7b68d27111 [Feature] Add a test for Layer-wise Prefill (#8231) Xiaoze Fan 2025-07-21 22:06:15 +08:00
• 74f59ae555 chore: upgrade sgl-kernel 0.2.6.post1 (#8202) Yineng Zhang 2025-07-21 02:10:24 -07:00
• 6936be3221 Remve router gemm output dtype conversion (#8204) Ke Bao 2025-07-21 15:37:00 +08:00
• 9b5de6cb06 [router] upgade router version to 0.1.6 (#8209) Simo Lin 2025-07-20 23:13:20 -07:00
• 5c8365a051 [router] add ut for pd router (#8208) Simo Lin 2025-07-20 23:12:52 -07:00
• 8430bfe3e9 [Refactor] simplify multimodal data processing (#8107) Xinyuan Tong 2025-07-20 21:43:09 -07:00
• c9e8613c97 Apply fused sorted token ids padding (#8193) Ke Bao 2025-07-21 11:19:48 +08:00
• 429bb0efa2 chore: bump sgl-kernel v0.2.6.post1 (#8200) Yineng Zhang 2025-07-20 19:50:28 -07:00
• 7eebd44047 [fix] fix modelopt fp4 on b200 (#8195) JieXin Liang 2025-07-21 08:39:57 +08:00
• 93d124ef5a [feature] enable NPU CI (#7935) ronnie_zhengandEven Zhou 2025-07-20 23:12:42 +03:00
• 1fc455e8b6 [router] add ut for pd request, metrics and config (#8184) Simo Lin 2025-07-20 10:53:42 -07:00
• 465968b2e3 Fix dtype error in CI (#8197) Ke Bao 2025-07-21 00:27:55 +08:00
• 750838adc4 fix: fix the bug of loading Internvl3 (#8067) GuoYipinandXinyuan Tong 2025-07-20 22:22:54 +08:00
• 99aefa037e Fix eagle3 cuda graph (#8163) Jay Zhou 2025-07-20 00:28:06 -07:00
• bbcfbc1a02 feat: add h200 tp 16 kimi k2 moe config (#8183) Qiaolin Yu 2025-07-20 02:30:08 -04:00
• 83c104b188 Feat: Support for Persimmon Model (#7983) Praneth Paruchuri 2025-07-20 11:37:47 +05:30
• 2db6719cc5 feat: update nccl 2.27.6 (#8182) Yineng Zhang 2025-07-19 22:55:45 -07:00
• 55381a46ac Revert "[Feature] Simple Improve Health Check Mechanism for Production-Grade Stability" (#8181) Lianmin Zheng 2025-07-19 22:41:30 -07:00
• a589a07167 fix moe gate dtype, fix tbo, fix fake dispatch (#7825) Atream 2025-07-20 13:13:46 +08:00
• f62d75b6a1 feat: add b200 tp 16 kimi k2 moe config (#8178) Yineng Zhang 2025-07-19 20:04:12 -07:00
• 0f9b11e310 feat: add h200 tp 16 kimi k2 moe config (#8176) Yineng Zhang 2025-07-19 20:04:02 -07:00
• 877e35d775 Add get_hidden_dim to qwen3.py for correct lora (#7312) Pavel Logachev 2025-07-20 05:31:16 +03:00
• cbdfb77123 Enable FlashInfer support encoder models and add head_dim padding workaround (#6230) Clay 2025-07-20 10:30:16 +08:00
• 282eb59ff3 Add bf16 output option for dsv3_router_gemm kernel (#7999) Baizhou Zhang 2025-07-19 18:49:37 -07:00
• 4540a4666a [Feature] Simple Improve Health Check Mechanism for Production-Grade Stability (#8115) ybyang 2025-07-20 09:10:00 +08:00
• abda2542d5 Fix tuning_fused_moe_triton.py (#8175) Cheng Wan 2025-07-19 17:33:50 -07:00
• 8cddfa56a1 Clean warning logs for gate_proj loading in Lora (#8172) Baizhou Zhang 2025-07-19 15:56:50 -07:00
• 4e3defe5a7 Support start up LoRA server without initial adapters (#8019) Lifu Huang 2025-07-19 15:38:09 -07:00
• 60468da4e2 bugfix: fix sglang crash in NVIDIA MIG container (#8167) Garry Fang 2025-07-20 05:41:27 +08:00
• 41d33e4736 [router] add ut for worker and errors (#8170) Simo Lin 2025-07-19 14:38:33 -07:00
• bfdd226f35 Fix Dockerfile.gb200 (#8169) kyleliang-nv 2025-07-19 14:37:53 -07:00
• 3de617a75b Fix LoRA buffer contamination during adapter eviction (#8103) Lifu Huang 2025-07-19 13:14:08 -07:00
• bb0e8a32b5 Clean up server args (#8161) Lianmin Zheng 2025-07-19 11:32:52 -07:00
• 1b427dae02 Update README.md (#8171) Lianmin Zheng 2025-07-19 11:04:19 -07:00
• f3d9736156 Fix suffix mismatch for the metrics. (#8168) Charles Chen 2025-07-20 01:11:24 +08:00