Commit Graph

Select branches

Hide Pull Requests

bot/update-est-time-20260928

dsv41-pd

dsv41-pd-visioncp

main

Mono Color

• 53e2cd464f [router] remove all tokenizer metrics for performance (#9474) Chang Su 2025-08-21 18:35:24 -07:00
• 9708d353b7 Support MHA with chunked prefix cache for flashinfer/flashmla backend, support page size > 1 for MHA chunked prefix (#8616) Yongfei Xuandxuyongfei.xyf 2025-08-22 09:19:44 +08:00
• 704ced1b2e [AMD] Remove the deprecated C10_WARP_SIZE (#9356) Hubert Lu 2025-08-21 18:16:35 -07:00
• 3cc3d9b950 Add Support for Page Size greater than 1 for Flashinfer MLA Backend (#8593) Pavani Majety 2025-08-21 18:15:06 -07:00
• 0b3a5b1151 Update reasoning parser doc (#9468) Xinyuan Tong 2025-08-22 08:25:30 +08:00
• 6c855db82c Revert "bugfix: Fix output_ids extraction in detokenizer_manager" (#9467) Xinyuan Tong 2025-08-22 08:24:25 +08:00
• 0f9318f7d0 feat: update auto_choose_speculative_params (#9470) Yineng Zhang 2025-08-21 17:12:12 -07:00
• 849957bc76 fix: tmp revert gpt oss tp sharding on hopper (#9469) Yineng Zhang 2025-08-21 17:03:21 -07:00
• cded039b57 [FA3] Init Spec Page Table only when Spec is enabled to save ~40MB (#9455) Stefan He 2025-08-21 15:11:38 -07:00
• 275f9df381 feat: add fused moe config for GLM-4.5-Air-FP8 on B200 (#9463) zixuanzhang226 2025-08-21 15:10:20 -07:00
• e8449ab515 Add deepseek v3.1 thinking parser support and update docs (#9464) Xinyuan Tong 2025-08-22 06:09:40 +08:00
• 4746aaea41 fix: support fb fp8 (#9462) Yineng Zhang 2025-08-21 14:31:43 -07:00
• 10d34f74e2 fix: should return a invalid request response when schema missing (#9461) gongwei-130 2025-08-21 14:06:50 -07:00
• 9ba7253094 accomendate reasoning_effort set in chat_template_kwargs (#9458) gongwei-130 2025-08-21 13:22:03 -07:00
• 9c8e4f69c3 [5/n]decouple quantization implementation from vLLM dependency (#9454) Hongbo Xu 2025-08-22 03:52:07 +08:00
• 78ae175866 [router] add tokenizer benchmark (#9427) Simo Lin 2025-08-21 11:09:39 -07:00
• dae9a80f43 [fix] Fix mxfp4 weight loading bug with TP sharding in GPT-OSS (#9433) hlu1andXinyuan Tong 2025-08-21 03:50:51 -07:00
• e85cb1ce9d Fix quant kernel test errors and benchmark wrong output speeds (#7604) fzyzcjy 2025-08-21 18:48:41 +08:00
• 55d336cb08 Refactor weight offloading logic (#8521) fzyzcjy 2025-08-21 18:48:13 +08:00
• de4990a5b2 [Bug] Fix w4afp8 moe kernel (#9392) Yuhao Yao 2025-08-21 18:45:18 +08:00
• 029e0af31d ci: enhance xeon ci (#9395) DiweiSun 2025-08-21 18:35:17 +08:00
• 64574ef8c0 Enables speculative decoding for the trtllm_mla attention backend (#9238) pranavm-nvidia 2025-08-21 01:18:21 -07:00
• 18da2c96ec [NVIDIA] Fix trtllm fp4 moe backend when used in MTP (#9384) Kaixi Hou 2025-08-21 00:54:01 -07:00
• 9b5f0f64f5 Fix tiny misalign with previous truncation setting in tokenizer_manager (#9430) Liangsheng Yin 2025-08-21 14:05:35 +08:00
• 70bb066ee4 Fix FP4 inference corruption issue in glm4.5-air model (#9346) Azure 2025-08-21 13:13:47 +08:00
• 2c4b4b786b [feature] Ascend NPU graph support (#9399) 6 people 2025-08-21 07:13:27 +03:00
• 7cd2ee06d7 feat: Add Triton fallback option and SM120 MoE configs for FP8 models (#9251) Martin Vit 2025-08-21 04:33:15 +02:00
• eb19ccadae [bug] fix errors related to context length in SD (#9388) Liangsheng Yin 2025-08-21 10:32:34 +08:00
• 25ef53f05f [PD] Fix nvlink transport accuracy through transferring metadata with tcp (#9261) Shangming Cai 2025-08-21 10:29:10 +08:00
• c674bf9c6b Fix biased_grouped_topk_cpu (#9420) Cao E 2025-08-21 10:18:48 +08:00
• af1973b871 Fix max_seq_len_k in trtllm_mha attention backend (#9416) Qiaolin Yu 2025-08-20 19:17:13 -07:00
• 5cfbb4c136 [router] add glm and step3 reasoning parser (#9415) Chang Su 2025-08-20 18:33:10 -07:00
• e65231022f [router] add tokenizer integration test with real mini tokenizer (#9413) Chang Su 2025-08-20 17:56:23 -07:00
• 3828db4309 [router] Add IGW (Inference Gateway) Feature Flag (#9371) Keyang RuandYineng Zhang 2025-08-20 17:38:57 -07:00
• 88fbc31b50 Support trtllm_allreduce_fusion in flashinfer for cuda<12.8 (#9339) strgrbandZhang Kaihong 2025-08-21 07:54:30 +08:00
• 8f5b9910c1 Add support for Qwen3-seq-cls (#9357) nathan 2025-08-21 01:51:56 +02:00
• ef3004d90a misc: parse bench_serving result as markdown table (#9377) Mick 2025-08-21 07:44:20 +08:00
• 84719b527a fix: InternS1 don't recognize image, updates image token for InternVL processor (#9381) Xinyuan Tong 2025-08-21 07:43:03 +08:00
• e99729c9f3 Fixed the issue where eagle3 TPOT was not as good as without eagle3. (#9404) jiapingW 2025-08-21 07:42:01 +08:00
• c10b8e6a0f Support DP attention with GPT-OSS (#9359) Nicolas Castet 2025-08-20 18:36:31 -05:00
• d4bce29721 Fix incorrect logic in chat template handling. (#9336) Lifu HuangandXinyuan Tong 2025-08-20 16:25:36 -07:00
• b0980af89f Support pinning adapter via server args. (#9249) Lifu Huang 2025-08-20 16:25:01 -07:00
• 24eaebeb4b Fix FlashInfer GPU <-> CPU sync (#9409) Nathan Wang 2025-08-20 18:26:12 -04:00
• a91e90d9a3 [2/2] Fuse routed scaling factor into select_experts (#8690) Trevor Morris 2025-08-20 15:10:16 -07:00
• f96413c444 Refactor allreduce add rmsnorm pattern (#9278) Xiaoyu Zhang 2025-08-20 17:03:08 +08:00
• 08ebdf79d0 Fix the --allow-auto-truncate argument in tokenizer manager. (#9391) Liangsheng Yinandgemini-code-assist[bot] 2025-08-20 16:56:47 +08:00
• 42c8704560 Add PDL support for quant kernel and rope kernel (#9106) fzyzcjy 2025-08-20 16:56:29 +08:00
• c9bf3877a0 Reduce overhead for fa by not calling heavy CUDA property check (#7375) Yichen Yan 2025-08-20 16:26:28 +08:00
• de2dd73831 Revert "[feature] Rework Ascend NPU graph support" (#9385) Even Zhou 2025-08-20 15:35:10 +08:00
• 1ec9769753 [Docs] Update contribution guide (#9383) Lianmin Zheng 2025-08-19 23:37:45 -07:00
• d8ed60f254 [CI] Fix disaggregation failure tolerance CI (#9378) Shangming Cai 2025-08-20 14:31:08 +08:00
• f1b0eda55c [readme] Add SGLang x AMD SF meetup information (#9380) Mingyi 2025-08-19 22:25:09 -07:00
• f20b6a3f2b [minor] Sync style changes (#9376) Lianmin Zheng 2025-08-19 21:35:01 -07:00
• 3680d6f88b [feature] Rework Ascend NPU graph support (#9350) 6 people 2025-08-20 11:32:27 +08:00
• f515449582 Fix gpt-oss response api streaming issue (#9368) Keyang Ru 2025-08-19 20:19:42 -07:00
• e0ce171d79 Fix triton backend eagle illegal memory access (#9344) Ke Bao 2025-08-20 11:16:26 +08:00
• fe43e889f8 Fix mini lb timeout issue (#9369) fzyzcjy 2025-08-20 11:15:16 +08:00
• 5ae5ecaa15 [router] Implement OpenAI Responses API specification (#9367) Keyang Ru 2025-08-19 20:14:47 -07:00
• 5fbad308cd [router] add tokenizer chat template support (#9370) Simo LinandChang Su 2025-08-19 20:14:02 -07:00
• 7638f5e44e [router] Implement gRPC SGLangSchedulerClient (#9364) Chang Su 2025-08-19 16:44:11 -07:00
• b45f753cba [router] adds reasoning parser pooling and thread-safe (#9360) Simo Lin 2025-08-19 13:35:39 -07:00
• c5057262fa [Router] Add validation module for API parameters (#9335) Keyang Ru 2025-08-19 13:25:53 -07:00
• 46fe8b8cb2 [CI] Fix lint issues (#9361) Chang Su 2025-08-19 13:05:36 -07:00
• 0b95a01a8f [router] add tiktokenizer and sequence in router (#9354) Simo LinandChang Su 2025-08-19 10:46:28 -07:00
• a3b810ebdb fix: enable multi-GPU Triton fused MoE tuning (#6295) mpashkovskiy 2025-08-19 20:16:58 +03:00
• 94959237bf [router] add dsr1, kimi, and qwen reasoning parser (#9353) Simo Lin 2025-08-19 10:15:24 -07:00
• f4fafacc5d Revert "[feature] Ascend NPU graph support (#8027)" (#9348) Even Zhou 2025-08-20 01:11:23 +08:00
• 01d47a27b6 [Bugfix] fix kv buffer register & dp attention & deepepmoe (#9327) chenxu140 2025-08-20 01:09:48 +08:00
• ecc9f3e47a [Minor] Fix the style of sgl-kernel (#9332) Lianmin Zheng 2025-08-18 23:45:00 -07:00
• 7e8187e004 docs: fix spec (#9326) Yineng Zhang 2025-08-18 19:35:46 -07:00
• e483ab6d20 enable marlin fp8 blockwise (#8990) Enrique Shockwave 2025-08-19 02:53:15 +01:00
• 720cd308ba Add CMakeLists.txt binary_dir (#7019) EduardDurech 2025-08-19 03:36:33 +02:00
• ce67b2d586 [router]restructure protocol modules for better organization (#9321) Keyang Ru 2025-08-18 18:07:58 -07:00
• 3c2c9f6c9e [Bug] Fix input arguments of flashinfer_trtllm_moe (#9317) Jiaqi Gu 2025-08-18 18:03:19 -07:00
• a31ea44824 support for interns1-mini (#9299) zxy 2025-08-19 08:56:04 +08:00
• 439df4548a [router] Add spec for sglang scheduler (#9322) Chang Su 2025-08-18 17:20:20 -07:00
• 5626e20b2b Tiny fix CI (#9306) fzyzcjy 2025-08-19 07:54:36 +08:00
• c6c379ab31 [AMD] Reorganize hip-related header files in sgl-kernel (#9320) Hubert Lu 2025-08-18 16:53:44 -07:00
• c2fbf60f39 [GLM4.1V and GLM4.5V] Add vision transformer num_dummy_head support: max tp=4 -> max tp=8 (#9059) Binyao Jiang 2025-08-18 14:40:13 -07:00
• 98b44e9e56 [PD] Propagate internal server errors from aborted requests to clients instead of blindly returning 200's (#8936) datdo-msft 2025-08-18 14:23:46 -07:00
• 6805f6da40 upgrade xgrammar 0.1.23 and openai-harmony 0.0.4 (#9284) Swipe4057 2025-08-19 00:02:00 +03:00
• ca533580f2 [Docs] Correct and clarify notes in Engine docstring (#9313) 江家瑋 2025-08-19 04:24:19 +08:00
• 886454e8e7 [MISC] use dynamic choices for tool-call-parser argument (#9316) Keyang Ru 2025-08-18 13:02:10 -07:00
• 0cf3fbeb18 should return invalide request for empty prompt (#9315) gongwei-130 2025-08-18 11:44:11 -07:00
• 2256d62d36 Modelopt quant config adaptation (#8829) Zhiyu 2025-08-18 11:27:30 -07:00
• 6cdcbcc674 [fix] fix enable_pdl for blackwell (#9011) JieXin Liang 2025-08-19 01:16:08 +08:00
• c480a3f6ea Minor style fixes for sgl-kernel (#9289) Lianmin Zheng 2025-08-18 09:38:35 -07:00
• 6e316588f8 [router] add reasoning parser base structure (#9310) Simo LinandChang Su 2025-08-18 09:26:09 -07:00
• 24247b4168 [router] add tokenizer metrics (#9307) Simo LinandChang Su 2025-08-18 09:25:51 -07:00
• 4c0bb411e5 Further fix memory pool leak error (#9298) fzyzcjy 2025-08-18 15:58:06 +08:00
• 968e181826 Fix triton_fused_moe unit test and benchmark (#9276) Yuan Luoandluoyuan.luo 2025-08-18 15:54:33 +08:00
• d08663eec1 [router] tokenizer factory, hf tokenizer, and stop sequence detector (#9293) Simo LinandChang Su 2025-08-17 22:38:38 -07:00
• 716e682721 [Fix] Add undefined update_tensor_inplace function (#6307) b8zhong 2025-08-17 23:11:00 -04:00
• 84b30d9e00 Set the default attention backend for GLM-4.5v to fa3 (#9245) zifeitong 2025-08-17 16:34:19 -07:00
• ff0cf51c8e [router] introducing tokenizer trait (#9287) Simo Lin 2025-08-17 16:30:01 -07:00
• a1c7f742f9 chore: bump sgl-kernel v0.3.6.post1 (#9286) Yineng Zhang 2025-08-17 16:26:17 -07:00
• ebbb75e917 [CPU] Fix TP padding issue on Phi-4 (#8289) blzheng 2025-08-18 07:25:26 +08:00
• b341b7dbce [router] introduce prefill response draining for http compliance (#9281) Simo Lin 2025-08-17 14:23:04 -07:00
• b498cd21d7 Tiny make fp4 moe method parameters more static (#8520) fzyzcjy 2025-08-18 04:26:02 +08:00
• 0fc54b971e [fix]: fix cutlass moe ut and and Opt H20 cutlass groupGemm performance (#9272) kousakawangandwanghanpei 2025-08-18 04:09:49 +08:00