Commit Graph

Select branches

Hide Pull Requests

bot/update-est-time-20260928

dsv41-pd

dsv41-pd-visioncp

main

Mono Color

• 9768c50d90 [router] restructure tool parser module folder (#9693) Simo Lin 2025-08-27 06:05:53 -07:00
• fd71b11b1d move is_sm90_supported/is_sm100_supported to python/sglang/srt/utils.py (#9679) Lianmin Zheng 2025-08-27 03:34:29 -07:00
• ae7428a8a7 fix mooncake store mla zero copy meta (#9678) huangtingwei 2025-08-27 15:43:16 +08:00
• a3aee7c377 fix: HiRadixCache: fix prefetch completion race (#9397) Pablo Iyu Guerrero 2025-08-27 09:43:01 +02:00
• 79e6a8a6ac support cuda 13.0 and trtllm kernel by Aug 25 2025 (#9495) Rain Jiang 2025-08-26 23:13:27 -07:00
• 8f7b1c31e8 Add A100 fused MoE kernel configs for Dpsk (#9677) ehuaa 2025-08-27 11:49:48 +08:00
• b9683be653 Support DeepSeek-V3.1 tool call (#9446) Xu WenqingandXinyuan Tong 2025-08-27 11:22:19 +08:00
• a85363c199 [docs] Instructions for bench_serving.py (#9071) 6 people 2025-08-27 09:30:57 +08:00
• b21fdd5373 feat: (chat-template matching) enhance multimodal model detection with config.json (#9597) Kevin Tuan 2025-08-27 08:55:40 +08:00
• c04c17edfa refactor(hicache): Introduce generic HiCacheStorageConfig for improved configuration management (#9555) hzh0425andTeng Ma 2025-08-27 08:55:20 +08:00
• 16a6d21b95 chore: enhance bench_serving for vlms with a new dataset of configurable image count and resolution (#9583) Mickandyhyang201 2025-08-27 08:42:54 +08:00
• a530b3ffdc [RL] fix register the same ops multiple times (#9564) Stefan He 2025-08-26 16:24:44 -07:00
• 603b3446dc Fix FA3 swa spec verify topk>1 (#9658) Ke Bao 2025-08-27 06:03:14 +08:00
• b6c14ec0b4 add response_format support for completion API (#9665) cicirori 2025-08-27 00:01:29 +02:00
• 43de1d7304 HiCache Storage fix host memory leak (#9648) Zhiqiang Xie 2025-08-26 10:49:40 -07:00
• 79ce3688bb BugFix(hicache): Fix host indices out of bound error (#9637) hzh0425andZhiqiang Xie 2025-08-27 01:42:23 +08:00
• 44ffe2cb72 Install py-spy by default for containers for easier debugging (#9649) fzyzcjy 2025-08-27 01:40:52 +08:00
• 1a0896e9c0 [doc] add kimik2 --tool-call-parser (#9647) Xiaotong Jiang 2025-08-26 10:39:40 -07:00
• 90313fb09a [router] add token bucket rate limiter (#9656) Chang Su 2025-08-26 10:36:26 -07:00
• 3578eb1e9b [router] address worker load tracking consistency (#9523) Simo Linandfzyzcjy 2025-08-26 06:40:51 -07:00
• 0936c766ed Fix kimi k2 function calling format (#9606) Xiaotong Jiang 2025-08-26 00:50:59 -07:00
• 0ef583b7de fix: allow user to specify function as role (#9635) GavinZhu-GMI 2025-08-26 15:47:20 +08:00
• f7881a27f9 Add reasoning_effort param in TiktokenTokenizer.apply_chat_template (#9630) 3 people 2025-08-26 15:44:20 +08:00
• fdff3167c5 [docs] Update README with additional highlights and resources for SGLang x AMD SF Meetup (#9640) Mingyi 2025-08-26 00:40:39 -07:00
• cbc0e4d779 Fix lint for router (#9636) Stefan He 2025-08-26 00:38:53 -07:00
• 4cd08dc592 model: Support nvidia/Llama-3_1-Nemotron-Ultra-253B-v1 (#9301) Netanel Haber 2025-08-26 10:33:40 +03:00
• f92b729d52 [new feat] ascend backend support fia fusion kernel (#8328) ZhengdQinandEven Zhou 2025-08-26 14:13:08 +08:00
• e2e378caba [router] add ut for mistral, llama, pythonic, and streaming tool parser (#9632) Simo LinandChang Su 2025-08-25 22:02:15 -07:00
• dc1decc6af [router] add llama tool parser (#9629) Simo LinandChang Su 2025-08-25 20:43:36 -07:00
• 03680f33be [router] add pythonic parser (#9628) Simo LinandChang Su 2025-08-25 20:40:06 -07:00
• d4c5e53401 [router] add qwen tool parser (#9623) Simo LinandChang Su 2025-08-25 20:32:05 -07:00
• 817c62a077 [router] add mistral tool parser (#9622) Simo LinandChang Su 2025-08-25 20:09:51 -07:00
• 0ff7241995 Improve bench_one_batch_server script (#9608) Liangsheng Yinandgemini-code-assist[bot] 2025-08-26 10:38:37 +08:00
• 80dc76e11a [Fix] HiCache Bugfix & Mooncake Error Handling Enhance (#8901) ykwdandZhiqiang Xie 2025-08-26 10:05:10 +08:00
• 9b08d975a0 [docs] Refactor, remove compiled results and add gpt-oss (#9613) Chayenneandzhaochenyang20 2025-08-25 15:27:06 -07:00
• a0a77d937b Fix Harmony reasoning parser for and auto-separation for gpt-oss models (#9190) 7 people 2025-08-26 00:26:26 +02:00
• 24a8cee66d Fix GLM45v launch server cuda torch compile bug (#9554) Binyao Jiang 2025-08-25 13:46:28 -07:00
• 3affa9dcc3 Fix GLM45 tool call multi-turn bug (#9500) Binyao Jiang 2025-08-25 13:46:13 -07:00
• ea0696b924 [Performance] Batch Send from Tokenizer Manager. (#9436) Sundara Raman Ramachandran 2025-08-25 10:43:54 -07:00
• 3aec3d4f8b [Doc] add LWS(LeaderWorkerSet) use case in sgl-router README (#9568) Bruce-x-1997andbruce.xu 2025-08-25 23:32:31 +08:00
• e3e97a120b chore: bump v0.5.1.post2 (#9592) Yineng Zhang 2025-08-25 03:45:09 -07:00
• 051068673c chore: update config (#9591) Yineng Zhang 2025-08-25 03:41:09 -07:00
• 9dcdf5da03 Tiny fix wrong comments (#9589) fzyzcjy 2025-08-25 18:08:10 +08:00
• f8b757bcac fix: resolve tuning fused moe issue (#9587) Yineng Zhang 2025-08-25 01:41:15 -07:00
• ebd9dbe71b fix: revert #8593 (#9581) Yineng Zhang 2025-08-25 01:29:06 -07:00
• 938e986e15 chore: upgrade flashinfer 0.2.14.post1 (#9578) Yineng Zhang 2025-08-25 00:12:17 -07:00
• 17d5eda887 bugfix for undefined logging functions in HarmonyBrowserTool & HarmonyPythonTool (#9229) Yuhao Zhou 2025-08-25 15:10:35 +08:00
• 71a7f1d86f Offload tensors by sharding on GPU (#9536) fzyzcjy 2025-08-25 15:02:49 +08:00
• 433266c125 Reintroduce memory usage fix (#9535) fzyzcjy 2025-08-25 15:02:31 +08:00
• fda4792620 Update CUTLASS 4.2 & Enable K-Major Scale Factor for SM90 FP8 Blockwise Group GEMM (#9559) Qi Yuhang 2025-08-25 14:24:43 +08:00
• a0b22f2f17 remove redundant rank0_log function. (#9560) miterandlinhuang 2025-08-25 14:17:55 +08:00
• b5c6529e17 [PD] Improve disaggregation metrics output: update the metrics to keep reflecting real stats (#7317) SCDESPERTATE 2025-08-25 14:16:43 +08:00
• ca4b86c564 fix: Update OpenAI client base URL in documentation (#9576) Xinyuan Tong 2025-08-25 14:06:57 +08:00
• dd6ec02965 Add target module validation for init adapters (#9429) Beichen Ma 2025-08-24 20:24:50 -07:00
• bf863e3bbf fix: use sgl-kernel 0.3.5 (#9565) Yineng Zhang 2025-08-24 15:46:47 -07:00
• 9e169ea8b5 [router] add right rustls dependency in sgl-router cargo.toml (#9498) Bruce-x-1997andbruce.xu 2025-08-25 00:03:15 +08:00
• e0ab167db0 chore: bump v0.5.1.post1 (#9558) Yineng Zhang 2025-08-24 01:14:17 -07:00
• c807cd7c75 chore: update configurer (#9557) Yineng Zhang 2025-08-24 01:05:00 -07:00
• 327f7b7c87 fix(grok): remove duplicate replicate_lm_head configuration (#9549) Vincent Zhong 2025-08-23 22:49:24 -04:00
• 80425e59bb [doc] deepseekv31 support (#9544) Xiaotong Jiang 2025-08-23 16:54:58 -07:00
• af9d4eb038 [readme] Include additional resources for the SGLang x AMD SF Meetup event (#9547) Mingyi 2025-08-23 16:51:16 -07:00
• fb107cfd75 feat: allow use local branch to build image (#9546) gongwei-130 2025-08-23 16:38:30 -07:00
• 97a38ee85b Release 0.5.1 (#9533) Lianmin Zheng 2025-08-23 07:09:26 -07:00
• 86d10d220f Update grok.py and tiktoken tokenizer (#9532) Lianmin Zheng 2025-08-23 05:40:18 -07:00
• 83871aa12d feat(hicache): Supports 3fs-hicache compatibility with dp-attention (#9372) hzh0425 2025-08-23 17:08:32 +08:00
• b1b3f0b38f Partially unify triton per token group quant kernels (#9485) fzyzcjy 2025-08-23 17:07:31 +08:00
• 34e5e11f0f Tiny make device_loading_context more static (#9478) fzyzcjy 2025-08-23 17:07:15 +08:00
• 2600fc0d47 Overlapped weight offload (#8034) fzyzcjy 2025-08-23 17:06:46 +08:00
• ccd3fb946e [fix] Fix mxfp4 triton MoE tp bug (#9473) hlu1 2025-08-23 01:48:40 -07:00
• c9dd70fbde tool-call(dsv3): Improve deepseek-v3 chat template and tool_choice = required (#9525) Chang Su 2025-08-23 01:46:56 -07:00
• 6b2b8bf0e1 fix: blackwell dsv3 fp8 issue temporary solution (#9530) Yineng Zhang 2025-08-23 01:33:21 -07:00
• 4edbe0d534 [benchmark] Add benchmark scripts for ceval and boolq (#8946) 5 people 2025-08-23 15:40:15 +08:00
• 0374304a2c Add enable_flashinfer_mxfp4_bf16_moe for higher precision and slower moe backend (#9004) fzyzcjy 2025-08-23 15:38:40 +08:00
• 127d4b0d5e Support GC Freezing to improve latency & throughput (#9241) 3 people 2025-08-22 22:43:09 -07:00
• 7e880286b5 Add support for extensions of interface and pre-registrations to NIXL HiCache (#9211) Moein KhazraeeandZhiqiang Xie 2025-08-22 20:06:13 -07:00
• 446c8e4cdb [router] ignore client error when record failure in pd_router (#9503) Bruce-x-1997andbruce.xu 2025-08-23 05:19:45 +08:00
• 5ef545e678 [router] Move all protocols to spec.rs file (#9519) Keyang Ru 2025-08-22 14:18:47 -07:00
• c4500233ff Add Qwen3-30B-A3B-Thinking-2507 support on AMD GPUs. (#9456) sogalin 2025-08-22 13:14:42 -07:00
• f445a1d9a3 [AMD] Fix Llama 4 FP8 accuracy issues on MI300X (#7699) Hubert Lu 2025-08-22 13:13:45 -07:00
• e5638573c1 [NVIDA] [1/N] Nvfp4 Masked Gemm: Add quant op for the flashinfer grouped gemm (#9200) Kaixi Hou 2025-08-22 12:19:45 -07:00
• f556ac8bd8 [router] add json tool parser (#9516) Simo Lin 2025-08-22 12:13:04 -07:00
• 110a65989b [MTP] Force greedy sampling on AMD (#9127) datdo-msft 2025-08-22 11:14:43 -07:00
• 49f9d02538 [router] tokenizer arch doc (#9513) Simo Lin 2025-08-22 09:52:33 -07:00
• 0f587e80d3 Use Tensor Core Decode when gqa group size >= 4 (#8624) Wenxuan Tanandgemini-code-assist[bot] 2025-08-22 10:25:15 -05:00
• 6078d5fcc0 [HiCacheStorage] backup optimization for MLA model (#8865) huangtingweiandZhiqiang Xie 2025-08-22 18:03:51 +08:00
• 70cf4abccc 3fs zerocopy (#9109) pansichengandZhiqiang Xie 2025-08-22 17:56:38 +08:00
• cebf45994b [bugfix] Make --enable-hierarchical-cache and --disable-radix-cache mutually exclusive (#9452) Xuchun Shang 2025-08-22 17:49:52 +08:00
• 9c0c1e30b2 Disable torch.compile for get_last_loc_large_page_size_large_top_k (#9507) Qiaolin Yuandispobock 2025-08-22 02:05:02 -07:00
• a1f011d09a minor: determine mm attn backend based on platforms (#9303) Mick 2025-08-22 16:08:41 +08:00
• 9ec314c6ac Support speculative decoding in the trtllm_mha attention backend (#9331) Qiaolin Yuandispobock 2025-08-21 23:53:35 -07:00
• fedfe91c1a [Docs] Add doc and quick demo for gpt-oss responses api & buildin tools (#9497) Xinyuan Tong 2025-08-22 14:51:52 +08:00
• 988accbc1e Update docker file for supporting PD-Disaggregation on MI300x (#9494) 3 people 2025-08-22 14:48:40 +08:00
• b6b2287e4b chore: bump sgl-kernel v0.3.6.post2 (#9475) Yineng Zhang 2025-08-21 23:02:08 -07:00
• 243e745d07 Add trtllm_mla and cutlass_mla for ragged fmha for chunked prefill (#9480) Elfie Guo 2025-08-21 23:01:36 -07:00
• 61a0e600df torch.compile() mrope (#9487) timmy-feng 2025-08-22 02:01:08 -04:00
• 0f8cee8cd3 [router] fix router load guard tracking for streaming (#9491) Simo Lin 2025-08-21 22:48:29 -07:00
• 816c4c8572 [router] add tool parser base structure and partial json parser (#9482) Chang Su 2025-08-21 22:08:56 -07:00
• 13ec8d427e [Docs]Update reasoning parser doc & fix outdated link (#9492) Xinyuan Tong 2025-08-22 13:08:28 +08:00
• 05bd789791 [docs]: fix reasoning context in docs (#9483) Chayenne 2025-08-21 20:04:12 -07:00
• 5fd311d33e [code clean] add H20 cutlass groupGemm default config (#9333) kousakawangandwanghanpei 2025-08-22 10:23:29 +08:00