Commit Graph

Select branches

Hide Pull Requests

bot/update-est-time-20260928

dsv41-pd

dsv41-pd-visioncp

main

Mono Color

• 229f236da8 [router] migrate app context to builder pattern 2/n (#12089) Simo Lin 2025-10-24 15:04:32 -07:00
• 5983e5bd1b [router] migrate app context to builder pattern 1/n (#12086) Simo Lin 2025-10-24 13:35:22 -07:00
• 4b046a72d3 docs(server-arguments): add allowed options for each argument (#11560) Jonah Bernard 2025-10-24 14:49:20 -04:00
• 770d63123c [router] fix ut router config init to use build pattern (#12084) Simo Lin 2025-10-24 11:16:31 -07:00
• 14203432b4 fix(compile_utils, ep_moe): update environment variable and dtype check (#12034) ishandhanani 2025-10-24 11:00:12 -07:00
• d7f0d88fa2 [router] implement response api get input item function and refactor input/output store (#11924) Keyang Ru 2025-10-24 10:44:21 -07:00
• fc86b18b3e adjust dynamic vs static outputs comparison in test_lora_update.py (#11884) Glen Liu 2025-10-24 13:35:34 -04:00
• 0bfa394aff [Fix]: HiCache hasher failed when EAGLE mode enabled (#12025) Yuanhang Sun 2025-10-24 23:53:13 +08:00
• e04340bf48 Fix multi processing serializer bug (#11958) fzyzcjy 2025-10-24 22:53:45 +08:00
• 8470133852 [b200] fix piecewise cuda graph launch bug (#12067) Xiaoyu Zhang 2025-10-24 22:36:39 +08:00
• 93ef9a094d [Profiler] expand '~' for torch_profiler_output_dir (#11999) Muqi Li 2025-10-24 17:20:46 +08:00
• b04cd3d487 Add 'gguf' to project dependencies (#12046) Muqi Li 2025-10-24 17:16:19 +08:00
• 7ef5d8afd4 Revise POINTSV15Chat model (#12049) Yuan Luoandluoyuan.luo 2025-10-24 17:09:45 +08:00
• 71d41212e4 Fix dpsk-r1-fp4 launching crash (#12063) Qiaolin Yu 2025-10-24 02:04:50 -07:00
• b9fb74f3bc fix: bench_serving ITL calculation when using spec-decoding (#12064) Xinyuan Tong 2025-10-24 02:02:44 -07:00
• e15b63a182 [Fix] fix missing ipc_name of __getitem__ in some IO structs (#12053) ybyang 2025-10-24 16:59:14 +08:00
• 4060ed37cb Refactoring GLM-4.5 and GLM-4.5V related implementations (#11800) Yuxuan Zhang 2025-10-24 16:22:36 +08:00
• 2342605ef0 Tiny cleanup send_single (#12056) fzyzcjy 2025-10-24 14:53:42 +08:00
• dbf17a8313 [router] Add mTLS Support for Router-to-Worker Communication (#12019) Simo LinandChang Su 2025-10-23 23:14:09 -07:00
• 0f0c430e93 Install numactl in Dockerfile for GH200/GB200/GB300 (#11853) fzyzcjy 2025-10-24 12:39:10 +08:00
• 8e797a47f0 fix: the hardcode hf repo name comparison for deepseek-ocr (#12031) Rain Jiangandgemini-code-assist[bot] 2025-10-23 21:37:56 -07:00
• aa3003f116 Add gguf dependency for cpu/xpu (#12041) Zaili Wang 2025-10-24 12:13:17 +08:00
• 4793ec7d1a Opt MHA chunked prefix: merge prefix and extend kv cache to run mha once (#10953) Yongfei Xu 2025-10-24 11:58:10 +08:00
• 92009bd28e fix: fix MMMU loading issue (#11759) Zaili Wangandgemini-code-assist[bot] 2025-10-24 11:21:38 +08:00
• 4ef981e2b6 Revert "[Fix] Fix lint to pass CI" (#12042) Baizhou Zhang 2025-10-23 19:44:58 -07:00
• 69ed8b67a8 [Fix] Fix lint to pass CI (#12037) Baizhou Zhang 2025-10-23 19:39:38 -07:00
• 1801cd199f support more model in piecewise cuda graph (#11745) narutolhy 2025-10-23 19:31:39 -07:00
• ffc722a690 Revert "lang: support direct video inference" (#12038) Lianmin Zheng 2025-10-23 19:21:31 -07:00
• 49afb3d9d9 Fix(security): block unsafe pickle deserialization to mitigate CVE-2025-10164 (#11909) 3 people 2025-10-24 10:12:40 +08:00
• f80371ff8c Use flashinfer_trtllm moe runner backend to gain around 10% perf on b200 fp8 dpsk (#11816) b8zhong 2025-10-23 19:12:15 -07:00
• 62eff37ba1 Refactor Triton-kernel MoE runner integration (#11795) Jonah Bernard 2025-10-23 21:47:28 -04:00
• 47e12e082e Enable Llama 4 + TRTLLM MHA (#12003) b8zhong 2025-10-23 18:22:58 -07:00
• 823b442945 lang: support direct video inference (#9936) MickandLianmin Zheng 2025-10-24 09:12:39 +08:00
• 14a4d80e57 [8/n] decouple quantization impl from vllm dependency - gguf srt (#11964) Fan YinandPeng Zhang 2025-10-24 09:12:00 +08:00
• 1053e1be17 chore: bump SGLang version to 0.5.4 (#12027) sglang-botandsglang-bot 2025-10-24 09:01:40 +08:00
• 9a71500cfb Fixed aarch64 flash-mla (#12009) nvjullin 2025-10-24 08:47:04 +08:00
• 6d6e24bcc4 [router] Add builder pattern for RouterConfig with zero duplication (#12030) Simo Lin 2025-10-23 16:46:10 -07:00
• 2c057fbfa8 Update Github action title for kernel build (#12029) Kangyan-Zhou 2025-10-23 13:39:40 -07:00
• dbd9435dc1 Fix mamba radix cache eviction logic in alloc_req_slots (#11616) Roger Young 2025-10-24 04:07:43 +08:00
• 8ae9d4bb41 Revert "[ROCm] Remove vLLM rope dependency & use AITER impl" (#12028) b8zhong 2025-10-23 12:42:59 -07:00
• 1c304aa9bc Log iteration # for prefill and decode (#9366) Nicolas Castet 2025-10-23 14:28:03 -05:00
• 770529a731 model: support deepseek-ocr (#11891) 5 people 2025-10-24 03:15:17 +08:00
• 39c237f02c Add AWQ quantization support for NPU. (#10158) 4 people 2025-10-24 03:08:05 +08:00
• 28b8a4064d [router][CI] Clean up imports and prints statements in sgl-router/py_test (#12024) Chang Su 2025-10-23 11:56:57 -07:00
• 8bd26dd4e6 ci: fix night-ci with push retry mechanism (#11765) Mick 2025-10-24 02:31:05 +08:00
• ab07cd3e5a [Auto Sync] Update test_deterministic_utils.py (20251023) (#12022) Lianmin ZhengandStefan He 2025-10-23 11:20:45 -07:00
• a98496834b Feature/nano v2 offline modelopt fp8 and nvfp4 (#12018) Netanel Haber 2025-10-23 21:16:46 +03:00
• a4b637d87a [router] change ci names and update log level in ci (#12021) Simo Lin 2025-10-23 10:36:19 -07:00
• 96a5e4dd79 [Feature] Support loading weights from ckpt engine worker (#11755) 5 people 2025-10-24 00:23:30 +08:00
• b0b4f71679 [Fix] memory leak by overlap + retract (#11981) cctryandLiangsheng Yin 2025-10-23 07:59:23 -07:00
• 6c18addb6f Revert "Support nvidia/NVIDIA-Nemotron-Nano-9B-v2-FP8/NVFP4" (#12015) Liangsheng Yin 2025-10-23 21:27:58 +08:00
• 32852fe9e9 Move memory runtime checker to mixin class (#12014) Liangsheng Yin 2025-10-23 20:53:26 +08:00
• 53c2934dce [Router] Consolidate ConnectionMode enum to core module (#11937) Arthur Cheng 2025-10-23 05:15:49 -07:00
• e321c97113 [router] Add comprehensive E2E tests for Response API (#11988) Keyang Ru 2025-10-23 05:13:51 -07:00
• d6fee73d1f Support nvidia/NVIDIA-Nemotron-Nano-9B-v2-FP8/NVFP4 (#11866) Netanel Haber 2025-10-23 12:29:02 +03:00
• 36a4cad7b0 Support overlap-spec-v2 with trtllm_mla attention backend (#11821) Qiaolin Yu 2025-10-23 01:55:35 -07:00
• 65d376b491 aiter update to v0.1.6.post1 (#12004) HAI 2025-10-22 23:53:05 -07:00
• c23eda8589 Fix incorrect KV indices creation when page_size=32 in TRTLLM MLA backend (#11985) yinghui 2025-10-22 22:44:45 -07:00
• 138ff23187 Allow to disable batch decoding. (#11944) Jue WANG 2025-10-22 21:57:12 -07:00
• 13fb8b5489 [CPU] Optimize FP16 decode_attention_cpu (#10652) blzheng 2025-10-23 12:39:51 +08:00
• 81fd2b0ee0 fix(deepep): resolve benchmark failure on 4×IB-card setup by aligning tuning config with DeepEP commit bdd119f8 (#11965) Zhengyi Lai 2025-10-23 12:20:54 +08:00
• 007b849b0e [CPU] misc updates (#11906) Zaili Wang 2025-10-23 12:10:05 +08:00
• 8612811d85 Bump grace blackwell DeepEP version (#11990) fzyzcjy 2025-10-23 12:08:12 +08:00
• e7aa4664b3 [NVIDIA] Build CUDA 13 (#11299) 3 people 2025-10-22 20:03:12 -07:00
• 4d4feccbb2 [ROCm] Remove vLLM rope dependency & use AITER impl (#11322) b8zhong 2025-10-22 19:17:34 -07:00
• 99c92ff24b [AMD] Support a new flag to disable quant on parallelLinear layer if required (#11811) jacky.cheng 2025-10-23 10:16:15 +08:00
• 6ade6a02d4 [grpc] Support gRPC standard health check (#11955) Chang Su 2025-10-22 16:59:09 -07:00
• 983ef22cf3 [Doc] Update deterministic inference flag in server_arguments.md (#11978) Baizhou Zhangandgemini-code-assist[bot] 2025-10-22 16:12:15 -05:00
• 164302c7df Implement BGE-M3 Sparse Embeddings in SGLang (#10869) 3 people 2025-10-22 22:46:16 +02:00
• 5dccf69713 [router] create worker removal step and clean up worker manager (#11921) Simo Lin 2025-10-22 13:26:06 -07:00
• eec9e471ca [NVIDIA] Update to leverage flashinfer trtllm FP4 MOE throughput kernel (#11563) jiahanc 2025-10-22 13:11:16 -07:00
• 6d535b719f Revert "Recapture cuda graph after model weight update to resolve IMA error " (#11980) Lianmin Zheng 2025-10-22 11:50:26 -07:00
• fdcb1d13c5 [BUG] AttributeError: 'DeepEPMoE' object has no attribute 'use_w4a… (#11977) yuho 2025-10-23 02:29:55 +08:00
• d7e834d6ba [6/n]decouple quantization implementation from vLLM dependency (#10750) Hongbo Xu 2025-10-23 02:07:55 +08:00
• 200a3c0bb1 [Documentation] add doc for deterministic inference (#11956) Minglei Zhu 2025-10-22 10:36:15 -07:00
• 77258ce039 [router] Support multiple worker URLs for OpenAI router (#11723) Keyang Ru 2025-10-22 09:27:58 -07:00
• 1d097aac87 [Fix] Remove unused import from triton_kernels_moe.py (#11967) Fan YinandShangming Cai 2025-10-22 21:02:57 +08:00
• 7fceeef599 Fix flaky hicache test with mooncake backend (#11953) Shangming Cai 2025-10-22 21:00:47 +08:00
• 88568c01eb [model] Support POINTSV15Chat (#9651) 4 people 2025-10-22 16:58:17 +08:00
• 904655c5fd [2/N] Added the core structure of elastic EP and the eplb algorithm with faulty rank (#10606) 3 people 2025-10-22 16:13:31 +08:00
• e028af6998 Fix mooncake dispatcher (#11908) Xun Sun 2025-10-22 16:11:49 +08:00
• 80b2b3207a Enable native ModelOpt quantization support (3/3) (#10154) Zhiyu 2025-10-21 21:44:29 -07:00
• 4b65ed42cc [NVIDIA] upstream FA4 and fix cccl path (#11929) Johnny 2025-10-21 21:18:25 -07:00
• 23afdfd1c2 [sgl-kernel] support flashmla libtorch (#11717) Fan Yin 2025-10-22 12:17:50 +08:00
• 9d61205dac [lint] improve ruff check (#11922) Liangsheng YinandXiaoyu Zhang 2025-10-22 11:32:50 +08:00
• 590bc4b7a7 [router][grpc] Fix background tasks stored with wrong id (#11945) Chang Su 2025-10-21 18:38:51 -07:00
• 63cfe1b032 [router] Add gRPC E2E test suite (#11790) Keyang Ru 2025-10-21 17:51:21 -07:00
• 70f6309cd4 [router][grpc] Support v1/responses API (#11926) Chang Su 2025-10-21 17:41:48 -07:00
• 704160017d fix: resolve flashinfer 0.4.1 import (#11940) Yineng Zhang 2025-10-21 17:19:57 -07:00
• 87a92e459a Fix openai input_text type compatibility (#11935) Keyang Ru 2025-10-21 16:10:35 -07:00
• c461e7714d [Auto Sync] Update forward_batch_info.py (20251021) (#11934) 3 people 2025-10-21 15:52:15 -07:00
• fde2decf8b [BugFix][Qwen3-VL]: add metadata for video in qwen3-vl (#11377) Zheng Wengang 2025-10-22 06:36:01 +08:00
• 9792b9d7e3 chore: upgrade flashinfer 0.4.1 (#11933) Yineng Zhang 2025-10-21 14:46:31 -07:00
• ef4a8097b8 Rename flashmla kernel options of nsa backend for better readability (#11876) Baizhou Zhang 2025-10-21 15:14:16 -05:00
• ebff4ee648 Update sgl-kernel and remove fast hadamard depedency (#11844) Baizhou Zhang 2025-10-21 15:13:54 -05:00
• 2b1da821b5 [NVIDIA] Add new SMs support for Spark & Thor (#11287) Serge Panev 2025-10-21 11:02:24 -07:00
• 97710ccd1a Fix flush cache API for spec v2 (#11918) Liangsheng Yin 2025-10-21 23:01:16 +08:00
• f3cd5d2510 [CI] Fix b200 flashinfer installation (#11915) Shangming Cai 2025-10-21 22:28:50 +08:00
• c61b0b294c [quantization][MoE] fix the check for tp_size / moe_ep_size / moe_intermediate_size / weight_block_size_n (#11702) Kai-Hsun Chen 2025-10-21 06:25:28 -07:00
• e8640ee9be [smol] [perf] Inverse perm improvement (#11482) Vincent Zhong 2025-10-21 07:18:10 -04:00