Commit Graph

Select branches

Hide Pull Requests

bot/update-est-time-20260928

dsv41-pd

dsv41-pd-visioncp

main

Mono Color

• 160a06cab2 [Feature] Xiaomi MiMo-V2-Flash day0 support (#15207) +616 people 2025-12-19 11:40:07 +08:00
• a0985dd5e5 [diffusion] profiling: add bench_serving.py and VBench (#15410) Mick 2025-12-19 10:57:39 +08:00
• f6c9db4bc4 [DLLM] Fix dLLM regression (#15371) Zehuan Li 2025-12-19 10:55:46 +08:00
• e88e75a9df [Deepseek V3.2] Fix Deepseek MTP in V1 mode (#15429) b8zhong 2025-12-18 17:56:07 -08:00
• 4b4050e27e chore: update CI_PERMISSIONS (#15431) Yineng Zhang 2025-12-18 17:08:51 -08:00
• b2803ff207 [DLLM] Add CI for diffusion LLMs (#14723) Zehuan Li 2025-12-19 08:54:03 +08:00
• 9e0ef04e5b Support using different attention backend for draft decoding. (#14843) Charles Chen 2025-12-18 16:49:11 -08:00
• 216067c0cb feat(dsv32): better error handling for DeepSeek-v3.2 encoder (#14353) Jimmy 2025-12-19 08:34:27 +08:00
• e72b02db28 tiny fix lint on main (#15424) b8zhong 2025-12-18 15:55:02 -08:00
• 29e8f7f9e5 multimodal: precompute hash for MultimodalDataItem (#14354) Feng Su 2025-12-19 07:27:59 +08:00
• e0963a6cb1 [AMD] Clear pre-built AITER kernels and warmup to prevent segfaults and test timeouts (#15318) sunxxuns 2025-12-18 17:00:09 -05:00
• e0026f7c92 [Performance] optimize NSA backend metadata computation for multi-step speculative decoding (#14781) Johnsonms 2025-12-18 13:48:27 -08:00
• 9749d3e346 Update benchmarks to use HF token from environment. (#15421) Frankandgemini-code-assist[bot] 2025-12-18 13:47:27 -08:00
• 2b0ddf89f5 [Tiny]Add warning for deepgemm on Blackwell (#15352) Baizhou Zhang 2025-12-18 13:45:26 -08:00
• 17e81c750a [PP] Add dynamic chunking PP test (#15395) Shangming Cai 2025-12-19 01:09:40 +08:00
• 88a405cc10 Support EPLB balancedness prometheus metric without GPU->CPU synchronize (#15401) fzyzcjy 2025-12-18 22:24:23 +08:00
• 602fe3b296 Super tiny add moe_ep_rank to prometheus labels (#15407) fzyzcjy 2025-12-18 22:23:20 +08:00
• ad9616f13a Tiny extract ModelRunnerOutput (#15400) fzyzcjy 2025-12-18 22:18:45 +08:00
• c5f4e20f2f Support GPU execution time breakdown by forward mode metrics (#15396) fzyzcjy 2025-12-18 22:18:17 +08:00
• 2c196f95a7 Tiny fix unknown route in prometheus metrics (#15404) fzyzcjyandgemini-code-assist[bot] 2025-12-18 22:17:46 +08:00
• 9a7641d7bf [diffusion] profiling: include per-denoising-step timings in perf-dump-path (#15397) Xiaoyu Zhang 2025-12-18 18:43:33 +08:00
• 793c96c3d2 [perf]optimize w4afp8 kernel on deepseek-v3-0324 (#12921) Bruce-x-1997 2025-12-18 18:13:22 +08:00
• d1f0063262 Clean up __init__ function of the scheduler and event loop for PD (#15298) Lianmin Zheng 2025-12-18 01:35:14 -08:00
• 4792d1f452 [sgl-kernel][1/2] Fused qk_norm_rope for GLM4.6 (#15141) Kevin_Xiong 2025-12-18 17:07:04 +08:00
• fea2d5211d [bug fix][pp] fix inconsistent latency between tp (#15379) Xuchun ShangandShangming Cai 2025-12-18 16:59:56 +08:00
• 56d12b4aea Fix warp illegal instruction in kimi k2 thinking PCG (#15306) Xiaoyu Zhang 2025-12-18 16:58:23 +08:00
• 374ad4cce0 Fix gpt-oss yarn with truncate argument (#14270) Liangsheng Yin 2025-12-18 16:31:15 +08:00
• 8b0a68f1bf Monkey patch deepseek-ocr's v_head_dim (#15384) Liangsheng Yinandgemini-code-assist[bot] 2025-12-18 16:28:07 +08:00
• 70607e55e8 [model-gateway] Replace PolicyRegistry RwLock with DashMap for lock-free policy lookups (#15361) Simo Lin 2025-12-17 22:11:20 -10:00
• ee1ca51de8 [PP] Fix dynamic chunking strategy for PP (#15372) Shangming Cai 2025-12-18 14:24:55 +08:00
• ef7c29acd7 Fix issue: ENABLE_BELOW_SM90 cannot be enabled on aarch64 CPU (#12967) MarcoDWei 2025-12-18 13:26:42 +08:00
• 58c840db6b Split test_piecewise_cuda_graph.py to optimize CI resource usage (#15290) Alison Shao 2025-12-17 21:20:21 -08:00
• 3d42b7e7b0 unified management of environment variables for vlm cuda ipc transport (#14501) Yuhao Yang 2025-12-18 12:28:06 +08:00
• 9970ee34e8 Mistral Large 3 NVFP4 TRTLLM MoE support (#15049) elvischenv 2025-12-18 11:11:42 +08:00
• 9e7656be80 fix: adjust time for test_epd_disaggregation.py (#15354) Douglas Yang 2025-12-17 18:30:52 -08:00
• 9d4f066fb9 Add doc for qwen3 next (#15337) Yi Zhang 2025-12-18 09:53:07 +08:00
• 41683536d3 feat: DeepSeek-V3.2 Streaming tool call output (#15278) 3 people 2025-12-18 01:43:58 +00:00
• 891ee8221f Feature/trtllm mha workspace size configurable #15089 (#15131) Dongjie Zou 2025-12-17 20:07:23 -05:00
• 8fa3dc36c5 [VLM] Support cos sin cache for Qwen3-VL & GLM-4.1V (#15205) Yuan Luoandluoyuan.luo 2025-12-18 08:39:50 +08:00
• d20699a33c [Deepseek V3.2] Support Overlap Spec + NSA (#15307) b8zhongandBrayden Zhong 2025-12-17 13:35:39 -08:00
• 169a75dfb8 Add request-level timestamp for when prefill finishes (#14860) Scott Lee 2025-12-17 13:34:46 -08:00
• 4128d4f5cb [CI] Migrate LoRA tests to test/registered/lora/ (#15176) Alison Shao 2025-12-17 13:19:42 -08:00
• 011d8d8970 Reserve more memory for DeepSeekOCR model and adjust server start timeout for DeepGEMM to reduce flakiness (#15277) Kangyan-Zhou 2025-12-17 13:13:05 -08:00
• 5290cef97c Fix condition check for require_gathered_buffer (#15328) Cheng Wan 2025-12-17 12:42:47 -08:00
• 726fe3e72e fix: prevent points regex from matching checkpoints/endpoints (#15120) 许霖 2025-12-18 04:38:18 +08:00
• 5d087891c9 Fix dp run error with fp8-kv enable in high concurrency test (#15241) kkandwunhuang 2025-12-18 03:49:18 +08:00
• 8451e22758 [DeepSeek-V32]Update nightly performance benchmark (#15308) Baizhou Zhang 2025-12-17 11:25:31 -08:00
• 53e151945a [model-gateway] optimize worker registry and reduce lock contention in grpc client fetch (#15336) Simo Lin 2025-12-17 08:49:10 -10:00
• d747147a26 [model-gateway] reduce cpu overhead (#15316) Simo Lin 2025-12-17 07:42:01 -10:00
• 0c00220795 tiny unify environ usage (#15335) Liangsheng Yin 2025-12-17 23:31:43 +08:00
• eeb2b9b259 [PP] Minor code cleanup for Pipeline Parallelism (#15329) Shangming Cai 2025-12-17 23:31:25 +08:00
• c4aed389da Super tiny rename failure_count for consistency (#15186) fzyzcjy 2025-12-17 21:21:07 +08:00
• 9d04b570bc Tiny dump native stacktraces in watchdog (#15222) fzyzcjy 2025-12-17 21:20:41 +08:00
• 3e690cce53 Add realtime token counter metrics (#15198) fzyzcjy 2025-12-17 21:20:24 +08:00
• b12b40dea4 Add cuda_graph_forward_passes_total and num_retracted_reqs_total (#15189) fzyzcjy 2025-12-17 21:19:42 +08:00
• 6c4bf8a0be [diffusion] profiling: enhance trace export with gzip and integrity check (#15326) 4 people 2025-12-17 20:44:55 +08:00
• 533851fbcb [diffusion] ci: add flux2 tp2 test into ci to avoid breaking tensor parallel (#15237) Xiaoyu Zhang 2025-12-17 20:44:21 +08:00
• 0071fe9c40 [bug fix][pp] fix weight load for qwen2.5-vl (#15138) Xuchun Shang 2025-12-17 18:33:50 +08:00
• ffa7e03506 [Piecewise CUDA Graph] Support INT8 (#14918) b8zhong 2025-12-17 02:20:57 -08:00
• 712f44ee2b fix qwenvl compressed tensors quantization weight loader (#11914) LHXuuuandPeng Zhang 2025-12-17 18:01:43 +08:00
• 8c34e18140 Fix the accuracy issue when running mxfp4 dsv3 model and enable ep (#15304) kkandwunhuang 2025-12-17 17:53:22 +08:00
• e9abb52576 [bug fix][pp] fix qwen3 model load (#15223) Xuchun Shang 2025-12-17 17:26:22 +08:00
• 888594333e Fix gpu-fault when running mtp in eager mode (#15233) kk 2025-12-17 17:25:06 +08:00
• feb8e30b9d [Hotfix] Fix required enable_mamba_track argument for Flashinfer autotune path (#15314) elvischenv 2025-12-17 17:03:24 +08:00
• 45a959d3e9 [PP] Add pp support for Qwen3-VL (#12333) 3 people 2025-12-17 16:03:58 +08:00
• cdce516331 [diffusion] api: add sampling parameters and model info endpoint to OpenAI API (#15071) 4 people 2025-12-17 15:33:18 +08:00
• 79ab57bd7a Revert "direct register custom op for mm_fp4 (#13699)" (#15284) b8zhong 2025-12-16 23:09:44 -08:00
• 4b8901ac0f Update FP4 GEMM Benchmark (#14449) b8zhong 2025-12-16 23:04:56 -08:00
• 435d1c83c1 [Perf] Enable Flashinfer autotune by default (#14357) elvischenv 2025-12-17 15:01:39 +08:00
• 2bdbaef18e [DeepSeekV3.2] Add pure TP+MTP test (#15088) Ashton ChewandBaizhou Zhang 2025-12-16 21:48:12 -08:00
• 31d48d7f6f Add Ollama-compatible API endpoints + Smart Router (#14376) 3 people 2025-12-16 20:43:38 -08:00
• 0129c911e0 fix(function_call): fallback to decode when batch decode options differ (#15155) luqitao 2025-12-17 12:21:08 +08:00
• 03f9eb2564 [BugFix] fix gptq_marlin_gemm has no parameter called b_bias (#13571) 3 people 2025-12-17 10:03:55 +08:00
• 7ec678eb09 [Test] Update LoRA eviction policy tests to match current behavior (#15283) Alison Shao 2025-12-16 17:55:15 -08:00
• 9d64a7b24f Minor style fixes to the scheduler.py (#15218) Lianmin Zheng 2025-12-16 17:09:44 -08:00
• 46ad4b986d fix: moving decorator to header (#15297) Douglas Yang 2025-12-16 17:05:45 -08:00
• 71cb90378b [NPU] fix for NPU memory settings logic (#15258) Even Zhou 2025-12-17 09:04:22 +08:00
• c8c64876a7 [NVIDIA] Fixes for NVFP4 all-gather with spec decoding (#15280) Trevor Morris 2025-12-17 00:58:39 +00:00
• 0861dca81f Revert "[misc] Upgrade cutedsl to 4.3.1 (#14857)" (#15293) Yineng Zhang 2025-12-16 16:31:32 -08:00
• da58df6b3d fix: skipping TestEPDDisaggregationOneEncoder test (#15292) Douglas Yang 2025-12-16 16:31:20 -08:00
• 49237e26fb Fix test_pp_single_node.py estimated time from 800s to 500s (#15291) Alison Shao 2025-12-16 16:27:09 -08:00
• d92c1f8cbd Fix lora doc (#15282) Baizhou Zhang 2025-12-16 15:53:38 -08:00
• 930705863f [sgl-kernel] Update flashmla to include fp8 sparse_mla optimizations (#15242) hlu1 2025-12-16 15:12:19 -08:00
• ccc8f3b266 support non disturbing remote instance weight loader v2 (#14997) amysaq2023 2025-12-17 06:39:56 +08:00
• a4c762811a Remove incorrect BlockRemoved event emission during node splits (#14934) Neal Vaidya 2025-12-16 14:11:24 -08:00
• 28a19e494b Fix lint (#15281) Baizhou Zhang 2025-12-16 13:36:01 -08:00
• 0261c4aff7 [misc] Upgrade cutedsl to 4.3.1 (#14857) Baizhou Zhang 2025-12-16 12:11:56 -08:00
• 8ac350f335 [AMD] Support fused_rms_mxfp4_quant in the prefill stage for DeepSeek-R1-MXFP4 (#14975) jacky.cheng 2025-12-17 04:03:58 +08:00
• 99401e7b1a Fix accuracy issue when using a16w16 mla_decode_fwd (#14936) kk 2025-12-17 01:55:18 +08:00
• 6682475124 [CI] Improve flaky 4 GPU test success rate (#15234) Shangming Cai 2025-12-17 00:02:26 +08:00
• f95729b06f [diffusion] doc: update profiling.md (#15270) Mick 2025-12-16 23:48:32 +08:00
• 9f4ed93dd8 [diffusion] multi-platform: use current_platform.device_type to replace hard-coded cuda device (#15232) R0CKSTAR 2025-12-16 22:17:33 +08:00
• ecb401ed42 Enhance runtime memory check in CI (#15192) Liangsheng Yin 2025-12-16 21:40:38 +08:00
• b399e3ac4f Support piecewise cuda graph for fused marlin moe (#15100) Ke Bao 2025-12-16 20:05:32 +08:00
• e27635a02d [CPU] Add 4D input support for ROPE in sgl-kernel (#9337) blzheng 2025-12-16 17:27:39 +08:00
• 272c5fe43e Increase timeout for TestDeepseekV3MTP for potential DeepGEMM cold start (#15239) Kangyan-Zhou 2025-12-16 00:26:10 -08:00
• 3c8dc448b2 fix: removing latest-sglang=1 (#15220) Douglas Yang 2025-12-16 00:14:48 -08:00
• 5e96beb3e5 Adding tool calling and reasoning parser support for Intern-S1 (#14866) 5 people 2025-12-16 03:01:58 -05:00
• 9327482baa [bugfix][quark] Fixed an issue where per_token could not be properly recognized when the token count was 1. (#14415) haoyangli-amd 2025-12-16 14:54:31 +08:00
• 36fcf71fff [Qwen3-next] Add PD disaggregation support for mamba with extra_buffer (#15180) Shangming Caiandybyang 2025-12-16 14:36:00 +08:00