Commit Graph

Select branches

Hide Pull Requests

bot/update-est-time-20260928

dsv41-pd

dsv41-pd-visioncp

main

Mono Color

• 55dfb539cf [Auto Sync] Update scheduler_metrics_mixin.py, collector.py (20251104) (#12647) 3 people 2025-11-04 13:56:14 -08:00
• 42889acbd0 [hotfix] Fix deepep w4a8 bug (#12642) Baizhou Zhang 2025-11-04 13:55:59 -08:00
• 211f4070e5 fix: Lazy import mooncake-ep to fix extra gpu contexts being created (#12641) Trevor Morris 2025-11-04 12:28:36 -08:00
• befa41a152 Fix output_ids inconsistency (#12628) Liangsheng Yin 2025-11-05 01:43:08 +08:00
• 30b26ee9d0 Add io struct naming check back (#12634) Liangsheng Yin 2025-11-05 01:15:01 +08:00
• aa797d013d [Test] Merge all constrained decoding tests. (#12633) Liangsheng Yin 2025-11-05 00:43:06 +08:00
• 7cee07a067 Fix skip layer in get_quant_method (#12632) Ke Bao 2025-11-04 23:27:46 +08:00
• bb517fe393 [HotFix] Disable torch dynamo for mrope_triton kernel (#12593) Yuan Luoandluoyuan.luo 2025-11-04 23:26:56 +08:00
• 0e82fd3df4 [router][grpc] Fix model validation, tool call check, streaming logic and misc in responses (#12616) Chang Su 2025-11-04 02:21:58 -08:00
• b7d7041190 Add sanity checks when a test file is not added to CI (reland) (#12594) fzyzcjy 2025-11-04 18:04:26 +08:00
• ff0b64e1e6 Ensure GPU work is finished when release memory occupation call is finished (#12592) fzyzcjy 2025-11-04 18:01:27 +08:00
• d84790db39 Support aggregating engine metrics in sgl-router (#11456) fzyzcjy 2025-11-04 17:59:50 +08:00
• 0678beaaee [sepc-v2] Fix imcompatibility with constrained decoding (#12615) Liangsheng Yin 2025-11-04 17:27:31 +08:00
• c2d4716da0 chore: bump mooncake version to 0.3.7.post2 (#12599) Shangming Cai 2025-11-04 17:09:31 +08:00
• c14cc47e39 [Deterministic] Optimize bmm_batch_invariant op (#12522) Minglei Zhu 2025-11-04 00:33:31 -08:00
• dbcf85b7f0 Add --speculative-moe-runner-backend server arg (#10183) Trevor Morris 2025-11-04 00:20:56 -08:00
• 83804bc626 [router][grpc] Restructure modules and code clean up (#12598) Chang Su 2025-11-03 23:59:27 -08:00
• d5fa019c36 feat: limit peak memory usage when computing logprobs (#6318) Zhao Chenand赵晨阳 2025-11-04 15:53:20 +08:00
• fef3a6b63b Restore torch defaults between sgl-kernel tests (#11131) Ben Barsdell 2025-11-04 18:51:23 +11:00
• 173e0f704f Enable memory saver for hybrid model (#11974) Junrong Lin 2025-11-04 14:55:26 +08:00
• f600866a44 Improve the metrics for PD (#12580) 3 people 2025-11-03 22:10:57 -08:00
• 93be7e863e fix: respect --ignore-eos in PD case for benchmarking (#12597) ishandhanani 2025-11-03 21:44:14 -08:00
• 60b0754cc9 Tiny fix ExpertDistributionReq error (#11760) fzyzcjy 2025-11-04 13:39:25 +08:00
• 0b24af4d79 test: support return logprobs in bench_offline_throughput test (#12462) Zhao Chenandgemini-code-assist[bot] 2025-11-04 13:38:48 +08:00
• a209fb05c1 [Qwen3 VL] Add LoRA support for Qwen 3 VL (#12165) Jonah Bernard 2025-11-03 23:32:54 -05:00
• 48d6bea1ea [GDN/SWA] mamba and swa radix cache edge case fix (#12111) Hanming Luandyizhang2077 2025-11-03 19:03:37 -08:00
• 1689c0e35f [Doc] fix miss index for production request trace (#12547) Teng Ma 2025-11-04 09:57:09 +08:00
• 193fbb0bce Super tiny add UT for copy_to_gpu_no_ce (#12270) fzyzcjy 2025-11-04 09:40:51 +08:00
• e607850fcf Enable mixed type LayerNorm kernel for NSA indexer (#12044) akhilg-nv 2025-11-03 16:50:41 -08:00
• 15efbcb4e7 [chore] Fix update_kernel_whl_index script for multiple cuda version (#12519) Baizhou Zhang 2025-11-03 16:34:14 -08:00
• 243c064df2 Remove the dependency of nccl.h in symmetric memory (#12571) Lianmin Zheng 2025-11-03 16:11:00 -08:00
• 0b41a293fa [router][grpc] Consolidate error messages build in error.rs (#12301) Chang Su 2025-11-03 14:41:45 -08:00
• d31d48b341 update usage of trtllm_fp8_per_tensor_scale_moe (#12569) b8zhong 2025-11-03 14:25:32 -08:00
• 8834260739 Super tiny dump server info such as args in bench for post analysis (#12550) fzyzcjy 2025-11-04 06:24:08 +08:00
• fd7a72d62d Super tiny allow profile activities in bench_serving (#12549) fzyzcjy 2025-11-04 06:23:18 +08:00
• 21a8fa16ea tiny optimize for bench serving (#12553) Yi Zhang 2025-11-04 06:13:18 +08:00
• 7a21d8b276 Reduce the overhead of nccl symmetric memory (#12524) Lianmin ZhengandNicolas Castet 2025-11-03 11:56:27 -08:00
• d36639eec7 [ROCm] Update Mooncake to v0.3.7.post1 and add -DUSE_HIP=ON to rocm.Dockerfile (#12560) R0CKSTAR 2025-11-04 03:33:30 +08:00
• 6ef23b9833 [Test] Add parameters to SRTRunner (#12227) Jonah Bernard 2025-11-03 14:20:56 -05:00
• 385599cb04 Fix error when calling quantization (#12548) fzyzcjy 2025-11-04 02:17:43 +08:00
• 952fbe47cb fix: fix the bug which leads qwen2_5_vl to crash with mixed_chunk (#11330) 3 people 2025-11-03 18:26:03 +01:00
• edb2569356 [hot-fix] Fix broken CI (#12564) Liangsheng Yin 2025-11-04 00:03:25 +08:00
• 3529c061bb [spec v2] Fix output repetition by speculative sampling error (#12561) Liangsheng Yin 2025-11-03 23:00:17 +08:00
• ffb32a8548 Conditionally recapture cuda graph after model weight update from disk (#12060) harrisonlimh 2025-11-03 05:51:27 -08:00
• 14d8064803 fix: Fix KTransformers hybrid inference with int8 quantization and format (#12536) Atream 2025-11-03 20:59:39 +08:00
• ab8b83f71d chore: upgrade mooncake 0.3.7.post1 (#12541) Shangming Cai 2025-11-03 16:34:32 +08:00
• de0b10cf5c fix: move dummy format loader check before quantization checks (#12532) yinghui 2025-11-02 23:41:30 -08:00
• 6e29446e45 [hotfix] Remove flashinfer-jit-cache from pyproject (#12530) Baizhou Zhang 2025-11-02 22:11:05 -08:00
• 0c3543d7d5 chore: upgrade flashinfer 0.5.0 (#12523) Yineng ZhangandBaizhou Zhang 2025-11-02 20:54:12 -08:00
• 6a3b9fd00f Update setup_github_runner.md Kangyan-Zhou 2025-11-02 20:44:09 -08:00
• 65f1d065c5 [Bug] Fix Intern-S1 model accuracy and support /generate interface with input_ids (#12367) Haian Huang(深度眸) 2025-11-03 12:22:33 +08:00
• 9434a0e50f [Refact] Remove hardcoded KV cache dimension in MLATokenToKVPool (#12502) Johnsonms 2025-11-02 19:49:53 -08:00
• 20315697f4 move all get_stream in sgl_kernel to c++ to reduce the launch overhead (#12521) Lianmin Zheng 2025-11-02 13:15:05 -08:00
• c9db79117f Super tiny fix naming in bench serving scripts (#12515) fzyzcjy 2025-11-03 04:43:10 +08:00
• 15ed27d7d4 [Fix] concat_mla_absorb_q_kernel fails for long inputs (#12453) bingps 2025-11-03 03:52:06 +08:00
• 66fb9b1307 [ServerArgs] allow --mamba-ssm-dtype extend (#12481) Hanming Lu 2025-11-02 11:50:04 -08:00
• 819fc59123 Add prefix for torch symm mem (#12506) Yuan Luoandluoyuan.luo 2025-11-03 03:23:05 +08:00
• 7efd8b3d1f [FEAT] Shared mem pool based cuda ipc for multi-modal data transport (#11917) 3 people 2025-11-02 16:46:37 +08:00
• a920b9dace chore: bump sgl-kernel version to 0.3.16.post5 (#12511) sglang-bot 2025-11-02 14:08:11 +08:00
• 76196b3cbf feat: Add FP4 (E2M1) KV Cache Support with Quantization Utilities for MLA (#10078) Ho-Ren (Jack) ChuangandYichen Wang 2025-11-01 22:24:58 -07:00
• 95191ebdca Migrate weak_ref_tensor to sgl-kernel (#12505) Xiaoyu Zhang 2025-11-02 10:55:39 +08:00
• 9a512cf95b [CI] Move some Lora/Deterministic CI tests to nightly (#12507) Baizhou Zhang 2025-11-01 19:54:22 -07:00
• 3451fc3280 [Feature] Qwen3-Next & FLA: Support MTP topk>1; Up to 6% faster (#11133) Binyao JiangandStefan He 2025-11-01 19:47:56 -07:00
• c550ab9125 [Ascend] Add Ascend NPU support for sglang.check_env & rework proposal (#11052) Zhihao Lyuandronnie_zheng 2025-11-02 10:26:45 +08:00
• 086f0b79fc [CI][NPU] remove pypi mirror site that hangs ci dependency installation (#12499) Even Zhou 2025-11-02 10:25:53 +08:00
• 0afd68321b Update Mooncake EP's a2a interface (#12391) Xun Sun 2025-11-02 09:48:47 +08:00
• 6f858930c8 [Bug] test_flashattn_mla_backend errors in Hopper #12487 (#12488) Johnsonms 2025-11-01 18:28:06 -07:00
• 229256c505 [Deterministic] add deepseek v3 deterministic inference CI test (#12412) Minglei Zhu 2025-11-01 18:10:32 -07:00
• 6b634493c3 [HICache / PD]: Support offloading incremental KV cache in decode side. (#11966) hzh0425 2025-11-02 05:59:37 +08:00
• 756ad9ceb1 Reduce docker image size. mount cache when use pip/cargo build (#12238) ybyang 2025-11-02 05:58:04 +08:00
• d2a8f71c2f [feat] Add SGLANG_TOOL_STRICT_LEVEL for tool-call behavior control (#12423) Xinyuan Tong 2025-11-01 13:15:02 -07:00
• 2b7bf11bd2 [Hotfix] Remove extra comment in sgl-kernel README (#12500) Baizhou Zhang 2025-11-01 12:22:45 -07:00
• 566ade0388 [CI] Build aarch64 kernels for sgl-kernel test (#12480) Baizhou Zhang 2025-11-01 11:55:42 -07:00
• 69193f7122 Filter tokenizer warning for kimi models (#12485) Ke Bao 2025-11-01 16:27:31 +08:00
• d5b6e50fe8 perf: trtllm mla performance minor improvements (#12435) yinghui 2025-10-31 22:48:02 -07:00
• 9632e48f5d [hot fix] Remove from python.sglang.xxx (#12483) Liangsheng Yin 2025-11-01 11:00:05 +08:00
• 59cce5941a Use sgl fp4 quant kernel by default (#12482) Qiaolin Yu 2025-10-31 19:51:28 -07:00
• 795e98f8a6 Forward unknown tool calls instead of dropping (#12226) Surya-Gunukula 2025-10-31 19:10:35 -07:00
• 358ae3563d Tiny fix eos handling for PD disaggregation (#12334) Shangming Cai 2025-11-01 08:57:10 +08:00
• 41c10e67fc chore: bump SGLang version to 0.5.4.post2 (#12439) sglang-bot 2025-11-01 08:38:50 +08:00
• 0bfe1d145c fa3 & trtllm_mha spec overlap (#11874) Xinyuan Tong 2025-10-31 17:38:13 -07:00
• 5f98b7fe61 [CI] Fix kernel installation on aarch runners (#12475) Baizhou Zhang 2025-10-31 14:25:27 -07:00
• a4bf5c6ad2 Support Kimi Linear (#12469) Ke Baoandyizhang2077 2025-11-01 05:03:35 +08:00
• 30ad107028 Try to allow NCCL cumem for multi node nvlink case (#11987) fzyzcjy 2025-11-01 03:48:25 +08:00
• a80bcb5a68 Add env var to disable FA4 warmup (#12430) yinghui 2025-10-31 12:25:00 -07:00
• f7f9e41b36 Fix run benchmark (#12473) Ke Bao 2025-11-01 02:39:48 +08:00
• 263eab9f5d fix: dummy health check server not accessible on non-zero rank nodes (#12297) ishandhanani 2025-10-31 11:34:57 -07:00
• 25257d8e00 Tiny assert no running requests when releasing memory to avoid IMA (#12341) fzyzcjy 2025-11-01 01:28:53 +08:00
• cf0c24150a add served model name in bench serving (#12428) daniel, chen 2025-11-01 01:28:11 +08:00
• 5538e05cb1 fix default env var for mooncake store (#12429) huangtingwei 2025-11-01 01:25:33 +08:00
• c30ebb9300 [VLM] Optimize async mm data process mechanism (#12066) Yuan Luoandluoyuan.luo 2025-11-01 01:24:53 +08:00
• 41efcaeb45 [Feature] PD-Multiplexing Context and Scheduler, lazy import spatial. (#12275) ykcombat 2025-11-01 00:40:01 +08:00
• 70562969b9 [Bug] OOM (Out-of-Memory) errors for extreme testing scenarios (min_tokens=2) (#11757) 0xNullPath 2025-11-01 00:28:41 +08:00
• b57dc169eb [Test] Add Functional Tests for Penalty Parameters (#11931) Neelabh Sinha 2025-10-31 09:10:17 -07:00
• 0095e01874 Fix lint in deepseek-ocr (#12470) Ke Bao 2025-11-01 00:08:19 +08:00
• 684864814b Feat: deepseek-ocr logits processor (#12415) Xinyuan Tongandxinyuant 2025-10-31 08:35:22 -07:00
• 410225b719 [Bug fix] Fix severe memory waste issue with torch.empty pin_memory (#12266) sjtu_shenhai 2025-10-31 21:30:37 +08:00
• 2c9aebea70 Simplify watchdog (#12463) Liangsheng Yin 2025-10-31 21:17:38 +08:00
• bc741073a3 fix:watchdog thread exception (#12328) Kindyaa 2025-10-31 20:54:50 +08:00
• 2f6af1a3de Enable bailing_moe to support TP=16 (#12369) Yuhong Guo 2025-10-31 19:32:49 +08:00