Commit Graph

Select branches

Hide Pull Requests

bot/update-est-time-20260928

dsv41-pd

dsv41-pd-visioncp

main

Mono Color

• 50b6842b4b fix: Add default value for backend in sample_mmmu_requests (#12256) Zaili Wang 2025-10-31 19:31:40 +08:00
• 2d5605e89b Fix ci install to allow prerelease (#12449) Lianmin Zheng 2025-10-31 02:22:15 -07:00
• 300b4c212e [CI] fix possible port conflicts. (#12452) Liangsheng YinandLianmin Zheng 2025-10-31 17:20:20 +08:00
• c0652d907b Clean up sgl kernel (#12413) Lianmin ZhengandByron Hsu 2025-10-31 01:13:34 -07:00
• 2e48584b62 [bugfix] set is_prefill_only=false when mixed_chunk (#10889) Bruce-x-1997 2025-10-31 15:12:21 +08:00
• 57cc5385c0 [CI] Add more bins for 1-gpu CI test (#12422) Baizhou Zhang 2025-10-31 00:05:01 -07:00
• 5cc0d25a0d Add trait for BasePrefixCache (#12436) Liangsheng Yin 2025-10-31 14:03:27 +08:00
• a076ec1a7a Revert "fix llama4 kv cache layout" (#12437) b8zhong 2025-10-30 22:33:37 -07:00
• 72b5f3d0bb [ci] install released version router (#12410) Keyang Ru 2025-10-30 21:59:08 -07:00
• 2f766f381d [Bugfix]: distinguish processors for deepseek_vl2 and deepseek_ocr to p… (#12384) bppps 2025-10-31 12:51:30 +08:00
• 069e490bcb feat: support trtllm_mha FP8 query attention kernel (#12307) elvischenv 2025-10-31 12:39:35 +08:00
• ab95d35fcb feat: Add Non-intrusive Tensor Dumping for Model Inference (#10566) Yuhong Guo 2025-10-31 12:04:48 +08:00
• 34c286b811 Fix the shared expert & routed expert overlap in Llama 4 (#12405) b8zhongandBrayden Zhong 2025-10-30 20:14:49 -07:00
• 9416ee60dd Reserved abortion API when retracting (#12425) Liangsheng Yin 2025-10-31 10:56:05 +08:00
• d4a09ec9dc [CI] fix tests' time estimation (#12401) Liangsheng Yin 2025-10-31 10:20:35 +08:00
• 662725b936 [DeepSeekV32] Bug fix to ensure page_table and result in same type (#12300) Johnsonms 2025-10-30 18:56:51 -07:00
• 82cfcd3bb8 [Refactor] tuning_fused_moe for MLLM and small refactor (#11224) Xinyuan TongandCursor Agent 2025-10-30 17:54:14 -07:00
• 6c1a3f0cb2 enable cudaProfilerApi for one batch benchmarking (#11116) lpc0220 2025-10-30 15:43:59 -07:00
• 6237754842 [router] 0.2.2 release (#12399) Simo Lin 2025-10-30 14:40:13 -07:00
• 96ac24c09e [router] web_search_preview tool basic implementation (#12290) Keyang Ru 2025-10-30 14:37:28 -07:00
• c0d02cf4d1 [NVIDIA] Add CI workloads for GB200 (#12242) Kaixi Hou 2025-10-30 14:32:03 -07:00
• 7d121448b2 [Bug fix][PD Dissaggregation] fix prefill hanging issue with PP and DP Attention, (#12368) popsiclexu 2025-10-31 05:22:47 +08:00
• 6a63a9852e minor code sync (#12403) Lianmin Zheng 2025-10-30 12:49:31 -07:00
• 4d2f17bd0b [router] Function call support for openai router Responses API (#12386) Keyang Ru 2025-10-30 12:45:10 -07:00
• 7cd716f776 Update news section in README.md (#12409) Lianmin Zheng 2025-10-30 12:41:21 -07:00
• b7fdde4bb4 [ci] Fix ci_install_deepep (#12375) Baizhou Zhang 2025-10-30 11:39:14 -07:00
• 69bf801173 [router] Fix safety_identifier missing (#12404) Keyang Ru 2025-10-30 10:40:50 -07:00
• d8fcbaa38d [CI Monitor] Fix ci_monitor perf analyzer bug (#12281) Xiaoyu Zhang 2025-10-31 00:47:12 +08:00
• 2cf3d0f89f [router] use safety_identifier replace user on chat history storage (#12185) rongfu.leng 2025-10-30 23:47:22 +08:00
• 1ed1abfd45 feat: add EP support in tuning (#12012) Chen1022 2025-10-30 22:58:50 +08:00
• ecb9fa14e6 [docker] clean up main dockerfile for router and dev configurations (#12364) Chang Suandybyang 2025-10-30 10:46:09 -04:00
• 700daa3463 [router] harmony responses api streaming support (#12395) Simo Lin 2025-10-30 07:30:48 -07:00
• 39cee0fed2 [sgl-kernel] upd deepgemm hash to rebased commit (#11960) Fan Yin 2025-10-30 22:19:00 +08:00
• 04e5b6faa7 Revert "Triton fused_moe_kernel support ep moe tuning" (#12377) Xiaoyu Zhang 2025-10-30 22:12:06 +08:00
• ce6b17c0f9 [Feature] Support DeepSeek MTP on NPU (#11897) Even Zhouandliupeng374 2025-10-30 21:56:56 +08:00
• cafebef154 [NPU] bugfix for Qwen3-Next and performance update (#11969) Even Zhou 2025-10-30 21:52:16 +08:00
• 73dfd2dfb1 [Test] Enhance radix cache test for spec cases (#12394) Liangsheng Yin 2025-10-30 18:46:28 +08:00
• df5192cffa Enable fast silu-and-mul-and-quant fused kernel (#11806) fzyzcjy 2025-10-30 18:15:39 +08:00
• 78c43d8847 [Feature] Initial eagle3 support for Deepseek-like models (#12319) JensenFire 2025-10-30 17:26:09 +08:00
• 7e28c67d19 Fix DeepSeek chat templates to handle tool call arguments type checking (#11700) (#12123) Kangyan-Zhou 2025-10-30 01:39:25 -07:00
• 3edba9bcc0 [fix] added image token as prefix for deepseek-ocr (#12358) Tanjiro 2025-10-30 14:07:03 +05:30
• e5ec976402 [Bug fix][PP] fix deadlock with tie_word_embeddings (#12362) Xuchun Shang 2025-10-30 16:03:28 +08:00
• 621dfb8886 Import flash_mla from sgl-kernel (#12135) Baizhou Zhang 2025-10-29 23:54:21 -07:00
• fb52d35f63 Super tiny fix AMD ci (#12378) fzyzcjy 2025-10-30 14:25:18 +08:00
• 2b71531a70 Allow benchmarking tool to handle empty response (#12174) Kangyan-ZhouandClaude 2025-10-29 20:26:23 -07:00
• 25c5049870 Super tiny add tag for benchmark scripts (#12340) fzyzcjy 2025-10-30 11:19:14 +08:00
• 8e2ac2e628 [NPU] fix pp_size>1 (#12195) Makcum888e 2025-10-30 06:18:36 +03:00
• 17a57fd862 [Perf] Optimize multimodal mm_inputs process in scheduler (#11910) Yuan Luoandluoyuan.luo 2025-10-30 10:48:10 +08:00
• 32438eba45 [Ckpt Engine] feat: new sglang entrypoint support for update (#12216) Teng Ma 2025-10-30 10:39:27 +08:00
• fed02a49c2 [bugfix] fix deepseekvl2 and deepseek_ocr model type conflict (#12050) FlyPandaandherta 2025-10-30 09:54:32 +08:00
• 03b3e89aea [router] Harmony Pipeline: Chat Completion & Responses API with MCP Support (#12153) Simo LinandChang Su 2025-10-29 18:52:56 -07:00
• 9ff9fa7f95 Fuse wk and weight_proj in Indexer for DeepSeekV3.2-FP4 (#12094) Trevor Morris 2025-10-29 18:42:56 -07:00
• 7ed8ba05cb [CI] Add Llama 3.1 8B FP4 to B200 CI (#12182) b8zhong 2025-10-29 14:56:41 -07:00
• df08f346ce Fix Duplicate Classmethod in spec_info.py (#12354) Stefan He 2025-10-29 13:57:10 -07:00
• db15148c8a [router] update router docker to use maturin and build from local (#12350) Chang Su 2025-10-29 16:00:57 -04:00
• 5259becd3c [bug] fix router installation to include additional dependency (#12348) Simo Lin 2025-10-29 12:45:18 -07:00
• ed1044ac1b support cutlass fp4 kernel in sm120 (#11737) AichenF 2025-10-30 03:25:16 +08:00
• d717e73ea8 [router] refactor mcp to use LRU and fix pooling bug (#12346) Chang Su 2025-10-29 15:18:54 -04:00
• a18161875c Fix Flashinfer Backend for SM120 Usage (#12325) weiliang 2025-10-30 02:51:55 +08:00
• e39628fd07 [2/2] Deepseek deterministic: support deepseek v3 deterministic inference on 8 x H200 (#12095) Minglei Zhu 2025-10-29 11:49:04 -07:00
• bacb3825fe fix: llama 4 + trtllm gen + fp8 kv cache incompatibility (#12347) b8zhong 2025-10-29 11:31:02 -07:00
• b53d9e11c6 [bug] fix router pypi license file (#12345) Simo Lin 2025-10-29 10:05:00 -07:00
• 8a6838212a [Fix] fix type issue of env flag value MODELOPT_MAX_TOKENS_PER_EXPERT (#11709) zejunchen-zejun 2025-10-30 00:44:05 +08:00
• 52694b60da Triton fused_moe_kernel support ep moe tuning (#12343) Xiaoyu Zhang 2025-10-29 23:16:09 +08:00
• 400bddf24c [router] fix router release workflow and add build test in PR (#12315) Chang Su 2025-10-29 11:11:45 -04:00
• 1e90fe2ed1 [Bug fix] trace: fix import error in mini_lb if sgl-router image does not install sglang (#12338) Feng Su 2025-10-29 22:45:08 +08:00
• caa5d2967c feat: return partial generation results when aborting requests in waiting queue (#11673) Yuhong Guo 2025-10-29 22:03:00 +08:00
• 750940ae36 Eagle3 DP attention for Qwen3 MoE (#12002) Rain H 2025-10-29 20:25:17 +08:00
• 42f8ea4030 [Test] Fix session control test (#12336) Liangsheng Yin 2025-10-29 18:28:04 +08:00
• 14cbe42fd3 Refactor abortion in event loop (#12312) Liangsheng Yin 2025-10-29 18:25:20 +08:00
• 685c06451f [ci] Try fixing broken CIs (#12317) Baizhou Zhang 2025-10-29 01:13:51 -07:00
• 1357397a34 feat: preview filename from tuning_fused_moe_triton.py (#12276) Liana KolevaandXiaoyu Zhang 2025-10-29 01:12:25 -07:00
• 42e1a72efb [Deepseek V3.2] Enable flashmla_auto with MTP (#12294) hlu1 2025-10-28 23:51:20 -07:00
• 83a7c89c3f followup fix for llama 4 trtllm flashinfer backend (#12314) b8zhong 2025-10-28 22:17:08 -07:00
• 0380ca82ef Add Batch‑Invariant RMSNorm (#12144) Yuzhen Zhou 2025-10-29 00:05:57 -04:00
• ec92b0cefe EPLB: prefer to use physical experts in the same gpu or node (#10874) Yingchun Lai 2025-10-29 12:01:11 +08:00
• e03b6beeb1 doc: improve modelopt error description (#12269) Liana Koleva 2025-10-28 20:57:55 -07:00
• 5e36a0b455 [metrics][EPLB]: Support selected count of physical experts on each GPU (#9825) Yingchun Lai 2025-10-29 11:56:19 +08:00
• 0297773a2f a tiny fix for support deepseek bf16 weights (#12313) Gao016andgaochang 2025-10-29 11:46:44 +08:00
• 587deb15a7 [hotfix] Fix pytest not found in CI (#12311) Baizhou Zhang 2025-10-28 20:07:36 -07:00
• 83087247d1 [hotfix] missing w13_weight_fp8 and w2_weight_fp8 in UE8M0 requantization (#12259) Cheng Wan 2025-10-28 19:10:38 -07:00
• 334543ff3b Add continuous_usage_stats support for streaming responses (#12241) Xiaoyu Zhang 2025-10-29 10:01:23 +08:00
• c143f416ce fix: Llama 4 BF16 load on Blackwell (#12308) b8zhong 2025-10-28 18:59:01 -07:00
• b48354c537 [router][grpc] Fix inconsistent behavior of conversation_id not found (#12299) Chang Su 2025-10-28 19:24:43 -04:00
• 29195aaa6e Super tiny fix expert distribution dump error (#12271) fzyzcjy 2025-10-29 06:20:55 +08:00
• 0ee831dee0 Update deepseek_v32.md (#12296) hlu1 2025-10-28 14:52:38 -07:00
• 8d6ab1cb88 fix seqlen bug for trtllm_mla's draft_extend (#12295) bmac3 2025-10-28 14:47:47 -07:00
• 84a9d0eab2 [router] support arm, windows, mac, linux, reduce wheel size and number (#12285) Simo Lin 2025-10-28 12:18:51 -07:00
• 737b58d6dc [rust][ci] Add end-to-end tests for Oracle history backend (#12233) Keyang Ru 2025-10-28 10:57:32 -07:00
• 77225d602a Use Flashinfer TRT-LLM as Llama 4 compatible MoE backend (#11928) b8zhong 2025-10-28 10:39:43 -07:00
• 9c6e25d2a6 doc for logit_bias (#12188) ybyang 2025-10-29 01:32:12 +08:00
• 2a3763c335 Tiny fix sgl-kernel related CI installing the wrong binary (#12283) fzyzcjy 2025-10-29 01:29:06 +08:00
• fdd00295b5 Fix 'BypassedTopKOutput' object has no attribute 'topk_weights' for DeepEP (#12231) Trevor Morris 2025-10-28 09:28:25 -07:00
• 25e73640f4 [router] upgrade grpc dependency and py 3.13 3.14 support (#12284) Simo Lin 2025-10-28 08:51:32 -07:00
• 0da9845ef1 Modify rocm.Dockerfile (#12274) sogalin 2025-10-28 23:36:32 +08:00
• 9288544180 [router] Fix type unmatch during validation (#12257) Keyang Ru 2025-10-28 06:04:54 -07:00
• 64cf868eba chore: cleanup quant deps (#12268) Yineng Zhang 2025-10-28 02:03:57 -07:00
• ea39952797 Revert "[Feature] PD-Multiplexing Context and Scheduler." (#12267) Yineng Zhang 2025-10-28 02:00:37 -07:00
• 41a113356a Fix potential eos bug on decode instance when PD is enabled (#12206) Shangming Cai 2025-10-28 16:29:02 +08:00
• a1f2dc90e4 [Bug fix] [PP] fix wrong dtype for quantified model (#12247) Xuchun Shang 2025-10-28 16:27:24 +08:00