Commit Graph
14395 Commits
Author SHA1 Message Date
eb4b167d27 [Feat] KVShard: PD transfer and e2e test (#38473)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Zhangheng <hzh0425@apache.org>
2026-10-20 22:48:43 +08:00
7656071309 [Feat] platform: Add initial T-HEAD SAIL platform support [2/N] (#43826)
Co-authored-by: Zhangheng <hzh0425@apache.org>
Co-authored-by: yzshi <163985193+Yunzshi310@users.noreply.github.com>
2026-10-20 22:19:07 +08:00
Jensenandronnie_zheng 10d5293546 [Perf] [NPU] glm5.2: Support dynamic INT8 MLAProlog for attention (#43469)
Co-authored-by: ronnie_zheng <zl19940307@163.com>
2026-10-20 16:33:08 +03:00
Jensen 3bf4cbc6cf [Perf] [NPU] mem_cache: Increase GLM-5.2-W4A8C8 KV cache capacity with compact DSA indexer allocation (#44436) 2026-10-20 16:30:01 +03:00
gjsheu aa91108f64 [Fix] [NPU] glm5.2: support pcp v2 with interleave & zigzag (#40165) 2026-10-20 21:26:30 +08:00
gjsheu ee2f95a522 [Fix] [NPU] glm5.2-mtp: L2 cache adaptation (#41642) 2026-10-20 21:25:37 +08:00
Kurkur 07eb671b6a [Perf] [NPU] attn: add not needs_cpu_seq_lens for dsa modules (#38461) 2026-10-20 19:19:34 +08:00
Shijin ZhangandXinyuan Tong ca473ef3e4 [Feat] glm-5.3: Render GLM-5.3 and GLM-5.3-Flash chat prompts with a Python chat template (#45154)
Signed-off-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
2026-10-20 19:11:40 +08:00
Yuxuan Zhou 306ccf6f57 [Perf] spec: Let the FlashAttention backend capture the draft-extend CUDA graph (#40762) 2026-10-20 03:05:42 -07:00
Sihan Chen 9db58ab547 [Fix] eplb: Fix Qwen3 MoE output corruption after EPLB rebalance (#39580)
Signed-off-by: Spycsh <sihan.chen@intel.com>
2026-10-20 03:00:39 -07:00
a70b1b73d3 [Fix] tracing: Fix OTel tracing context lost in multi-tokenizer router processes (#38211)
Co-authored-by: Ma <T9FOPWP@cn.vwgroup.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
2026-10-20 17:57:16 +08:00
16eb87c06f [Fix] openai: distinguish s3 served-model-name from LoRA adapter syntax (#44450)
Signed-off-by: Muqi Li <muqi1029@gmail.com>
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
2026-10-20 17:11:06 +08:00
Xiaoyu ZhangandClaude Opus 5.5 2229345dff [Doc] kda_kernels: Point the SM121 QSA kernel provenance at its SGLang PR (#45359)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 16:50:58 +08:00
Ho-Ren (Jack) Chuang f8fd41c36e [Perf] moe: Extend FlashInfer CUTLASS MoE autotune coverage to prefill chunks (#43363) 2026-10-20 01:41:16 -07:00
Xiaoyu ZhangandClaude Opus 5.5 0788f1fdd0 [Refactor] kernels: Register agent-generated kernels under their KDA / Cake backend (#45277)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 16:35:11 +08:00
Mohammad Miadh AngkadandMohammad Angkad 78185f4b3f [Fix] [NVIDIA] modelopt-nvfp4: Read w13 global scales in the TRT-LLM [up, gate] order (#44858)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-10-20 07:38:45 +00:00
XXXXRT666andAlex Nails 1739d962a4 [Refactor] [Apple] MLX: Use Native Extension Builder for Metal Kernels (#44367)
Co-authored-by: Alex Nails <alex.nails@radixark.ai>
2026-10-20 00:35:29 -07:00
Shuwen Wang 15e8e687db [Chore] environ: Remove retired unified radix tree environment variable (#45090) 2026-10-20 07:02:43 +00:00
1ec902e00f [Fix] cache: Support DeepSeek-V4.1 external linker pools (#44623)
Co-authored-by: Zhangheng <hzh0425@apache.org>
Co-authored-by: Shuwen Wang <47200617+alphabetc1@users.noreply.github.com>
2026-10-19 23:40:25 -07:00
Michael 407bfc94f9 [Test] [AMD] diffusion: Make multimodal_gen unit tests ROCm-aware (#44599) 2026-10-20 14:24:37 +08:00
Xinyuan Tong 4b83f215f4 [Feat] entrypoints: add a chat encoding adapter registry (#45300) 2026-10-20 14:10:01 +08:00
lpdinkandZhangheng 4a651146de [Feat] hicache: Notify storage backends when the initial host-pool registration burst completes (#44030)
Co-authored-by: Zhangheng <hzh0425@apache.org>
2026-10-20 14:09:13 +08:00
Michael 36978d6c4d [Test] [AMD] diffusion: Skip the Kandinsky6 FA dispatch test on ROCm (#44421) 2026-10-20 13:54:23 +08:00
Liangsheng Yin 472f0d540b [Fix] deepseek-v4.1: Keep the padded query buffer inside decoder tail graphs (#45301) 2026-10-19 22:41:53 -07:00
Khoa PhamandClaude Fable 5.1 476fe86a12 [Perf] deepseek-v4.1: Score a request's verify rows as one request in the full-scan indexer (#45251)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-19 22:25:04 -07:00
35c873cff4 [Fix] observability: Log scheduler stats once per attention CP group (#45319)
Co-authored-by: Hun-ho Kim <hunho.kim@samsung.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 22:22:31 -07:00
weireweireandweireweire c7687a0e7c [Fix] deepseek-v4: Keep the draft MoE out of the target's expert placement (#45299)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
2026-10-19 22:20:26 -07:00
602c92ccd8 [Perf] DeepSeek-V4.1: Raise the SM100 swap-AB decode cap to 40 rows (#44632)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
Co-authored-by: nbommi19 <227267412+nbommi19@users.noreply.github.com>
2026-10-19 22:20:10 -07:00
parkeychenandparkeychen 43a896933e [Fix] deepseek-v4: Refresh attn_sink after loading weights (#41630)
Co-authored-by: parkeychen <parkeychen@tencent.com>
2026-10-19 21:59:32 -07:00
ashwini rathiandClaude Opus 5.5 d1c3346f74 [Fix] [Intel] spec: Fix XPU nightly dflash stop_regex and DSpark capture check (#45033)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 12:27:36 +08:00
72d886efb8 [Perf] DeepSeek-V4.1: Run the small-batch decode fusions at TP2 (#43838)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
Co-authored-by: nbommi19 <227267412+nbommi19@users.noreply.github.com>
Co-authored-by: Po-Han Huang <pohanh@nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
2026-10-19 21:17:05 -07:00
Shuwen Wang c53b98d1fe [Fix] test-utils: Allow expected server exits in canary tests (#45172) 2026-10-19 21:07:03 -07:00
wxzhoucsandXiaoyu Zhang e3205080ce [Fix] qwen3.8: Fix batch-invariant kernels for Qwen3.8 Flash-Next (#42087)
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
2026-10-20 11:32:40 +08:00
1140a8dfd9 [Fix] [AMD] aiter: Fix garbage decode output for non-sliding-window models on the vectorized_5d KV layout (#43335)
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
2026-10-20 11:31:20 +08:00
Cao EandMa Mingfei 3c07ff2654 [Perf] [XPU] fp8: Optimize FP8 scale layouts (#40052)
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
2026-10-20 11:24:16 +08:00
huangtingwei 3eb4bd598f [Perf] hicache: Unify PP prefetch hit queries and buffer allocation (#43344) 2026-10-20 11:05:00 +08:00
kpjeeja 8683f8ad4a [Feat] [Intel] hicache: Enable L1<->L2 support on XPU (#32503) 2026-10-20 10:42:21 +08:00
Liangsheng Yin 5e630fa571 [Perf] deepseek-v4.1: Drop host syncs from encoder-replay folding and tail metadata (#45275) 2026-10-19 19:39:39 -07:00
Mohammad Miadh AngkadandMohammad Angkad af856717c2 [Fix] mamba: Skip zero-width conv pools in the MTP verify scatter (#45212)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-10-19 19:23:17 -07:00
Vladimir Serovandronnie_zheng 55b2aef7ad [Fix] [NPU] LoRA: Update for follow-up with changes in sgl-kernel-npu (#40908)
Co-authored-by: ronnie_zheng <zl19940307@163.com>
2026-10-20 05:16:05 +03:00
Gabriel Wuandnumber-eleven11 67d9ee156c [Fix] [SM120] quantization: Requantize block-FP8 MoE weights to UE8M0 for DeepGEMM (#43556)
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: number-eleven11 <1635219023@qq.com>
2026-10-19 19:06:21 -07:00
Liangsheng Yin ce329c85fa [Perf] deepseek-v4.1: Skip the request-window history check when invariant checks are off (#45276) 2026-10-19 18:38:44 -07:00
79ed13fc2c [Fix] glm47: Disable tool grammar for tool_choice="none" (#41233)
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: Po-Han Huang <pohanh@nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 09:36:06 +08:00
Lee Nau 1897575ce1 [Fix] runtime: Version FlashInfer caches for default-tactic tuning. (#44514) 2026-10-19 18:33:49 -07:00
Anihcandsunguozhong ebc71e650e [Perf] [NPU] mem_cache: Add optional ScatterPa KV cache writes (#40535)
Co-authored-by: sunguozhong <sun_gz2772@163.com>
2026-10-20 09:33:34 +08:00
71ee2dc950 [Fix] inkling: Return 400 instead of 500 for unfetchable/undecodable multimodal input (#40899)
Co-authored-by: apex-mochen <265981728+apex-mochen@users.noreply.github.com>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
Co-authored-by: apex-mochen <2756823972@qq.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Po-Han Huang <pohanh@nvidia.com>
2026-10-20 09:18:18 +08:00
Xiaoyu ZhangandClaude Opus 5.5 693d333306 [Refactor] kernels: Regroup operators by computation and dissolve ops/muse_spark (#45112)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 08:47:57 +08:00
Liangsheng Yin ab580b2e06 [CI] radix-cache: Split unified radix cache unit tests into topic files and a shared kit (#45236) 2026-10-19 17:00:42 -07:00
Cheng Wan 6b90e5716d [Refactor] moe: Scope shared-expert configuration to each model (#44913) 2026-10-19 16:59:37 -07:00
Cheng Wan 964ab2b280 [Refactor] moe: Split the two meanings of num_fused_shared_experts (#44912) 2026-10-19 16:57:15 -07:00