Commit Graph
6055 Commits
Author SHA1 Message Date
YAMY 982aa8acfc [Bugfix] Load Qwen3.5 MTP embedding under PP (#37471) 2026-09-02 15:19:40 -07:00
YAMY 3c9cea8f10 [EAGLE] Prune draft-extend logits to selected rows (#35546) 2026-09-02 15:10:08 -07:00
YAMY fe45af1e6f perf(gdn): select ReplaySSM verify loop unrolling by shape (#36970) 2026-09-02 15:07:49 -07:00
cctry 3a855b050a fix(disagg): poll receivers during decode preallocation (#37483) 2026-09-02 14:26:40 -07:00
Liangsheng Yin 19c7679e9e [mem_cache] Make free_swa sync-free on page_size == 1 (#36723) 2026-09-02 14:18:22 -07:00
JoeandBBuf acea43079f Fix native MoE handling of noncontiguous top-k IDs (#36407)
Co-authored-by: BBuf <1182563586@qq.com>
2026-09-02 14:16:19 -07:00
f8cbf000f4 [AMD] Enable FP4 indexer for Deepseek V4 (#37353)
Co-authored-by: 1am9trash <1am9trash@gmail.com>
Co-authored-by: AMD-yanfeiwang <256076023+AMD-yanfeiwang@users.noreply.github.com>
Co-authored-by: Thomas Wang <thomawan@amd.com>
2026-09-02 09:45:08 -07:00
Po-Han Huang (NVIDIA) d585cec4bd Fix nondeterministic FlashInfer GDN alignment test (#37343) 2026-09-01 23:46:58 -07:00
Lianmin Zheng f8f04bafa8 Rust server: align launcher and request validation behavior (#37327) 2026-09-01 23:42:36 -07:00
jasonjk-park 83bd2c473f Allow custom policy for adaptive speculative decoding (#37274) 2026-09-01 23:15:21 -07:00
Liangsheng Yin 01c3a5f54f [misc] Resolve SWA ownership at enqueue time for grouped free() (#36646) 2026-09-01 23:06:55 -07:00
Liangsheng Yin 832d029870 [mem_cache] Split duplicate insert frees at the SWA eviction floor (#37481) 2026-09-01 23:01:40 -07:00
YAMY a6a19f9290 [Bugfix] Skip absent radix lock during cache cleanup (#37494) 2026-09-01 22:50:19 -07:00
Khoa PhamandClaude Opus 5 c66a285c94 [Kernel] GLM 5.3 Flash related kernels (ported from #36507) (#37477)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-01 22:17:09 -07:00
Wes 2d9c64394f Fix reasoning metrics and add TPOT to bench_multiturn (#35443) 2026-09-02 11:28:47 +08:00
Xiaoyu Zhang 403a15c163 [CI] Batch CPU test workers (#37252) 2026-09-02 10:35:14 +08:00
26f760d5c0 [CPU] Support FP8 KV cache (#32733)
Co-authored-by: Zheng, Beilei <beilei.zheng@intel.com>
Co-authored-by: mingfeima <mingfei.ma@intel.com>
2026-09-02 10:20:54 +08:00
ashwini rathiandP V R K Jyothendra Varma e874ae64cd [XPU][CI] Enable nightly-xpu-8-gpu suite: declare + wire runner job (#37230)
Signed-off-by: P V R K Jyothendra Varma <polisetty.v.r.k.jyothendra.varma@intel.com>
Co-authored-by: P V R K Jyothendra Varma <polisetty.v.r.k.jyothendra.varma@intel.com>
2026-09-02 10:14:40 +08:00
kkandwunhuang dc276264cb [AMD] Perf Kimi-K3 fuse ROCm KDA decode boundary (#34198)
Co-authored-by: wunhuang <wunhuang@amd.com>
2026-09-01 18:35:48 -07:00
Xiaoyu ZhangandCursor f4c17fed07 [Diffusion] Fuse FLUX.2 NVFP4 FC1, SwiGLU, and FC2 quantization (#37096)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-02 08:21:14 +08:00
c593527f33 [Kernel] Add KDA NVFP4 GEMM for Qwen3.x on SM120 (#36865)
Co-authored-by: Song Bian <biansonghz@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-02 08:16:54 +08:00
33428d3dae Fix GPU kernel ordering and MXFP8 quantization dispatch (#37331)
Co-authored-by: Pranjal Shankhdhar <pranjalssh@meta.com>
Co-authored-by: Chengze Fan <fancz2002@gmail.com>
2026-09-01 17:09:44 -07:00
YAMY 221a6273ce [Kernel] Avoid zero-bias allocation in fused softmax routing (#36811) 2026-09-01 16:29:57 -07:00
Yikai Zhangandamdpilot-upstream-sync 9978aaec8b [ROCm][Bugfix] Cap the DSA MQA-logits budget at AITER's buffer_store limit (#36960)
Co-authored-by: amdpilot-upstream-sync <amdpilot-upstream-sync@users.noreply.github.com>
2026-09-01 15:40:12 -07:00
b24c8f10e7 [FlashInfer] Avoid D2H sync for sliding-window lengths (#32218)
Co-authored-by: llilian73 <204300658+llilian73@users.noreply.github.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
2026-09-01 14:08:22 -07:00
Cheng Wan 0b1ce3d140 [Feature] Unified memory: support decode context parallelism for Kimi-Linear (#36890) 2026-09-01 12:44:26 -07:00
Shuwen Wang a2b8681d1d [CI][MLX] Restore the mamba_branching_seqlen attribute the MLX runner reads off a request (#37453) 2026-09-01 11:16:45 -07:00
cctryandcctry 9a05b470fa [Memory] Size the CUDA graph pool from warmup measurements and fix graph-pool borrowing (#36911)
Co-authored-by: cctry <cctry@fb.com>
2026-09-01 09:32:38 -07:00
DarkSharpnessandClaude Opus 5 cb6dd58fbe [Kernel] Replace dsv3_router_gemm with the unified tiny GEMM (#34693)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-01 23:00:21 +08:00
Xiaoyu Zhang 5993f91f84 [Kernel] Register merged diffusion agent kernels with KDA backend (#37385) 2026-09-01 22:51:10 +08:00
DarkSharpnessandClaude Opus 5 b6c06e1efb [DSA] Drop the redundant 512 from the top-k transform entry-point names (#36831)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-01 22:37:03 +08:00
Even Zhou 00689c0c94 [CI] Add entrypoint to hc_combine test for standalone execution (#37406) 2026-09-01 22:08:00 +08:00
3ae54c6ca2 test(npu): add DSV4-Flash / GLM-5.2 / Kimi-K3 gpqa accuracy cases (#37431)
Co-authored-by: Sugar920 <Sugar920@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-09-01 22:00:04 +08:00
datdo-msftandShangming Cai 49db27528a fix(test): deflake zmq load-snapshot round-trip tests (#35787)
Co-authored-by: Shangming Cai <csmthu@gmail.com>
2026-09-01 18:14:55 +08:00
Jialin Ouyang a77283fb02 [Rust] Rename mem-cache to sglang-radix-tree (#37290) 2026-09-01 03:03:44 -07:00
eeecho b68702be99 [DSV4] hc-prenorm: fuse the combine step into a Triton kernel (#35118) 2026-09-01 02:04:05 -07:00
Ziang Li 5edcd0a445 [FlashInfer V0.6.18] feat(dsv4): support --dsa-topk-backend flashinfer with fused top-k (#33237) 2026-09-01 01:18:10 -07:00
Liangsheng Yin 3484f7f836 [mem_cache] Add free_kv_row to release a request's kv row by row range (#36721) 2026-09-01 01:14:43 -07:00
Xiaoyu ZhangandCursor 1c3ad92438 [Diffusion] Fuse FLUX.2 ModelOpt FP8 producers and QKV packing (#37162)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-01 16:14:29 +08:00
Mohammad Miadh Angkad 03b33cbe5d [CI] Fix hybrid wrapper test fake missing kv_index_translator (#37374) 2026-09-01 00:55:05 -07:00
chien-an-chen 8a191554e3 [AMD] Enable 12-head MLA aiter fp8 Gluon decode (batched bh16bn128). (#34647) 2026-08-31 23:47:45 -07:00
Ma Mingfei 5e79110122 [Fix][CPU] fix xeon ci failure by test_qwen35_flashinfer_fusion (#37338) 2026-09-01 14:17:35 +08:00
Liangsheng Yin 959ca033eb refactor(hicache): simplify decode offload state bookkeeping (#37299) 2026-08-31 23:09:41 -07:00
Mick ae2bd5728b [vlm] fix: contain multimodal feature transport failures (#37047) 2026-09-01 13:46:38 +08:00
YAMY 33ed29a0ee test: update hybrid attention runner fixtures (#37345) 2026-08-31 22:00:50 -07:00
Liangsheng Yin 3b14f37b74 [Fix] Use real ReqKvInfo in unit-test req mocks (#37339) 2026-08-31 20:25:51 -07:00
Yuan Luoandluoyuan.luo 5b04408784 [MoE] Add FlashInfer SM90 MXFP4 W4A8 CUTLASS MoE (#34967)
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
2026-08-31 20:04:41 -07:00
22337e9c56 fix(unified-memory): forward the KV-index translator through every wrapper backend (#37307)
Co-authored-by: Caihua Li <caihua.li@bytedance.com>
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
2026-08-31 19:55:21 -07:00
Mohammad Miadh Angkad e6f21cdadc [Cohere Command-A-Plus] Optimize decode and BCG capture on SM10X (#36624) 2026-08-31 19:37:47 -07:00
71cee04ebe [Diffusion] Optimize Qwen-Image TP collectives and attention (#36680)
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-01 10:28:37 +08:00