Commit Graph
14345 Commits
Author SHA1 Message Date
Cheng Wan 8fc3ce48da [Refactor] Choose prepare_attn / prepare_mlp steps and fused kernels at construction (#41252) 2026-09-26 02:38:03 -07:00
7bc988446d [DeepSeek V4.1] Add DeepSelect JIT kernel. (#40556)
Co-authored-by: DarkSharpness <2040703891@qq.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-26 17:36:48 +08:00
Hexu Zhaoandhexu.zhao a5c8afab48 [Perf] Mamba2 selective_state_update up to 2x faster on B200 via 8x1 launch config for dstate 128 (+7.5% Nemotron-3-Super serving) (#41223)
Co-authored-by: hexu.zhao <zhaohexu2001@gmail.com>
2026-09-25 23:27:01 -10:00
Sundara Raman Ramachandran 3ed56a341b [Score API] Setwise scoring: CausalLM support (batched + --enable-mis) (#41188) 2026-09-25 23:02:44 -07:00
Mohammad Miadh AngkadandMohammad Angkad 4bb645b8e2 [DSA] Fix the pooled-indexer breakable prefill bridge under DP attention (#41311)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-09-25 22:52:22 -07:00
8772916e06 [DSv4.1] Move the low-ratio index top-k into dsv4/low_ratio_indexer (#41125)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-09-25 22:22:57 -07:00
Xiaoyu Zhang 38ec649048 [Refactor] Restore logical kernel groups and test organization (#41243) 2026-09-26 12:30:44 +08:00
MickandMick Qian e538489e0e [diffusion] fix: copy small files into overlay materialized trees instead of linking them (#41298)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-09-26 12:22:47 +08:00
WMCandjacky.cheng dfdbff931d [diffusion] fix: fix Qwen-Image 2.1 default RGBA output (#41150)
Co-authored-by: jacky.cheng <yichiche@amd.com>
2026-09-26 12:21:24 +08:00
Lianmin ZhengandLu Fang 3ecb6cd729 [MemCache] Unify component eviction cursors and lock receipts (#41276)
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com>
2026-09-25 20:53:35 -07:00
Olga Miroshnichenko 0e2aac500b [ROCm][Perf] aiter: page-level KV view for gfx950 fp8 page-64 asm prefill (#36505) 2026-09-25 19:23:46 -07:00
Shuwen Wang 4273e7de77 [PD] fix: cache resumed decode-radix requests from root instead of an unlocked re-match (#41261) 2026-09-26 10:21:41 +08:00
Mohammad Miadh AngkadandMohammad Angkad 63d320c723 [DSA] Chunk the kpool indexer MQA logits by query rows under a free-memory budget (#40854)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-09-25 19:13:13 -07:00
MickandMick Qian 2ee82fe158 [diffusion] model: support Anima Base v1.0 (#41011)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-09-26 09:53:51 +08:00
Liangsheng YinandDarkSharpness 0967a013c2 [DSv4.1] Move the ratio-1/2 index top-k ops into kernels/ops/attention/dsv4 (#41291)
Co-authored-by: DarkSharpness <2040703891@qq.com>
2026-09-25 18:28:04 -07:00
Xiaoyu Zhang 85374532f2 [diffusion] Fuse lossless Klein packed QK RMSNorm and RoPE on Hopper (#40490) 2026-09-26 09:24:21 +08:00
50e50e13ab [DCP] Use logical token capacity for PD admission and load reporting (#39731)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Khoa Pham <khoa.pham@radixark.ai>
2026-09-25 17:39:47 -07:00
Liangsheng Yin 2afd343934 [misc] Call all_gather_single / reduce_scatter_single to drop torch deprecation warnings (#41284) 2026-09-25 16:14:13 -07:00
Liangsheng YinandBBuf efd9a40bb8 [PD] Share one head-slice helper across mooncake, mori, and nixl (#39660)
Co-authored-by: BBuf <1182563586@qq.com>
2026-09-25 16:13:18 -07:00
Liangsheng Yin 516ab77619 [Test] Route all sgl-eval benchmarks through run_sgl_eval and deprecate run_eval (#41280) 2026-09-25 15:48:38 -07:00
Kevin MiandClaude Opus 5.5 27e883a20d dsv4.1-amd: gfx950 MXFP8 matmul kernels and fp8-grid producers (#41018)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-25 15:45:40 -07:00
Erik WijmansandEthan Su c421d16563 [sglang][lora] Support DP attention in LoRA backends (#36389)
Co-authored-by: Ethan (Yusheng) Su <yushengsu@radixark.ai>
2026-09-25 12:03:47 -10:00
cf5df82680 [KVCache] Support lmcache unified radix cache (#38652)
Signed-off-by: chunxiaozheng <1179548172@qq.com>
Co-authored-by: Yuwei An <ayw.sirius19@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-25 14:08:45 -07:00
Ethan (Yusheng) Su d3e3945e41 [LoRA] Size dense row/column-parallel LoRA buffers from the base linear's real shard (#39379) 2026-09-25 14:01:08 -07:00
Liangsheng Yin f88572a57c [Test] Add in-process sgl-eval adapter and move validated GSM8K tests to it (#41216) 2026-09-25 13:59:17 -07:00
Liangsheng Yin f337f01b80 [Test] Remove dead eval modules and point GSM8K/MMLU docs to sgl-eval (#41215) 2026-09-25 12:50:31 -07:00
Nan Jiang 24b6930076 [Sampling] Stream sampling masks as per-request arrays (#40986) 2026-09-25 12:20:25 -07:00
6d35cee7fc Fix multimodal feature offload races (#40621)
Co-authored-by: cctry <cctry@fb.com>
Co-authored-by: Yongji Wu <yongji@meta.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
2026-09-25 12:18:08 -07:00
metamergebotandYonghao Zhuang 671000aee6 [unified-memory] Honor move gates in float relocation and size auto HiCache from host capacity (#41248)
Co-authored-by: Yonghao Zhuang <yhzhuang@meta.com>
2026-09-25 12:17:28 -07:00
31b931edc3 fix(multimodal): return 400 for corrupt image inputs (#28131)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Co-authored-by: Justin Tong <justintong0323@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
2026-09-25 19:51:33 +02:00
0154f72b48 [ROCm][Bugfix] Keep quantization for mixed Quark Qwen3.5 MTP checkpoints (#39064)
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: jacky.cheng <yichiche@amd.com>
2026-09-25 10:31:29 -07:00
Even Zhou 67bb6a58d0 Revert "[NPU] Fuse FIA KV-cache K/V writes into one npu_scatter_pa_kv_cache call" (#41132) 2026-09-25 23:56:22 +08:00
Xiaoyu Zhang 2f5c9ac43d [diffusion] Enable lossless Cosmos3 Super T2I QK fusion on Hopper TP2 (#40486) 2026-09-25 18:01:06 +08:00
Jianghaiandmickqian a749d84303 [Diffusion] Fix AttributeError in grouped forward_batch by installing the residency manager (#34417)
Co-authored-by: mickqian <mickqian@users.noreply.github.com>
2026-09-25 17:27:38 +08:00
Cheng Wan 3450d68d4e [Refactor] Decide an FFN exit's completion once and declare the group it owes (#41200) 2026-09-25 01:50:35 -07:00
Cheng Wan 26a3214cda [Refactor] Take a layer's last-layer fact from its scatter-mode plan (#41199) 2026-09-25 01:49:33 -07:00
Cheng Wan 8f5a636f8d [Refactor] Build prepare_mlp and the layout moves from named steps (#41191) 2026-09-25 01:48:30 -07:00
Cheng Wan b7f6d04a9a [Refactor] Move Step-3.5, GLM5-Next, Dots3, MiniMax-M3 and Qwen3.5 onto ffn_exit (#41198) 2026-09-25 01:47:43 -07:00
Cheng Wan 963e9fb42d [Refactor] Leave the FFN reduction to the next layer under attention DP (#41197) 2026-09-25 01:47:03 -07:00
Cheng Wan 9d7f44bdbb [Refactor] Carry a deferred FFN all-reduce as UnreducedOutput and complete it in the next layer without the fused kernel (#41196) 2026-09-25 01:46:03 -07:00
Cheng Wan 402df23188 [Fix] Stop counting a deferred FFN sum more than once: replicated TP1 shared expert, dense reduce_scatterv (#41195) 2026-09-25 01:45:00 -07:00
Cheng Wan 1409f469be [Fix] Plan NextN / MTP draft layers as one-layer models and fix the Bailing V2 NextN draft (#41194) 2026-09-25 01:43:55 -07:00
Cheng Wan 51c92e8df1 [Fix] Complete the all-reduce when the flashinfer fused norm declines a batch (#41193) 2026-09-25 01:42:16 -07:00
kk 515f5be77e [AMD] Integrate Aiter MegaMoEv2 for DeepSeek-V4 (#35619) 2026-09-25 00:47:08 -07:00
Shuwen Wang 434c2e3adc [HiCache] fix: Drain pending backups before internal Mamba write-back (#41092) 2026-09-25 15:06:24 +08:00
Shuwen Wang cbe137727e [HiCache] Give trailing sidecar storage transfers a contiguous prefix_keys chain (#40456) 2026-09-25 15:06:05 +08:00
MickandMick Qian 0fb699ac03 [diffusion] model: support Ming-Image Design and Design-Layer (#41067)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-09-25 14:44:37 +08:00
metamergebotandXingyu Liu 9c8340a2e5 [DeepEP v2] Let a model package supply its per-rank prefill dispatch bound (#41201)
Co-authored-by: Xingyu Liu <38244988+charlotte12l@users.noreply.github.com>
2026-09-24 23:39:03 -07:00
Xinyi Song ec070ec8c8 [AMD] Fix int32 offset overflow in Triton DSv4 KV store kernels (#41159) 2026-09-24 23:11:59 -07:00
Ting SUNandXinyuan Tong 3e7e652900 fix(openai): reject request-supplied chat_template by default (#28135)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
2026-09-25 07:48:34 +02:00