Commit Graph
12111 Commits
Author SHA1 Message Date
Zhanghengand晟海 5d92e60783 [Unified Cache Linker][3/N]: Add backend-independent linker core (#37151)
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
2026-08-31 14:07:39 +08:00
Mick 62c470697e [diffusion] chore: enforce component attention backend application (#36907) 2026-08-31 14:00:02 +08:00
Mick 28690f5aa5 [diffusion] chore: detect quantized transformer replacements (#36916) 2026-08-31 13:55:50 +08:00
Liangsheng Yin 5d12ad4fd7 [mem_cache] Move mamba state and retraction_backup into ReqKvInfo (#37164) 2026-08-30 22:03:01 -07:00
Aurick Qiao 9a9e167179 [Bugfix] Fix full prefill CUDA graph padding and EAGLE capture (#35588) 2026-08-30 21:30:24 -07:00
YAMY 2ea6d17eab fix(staging): make empty staging rings reusable (#37166) 2026-08-30 20:59:57 -07:00
YC Yen-Ching TsengandAlex Nails 5972211977 [AMD] Fix the QuickReduce bf16 cast failing to build for CDNA (#37132)
Co-authored-by: Alex Nails <alex.nails@radixark.ai>
2026-08-30 20:57:29 -07:00
Mick 881cbfe54c [diffusion] feat: add exact component precision overrides (#36991) 2026-08-31 11:12:07 +08:00
Pavan Sivaram GirijalaandYanbingJiang 4dc7dc8518 [Fix] Transformers-fallback (GPT-NeoX) + KV pool config (DeepSeek-VL2) (#35244)
Co-authored-by: YanbingJiang <yanbing.jiang@intel.com>
2026-08-31 11:05:50 +08:00
Piotr MazurekandKhoa Pham 046454404a [Spec] Add LFM2 and LFM2-MoE DSpark speculative decoding support (#31041)
Co-authored-by: Khoa Pham <khoa.pham@radixark.ai>
2026-08-30 20:03:59 -07:00
jambow0320andShangming Cai 7700602278 [PD] Align defensive protocol behavior across Mooncake, NIXL, and Mori (#35281)
Co-authored-by: Shangming Cai <csmthu@gmail.com>
2026-08-31 10:57:56 +08:00
Cui LilyandMa Mingfei 3139ceaeec [XPU] Use SYCL kernels for topk_transform on XPU (#33318)
Signed-off-by: Cui, Lily <lily.cui@intel.com>
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
2026-08-31 10:33:19 +08:00
YanbingJiang a9d5ca723a [CPU] Fix weight missing issue in fused_input_proj_cpu for GPTQ INT4 for Qwen 3.5 (#35805) 2026-08-31 10:14:41 +08:00
Mohammad Miadh Angkad 4f761e8649 [Deps] Bump FlashInfer to 0.6.18 (#36954) 2026-08-30 19:02:39 -07:00
Ali Ihsan Nergiz 0da6a66856 [MLX] Fix startup crash when reporting preloaded weights (#37035) 2026-08-30 17:36:13 -07:00
Xiaoyu Zhang bb5e619860 [diffusion] perf: absorb Qwen-Image output projection biases (#37116) 2026-08-31 08:25:37 +08:00
Liangsheng Yin 4bb8de34cc [mem_cache] Share one ReqKvInfo between a streaming session slot and its request (#37108) 2026-08-30 16:36:48 -07:00
4bea51d885 feat(unified-memory): dense KV views for uniform-row MHA/SWA models (#34602)
Co-authored-by: Caihua Li <caihua.li@bytedance.com>
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
2026-08-30 15:10:12 -07:00
Liangsheng Yin 007ef5e23a [mem_cache] Move req_pool_idx into ReqKvInfo (#37094) 2026-08-30 14:46:21 -07:00
Yuzhen ZhouandJiajun Li 8a87079dbb Fix stale GLM MoE routing after runtime weight updates (#35883)
Co-authored-by: Jiajun Li <jiajun.li@radixark.ai>
2026-08-30 14:13:34 -07:00
Xiaoyu Zhang 5ab97c4f44 [Diffusion] Cache Qwen-Image modulation across serial CFG branches (#37090) 2026-08-31 01:40:29 +08:00
Xiaoyu Zhang 8c28cdd116 [Diffusion][Kernel] Fuse Wan2.2 NVFP4 bias + GELU on Blackwell (#37075) 2026-08-31 01:37:35 +08:00
Zhanghengand晟海 6a9366f036 [Unified Cache Linker][2/N]: Add device pool assembly for external linkers (#37098)
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
2026-08-30 23:54:00 +08:00
YAMY 84e56982b6 [Fix] Fix transformer loader fallback test fixture (#37142) 2026-08-30 08:28:11 -07:00
Zhanghengand晟海 c9eb475a88 [Unified Cache][1/N]: Support cache contract for external linker (#37091)
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
2026-08-30 22:24:13 +08:00
Mick fe694986a2 [diffusion] chore: make malformed component execution options fail-fast (#37049) 2026-08-30 21:06:19 +08:00
Mick e6a6492057 [vlm] fix: preserve per-request vit graph metadata for qwen-vl (#37043) 2026-08-30 21:02:14 +08:00
WenhaoZhang e9a7157615 [diffusion] feat: allow cache-dit with dit layerwise offload (#35858) 2026-08-30 20:55:41 +08:00
Mick 26c754e06e [vlm] fix: recover multimodal decode and processor failures (#36983) 2026-08-30 20:50:19 +08:00
Mick aa483ab782 [diffusion] feat: support streaming native vae weights directly to gpu (#37004) 2026-08-30 20:48:09 +08:00
Cheng WanandClaude Opus 5 e51a3ae65e [Config] Round 5.2: the per-model declarations get their own modules (#37087)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-30 02:24:35 -07:00
Cheng Wan 7e751153eb [Config] Round 5.1: the published-side readers ask the bags, and a platform fact gets one address (#37086) 2026-08-30 02:18:33 -07:00
Mick a6e4021368 [diffusion] chore: reject incompatible transformer fallback (#36917) 2026-08-30 16:33:14 +08:00
Jason Wiemels e635577431 [rotary] Fix the fused Qwen3.5 RoPE kernel discarding mrope height and width (#34446) 2026-08-30 15:37:40 +08:00
Oguz Ulgen d249672ad3 Scatter mm embeddings with row index_copy_ instead of masked_scatter_ to cut transient GPU memory (#37070) 2026-08-30 00:15:39 -07:00
Lin Junrong 512df615de [Bugfix] Hold references to fire-and-forget tasks in disaggregation (#33048) 2026-08-30 14:26:34 +08:00
Liangsheng Yin 5ec959965b [mem_cache] Settle extend kv_committed_len inside alloc_for_extend (#37085) 2026-08-29 23:14:31 -07:00
Liangsheng Yin 0438b16154 [mem_cache] Move kv_committed_len into ReqKvInfo (#37078) 2026-08-29 22:42:25 -07:00
EchO 5b7c62d5d6 fix(frontend): bound stop strings and regex patterns (#37029) 2026-08-29 22:25:26 -07:00
Alex Nails 78fa921189 [ROCm] Fix QuickReduce fp16 saturation corrupting bf16 all-reduces (106M non-finite -> 0, +0.3%) (#34484) 2026-08-29 22:08:17 -07:00
Baizhou Zhang a7ee399904 Bump sgl-deep-gemm to v0.1.6 (#36998) 2026-08-29 19:51:02 -07:00
Liangsheng Yin 6be767c2d2 [mem_cache] Move cache_protected_len and swa_evict_floor into ReqKvInfo (#36982) 2026-08-29 19:37:53 -07:00
ed39568e79 [MoE] Add extension points for custom runner backends (#32665)
Co-authored-by: Alex Nails <alex.nails@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-29 19:03:41 -07:00
EchOandGuanyi Chen ca8ff035c3 fix(hicache): isolate decode offload state per request (#37026)
Co-authored-by: Guanyi Chen <939416532@qq.com>
2026-08-29 18:15:05 -07:00
karverma-amd fbecd75c83 [AMD] fix: do not emit a shared-expert marker twice on the per-rank slot path (#36515) 2026-08-29 18:09:33 -07:00
Liangsheng Yin 9a489f8d2f [Test] Move gpqa and aime25 onto sgl-eval, drop unused eval paths (#36979) 2026-08-29 17:36:13 -07:00
Neeraj KashyapandBaizhou Zhang 032fe91bf1 Handle unlimited tokenizer context lengths (#37054)
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
2026-08-29 17:29:11 -07:00
f60bc73c58 [Spec] Fix Dspark and Dflash state divergence across TP rank (#33614)
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
2026-08-29 17:27:43 -07:00
a1fe4e30a9 [Kernel] Fix SM90 FP8 decode regression with benchmarked M/K/N routing (#37018)
Co-authored-by: John Doe <johndoe@example.com>
Co-authored-by: BBuf <1182563586@qq.com>
2026-08-29 16:59:30 -07:00
Alex NailsandClaude Opus 5 00fbb6e8ac [Perf] Tune the W4AFP8 DeepEP low-latency requant launch geometry (#35760)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-29 16:03:01 -07:00