Commit Graph
14279 Commits
Author SHA1 Message Date
e9b22a0818 [Perf] [AMD] monokernels: Let a plugin skip per-step attention metadata (#44601)
Co-authored-by: Kevin Mi <kevin.mi@radixark.ai>
Co-authored-by: kkHuang-amd <43161300+kkHuang-amd@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 14:04:57 +00:00
Cheng Wan 57db5089c2 [Perf] attention: Capture translated FA4 verify metadata for DSPARK (#44973) 2026-10-18 05:42:16 -07:00
Cheng Wan 2e9b2cacda [Perf] speculative: Fuse draft write-window gathering and KV translation (#44972) 2026-10-18 05:41:34 -07:00
Cheng Wan 2a83471676 [Perf] mamba: Fuse strided state updates and replay index translation (#44971) 2026-10-18 05:40:57 -07:00
Cheng Wan d318a546da [Perf] unified-memory: Reduce allocation and SWA retirement overhead (#44970) 2026-10-18 05:40:25 -07:00
Cheng Wan a5b21145e1 [Refactor] attention: Separate decode metadata from pool ownership (#44969) 2026-10-18 05:39:44 -07:00
3edb156dec [Feat] unified-memory: Fuse drafts with sliding-window and recurrent-state layers (#43326)
Co-authored-by: Caihua Li <caihua.li@bytedance.com>
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
2026-10-18 05:39:09 -07:00
caihuali95 19cb24f1e1 [Feat] unified-memory: Support asymmetric KV rows (#43325) 2026-10-18 05:37:50 -07:00
caihuali95andCaihua Li 1d17066bad [Perf] gdn: Select FlashInfer prefill for page-major pools (#43171)
Co-authored-by: Caihua Li <caihua.li@bytedance.com>
2026-10-18 05:37:24 -07:00
caihuali95andCaihua Li ba50ea3d77 [Perf] unified-memory: Free SWA segments without a host sync (#43170)
Co-authored-by: Caihua Li <caihua.li@bytedance.com>
2026-10-18 05:36:56 -07:00
caihuali95andCaihua Li 47eeddcd18 [Fix] swa: Retain a whole window behind insertion pages (#43168)
Co-authored-by: Caihua Li <caihua.li@bytedance.com>
2026-10-18 05:36:02 -07:00
WenhaoZhang e17fed6ea7 [Perf] diffusion: fused gate/up + SwiGLU MLP on comfy_kitchen INT8 for Qwen-Image 2.1 (#44196) 2026-10-18 20:33:31 +08:00
LindaandMick 680a2ff27e [Perf] [NVIDIA] diffusion/FLUX.3: Run dynamic LoRA through PrimsTS GEMMs (#44761)
Co-authored-by: Mick <mickjagger19@icloud.com>
2026-10-18 20:32:19 +08:00
WenhaoZhangandMick 1fd46ddd56 [Perf] diffusion: fused indexed modulation for FP32 AdaLN tables (#44180)
Co-authored-by: Mick <mickjagger19@icloud.com>
2026-10-18 20:31:36 +08:00
Shuwen Wang 8dee3f6bdb [Fix] [H200] ci: Split GLM-5.3-Flash serving recipes into separate tests (#44892) 2026-10-18 19:26:40 +08:00
3b932cde26 [Fix] VLM: harden Kimi image cache identity across preprocessing and radix hashing (#43043)
Co-authored-by: Rahul Chalamala <22563365+rchalamala@users.noreply.github.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
2026-10-18 17:39:37 +08:00
5a930cfd88 [Fix] glm4v: Clamp request video budgets to the deployment limit (#43871)
Co-authored-by: cipheraxat <cipheraxat@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 17:31:33 +08:00
Ilia Yastrebovandishandhanani 6c3db0f520 [Perf] NIXL: prepared dlists for hybrid-model state transfer (#44740)
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
2026-10-18 16:59:18 +08:00
Xinyuan Tong 80fef63786 [Fix] Inkling: frame constrained output in native message tokens (#44716) 2026-10-18 16:47:56 +08:00
Huang Long 889658a5be [Fix] disaggregation: Reuse startup decode graphs across PD role switches (#43760) 2026-10-18 16:16:46 +08:00
+14 d33dd0566a [Feat] srt: Add muse spark runtime, multimodal support and tests (3/5) (#44666)
Co-authored-by: 1am9trash <1am9trash@gmail.com>
Co-authored-by: Alex Nails <alex.nails@radixark.ai>
Co-authored-by: Chunan Zeng <zcnrex@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Ethan Che <eche@meta.com>
Co-authored-by: HAI <hixiao@gmail.com>
Co-authored-by: Jiajun Li <guapisolo@gmail.com>
Co-authored-by: Jiajun Li <jiajun.li@radixark.ai>
Co-authored-by: Jimmy Shiu <jimmysh341@gmail.com>
Co-authored-by: Jimmy Shong <69131491+Jiminator@users.noreply.github.com>
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
Co-authored-by: Wookje Han <52435620+wookjeHan@users.noreply.github.com>
Co-authored-by: Wookje Han <wookjeh@nvl72d079-T17.cm.cluster>
Co-authored-by: Wookje Han <wookjeh@nvl72d094-T17.cm.cluster>
Co-authored-by: Wookje Han <wookjeh@nvl72d157-T16.cm.cluster>
Co-authored-by: Wookje Han <wookjeh@nvl72d170-T04.cm.cluster>
Co-authored-by: Wookje Han <wookjeh@nvl72d214-T17.cm.cluster>
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: Yuhao Yang <yhyang201@users.noreply.github.com>
Co-authored-by: Zijie Chen <300606707+zijiecode@users.noreply.github.com>
Co-authored-by: ethche <ethche@gmail.com>
Co-authored-by: jacky.cheng <yichiche@amd.com>
Co-authored-by: jianan-gu <jianan.gu@intel.com>
2026-10-18 01:00:24 -07:00
Yuwei An 3a6e1fcf77 [Perf] DSv4.1: Bound encoder BCG metadata memory (#44963) 2026-10-18 00:33:35 -07:00
Richard WangandMick Qian de61ab53d4 [Perf] diffusion: Defer full GC collections while importing the runtime (#44606)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-10-18 15:31:10 +08:00
Zhaoyi Li f1cf2e1005 [Fix] [AMD] mori: Look up elided DSA state entries by the paired dst index (#44412) 2026-10-18 00:29:22 -07:00
Shuwen Wang ba3fc9441d [Fix] mamba: Release consumed lazy Mamba prefix locks before donation retry (#43559) 2026-10-18 15:16:31 +08:00
jacky.chengandMick Qian f96a7326e6 [Fix] [AMD] diffusion: Prevent ComponentResidencyError after compile warmup (#43952)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-10-18 00:13:25 -07:00
Shuwen Wang a11fa06be7 [Fix] hicache: Key MLA Mamba state per TP rank in the file backend (#41907) 2026-10-18 15:10:50 +08:00
Shuwen Wang ffc4573c98 [Fix] hicache: Key MLA Mamba state per TP rank in the SeaweedFS backend (#43816) 2026-10-18 15:09:58 +08:00
Shuwen Wang e6c2cd47a9 [Fix] hicache: Keep unified memory off the direct page_first_direct copies (#43333) 2026-10-18 15:09:26 +08:00
Liangsheng Yin df966f2345 [Feat] deepseek-v4.1: Run encoder SWA bounded replay on PD prefill workers (#44952) 2026-10-18 00:07:55 -07:00
Ziang Li 520df8a491 [Perf] weight-sync: Allocate packed receive buffers from metadata (#39264) 2026-10-18 00:04:10 -07:00
Michael 00f6cdcd1b [Fix] [AMD] kda: Reject match_cutedsl_decode on ROCm and skip its parity tests (#44089) 2026-10-18 00:01:40 -07:00
Michael 3626978a13 [Fix] [AMD] kda: Match the fused verify Q/K norm to its reference kernel (#43991) 2026-10-18 00:01:11 -07:00
Bowen Wang 3671503e44 [Feat] bench: report goodput and SLO attainment in bench_serving (#43967) 2026-10-18 13:31:48 +08:00
Liangsheng Yin d20d010574 [Fix] deepseek-v4.1: Exchange late-layer DP row counts before the forward (#44948) 2026-10-17 21:46:46 -07:00
140c57d5bf [Perf] diffusion: classify residual_gate_add operands once per call (#44478)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 11:51:52 +08:00
Liangsheng YinandVincent Zhong ff73d63121 [Perf] deepseek-v4.1: Run encoder SWA bounded replay in the breakable prefill graph (#44930)
Co-authored-by: Vincent Zhong <207368749+vincentzed@users.noreply.github.com>
2026-10-17 20:51:18 -07:00
Chunan Zeng a0b77db84d [Perf] DFLASH: build the draft num_token_non_padded scalar with torch.full (#44119) 2026-10-17 20:40:16 -07:00
Richard Wang 01f3cba70e [Fix] dLLM: Size decode graphs to the default max running requests (#40941) 2026-10-17 20:22:39 -07:00
531b426281 [Feat] diffusion/model: support Cosmos Dreams and Cosmos Multiview (#39475) (#39475)
Co-authored-by: Kedi Wu <kediw@nvidia.com>
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-10-18 11:15:57 +08:00
14ad64c411 [Perf] custom_op: call register_custom_op ops directly in plain eager execution (#44158)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 10:10:10 +08:00
MickandClaude Opus 5 d748bede96 [Feat] diffusion: selectable sequence-parallel strategies for LingBot causal attention (#34385)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-10-18 10:03:43 +08:00
8d02b69c11 [Perf] diffusion: Replay Triton launches without the runner's per-launch hook metadata (#44471)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 09:58:54 +08:00
949e1a2080 [Perf] diffusion: Cache the fused linear-GELU verdict per linear (#44643)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 09:55:29 +08:00
f53bd436c1 [Perf] distributed: cache the eager launch of custom all-reduce v2 per size (#44642)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 09:54:58 +08:00
Yuwei An b234b5c32e [Refactor] TCPCG: Consolidate attention eager regions and isolate Inkling policy (4/9) (#42288) 2026-10-17 16:06:24 -07:00
Yuwei An e4512a5a17 [Perf] DSv4.1: Reduce BCG metadata memory (#44920) 2026-10-17 16:05:28 -07:00
Liangsheng Yin 563cd60345 [Fix] [AMD] deepseek-v4.1: Reject attention DP with encoder SWA bounded replay on ROCm (#44934) 2026-10-17 15:47:41 -07:00
Yi (Vincent) Zhong 05ce0e980f [Perf] deepseek-v4.1: Skip wasted layer 20 work on earlier prompt tokens under decoder bounded replay (#42901) 2026-10-17 14:12:00 -07:00
05c1539098 [Feat] unified_cache: Support hybrid SWA + Mamba models in the Mooncake direct linker (#41343)
Co-authored-by: Zhangheng <hzh0425@apache.org>
Co-authored-by: Teng Ma <sima.mt@alibaba-inc.com>
2026-10-17 13:24:08 -07:00