Commit Graph
13989 Commits
Author SHA1 Message Date
Thomas Wang d6853f5066 [AMD] Add flydsl mxfp8 dense gemm kernel for dpsk v4.1 prefill (#43803) 2026-10-12 14:42:02 -07:00
Zhang, JiejingandJohnQinAMD 6966c5d027 [ROCm] Eliminate HiCache migration stalls and use exact-size host allocations (#42553)
Signed-off-by: JohnQinAMD <yanyuan.qin@amd.com>
Co-authored-by: JohnQinAMD <yanyuan.qin@amd.com>
2026-10-12 14:05:51 -07:00
ix-48 b60444c621 Fix autograd retention in CUDA graph capture tail hooks (#43854) 2026-10-12 12:31:18 -07:00
Liangsheng Yin 0efbb3d2b9 [DFlash] Decide the Domino candidate pool in one place and apply simulated acceptance inside the accept (#43791) 2026-10-12 11:35:57 -07:00
Liangsheng Yin 392039d752 [DSpark] Share draft params, logits dtype, confidence and the accept tail between graph and eager (#43790) 2026-10-12 11:35:41 -07:00
Shuwen Wang f92199b569 [Unified Cache] Abort requests whose external linker load fails instead of crashing the scheduler (#40896) 2026-10-13 00:03:21 +08:00
Sam ShleiferandClaude Fable 5 d88d5c325a Prebuild fixup_zero_kv JIT module at TRTLLM-MLA backend init (#37525)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-10-12 23:27:21 +08:00
Yuan Luoandluoyuan.luo 23a1f53fc3 [QSA] Bound the decode indexer's work by the context length (#43463)
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
2026-10-12 22:47:45 +08:00
279f1fad90 [diffusion] chore: bind worker CPUs to the GPU's NUMA node when memory policy is not permitted (#43756)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 21:09:41 +08:00
Артем Савкин d1357bf87b [NPU] Diffusion CI fixes (#43210) 2026-10-12 14:53:51 +03:00
ming_wang c6e822c29b [Fix] Adapt NPU Qwen-VL processor patch to transformers 5.19 resize() signature change (#43309) 2026-10-12 19:24:05 +08:00
Sasha SidorovandShangming Cai 7953751bfa [PD][Spec] Give a PD decode's first EAGLE draft step a proposal distribution (#41783)
Co-authored-by: Shangming Cai <csmthu@gmail.com>
2026-10-12 18:49:52 +08:00
8afe107c24 [XPU] Upgrade XPU backend to PyTorch 2.14 (#42003)
Co-authored-by: Ashwini Rathi <arathi@habana.ai>
Co-authored-by: Y <ashwini.rathi@intel.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-10-12 18:24:48 +08:00
CandCheng Wan 28621739fc [Fix] Make tokenizer request-state initialization atomic (#41911)
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
2026-10-12 03:10:57 -07:00
BingjiaWang 62ff466ac0 [DSV4.1][PD][1/N] Support prefill node use cp parallel strategy (#40323) 2026-10-12 01:05:21 -07:00
BingjiaWang 54409c2fa0 [BCG][DSV4-Flash][CP] Support DeepSeek V4 prefill with interleave CP and breakable CUDA graphs (#38603) 2026-10-12 01:03:10 -07:00
Anupa SajikumarandSiju Samuel 8d491fbc3e [XPU] Support decode context parallelism (DCP) on Intel XPU (#34355)
Co-authored-by: Siju Samuel <siju.samuel@intel.com>
2026-10-12 15:50:33 +08:00
Ke Bao a70a83bcd2 Keep NIXL Mamba checkpoint keys separate by TP rank (#43594) 2026-10-12 15:40:59 +08:00
Ke Bao b6ba148384 Capture KV host transfer buffers (#43711) 2026-10-12 15:40:06 +08:00
5ca0695aa6 [HiCache] Preserve all MTP KV and recurrent state (#40223)
Co-authored-by: alphabetc1 <2508695655@qq.com>
Co-authored-by: Shuwen Wang <47200617+alphabetc1@users.noreply.github.com>
2026-10-12 15:37:59 +08:00
Shuwen Wang f5f5003323 [HiCache] Support Mamba in buffer-only host memory mode (#36345) 2026-10-12 00:35:16 -07:00
Shangming CaiandClaude Fable 5.1 4195980f5a [PD] Poison the deferred abort ack when a Mooncake send fails (#43719)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-12 15:33:09 +08:00
da4bf1ef8e [Perf] Route BF16 mixed-batch decode tails through decode attention (#43238)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
2026-10-12 15:27:22 +08:00
Xinyuan Tong a0e6effe50 [Fix] Link cuda_ipc JIT against CUDA driver stubs dir (#43764) 2026-10-12 15:26:32 +08:00
208df292e5 [diffusion][xpu] FLUX.2: enable the bit-exact Triton SwiGLU on XPU (#41940)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-10-12 15:16:58 +08:00
Mohammad Miadh AngkadandMohammad Angkad 166b4ae7fc [Fix] Adapt to transformers 5.18 / 5.19 changes (#43688)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-10-11 23:24:57 -07:00
ashwini rathi 8ce76b88df [XPU CI] Optimize PR CI per-file timeouts and fail hanging tests fast (#42048) 2026-10-12 14:24:26 +08:00
Mohammad Miadh AngkadandMohammad Angkad 130e11d50e [Deps] Bump FlashInfer to 0.7.1 (#43494)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-10-11 23:23:11 -07:00
Liangsheng Yin 2240049194 [RL] Return a failed weight update when the cache flush is refused (#43758) 2026-10-11 21:57:58 -07:00
a297cd3da1 [Fix] scheduler: stop latching batch_is_full when nothing can run (#41872)
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
2026-10-11 21:56:19 -07:00
d434ea444d [DSV4.1] Reduce eager Engram extend preparation (#42159)
Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 21:31:55 -07:00
Liangsheng Yin a1001bcf3a [PD] Retract transferred decode requests on a retract pause (#43757) 2026-10-11 21:29:39 -07:00
ChengYM 0d0be8fc98 [AMD][MORI] Add MORI EP V2 support (#40666) 2026-10-11 20:54:13 -07:00
Shunkangz 20c49f12c9 Support kv cache sharding for DSA indexer (#40925) 2026-10-12 11:50:23 +08:00
37d2e5c34f [AMD][V4.1][*/N] Fix DP attention + TP MoE decode NaN and support bounded replay (#43077)
Co-authored-by: wunhuang <wunhuang@amd.com>
Co-authored-by: HAI <hixiao@gmail.com>
2026-10-11 20:47:13 -07:00
weireweire 296774da37 [Tokenizer] Require fastokens>=0.3.2 and include it in the all extra (#43104) 2026-10-11 20:41:53 -07:00
adenzhou1350 cd8ed1bb55 [Bugfix] Preserve available host memory for integrated MUSA devices (#43398)
Signed-off-by: Xucheng Zhou <aden1350@outlook.com>
2026-10-11 20:34:58 -07:00
77a15cf4e2 [MemCache] Add shared component size and path APIs to the tree core interface (#43722)
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: Jialin <Jialin@users.noreply.github.com>
2026-10-11 20:14:07 -07:00
e4cfc0c42a [MemCache] Split the SWA mapping move out of locked-full recovery as RecoverEvictedSWA (#43721)
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: Jialin <Jialin@users.noreply.github.com>
2026-10-11 20:11:52 -07:00
WangHemeng 0c52cb1519 [MoE] Skip block-padding rows in the TMA down activation and quant (#42212) 2026-10-12 11:10:30 +08:00
Alison Shao 1a7af3290d Fix Mistral-native format detection when the Hub is unreachable (#43446) 2026-10-11 19:59:19 -07:00
Chunan Zeng 3c74f68a9c [Fix] Initialize BF16 GEMM dispatch in the one-batch benchmark (#43702) 2026-10-12 02:53:02 +00:00
WenhaoZhangandMick Qian d0a098b2e0 [diffusion] perf: split and batch LTX-2.3 one-stage guidance passes under CFG parallel (#43684)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-10-12 09:36:01 +08:00
WenhaoZhang a954b395e6 [diffusion] perf: layerwise offload for the LTX-2.5 Gemma-4 text encoder (#43683) 2026-10-12 09:34:35 +08:00
WenhaoZhang a67228aa43 [diffusion] fix: shard Qwen-Image adaLN modulation and text FFN under TP (#43691) 2026-10-12 09:32:52 +08:00
Liangsheng Yin e6072935bf [DSpark] Apply SGLANG_SIMULATE_ACC_LEN inside the accept instead of switching paths (#43623) 2026-10-11 15:33:24 -07:00
Liangsheng Yin a65b776963 [DSpark] Add SGLANG_DSPARK_USE_GREEDY_DRAFT, an argmax draft accepted as one-hot (#43670) 2026-10-11 15:18:59 -07:00
Liangsheng Yin 3cc23b3b86 [DSpark] Merge the draft TP sync sites into dspark-draft (#43669) 2026-10-11 15:17:47 -07:00
Liangsheng Yin 00ba90fd1a [DSpark] Run the same markov block in the draft graph tail; drop SGLANG_DSPARK_FOLDED_SAMPLING (#43668) 2026-10-11 15:15:59 -07:00
Liangsheng Yin 48abbdb845 [DSpark] Draft the eager markov block through MarkovSampleStep (#43622) 2026-10-11 15:13:02 -07:00