Shuwen Wang
a11fa06be7
[Fix] hicache: Key MLA Mamba state per TP rank in the file backend ( #41907 )
2026-10-18 15:10:50 +08:00
Shuwen Wang
ffc4573c98
[Fix] hicache: Key MLA Mamba state per TP rank in the SeaweedFS backend ( #43816 )
2026-10-18 15:09:58 +08:00
Shuwen Wang
e6c2cd47a9
[Fix] hicache: Keep unified memory off the direct page_first_direct copies ( #43333 )
2026-10-18 15:09:26 +08:00
Liangsheng Yin
df966f2345
[Feat] deepseek-v4.1: Run encoder SWA bounded replay on PD prefill workers ( #44952 )
2026-10-18 00:07:55 -07:00
Ziang Li
520df8a491
[Perf] weight-sync: Allocate packed receive buffers from metadata ( #39264 )
2026-10-18 00:04:10 -07:00
Michael
00f6cdcd1b
[Fix] [AMD] kda: Reject match_cutedsl_decode on ROCm and skip its parity tests ( #44089 )
2026-10-18 00:01:40 -07:00
Michael
3626978a13
[Fix] [AMD] kda: Match the fused verify Q/K norm to its reference kernel ( #43991 )
2026-10-18 00:01:11 -07:00
Bowen Wang
3671503e44
[Feat] bench: report goodput and SLO attainment in bench_serving ( #43967 )
2026-10-18 13:31:48 +08:00
Liangsheng Yin
d20d010574
[Fix] deepseek-v4.1: Exchange late-layer DP row counts before the forward ( #44948 )
2026-10-17 21:46:46 -07:00
140c57d5bf
[Perf] diffusion: classify residual_gate_add operands once per call ( #44478 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-18 11:51:52 +08:00
Liangsheng Yin and Vincent Zhong
ff73d63121
[Perf] deepseek-v4.1: Run encoder SWA bounded replay in the breakable prefill graph ( #44930 )
...
Co-authored-by: Vincent Zhong <207368749+vincentzed@users.noreply.github.com >
2026-10-17 20:51:18 -07:00
Chunan Zeng
a0b77db84d
[Perf] DFLASH: build the draft num_token_non_padded scalar with torch.full ( #44119 )
2026-10-17 20:40:16 -07:00
Richard Wang
01f3cba70e
[Fix] dLLM: Size decode graphs to the default max running requests ( #40941 )
2026-10-17 20:22:39 -07:00
531b426281
[Feat] diffusion/model: support Cosmos Dreams and Cosmos Multiview ( #39475 ) ( #39475 )
...
Co-authored-by: Kedi Wu <kediw@nvidia.com >
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-18 11:15:57 +08:00
14ad64c411
[Perf] custom_op: call register_custom_op ops directly in plain eager execution ( #44158 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-18 10:10:10 +08:00
Mick and Claude Opus 5
d748bede96
[Feat] diffusion: selectable sequence-parallel strategies for LingBot causal attention ( #34385 )
...
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-10-18 10:03:43 +08:00
8d02b69c11
[Perf] diffusion: Replay Triton launches without the runner's per-launch hook metadata ( #44471 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-18 09:58:54 +08:00
949e1a2080
[Perf] diffusion: Cache the fused linear-GELU verdict per linear ( #44643 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-18 09:55:29 +08:00
f53bd436c1
[Perf] distributed: cache the eager launch of custom all-reduce v2 per size ( #44642 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-18 09:54:58 +08:00
Yuwei An
b234b5c32e
[Refactor] TCPCG: Consolidate attention eager regions and isolate Inkling policy (4/9) ( #42288 )
2026-10-17 16:06:24 -07:00
Yuwei An
e4512a5a17
[Perf] DSv4.1: Reduce BCG metadata memory ( #44920 )
2026-10-17 16:05:28 -07:00
Liangsheng Yin
563cd60345
[Fix] [AMD] deepseek-v4.1: Reject attention DP with encoder SWA bounded replay on ROCm ( #44934 )
2026-10-17 15:47:41 -07:00
Yi (Vincent) Zhong
05ce0e980f
[Perf] deepseek-v4.1: Skip wasted layer 20 work on earlier prompt tokens under decoder bounded replay ( #42901 )
2026-10-17 14:12:00 -07:00
05c1539098
[Feat] unified_cache: Support hybrid SWA + Mamba models in the Mooncake direct linker ( #41343 )
...
Co-authored-by: Zhangheng <hzh0425@apache.org >
Co-authored-by: Teng Ma <sima.mt@alibaba-inc.com >
2026-10-17 13:24:08 -07:00
YAMY and Claude Opus 5.5
3b34d9f8b6
[Perf] [NVIDIA] qsa: Keep sparse decode scratch in FP8 for FP8 KV pools ( #44114 )
...
With an FP8 KV pool on SM10x, keep the packed sparse-decode K/V scratch in FP8 instead of dequantizing to BF16 (BF16-q + FP8-KV TRTLLM-gen decode). Tests cover FP8 pool -> FP8 scratch.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-17 11:50:38 -07:00
4eb606b2ae
[Feat] diffusion: support opt-in pi0.5 completion-boundary action batching ( #44634 )
...
Co-authored-by: Jiahua Huang <35223412+dadiaokua@users.noreply.github.com >
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-17 20:49:44 +08:00
b15a78a881
[Fix] diffusion/lora: revalidate FLUX.2 FP8/NVFP4 fast paths after LoRA attach ( #40171 )
...
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Mick <mickjagger19@icloud.com >
2026-10-17 20:48:14 +08:00
23f8e6f7ec
[Fix] diffusion: keep ZMQ ingress sockets on their configured port ( #43732 )
...
Co-authored-by: Sylvan Liu <syliu@cse-ai-16.amd.com >
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Mick <mickjagger19@icloud.com >
2026-10-17 20:47:29 +08:00
+3
d4191a4624
[Fix] diffusion: stabilize FLUX.3 serving and reuse static conditioning ( #44493 )
...
Co-authored-by: Yihao Wang <42559837+AgainstEntropy@users.noreply.github.com >
Co-authored-by: Dom Brown <3886319+DomBrown@users.noreply.github.com >
Co-authored-by: niehen6174 <1639206518@qq.com >
Co-authored-by: Mick Qian <mickqian@radixark.ai >
Co-authored-by: Nikita Korobov <14355239+nekorobov@users.noreply.github.com >
Co-authored-by: Daniel Campora <961215+dcampora@users.noreply.github.com >
Co-authored-by: Richard Wang <11150595+rwang5203@users.noreply.github.com >
Co-authored-by: BBuf <1182563586@qq.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Fabio Camaiora <fcamaiora@nvidia.com >
Co-authored-by: Linda-Stadter <lstadter@nvidia.com >
2026-10-17 20:46:13 +08:00
9771e5a4f0
[Feat] diffusion: Per-task prompt enhancer config and the FLUX.3 Klein upsampler protocol ( #44662 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-17 20:45:07 +08:00
Mick and Mick Qian
e01dfd3cbc
[Refactor] multimodal: Replace padding strategies with helpers ( #44845 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-17 20:43:02 +08:00
Shuwen Wang
86e67f7d93
[Fix] unified_cache: Prevent misaligned Mamba checkpoints after speculative truncation ( #43387 )
2026-10-17 20:20:49 +08:00
SuperSong
025655068c
[Fix] unified-memory: Avoid gather scratch during Mamba and Conv compaction ( #44097 )
2026-10-17 04:56:41 -07:00
SuperSong
ee30dc4d40
[Fix] unified-memory: Bound joint reclaim and preserve cached prefixes ( #39294 )
...
Bound joint FULL/SWA/Mamba reclaim so feasible allocations recover capacity without evicting unnecessary cached prefixes, and impossible demands preserve the cache. Use exact unsharded paged-extend demand and update allocation regression fixtures.
2026-10-17 04:46:48 -07:00
Shuwen Wang
ae28306a74
[Fix] cuda_graph: Wait for in-flight layer-wise cache loads before prefill graph replay ( #44220 )
2026-10-17 19:13:57 +08:00
Shuwen Wang and Zhangheng
5fcb07da34
[Fix] hicache: Key MLA Mamba state per TP rank in the HF3FS backend ( #43817 )
...
Co-authored-by: Zhangheng <hzh0425@apache.org >
2026-10-17 11:08:17 +00:00
Woody-Hu and Mick
465ddf9758
[Feat] diffusion/VSA-H3: support sm80 (A100) with a raw-pointer Triton load path ( #43823 )
...
Co-authored-by: Mick <mickjagger19@icloud.com >
2026-10-17 18:47:19 +08:00
400c1dfe6e
[Perf] distributed: skip the DLPack re-wrap of custom all-reduce results ( #44251 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-17 18:44:31 +08:00
Khoa Pham and Claude Fable 5.1
a126e8d640
[Perf] deepseek_v4: fold the MegaAttention query layout into wq_b for block-FP8-as-MXFP8 checkpoints ( #44838 )
...
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-10-17 02:45:53 -07:00
Mick and Mick Qian
80f40522cd
[UX] diffusion: log LoRA merge decisions only on first evaluation ( #44376 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-17 16:47:07 +08:00
56a20d7e34
[Perf] kernels: Enter a JIT kernel's device guard only when the device differs ( #44644 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-17 16:43:27 +08:00
f75232cb42
[Fix] hicache: Order write-back host copies after the forward that produced the KV ( #44800 )
...
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com >
Co-authored-by: xiezhq-hermann <xiezhq-hermann@users.noreply.github.com >
2026-10-17 01:31:40 -07:00
0d13efb2b8
[Feat] rust-server: Serve /v1/tokenize and /v1/detokenize from the Rust frontend ( #44767 )
...
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com >
Co-authored-by: shodoco <shodoco@users.noreply.github.com >
2026-10-17 01:30:50 -07:00
a2953d7b7f
[Fix] deepseek-v4.1: Serve prompt-logprob requests under decoder SWA bounded replay ( #44850 )
...
Co-authored-by: Pedram Ghazi <pedram.ghazi@amd.com >
Co-authored-by: David Orman <ormandj@corenode.com >
2026-10-17 01:25:33 -07:00
McZyWu
7a8804edb5
[Fix] [NPU] qwen: Fix shared-expert routing with Ascend FuseEP ( #40516 )
2026-10-17 16:19:44 +08:00
Liangsheng Yin
ef26f6ad49
[Fix] deepseek-v4.1: Keep the cached KV of folded encoder replay rows ( #44857 )
2026-10-17 00:56:18 -07:00
Khoa Pham and Claude Fable 5.1
bce7b27aa6
[Perf] deepseek_v4: run FlashInfer autotune for block-FP8 models served as MXFP8 ( #44842 )
...
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-10-17 00:19:35 -07:00
01dc08e23f
[Refactor] dsv41: Shard Engram tables on hash-column boundaries ( #42787 )
...
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
Co-authored-by: hnyls2002 <lsyincs@gmail.com >
2026-10-17 00:15:49 -07:00
Khoa Pham and Claude Fable 5.1
14c4dda24e
[Perf] deepseek_v4: fuse block-FP8 shared experts into MegaMoE for any 32-aligned block size ( #44837 )
...
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-10-16 23:47:41 -07:00
Liangsheng Yin
9259553f09
[Fix] megamoe: Support EPLB rebalance on SM100 ( #44847 )
2026-10-16 22:22:40 -07:00