 
|
eb9c9ee99d
|
[Qwen4-Exp] Optional fp8 (e4m3) storage for the compressed QSA indexer cache (#39614)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: YAMY1234 <74099316+YAMY1234@users.noreply.github.com>
|
2026-09-29 22:11:46 -07:00 |
|
Cheng Wan
|
5781206833
|
[PD] Give FakeKVReceiver ensure_abort_notified (#41618)
|
2026-09-28 21:59:17 -07:00 |
|
  
|
7889fbab9a
|
[PD] Keep EAGLE DP graph and token metadata consistent (#32196)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
Co-authored-by: Khoa Pham <khoa.pham@radixark.ai>
|
2026-09-28 17:15:51 -07:00 |
|
Nan Jiang
|
6fa3fe69e2
|
[PD] Keep the sampling mask of a replayed rebootstrap token (#41235)
|
2026-09-28 12:02:27 -07:00 |
|
 Shangming CaiandClaude Fable 5
|
2a1c477648
|
[PD] Defer decode KV release on every transfer failure, not only decode-initiated aborts (#41404)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-09-29 02:19:51 +08:00 |
|
 Shangming CaiandClaude Fable 5
|
14ab74ef95
|
[PD] Fan drain abort ACKs out to every decode peer of the room (#41402)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-09-28 05:16:23 +08:00 |
|
Liangsheng Yin
|
d27efca353
|
[mem_cache] Never free the protected prefix on request release (#41312)
|
2026-09-26 22:00:02 -07:00 |
|
Xiaoyu Zhang
|
38ec649048
|
[Refactor] Restore logical kernel groups and test organization (#41243)
|
2026-09-26 12:30:44 +08:00 |
|
Liangsheng Yin
|
f35ec18d8b
|
[Test] Remove more unit tests that mirror implementation or never run in CI (#41297)
|
2026-09-25 19:22:14 -07:00 |
|
 Liangsheng YinandBBuf
|
efd9a40bb8
|
[PD] Share one head-slice helper across mooncake, mori, and nixl (#39660)
Co-authored-by: BBuf <1182563586@qq.com>
|
2026-09-25 16:13:18 -07:00 |
|
 
|
cf5df82680
|
[KVCache] Support lmcache unified radix cache (#38652)
Signed-off-by: chunxiaozheng <1179548172@qq.com>
Co-authored-by: Yuwei An <ayw.sirius19@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
2026-09-25 14:08:45 -07:00 |
|
Nan Jiang
|
24b6930076
|
[Sampling] Stream sampling masks as per-request arrays (#40986)
|
2026-09-25 12:20:25 -07:00 |
|
   
|
78b382b1ad
|
Support unified memory decode host pools (#39478)
Co-authored-by: yhzhuang <yhzhuang@fb.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: Yonghao Zhuang <yhzhuang@users.noreply.github.com>
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
|
2026-09-24 11:02:15 -07:00 |
|
 Shangming CaiandClaude Opus 5
|
cd11037009
|
[PD] Enable deferred decode-side KV release by default (#41023)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-24 22:35:11 +08:00 |
|
 cctryandcctry
|
efff836911
|
[PD] Add decode host receive for custom transfer backends (#40238)
Co-authored-by: cctry <cctry@fb.com>
|
2026-09-23 21:57:33 -07:00 |
|
Nan Jiang
|
c3685dff42
|
[Sampling] Add selected/support sampling logprob modes (#40932)
|
2026-09-23 21:38:19 -07:00 |
|
 YAMYandzhangxiaolei
|
6fd98c98b9
|
[Qwen3.8-Next] Pipeline-parallel serving and PD-prefill MTP for Qwen4-Exp (#40501)
Co-authored-by: zhangxiaolei <zhangxiaolei.666@bytedance.com>
|
2026-09-22 08:46:44 -07:00 |
|
cctry
|
8ef6d31cb9
|
[PD] Preserve abort ACKs until in-flight KV transfers drain (#40645)
|
2026-09-22 18:46:07 +08:00 |
|
Po-Han Huang (NVIDIA)
|
771c9d782d
|
[PD] Validate Mooncake EFA allocator compatibility (#39973)
|
2026-09-22 15:12:29 +08:00 |
|
 Guangda LiuandGuangda Liu
|
04c0913434
|
[HiSparse] Add MHA hisparse support for MiniMax M3 (#31446)
Co-authored-by: Guangda Liu <bingps@users.noreply.github.com>
|
2026-09-22 13:28:03 +08:00 |
|
 Khoa PhamandQiaolin Yu
|
018b73c7a0
|
[PD] Pack draft KV head slices for DCP transfers (#40500)
Co-authored-by: Qiaolin Yu <liin1211@outlook.com>
|
2026-09-21 21:11:27 -07:00 |
|
Cheng Wan
|
506698761d
|
[unified-memory] Hierarchical cache for every unified pool shape (#37507)
|
2026-09-21 16:50:37 -07:00 |
|
Cheng Wan
|
acac4dd9d9
|
[Refactor] Clean up parallel runtime comments (#40632)
|
2026-09-21 14:32:22 -07:00 |
|
Cheng Wan
|
970e946e4f
|
Retire the per-runner parallel record (#40343)
|
2026-09-21 12:26:40 -07:00 |
|
Cheng Wan
|
73f071db52
|
Deprecate the parallel getters the context answers, and ratchet them shut (#40342)
|
2026-09-21 12:25:32 -07:00 |
|
kpjeeja
|
b54d5b7c7b
|
disaggregation: Fix FakeKVSender queue accumulation (#28652)
Signed-off-by: KP, Jeeja <jeeja.kp@intel.com>
|
2026-09-21 14:27:05 +08:00 |
|
Khoa Pham
|
b3e4d198af
|
[PD] Bound cached-prefix DCP transfers by pack capacity (#40376)
|
2026-09-21 01:17:15 +08:00 |
|
Qiaolin Yu
|
f4c256354c
|
[kimi k3][pd disagg] support pp prefill + dcp decode with dspark (#40045)
|
2026-09-20 00:15:28 -07:00 |
|
  
|
3a64faa1f2
|
Fix disagg PP MTP for GLM-5.2 (#39378)
Co-authored-by: Julien Lin <jullin@nvidia.com>
Co-authored-by: YAMY1234 <74099316+YAMY1234@users.noreply.github.com>
Co-authored-by: Yangmin Li <yangminl@nvidia.com>
|
2026-09-19 13:55:07 -07:00 |
|
 
|
8189e3896b
|
[PD] Skip singleton transfer-status all-reduces (#40003)
Co-authored-by: Pranjal Shankhdhar <pranjalssh@users.noreply.github.com>
Co-authored-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
|
2026-09-18 22:41:50 -07:00 |
|
 
|
5e9342d16f
|
[PP][DeepSeek V4] Overlap communication and optimize SM120 prefill (#38792)
Co-authored-by: Yangmin Li <yangminl@nvidia.com>
Co-authored-by: YAMY <74099316+YAMY1234@users.noreply.github.com>
|
2026-09-18 21:18:04 -07:00 |
|
Cheng Wan
|
afe71f4b9e
|
Read process groups through the runtime context (#40068)
|
2026-09-18 17:40:32 -07:00 |
|
   
|
5931fd60ee
|
Support unified memory page-envelope transfers in PD (#39477)
Co-authored-by: yhzhuang <yhzhuang@fb.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: Yonghao Zhuang <yhzhuang@users.noreply.github.com>
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
|
2026-09-18 17:39:50 -07:00 |
|
Liangsheng Yin
|
6cc9090d1f
|
[mem_cache] Release up to owned_kv_len on radix cache insert (#40075)
|
2026-09-18 15:37:52 -07:00 |
|
       
|
a6cf05817f
|
dsv4.1: remaining model and runtime integration (#38798)
Co-authored-by: BBuf <1182563586@qq.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Xiaoyu Zhang <xiaoyu.zhang@radixark.ai>
Co-authored-by: Yuwei An <ayw.sirius19@gmail.com>
Co-authored-by: Khoa Pham <khoa.pham@radixark.ai>
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: Zhichen Zeng <zczeng@uw.edu>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
|
2026-09-18 02:55:30 -07:00 |
|
iridiumine
|
f86f60081d
|
[NPU] Adapt hicache for K3 hybrid models (#39415)
|
2026-09-18 14:48:20 +08:00 |
|
Liangsheng Yin
|
1f0c73e9bd
|
[DSV4] Generalize attention metadata, sparse prefill, and KV pool over compress ratios (#39921)
|
2026-09-17 15:55:19 -07:00 |
|
  
|
1f60ddef5d
|
[PD] Introduce runtime role switching between prefill and decode (#28403)
Signed-off-by: huanglong <huanglong@linux.alibaba.com>
Signed-off-by: inkcherry <mingzhi.liu@amd.com>
Co-authored-by: huanglong <huanglong@linux.alibaba.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Huang Long <121648372+LLLL114@users.noreply.github.com>
|
2026-09-18 01:45:12 +08:00 |
|
 Aurick QiaoandAurick Qiao
|
7b7620774c
|
Fix device context during NIXL backend initialization (#38774)
Co-authored-by: Aurick Qiao <6137920+aurickq@users.noreply.github.com>
|
2026-09-16 17:59:24 +08:00 |
|
 Shangming CaiandClaude Opus 5
|
3f8eb35ead
|
[PD] Do not admit intake-rejected requests to a PD handoff (#38935)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 12:44:49 +08:00 |
|
Jinyan Yi
|
dbd7281b06
|
fix(npu): fix hybrid KV transfer with PP prefill in PD disaggregation (#38402)
|
2026-09-16 09:10:17 +08:00 |
|
 
|
2929a39927
|
Use a shared byte budget for unified hybrid-SWA memory (#36729)
Co-authored-by: yhzhuang <yhzhuang@fb.com>
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
|
2026-09-15 15:27:00 -07:00 |
|
 Siyuan ChenandBJWang-ant
|
832ec39cc0
|
[Intra-node PD][DSV4] Pack all layers into one batch for INTRA_NODE_NVLINK path (#38984)
Co-authored-by: BJWang-ant <wangbingjia.wbj@ant-intl.com>
|
2026-09-15 21:08:33 +08:00 |
|
Yanbin Jiang
|
ebd37705e4
|
[PD][LoRA] Gate decode admission on adapter slots (#39332)
|
2026-09-15 13:11:04 +08:00 |
|
ishandhanani
|
3f871a246c
|
feat(agent sessions): attribute stored KV cache blocks to sessions (#37482)
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
|
2026-09-14 21:34:49 -07:00 |
|
cctry
|
dad8c074e7
|
Scope prefetch cache state to the request attempt (#39318)
|
2026-09-14 15:03:03 -07:00 |
|
 
|
4c71d14fba
|
[HiCache][LoRA] Simplify decode offload hash inputs (#39162)
Co-authored-by: Shuwen Wang <47200617+alphabetc1@users.noreply.github.com>
Co-authored-by: alphabetc1 <2508695655@qq.com>
|
2026-09-14 11:23:31 +08:00 |
|
Cheng Wan
|
2ec4bbcbd4
|
[unified-memory] PD disaggregation for every unified pool shape (#37506)
|
2026-09-13 19:16:43 -07:00 |
|
 Yanbin JiangandShuwen Wang
|
0b415fa573
|
[HiCache][LoRA] Isolate storage pages by extra key (#38577)
Co-authored-by: Shuwen Wang <47200617+alphabetc1@users.noreply.github.com>
|
2026-09-12 18:18:14 +08:00 |
|
YAMY
|
55e5e21c88
|
[Qwen3.8-Next] Add PD state transfer for Flash Next (#36651)
|
2026-09-11 22:31:45 -07:00 |
|