|
23ab10a63e
|
Support speculative decoding with unified SWA memory (#36403)
Co-authored-by: yhzhuang <yhzhuang@fb.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
|
2026-09-03 10:44:57 -07:00 |
|
|
|
2bb25dc18b
|
[Speculative Decoding] Add native UNO serving support (#37667)
Co-authored-by: drproduck <drproduck@MacBook-Air-2.local>
Co-authored-by: BBuf <1182563586@qq.com>
|
2026-09-03 20:08:41 +08:00 |
|
|
Alex NailsandAlison Shao
|
28262c20df
|
[CI][RFC] Replace black-jupyter with ruff-format (#37210)
Co-authored-by: Alison Shao <a.shao@wustl.edu>
|
2026-09-02 19:46:08 -07:00 |
|
|
|
87d60a2229
|
Improve CUDA graph and speculative execution output handling (#37329)
Co-authored-by: jiayisuse <jiayisuse@fb.com>
Co-authored-by: Yinghai Lu <yinghai@meta.com>
Co-authored-by: Hao Zhang <zhisbug@users.noreply.github.com>
Co-authored-by: Yichao Fu <yichaofu@meta.com>
|
2026-09-02 17:25:27 -07:00 |
|
|
YAMY
|
3c9cea8f10
|
[EAGLE] Prune draft-extend logits to selected rows (#35546)
|
2026-09-02 15:10:08 -07:00 |
|
|
cctry
|
ad6e830858
|
[Bugfix] Key CUDA graph dedup signatures on kernel function identity (#37657)
|
2026-09-02 14:24:31 -07:00 |
|
|
|
f8cbf000f4
|
[AMD] Enable FP4 indexer for Deepseek V4 (#37353)
Co-authored-by: 1am9trash <1am9trash@gmail.com>
Co-authored-by: AMD-yanfeiwang <256076023+AMD-yanfeiwang@users.noreply.github.com>
Co-authored-by: Thomas Wang <thomawan@amd.com>
|
2026-09-02 09:45:08 -07:00 |
|
|
Yichao Fu
|
66de38f30c
|
Decouple ragged CUDA graph request and token capacities (#37300)
|
2026-09-01 18:48:11 -07:00 |
|
|
Cheng Wan
|
0b1ce3d140
|
[Feature] Unified memory: support decode context parallelism for Kimi-Linear (#36890)
|
2026-09-01 12:44:26 -07:00 |
|
|
cctryandcctry
|
9a05b470fa
|
[Memory] Size the CUDA graph pool from warmup measurements and fix graph-pool borrowing (#36911)
Co-authored-by: cctry <cctry@fb.com>
|
2026-09-01 09:32:38 -07:00 |
|
|
|
22337e9c56
|
fix(unified-memory): forward the KV-index translator through every wrapper backend (#37307)
Co-authored-by: Caihua Li <caihua.li@bytedance.com>
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
|
2026-08-31 19:55:21 -07:00 |
|
|
|
98cb3535b7
|
feat(unified-memory): byte-budget sizing, feasibility floor, and a conservation verifier (#35158)
Co-authored-by: Caihua Li <caihua.li@bytedance.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
|
2026-08-31 15:09:28 -07:00 |
|
|
|
6580d5cd9a
|
weight cache: key daemon paths by GPU UUID (#36101)
Co-authored-by: siyu <liusy58@linux.alibaba.com>
Co-authored-by: Alex Nails <alex.nails@radixark.ai>
|
2026-08-31 01:44:45 -07:00 |
|
|
+1
|
3865efc9f7
|
[AMD] support gfx1250 on ROCM 10 (#36871)
Co-authored-by: HAI <hixiao@gmail.com>
Co-authored-by: Kao <akao@amd.com>
Co-authored-by: wunhuang <wunhuang@amd.com>
Co-authored-by: Thomas Wang <1am9trash@gmail.com>
Co-authored-by: Xinyi Song <86638975+RolaoDenthu@users.noreply.github.com>
Co-authored-by: Lin, Soga <soga.lin@amd.com>
Co-authored-by: kk <43161300+kkHuang-amd@users.noreply.github.com>
Co-authored-by: Bingxu Chen <bingxche@amd.com>
Co-authored-by: sogalin_codegen <39478626+sogalin@users.noreply.github.com>
Co-authored-by: Thomas Wang <thomawan@amd.com>
|
2026-08-31 01:19:11 -07:00 |
|
|
|
8bb776dc48
|
feat(unified-memory): read unified pool from attention backends fa3/flashinfer/trtllm_mha/flashmla (#34613)
Co-authored-by: Caihua Li <caihua.li@bytedance.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
|
2026-08-30 23:58:24 -07:00 |
|
|
|
29578d5578
|
refactor(unified-memory): translate the KV write location once, at ForwardBatch construction (#35245)
Co-authored-by: Caihua Li <caihua.li@bytedance.com>
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
|
2026-08-30 23:52:14 -07:00 |
|
|
YAMYandgithub-actions[bot]
|
b77cac06a9
|
[PP] Support prefill CUDA graph proxy tensors (#36248)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
|
2026-08-30 23:11:45 -07:00 |
|
|
Aurick Qiao
|
9a9e167179
|
[Bugfix] Fix full prefill CUDA graph padding and EAGLE capture (#35588)
|
2026-08-30 21:30:24 -07:00 |
|
|
Cheng WanandClaude Opus 5
|
e51a3ae65e
|
[Config] Round 5.2: the per-model declarations get their own modules (#37087)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
|
2026-08-30 02:24:35 -07:00 |
|
|
Cheng Wan
|
7e751153eb
|
[Config] Round 5.1: the published-side readers ask the bags, and a platform fact gets one address (#37086)
|
2026-08-30 02:18:33 -07:00 |
|
|
Cheng WanandClaude Opus 5
|
4d53767b09
|
config: the lazy imports that buy nothing become eager (#36975)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
|
2026-08-29 04:21:54 -07:00 |
|
|
Cheng WanandClaude Opus 5
|
1a3e152f03
|
config: six more runtime readers ask the bags (#36973)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
|
2026-08-29 04:19:14 -07:00 |
|
|
Cheng WanandClaude Opus 5
|
b65e677e48
|
config: the resolution callbacks into the record go to zero (#36972)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
|
2026-08-29 04:18:05 -07:00 |
|
|
+8
|
5f216fc33f
|
qwen 3.8 rebase (#35758)
Co-authored-by: cherichy <cherichy@outlook.com>
Co-authored-by: guangyunh-nv <guangyunh@nvidia.com>
Co-authored-by: jiahanc <jiahanc@nvidia.com>
Co-authored-by: jinyangyuan-nvidia <joyuan@nvidia.com>
Co-authored-by: Cheng Hang <chang@nvidia.com>
Co-authored-by: Yicheng Qiang <yqiang@nvidia.com>
Co-authored-by: Sam Li <lsam@nvidia.com>
Co-authored-by: Tom-Zheng <tizheng@nvidia.com>
Co-authored-by: Yangmin Li <yangminl@nvidia.com>
Co-authored-by: xiaoweiw-nv <xiaoweiw@nvidia.com>
Co-authored-by: Zheng Li <lizheng.cs@zju.edu.cn>
Co-authored-by: yizhang2077 <1109276519@qq.com>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: Zijie Xia <zijie.xia@radixark.ai>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
|
2026-08-28 20:41:34 -07:00 |
|
|
Cheng WanandClaude Opus 5
|
7bc3204117
|
config: three cache and pool readers take the bags (#36791)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
|
2026-08-28 10:21:32 -07:00 |
|
|
Aurick QiaoandYuwei An
|
26fd7fdaa2
|
[Feature] Support PP in full prefill CUDA graphs (#35451)
Co-authored-by: Yuwei An <ayw.sirius19@gmail.com>
|
2026-08-27 17:32:00 -07:00 |
|
|
Yuwei AnandClaude Fable 5
|
ff5578eb4e
|
[1/N][Mix] Mixed Chunk Prefill Base (#36288)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-08-27 12:59:56 -07:00 |
|
|
Cheng WanandClaude Opus 5
|
6ff2a20ccf
|
config: the record is not an object that gets passed around (#36622)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
|
2026-08-27 12:57:10 -07:00 |
|
|
Cheng WanandClaude Opus 5
|
fd40a331bf
|
config: a parallel size has one spelling; a patched scope declares its own (#36621)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
|
2026-08-27 12:56:42 -07:00 |
|
|
Cheng WanandClaude Opus 5
|
ca1d7ed8e6
|
config: a parallel leaf with no live counterpart is read bare (#36620)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
|
2026-08-27 12:56:11 -07:00 |
|
|
Cheng WanandClaude Opus 5
|
7c3b5a6732
|
config: every handler declares its cuda-graph decisions (#36725)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
|
2026-08-27 12:55:34 -07:00 |
|
|
Schwinn Saereesitthipitak
|
08315c56df
|
[Fix] Account resident weight memory in KV sizing (#34053)
Signed-off-by: Schwinn Saereesitthipitak <schwinns@nvidia.com>
|
2026-08-27 02:43:04 -07:00 |
|
|
|
b8a6adadfe
|
[Bug][Spec] fix startup crash and reduce CUDA graph memory usage for speculative adaptive (#35275)
Co-authored-by: alphabetc1 <2508695655@qq.com>
Co-authored-by: Shuwen Wang <47200617+alphabetc1@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-08-27 00:46:13 -07:00 |
|
|
weireweireandweireweire
|
ea48cb04cc
|
Pin scheduler metadata before asynchronous H2D copies (#35944)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
|
2026-08-26 22:21:47 -07:00 |
|
|
weireweireandweireweire
|
4d5d506486
|
Publish gated DSV4 DFLASH-family target-prefill read completion (#35947)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
|
2026-08-26 21:33:08 -07:00 |
|
|
Promisewjx
|
9d07b9e227
|
[Performance] Reduce idle DP work in breakable prefill CUDA graphs (#33871)
|
2026-08-27 10:09:54 +08:00 |
|
|
Haotong Zou
|
b658d64d5b
|
[CPU] Fix rotary_embedding_cpu fake for in-place layouts (#36443)
|
2026-08-27 09:48:07 +08:00 |
|
|
|
20621aa14b
|
[Model] Support Ling-3.0-flash (BailingMoeV3) (#33561)
Signed-off-by: JustinTong <justintong0323@gmail.com>
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: 得泽 <zhangkaihong.zkh@antgroup.com>
Co-authored-by: 翎悦 <vito.yy@antgroup.com>
Co-authored-by: 羽癫 <yudian.zy@antgroup.com>
Co-authored-by: tiwei.btw <tiwei.btw@antgroup.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
Co-authored-by: 文赋 <zibin.zb@antgroup.com>
Co-authored-by: JustinTong <justintong0323@gmail.com>
|
2026-08-26 17:27:23 -07:00 |
|
|
paulzhang-tm
|
8739d56a31
|
[Spec] Generalize hybrid SWA MTP draft pool routing (#35379)
|
2026-08-26 17:02:36 -07:00 |
|
|
Cheng Wan
|
937af8538b
|
config: the runtime readers take the published bags (#36254)
|
2026-08-26 05:08:25 -07:00 |
|
|
Cheng Wan
|
ae5feb4b9c
|
config: stop handing the record to code that does not read it (#36252)
|
2026-08-26 05:02:17 -07:00 |
|
|
Cheng Wan
|
d7b144f64e
|
config: publishing is the process entry's job (#36251)
|
2026-08-26 04:58:41 -07:00 |
|
|
Cheng Wan
|
8005df61d3
|
config: spell the parallel config tier at the call site (#36250)
|
2026-08-26 03:00:28 -07:00 |
|
|
Aurick QiaoandYuwei An
|
58ecbba0bd
|
[Feature] Coordinate FullCG prefill across DP-attention ranks (#35640)
Co-authored-by: Yuwei An <ayw.sirius19@gmail.com>
|
2026-08-26 02:16:02 -07:00 |
|
|
Mohammad Miadh AngkadandMohammad Angkad
|
a3c4936438
|
Sync FlashInfer autotune tactic choice across TP ranks (#35343)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
|
2026-08-26 16:30:47 +08:00 |
|
|
Baizhou ZhangandRyan Stewart
|
41e7612dee
|
[Model] Support Nemotron 3.5 Lightning speculative decoding (#36186)
Co-authored-by: Ryan Stewart <rystewart@nvidia.com>
|
2026-08-25 16:43:58 -07:00 |
|
|
YAMY
|
e9c9df6a52
|
[Performance] Tune FlashInfer EXTEND for DP prefill (#36219)
|
2026-08-25 08:29:57 -07:00 |
|
|
|
092d85eb87
|
[Feature] Add MiniCPM-SALA support (#30360)
Co-authored-by: Alex Nails <alex.nails@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-08-24 02:25:16 -07:00 |
|
|
Lianmin Zheng
|
b498efce52
|
chore: move cuda_vmm_utils.py under srt/utils/ (#36053)
|
2026-08-23 21:47:29 -07:00 |
|
|
Chunyuan WU
|
1daa94a069
|
[CPU] Fix NUMA/core binding for DP ranks (#32856)
|
2026-08-24 12:15:38 +08:00 |
|