 cctry
|
df986c4d5e
|
Consolidate CUDA VMM allocation helpers (#34199)
|
2026-08-10 18:11:11 -07:00 |
|
|
cctryandkangwangamd
|
df3aa20d89
|
Reland serving-time Triton load diagnostics (#33908)
Co-authored-by: kangwangamd <100359556+kangwangamd@users.noreply.github.com>
|
2026-08-07 14:51:43 -07:00 |
|
|
cctry
|
a3a1ebc7b7
|
Warn on risky serving-time Triton work (#33120)
|
2026-08-05 14:55:03 -07:00 |
|
|
cctry
|
96c89863a3
|
Measure prefill busy time between launches (#33595)
|
2026-08-05 11:26:27 -07:00 |
|
|
cctry
|
c8822fd990
|
Clarify post-capture KV reservation logs (#33598)
|
2026-08-04 15:06:11 -07:00 |
|
|
cctry
|
db0fe370b7
|
[PD] Fix false health-503 during decode retraction re-admission (#33118)
|
2026-08-03 09:44:25 -07:00 |
|
|
cctry
|
6cf661117d
|
[PD] Add a queues.prealloc_ready counter to the load snapshot (#33133)
|
2026-08-03 09:44:11 -07:00 |
|
|
cctry
|
3c1717d9b6
|
Follow up on #30157 post-merge review (#32672)
|
2026-07-29 15:03:59 -07:00 |
|
|
|
2c63a2f12b
|
Fix --hicache-size allocating ~2x host memory on hybrid SWA (#32373)
Co-authored-by: cctry <cctry@fb.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
|
2026-07-25 17:19:44 -07:00 |
|
|
cctryandJialin Ouyang
|
a690e5e0b3
|
Add stream label to TTFT metrics (#32363)
Co-authored-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
|
2026-07-24 17:44:04 -07:00 |
|
|
cctryandYinghai Lu
|
ce705bb6dc
|
Report accelerator type in /v1/loads (#32348)
Co-authored-by: Yinghai Lu <yinghai@meta.com>
|
2026-07-24 17:31:28 -07:00 |
|
|
cctry
|
8727d105db
|
Add prefill and decode load counters to LoadSnapshot (#32245)
|
2026-07-24 14:04:54 -07:00 |
|
|
cctryandZhiqiang Xie
|
d28e35b1a1
|
Fix num_running_reqs gauge on disagg prefill servers (#31495)
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
|
2026-07-16 15:43:06 -07:00 |
|
|
cctryandcctry
|
cb47a68717
|
[PD] Stride KV token->page indices on device before D2H copy (#31173)
Co-authored-by: cctry <cctry@fb.com>
|
2026-07-14 10:08:30 -07:00 |
|
|
cctry
|
1b4176cc46
|
[PD] Fix optimistic prefill inflight-queue hangs on parked/aborted reqs (#31075)
|
2026-07-13 20:34:44 -07:00 |
|
|
cctry
|
c616d5a55e
|
[PD] Improve optimistic prefill (#30951)
|
2026-07-12 15:31:53 -07:00 |
|
|
cctry
|
b86466d54b
|
Make KvVmmArena JIT stub unique per process (#30702)
|
2026-07-09 15:56:26 -07:00 |
|
|
cctryandcctry
|
1959335997
|
refactor(load-snapshot): build LoadSnapshot directly, drop legacy get_loads IPC (#30525)
Co-authored-by: cctry <cctry@fb.com>
|
2026-07-09 05:58:10 -07:00 |
|
|
cctry
|
096551eed6
|
Make CUDA graph disabling PD-role-aware (prefill/decode) (#30409)
|
2026-07-08 15:08:22 -07:00 |
|
|
cctry
|
2ad9a243f5
|
Size KV pool after CUDA graph capture (opt-in) (#30157)
|
2026-07-07 12:05:01 -07:00 |
|
|
cctry
|
fbe3110866
|
Make mem_fraction_static reserve disaggregation-mode aware (#29615)
|
2026-07-04 18:44:06 -07:00 |
|
|
cctry
|
697b400d70
|
Share one logits output buffer across prefill/decode/draft cuda-graph runners (#29779)
|
2026-07-01 23:30:56 -07:00 |
|
|
cctry
|
80ac11eda3
|
Fix capture-mode detection during breakable CUDA graph capture (#29866)
|
2026-07-01 22:12:56 -07:00 |
|
|
cctryandcctry
|
4b4b4af583
|
Budget EAGLE/STANDALONE draft KV pool in SWA pool configurators (#29622)
Co-authored-by: cctry <cctry@fb.com>
|
2026-06-30 00:58:33 -07:00 |
|
|
cctryandcctry
|
a5c45a12bb
|
CUDA graph executable dedup via cudaGraphExecUpdate (#29625)
Co-authored-by: cctry <cctry@fb.com>
|
2026-06-29 13:56:33 -07:00 |
|
|
cctry
|
dbe9e3b706
|
[PD] Early-send cached-prefix KV overlapping uncached prefill forward (#29316)
|
2026-06-25 13:31:53 -07:00 |
|
|
cctryandcctry
|
76db6c9d9e
|
[Refactor] Share CUDA graph memory pool across prefill and decode (#28973)
Co-authored-by: cctry <cctry@fb.com>
|
2026-06-24 10:18:20 -07:00 |
|
|
cctryandcctry
|
743ce88bc5
|
Fix flaky optimistic prefill retry test (#28995)
Co-authored-by: cctry <cctry@fb.com>
|
2026-06-23 00:18:01 -07:00 |
|
|
cctryandcctry
|
0c065671c9
|
[Spec] Redo: split init_backends; account draft weights in --mem-fraction-static (#28855)
Co-authored-by: cctry <cctry@fb.com>
|
2026-06-21 20:45:16 -07:00 |
|
|
cctry
|
9691a29fe0
|
[Spec] Split init_backends; account draft weights in --mem-fraction-static (#28683)
|
2026-06-21 01:22:26 -07:00 |
|
|
cctry
|
6d4ca9bc54
|
Cap SWA pool sizing with chunk cache (#28755)
|
2026-06-21 01:06:59 -07:00 |
|
|
cctry
|
fa71064147
|
fix: speculative draft worker clobbering target attention backend (#28559)
|
2026-06-18 01:21:14 -07:00 |
|
|
cctry
|
7976928c57
|
Abort during chunked prefill + PD peer-liveness abort (#28086)
|
2026-06-17 23:13:28 -07:00 |
|
|
cctry
|
cfa4aa988f
|
Revert "revert the head_dim assignment from PR 23862" (#28583)
|
2026-06-17 17:11:05 -07:00 |
|
|
cctry
|
5d6b35eabb
|
revert the head_dim assignment from PR 23862 (#28571)
|
2026-06-17 15:46:28 -07:00 |
|
|
|
33719cfb31
|
[PD] Optimize SWA allocation (#28085)
Co-authored-by: cctry <cctry@fb.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
|
2026-06-15 11:01:55 -07:00 |
|
|
cctry
|
75998d0421
|
Fix --mem-fraction-static not accounting for EAGLE draft model KV cache (#23862)
|
2026-06-12 10:35:48 -07:00 |
|
|
cctryandcctry
|
10219bd9d6
|
[PD] Fix negative prefill kv_transfer_alloc_ms under optimistic prefill (#27885)
Co-authored-by: cctry <cctry@fb.com>
|
2026-06-11 13:12:25 -07:00 |
|
|
cctry
|
b3e4c204fd
|
Don't write crash dump on graceful exit (#27405)
|
2026-06-05 17:06:39 -07:00 |
|
|
|
f6cd1a9822
|
Add num_waiting_uncached_tokens load metric (#27174)
Co-authored-by: cctry <cctry@fb.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
|
2026-06-03 18:29:49 -07:00 |
|
|
cctryandgemini-code-assist[bot]
|
b55570d38e
|
[PD] Optimistic prefill (#26780)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2026-06-02 01:16:14 -07:00 |
|
|
|
163bf1ba71
|
[PD] Fix KV transfer metrics (#24416)
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
|
2026-05-06 03:44:48 -07:00 |
|
|
cctry
|
660a77f221
|
Silence noisy health-check race log in TokenizerManager (#24466)
|
2026-05-05 21:06:43 -07:00 |
|
|
cctry
|
5c31b35db2
|
[hicache] Optimization for DMA copy (#8245)
|
2025-08-09 17:16:07 -07:00 |
|