 Bingxu ChenandCursor Agent
|
f113ece5cc
|
Revert "improve: combine vit calls for images from different reqs from one batch (#25910)" (#26442)
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
|
2026-05-29 11:13:34 -07:00 |
|
|
Bingxu Chen
|
5601b7139d
|
[core] Make overlap-schedule WAR barrier CUDA-only (#26646)
|
2026-05-29 01:02:31 -07:00 |
|
|
Arik
|
ace730db48
|
[AMD] Work around HIP TPOT regression from Event.wait() in MTP seq lens resolution (#26672)
|
2026-05-29 00:18:02 -07:00 |
|
|
LucQueenandZhengWG
|
36d0a6e08e
|
[EPD] Optimize the Mooncake backend (#22587)
Co-authored-by: ZhengWG <zwg0606@gmail.com>
|
2026-05-29 10:42:24 +08:00 |
|
|
|
3bdea78ad1
|
model: support Step-3.7-Flash (#26565)
Co-authored-by: yhyang201 <yhyang201@users.noreply.github.com>
Co-authored-by: luotingdan <luotingdan@stepfun.com>
|
2026-05-29 08:00:54 +08:00 |
|
|
shuwennandmaodoudou168
|
68706e615a
|
[SPEC] fix: use effective max draft tokens for adaptive spec initiali… (#26354)
Co-authored-by: maodoudou168 <maodoudou168@users.noreply.github.com>
|
2026-05-28 13:33:11 -07:00 |
|
|
Cameron Quilici
|
3ca8cb470f
|
[BugFix] preserve cached token details in multi-tokenizer output (#26590)
|
2026-05-28 13:31:55 -07:00 |
|
|
Liangsheng Yin
|
8e0ed75f2d
|
Remove dead fields and always-False plumbing across SB / FB / LogitsMetadata (#26551)
|
2026-05-28 03:15:04 -07:00 |
|
|
Liangsheng Yin
|
686ef50672
|
Group ScheduleBatch and ForwardBatch fields by data-flow role (#26022)
|
2026-05-28 00:11:20 -07:00 |
|
|
|
19663aafcd
|
Support batch size > 1 when enable CP (#23269)
Co-authored-by: Shunkang <182541032+Shunkangz@users.noreply.github.co>
Co-authored-by: Khoa Pham <khoa.pham@radixark.ai>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
|
2026-05-27 14:11:17 -07:00 |
|
|
Jialin Ouyang
|
98bc6f3c22
|
API Perf: Replace pydantic per-element validation with C loop validation (#26355)
|
2026-05-27 02:04:07 -07:00 |
|
|
Liangsheng Yin
|
216ed270e5
|
refresh resolve_seq_lens_cpu comments (#26463)
|
2026-05-27 00:51:43 -07:00 |
|
|
Liangsheng Yin
|
163b970127
|
[core] WAR barrier for overlap schedule buffer writes, without fwd occupancy cost (#26380)
|
2026-05-26 23:58:32 -07:00 |
|
|
xdtbynd
|
21d0e74aff
|
Disable torch.compile for NPU in speculative overlap utils (#26403)
|
2026-05-27 12:30:53 +08:00 |
|
|
Liangsheng Yin
|
1051a8456f
|
[core] Maintain req_pool_indices_cpu host mirror (like seq_lens_cpu) (#26425)
|
2026-05-26 18:59:54 -07:00 |
|
|
Yilong Zhaoandhappierpig
|
1a05b511e4
|
dp: refactor idle batch logic (#25025)
Co-authored-by: happierpig <zhaoyilong217@sjtu.edn.cn>
|
2026-05-26 14:22:25 -07:00 |
|
|
Yongfei Xu
|
98eb84497d
|
[PP] Skip PP output communication for pure chunked prefill batches (#26148)
|
2026-05-26 21:59:18 +08:00 |
|
|
fzyzcjy
|
d9c82934c8
|
Extract Scheduler init methods and add skills to enforce the splitting requirements (#26271)
|
2026-05-26 17:45:09 +08:00 |
|
|
roikoren755
|
e958f4561f
|
[feat] Support extra_buffer in Mamba2-based models (#15829)
Signed-off-by: Roi Koren <roik@nvidia.com>
|
2026-05-26 16:03:29 +08:00 |
|
|
Mick
|
8f2a4e70f8
|
[SRT] minor: reuse req input id array for unpadded ids (#26232)
|
2026-05-26 08:58:32 +08:00 |
|
|
Liangsheng Yin
|
8805f4cf16
|
Fail-fast on PD subprocess exit and scheduler exception (#26298)
|
2026-05-25 16:42:50 -07:00 |
|
|
Yuhui Liang
|
ca029e816b
|
Fix missing idle-batch handling in prepare_mlp_sync_batch_raw (#25404)
|
2026-05-24 23:53:54 -07:00 |
|
|
Mick
|
e1463bb2c2
|
[VLM] try to reuse precomputed padded input ids in scheduler instead of padding (#26097)
|
2026-05-25 10:27:04 +08:00 |
|
|
Lianmin Zheng
|
93fa577bb9
|
Clean up server startup log noise (#26205)
|
2026-05-24 14:35:15 -07:00 |
|
|
Dongjun Na
|
9d50cd9742
|
[observability] add ServerArgs.stat_loggers for pluggable metrics backend (#24610)
Signed-off-by: Dongjun Na <kmu5544616@gmail.com>
|
2026-05-24 22:41:28 +08:00 |
|
|
Mick
|
b6f71d5850
|
[VLM] avoid extra cuda-ipc staging for preprocessed input (#26096)
|
2026-05-24 19:48:06 +08:00 |
|
|
Mick
|
6447596501
|
[VLM] feat: replace small H2D calls with a single one for qwen-vl (#26167)
|
2026-05-24 18:27:17 +08:00 |
|
|
Mick
|
826a4de062
|
[srt] store req input ids as arrays (#26165)
|
2026-05-24 15:09:36 +08:00 |
|
|
Liangsheng Yin
|
1e59ed7443
|
compile _resolve_spec_extras gather kernels (#26129)
|
2026-05-23 02:34:41 -07:00 |
|
|
Mick
|
774b29dade
|
[VLM] feat: early-return in mm processor if the input is preprocessed (#26117)
|
2026-05-23 17:02:44 +08:00 |
|
|
Mick
|
19b60a4f9e
|
[VLM] reuse pretokenized ids from preprocessed input for qwen-vl (#26116)
|
2026-05-23 16:01:04 +08:00 |
|
|
Yueming Yuan
|
208397affc
|
[RL] [Spec v2] Use stop-aware seqlen for returned topk metadata (#26126)
|
2026-05-22 18:13:16 -07:00 |
|
|
Liangsheng Yin
|
763174fa6c
|
FutureMap: debug-assert that gather sees a stashed value (#26108)
|
2026-05-22 13:10:57 -07:00 |
|
|
Jialin Ouyangandjialino
|
06c23d55b5
|
perf: migrate Req token-id storage to array.array('q') in Scheduler (#25098)
Co-authored-by: jialino <jialino@fb.com>
|
2026-05-22 10:51:07 -07:00 |
|
|
zhangtao2-1andzhangtao
|
6baa859a86
|
Add MiniCPM5 tool call parser for XML-style function calls (#25600)
Co-authored-by: zhangtao <zhangtao2@modelbest.cn>
|
2026-05-22 23:09:40 +08:00 |
|
|
Liangsheng Yin
|
10751a4f0c
|
drop FutureIndices wrapper class (#26085)
|
2026-05-22 02:44:42 -07:00 |
|
|
Liangsheng Yin
|
c4b6b5ea1e
|
[core] step 2: drop seq_lens sentinel; SB maintains GPU as seq_lens_cpu mirror (#26020)
|
2026-05-22 00:12:51 -07:00 |
|
|
Yuhao YangandYaochen Han
|
fa6f4dfb35
|
improve: combine vit calls for images from different reqs from one batch (#25910)
Co-authored-by: Yaochen Han <814073252@qq.com>
|
2026-05-22 14:57:41 +08:00 |
|
|
fzyzcjy
|
3f0814974c
|
Fix disaggregation bootstrap server lifetime (#25982)
|
2026-05-22 14:10:57 +08:00 |
|
|
Liangsheng Yin
|
8b473aa0bc
|
[core] step 1: route non-spec seq_lens via FutureMap with per-mode bootstrap fixes (#25944)
|
2026-05-21 20:15:51 -07:00 |
|
|
Liangsheng Yin
|
44ec2ee18d
|
[core] Unify output_tokens_buf in FutureMap (#25922)
|
2026-05-21 13:56:44 -07:00 |
|
|
Liangsheng Yin
|
c9a0e55eb5
|
[Spec] Polish FutureMap after #25879: rename callback, async guard, cleanup (#25962)
|
2026-05-21 13:56:22 -07:00 |
|
|
Liangsheng Yin
|
baeac179f7
|
[Spec] Route seq_lens through FutureMap; drop verify_done.wait (#25879)
|
2026-05-21 01:51:40 -07:00 |
|
|
Xinyuan Tong
|
40faf44f7a
|
[auto-detect] match Ring-2.6/Ling XML kv tool-call format via vocab signature (#25366)
|
2026-05-20 23:34:52 -07:00 |
|
|
Charles Chen
|
847cbada9c
|
Support Gemma4 MoE NVFP4 (#25054)
|
2026-05-20 22:45:15 -07:00 |
|
|
Randall LinandCursor
|
791a2f057f
|
Add overridable hooks for custom chat serving implementations (#25807)
Co-authored-by: Cursor <cursoragent@cursor.com>
|
2026-05-21 11:21:25 +08:00 |
|
|
Liangsheng Yin
|
512d164916
|
Address overlap future token map by request-pool index (#25862)
|
2026-05-20 16:34:00 -07:00 |
|
|
Hanming Lu
|
371b6c9ea0
|
[Scheduler] fix chunked prefill not always being full (#25741)
|
2026-05-20 15:03:11 -07:00 |
|
|
Liangsheng Yin
|
34d3e23232
|
spec_v2: consolidate seq_lens_cpu/sum maintenance into helper (#25818)
|
2026-05-20 04:42:26 -07:00 |
|
|
Yuhong Guo
|
24d27c2035
|
[BugFix] Fix rid_to_state leak for aborted queued requests (#24070)
|
2026-05-20 01:32:44 -07:00 |
|