 Sam Shleifer
|
58974ca16c
|
[perf] Assemble flat prompt top logprobs scheduler-side as numpy arrays (#32223)
|
2026-07-31 18:00:51 -07:00 |
|
|
Yuwei AnandClaude Fable 5
|
3e0f7c3f30
|
[BCG][3/N] Enable bcg on dsa & deepep a2a backend (#31987)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-07-31 16:45:18 -07:00 |
|
|
Ethan (Yusheng) Su
|
3c5f115741
|
Split #32584 into 2/2: [LoRA] Shard attention LoRA by attn-TP and allow dynamic LoRA with dp attention (#32708)
|
2026-07-31 15:37:13 -07:00 |
|
|
Cheng Wan
|
55b6769b0e
|
config: read resolved config via namespace accessors (#33013)
|
2026-07-31 15:06:59 -07:00 |
|
|
Cheng Wan
|
4862edc85f
|
runtime_context: record the publishing process role (#33012)
|
2026-07-31 15:06:22 -07:00 |
|
|
Rain Jiang
|
4af8ddb576
|
support rust sglang server (#29799)
|
2026-07-31 11:56:31 -07:00 |
|
|
DAI0818andXiaoyu Zhang
|
afeaeccfa2
|
perf(hisparse): eliminate redundant swap output fill (#32483)
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
|
2026-07-31 12:09:44 +08:00 |
|
|
Shangming Cai
|
f6ff5e8bb0
|
[PD] Handle abort requests in PP mode (#32797)
|
2026-07-30 14:39:44 +08:00 |
|
|
Sam Shleifer
|
d0e69d3881
|
[feat] Optional base64 encoding for the flat prompt top logprob arrays (#31960)
|
2026-07-29 12:15:56 -07:00 |
|
|
ziang663andChao Shi
|
eefb434d17
|
[PD+PP] Honor PP consensus for bootstrap and prealloc (#31869)
Co-authored-by: Chao Shi <chao.shi@alibaba-inc.com>
|
2026-07-30 01:19:17 +08:00 |
|
|
Yoray Zack
|
62d0f81f16
|
[2/N] elastic-ep: Enable EPLB after scale-up (#30553)
|
2026-07-30 01:06:56 +08:00 |
|
|
huangtingweiandalphabetc1
|
50029f05a3
|
[HiCache] Merge HiCache event checks to reduce decode overhead (#30511)
Co-authored-by: alphabetc1 <47200617+alphabetc1@users.noreply.github.com>
|
2026-07-29 23:48:48 +08:00 |
|
|
Ke Bao
|
50b029257f
|
Skip mamba lock during decoding (#32228)
|
2026-07-29 21:53:28 +08:00 |
|
|
Ethan (Yusheng) Su
|
0a49226d19
|
[LoRA] 1/n Per-rank tensor serialization for load_lora_adapter_from_tensors under dp_size > 1 (#32580)
|
2026-07-28 14:28:56 -07:00 |
|
|
Talantan1102
|
5558dbad00
|
[NPU] Optimize DeepSeek-V4 performance (#31931)
|
2026-07-28 19:45:46 +08:00 |
|
|
Lianmin Zhengandcctry
|
c19a333944
|
[mm] Handle per-item embeddings in cache misses (#32498)
Co-authored-by: cctry <cctry@meta.com>
|
2026-07-27 17:16:17 -07:00 |
|
|
Mick
|
08af5aea57
|
optimize: optimize EmbeddingGemma prefill performance (#32383)
|
2026-07-27 17:34:29 +08:00 |
|
|
Sam Shleifer
|
5cc273a780
|
[feat] Opt-in flat response format for prompt top logprobs (#32078)
|
2026-07-26 23:44:34 -07:00 |
|
|
Mick
|
abb8f4b5e3
|
model: support EmbeddingGemma (#32375)
|
2026-07-27 10:40:47 +08:00 |
|
|
silencejade
|
72e415dfc8
|
[Bugfix] Fix prefill suspension caused by delayed negotiate_should_allow_prefill invocation (#32389)
|
2026-07-26 14:59:26 +08:00 |
|
|
Jialin Ouyang
|
cd145f840f
|
Radix Cache Split: Spin off TreeCore (#29901)
|
2026-07-25 14:31:59 -07:00 |
|
|
Ke Bao
|
69a3c54c70
|
Fix SWA admission livelock on cached-prefix resumes (#32379)
|
2026-07-25 22:37:35 +08:00 |
|
|
|
e943e609dc
|
[DSPARK] Grammar-constrained decoding, incl. tool_choice=auto (#31753)
Co-authored-by: shanemort1982 <shanemort1982@users.noreply.github.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
|
2026-07-25 05:20:25 -07:00 |
|
|
Hồ Sỹ Thếandhnyls2002
|
d021990bf5
|
[DFLASH] Support grammar-constrained decoding in speculative verify (#30096)
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
|
2026-07-25 04:36:46 -07:00 |
|
|
Xun SunandShangming Cai
|
9eb2dccbb7
|
[Elastic EP] Fix recovery lifecycle and add manual coverage (#31744)
Co-authored-by: Shangming Cai <csmthu@gmail.com>
|
2026-07-25 14:32:47 +08:00 |
|
|
|
f9c14e6bd4
|
[FEAT] Support fast engine recovery through weight cache (#27139)
Signed-off-by: Michael Qiu <qiudayu.qdy@antgroup.com>
Co-authored-by: liusy58 <liusy58@linux.alibaba.com>
Co-authored-by: Alex Nails <alex.nails@radixark.ai>
|
2026-07-25 14:31:21 +08:00 |
|
|
Liangsheng Yin
|
cff20a2fbb
|
[Spec] Consolidate the grammar sync decision into ScheduleBatch.grammar_needs_sync (#32353)
|
2026-07-24 20:42:38 -07:00 |
|
|
cctryandJialin Ouyang
|
a690e5e0b3
|
Add stream label to TTFT metrics (#32363)
Co-authored-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
|
2026-07-24 17:44:04 -07:00 |
|
|
|
962c076934
|
Decode input_audio media containers with PyAV & Update memory profiler (#31832)
Signed-off-by: Shiyan Deng <dsy842974287@meta.com>
Signed-off-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: Lucia Fang <116399278+luccafong@users.noreply.github.com>
|
2026-07-24 16:18:45 -07:00 |
|
|
cctry
|
8727d105db
|
Add prefill and decode load counters to LoadSnapshot (#32245)
|
2026-07-24 14:04:54 -07:00 |
|
|
Lu Fang
|
448662e85e
|
[mm] Accept per-item embedding lists from DataEmbeddingFunc (#31826)
|
2026-07-24 08:27:24 -07:00 |
|
|
Zheng Wengangandsiyu
|
364b5f23e6
|
[BugFix][EPD] Harden zmq_to_scheduler receiver failures; sync error info across TP (#31592)
Co-authored-by: siyu <liusy58@linux.alibaba.com>
|
2026-07-24 16:37:52 +08:00 |
|
|
Jun LiuandXinyuan Tong
|
58f417049d
|
[PD] Fix multi-tokenizer disaggregation metrics labels (#30412)
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
|
2026-07-24 16:27:21 +08:00 |
|
|
Hongkuan Zhou
|
d0b9689805
|
[Fix] Include disagg prefill waiting queue in FPM (#32122)
Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
|
2026-07-23 09:30:27 -07:00 |
|
|
Liangsheng Yin
|
9b853e6832
|
[Scheduler] Enable decode retraction ordering under speculative decoding (#32023)
|
2026-07-23 00:42:56 -07:00 |
|
|
Liangsheng Yin
|
98cc8d91cf
|
[Fix] Evict only the KV shortfall in evict_from_tree_cache (#32016)
|
2026-07-22 12:41:06 -07:00 |
|
|
Liangsheng Yin
|
5c6a29b3c3
|
[Fix] Unify pinned host pool release on graceful shutdown (#32029)
|
2026-07-22 12:29:34 -07:00 |
|
|
Cheng Wan
|
f5dcbe8f14
|
Revert RuntimeContext config-namespace reads/roles (#31813–#31817) (#32100)
|
2026-07-22 11:52:41 -07:00 |
|
|
Xiaoyu ZhangandClaude Opus 4.8
|
977ea336cd
|
[Kernel] Phase 4 batch-2: migrate JIT operator groups into kernels.ops (no shims) (RFC #29630) (#32015)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-07-22 17:49:51 +08:00 |
|
|
Cheng Wan
|
745b2ca45c
|
config: read parallel config leaves via get_parallel() (#31816)
|
2026-07-22 01:18:50 -07:00 |
|
|
Cheng Wan
|
11a4c2d057
|
config: read resolved config via namespace accessors (#31814)
|
2026-07-22 01:18:05 -07:00 |
|
|
Cheng Wan
|
1d0a6ee178
|
runtime_context: record the publishing process role (#31813)
|
2026-07-22 01:17:42 -07:00 |
|
|
Cheng Wan
|
e1479cc966
|
config: route runtime config adjustments through the namespace bags (#31812)
|
2026-07-22 01:17:22 -07:00 |
|
|
Hanming Lu
|
d03c8cee80
|
Negotiate PrefillDelayer only after KV-budget admission checks (#31835)
|
2026-07-21 12:07:30 -07:00 |
|
|
YAMY
|
7c9257529f
|
[Mamba] Support speculative decoding with extra_buffer_lazy (#30437)
|
2026-07-21 19:38:03 +08:00 |
|
|
Yuhao Yang
|
37a830b667
|
[Feature] Add DWDP (Distributed Weight Data Parallelism) for MoE prefill (#29778)
|
2026-07-20 23:59:54 -07:00 |
|
|
Yang Liu
|
9462c303a5
|
Fix stop boundaries for grammar-constrained speculative decoding (#31738)
|
2026-07-20 17:55:27 -07:00 |
|
|
Lianmin ZhengandJason Park
|
e7e8aaa73c
|
Overlap grammar (constrained decoding) with speculative decode verify (#31488)
Co-authored-by: Jason Park <jasonjk@fb.com>
|
2026-07-20 17:36:05 -07:00 |
|
|
Jinyan Yi
|
fce5c75a30
|
[NPU] Fix vit graph tnd cu seqlens (#31701)
|
2026-07-20 16:20:33 +08:00 |
|
|
|
02236fa38c
|
Add Inkling model support (#31681)
Co-authored-by: Chunan Zeng <zcnrex@gmail.com>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
Co-authored-by: Yanbin Jiang <jybsuper@gmail.com>
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: Qiaolin Yu <qiaolin.yu@radixark.ai>
Co-authored-by: Zhichen Zeng <zczeng@uw.edu>
Co-authored-by: Aurick Qiao <aurick@thinkingmachines.ai>
Co-authored-by: Joseph <jk@thinkingmachines.ai>
|
2026-07-19 22:57:37 -07:00 |
|