514b997e6c
Register CPU CI for 17 e2e tests and partition xeon base-c suite ( #35227 )
...
Co-authored-by: Zhang, Mingxu <mingxu.zhang@intel.com >
Co-authored-by: MingxuZh <109504044+MingxuZh@users.noreply.github.com >
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-08-24 10:49:38 +08:00
Cheng Wan and Claude Opus 5
340391a297
config: publish before the launcher reads effective configuration ( #35910 )
...
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-08-23 01:20:20 -07:00
Cheng Wan and Claude Opus 5
a43592dce5
config: pin two orderings resolution relies on ( #35909 )
...
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-08-23 01:19:44 -07:00
Cheng Wan and Claude Opus 5
362c2ee849
config: borrowed-record reads follow the config bags ( #35908 )
...
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-08-23 01:19:20 -07:00
Cheng Wan and Claude Opus 5
64aa859da2
config: constructing a config no longer resolves it ( #35907 )
...
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-08-23 01:18:53 -07:00
Cheng Wan and Claude Opus 5
4bc79a1b49
config: project the config bags from the resolution result ( #35906 )
...
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-08-23 01:18:24 -07:00
Cheng Wan and Claude Opus 5
0e22777572
config: record resolution writes in a declaration stash ( #35905 )
...
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-08-23 01:17:27 -07:00
Baizhou Zhang
3b5909de0e
[DeepSeek V4] Add W4A4 MegaMoE server flag ( #35918 )
2026-08-21 18:44:18 -07:00
9db4ba8da1
[DeepSeek-V4] Add Q8KV8 sparse MLA prefill runtime backend ( #32327 )
...
Co-authored-by: Ho-Ren (Jack) Chuang <horenchuang@bytedance.com >
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com >
2026-08-20 10:23:49 +08:00
Liangsheng Yin
1270204d2c
Revert "[Feature] Add DeepEPv2 (ElasticBuffer) MoE A2A backend" ( #35568 )
2026-08-19 14:14:38 -07:00
milesial
ed12d6827d
fix(disagg): allow fake transfer with decode DCP ( #35409 )
...
Signed-off-by: Alexandre Milesi <milesial@users.noreply.github.com >
2026-08-19 13:54:37 -07:00
Po-Han Huang (NVIDIA)
6f69f927da
[Scheduler] Add configurable decode interval after prefill ( #35017 )
2026-08-19 12:01:36 -07:00
MengYu and menyu
4f8ecf6ae9
[Feature] Add DeepEPv2 (ElasticBuffer) MoE A2A backend ( #29525 )
...
Co-authored-by: menyu <menyu@nvidia.com >
2026-08-19 11:52:45 -07:00
Jimmy Shong and Claude Fable 5
c863760ae1
[Fix] DCP: advertise the logical KV-event block size ( #35298 )
...
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-08-18 20:55:11 -07:00
Xingyu Liu and xingyuliu
7dcaf11987
[Fix] Select custom all-reduce v2 by topology capability ( #35061 )
...
Co-authored-by: xingyuliu <xingyuliu@fb.com >
2026-08-18 10:30:10 -07:00
Cheng Wan
cba3c5d5ac
config: the per-instance families read the bags ( #35026 )
2026-08-17 16:17:53 -07:00
cctry and cctry
2e7c85da68
[PD] Preserve decode KV across retraction in HiCache ( #34801 )
...
Co-authored-by: cctry <cctry@fb.com >
2026-08-17 08:49:11 -07:00
Lianmin Zheng
67e12131df
Build Rust extensions on demand in source checkouts ( #34994 )
2026-08-16 14:58:06 -07:00
Cheng Wan
61908870f6
config: spell out the one dynamic config read the census could not see
...
`_is_dsa_active` asked `getattr(server_args, "_is_dsa_model_arch", False)`, and
that name has never existed on `ServerArgs` -- it arrived as a placeholder with
the CP strategy abstractions (#27313 ), so the getattr default has always decided
the predicate. A dynamic read of a name nothing sets is the one shape the config
census cannot follow, and it looked like a live decision while being dead.
Spelled as the constant it evaluates to, with the placeholder written down: what
it should ask (whether this process runs a DSA model arch) is the CP path's
call, and its only consumer, `ContextParallelStrategy.per_layer_attn_cp_comm`,
has no readers yet.
That was the sole entry in the read ratchet's `_INERT_DYNAMIC_READS`, so the
exemption list is gone with it -- there is no way to exempt a read from the
baselines any more, which is the invariant worth having. The `counted()`
indirection it existed for goes too (verified the three shapes it guarded still
report: direct, `getattr`, and an attribute-parked alias).
2026-08-15 00:38:30 -07:00
ethche and Ethan Che
aeee1562e6
[Kernel] Enable Helion backend for Kimi Delta-Attention ( #32593 )
...
Co-authored-by: Ethan Che <eche@meta.com >
2026-08-14 22:22:46 -07:00
Mohammad Miadh Angkad
96db53ec70
[CI] Fix test_resolution_is_reproducible after cuda_ipc became opt-in ( #34746 )
2026-08-13 12:47:03 -07:00
Mick
69bf601e3c
fix: restore VLM nightly regression coverage ( #34662 )
2026-08-12 21:47:16 -07:00
gongwei1027
2c07ca5e8d
[Fix] Allow flashinfer_sparse_mla DSA backend for HiSparse on SM120 FP8 KV ( #33075 )
2026-08-11 15:05:04 -07:00
Mick
8267d76c2c
[VLM] replace deprecated image processor use_fast ( #34175 )
2026-08-12 00:14:07 +08:00
Cheng Wan and Claude Fable 5
a2199c1dee
config: pin that resolution is reproducible from the raw input ( #34094 )
...
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-08-09 14:44:39 -07:00
Mick
d747bd052e
feat(vlm): auto-select cuda vmm on multi-node mnnvl ( #33936 )
2026-08-08 16:00:58 +08:00
Oguz Ulgen and Yinghai Lu
7f6b4cb94b
Add CUDA VMM multimodal feature transport ( #33899 )
...
Co-authored-by: Yinghai Lu <yinghai@meta.com >
2026-08-07 13:39:54 -07:00
Ziang Li and Brayden Zhong
4ad990ba7d
[ModelOpt FP4] Support online MoE weight quantization ( #33115 )
...
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca >
2026-08-06 11:01:55 -07:00
Mohammad Miadh Angkad
1a15cf1536
Gate multimodal feature transport by model capability ( #33653 )
2026-08-06 06:48:32 -07:00
+26
abddb1c7e9
[Kimi] Support kimi-k3 ( #32541 )
...
Co-authored-by: DarkSharpness <76582120+DarkSharpness@users.noreply.github.com >
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com >
Co-authored-by: Mick <mickjagger19@icloud.com >
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com >
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com >
Co-authored-by: Ke Bao <ispobaoke@gmail.com >
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com >
Co-authored-by: Chunan Zeng <zcnrex@gmail.com >
Co-authored-by: Khoa Pham <khoa.pham@radixark.ai >
Co-authored-by: Ziyi Xu <ziyi.xu@radixark.ai >
Co-authored-by: Zijie Xia <37504505+zijiexia@users.noreply.github.com >
Co-authored-by: Yuwei An <ayw.sirius19@gmail.com >
Co-authored-by: zhangxiaohao <1024393531@qq.com >
Co-authored-by: Yangmin Li <yangminl@nvidia.com >
Co-authored-by: Julien Lin <jullin@nvidia.com >
Co-authored-by: Hao Phan <htphan@nvidia.com >
Co-authored-by: Thomas Wang <1am9trash@gmail.com >
Co-authored-by: RolaoDenthu <xinyisong0111@gmail.com >
Co-authored-by: pigeonsoup <32922982+pigeonsoup@users.noreply.github.com >
Co-authored-by: HaiShaw <hixiao@gmail.com >
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com >
Co-authored-by: Pranjal Shankhdhar <pranjal.ssh@gmail.com >
Co-authored-by: Lee Nau <lee.nau@gmail.com >
Co-authored-by: HMING <126185151+Hearum@users.noreply.github.com >
Co-authored-by: elvischenv <219235043+elvischenv@users.noreply.github.com >
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com >
Co-authored-by: Byron Hsu <byron+per@periodiclabs.ai >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
Co-authored-by: Thomas Wang <thomawan@amd.com >
Co-authored-by: Xinyi Song <86638975+RolaoDenthu@users.noreply.github.com >
Co-authored-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com >
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai >
Co-authored-by: BBuf <xiaoyu.zhang@radixark.ai >
Co-authored-by: Hanming Lu <hanminglu@meta.com >
Co-authored-by: Xinyi Song <xinyis10@illinois.edu >
2026-08-04 13:22:49 -07:00
Tao Li and litao.dream
a0b7bcf592
[Feature] Support return_hidden_states="last" ( #30177 )
...
Co-authored-by: litao.dream <litao.dream@bytedance.com >
2026-08-02 15:09:33 +08:00
056474cdb0
feat: Session-reference-aware Unified Radix Cache for agentic multi-turn workloads ( #29173 )
...
Co-authored-by: Ishan Dhanani <ishandhanani@gmail.com >
Co-authored-by: hzh0425 <hzh0425@apache.org >
Co-authored-by: ispobock <ispobaoke@gmail.com >
2026-08-02 13:43:55 +08:00
Cheng Wan
d3222bcc3a
[unified-memory] Support fa3, the default MLA backend on pre-Blackwell hosts ( #33046 )
2026-07-31 11:46:46 -07:00
Khoa Pham and Yangmin Li
2573190b93
feat: support Kimi Linear PD disaggregation with DCP ( #32837 )
...
Co-authored-by: Yangmin Li <yangminl@nvidia.com >
2026-07-31 02:14:09 -07:00
Cheng Wan
33c27d8e7f
[unified-memory] Let Kimi-Linear use the paged MLA attention backends ( #32972 )
2026-07-31 01:32:08 -07:00
Mick
a149717308
feat: log multimodal encoder DP tradeoffs ( #30903 )
2026-07-31 08:50:20 +08:00
Ethan (Yusheng) Su
ee1736f39a
[LoRA] Support LoRA under the breakable/full prefill CUDA graph ( #30988 )
2026-07-26 22:10:03 -07:00
35e25f5356
[Feature] DCP: A2A + FlashInfer-MNNVL comm backends and q-replicate (Helix) ( #21637 )
...
Co-authored-by: Hao Phan <htphan@nvidia.com >
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com >
Co-authored-by: kpham-sgl <khoa.pham@radixark.ai >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com >
2026-07-24 00:21:53 -07:00
ishandhanani and Connor Carpenter
21065bc862
feat: add native gRPC sidecar module launcher ( #31076 )
...
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com >
Signed-off-by: Connor Carpenter <connorc@nvidia.com >
Co-authored-by: Connor Carpenter <connorc@nvidia.com >
2026-07-22 06:39:08 -07:00
Sam (Kesen Li)
ec6a3163b7
[Feature] Add FP4 KV Cache Design and support SM120 GPUs ( #21601 )
2026-07-17 14:49:43 -07:00
db7e6807de
[BugFix] Preserve tokenizer worker fanout when skip_tokenizer_init is enabled ( #30682 )
...
Co-authored-by: hjzhang <zhanghjzzz@qq.com >
Co-authored-by: hnyls2002 <lsyincs@gmail.com >
2026-07-16 01:29:46 -07:00
Mick
947a14d617
feat: unify multimodal feature transport ( #30904 )
2026-07-15 17:42:38 +08:00
YAMY
41ad0d9c26
Allow prefill breakable CUDA graph for Qwen3.5 via multimodal opt-in allowlist ( #30620 )
2026-07-13 18:39:44 -07:00
xutizhou
eb31b5310c
Support Waterfill with MegaMoE backend ( #27350 )
2026-07-13 03:56:46 -07:00
Lianmin Zheng and Yongji Wu
81d273f73b
Handle coredump dirs and cache hit updates ( #30897 )
...
Co-authored-by: Yongji Wu <30348494+libertyeagle@users.noreply.github.com >
2026-07-12 00:37:43 -07:00
Liangsheng Yin
c53559ba10
[misc] Remove unit test cases that fail the admission criteria ( #30690 )
2026-07-09 15:31:28 -07:00
cctry
096551eed6
Make CUDA graph disabling PD-role-aware (prefill/decode) ( #30409 )
2026-07-08 15:08:22 -07:00
Alex Nails and Claude Opus 4.7
3d2e7cc601
[gRPC] Native server: launcher + HTTP + server args wiring (3/4) ( #23508 )
...
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com >
2026-07-07 14:57:25 -07:00
Cheng Wan
c861896721
[refactor] Resolve config declarations onto server_args at the end of __post_init__ ( #30297 )
2026-07-06 18:04:44 -07:00
Lianmin Zheng
24c42c90be
Clean up ServerArgs post-init dispatch ( #30186 )
2026-07-05 23:05:07 -07:00
