Commit Graph
6055 Commits
Author SHA1 Message Date
Yuan Luoandluoyuan.luo 23a1f53fc3 [QSA] Bound the decode indexer's work by the context length (#43463)
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
2026-10-12 22:47:45 +08:00
279f1fad90 [diffusion] chore: bind worker CPUs to the GPU's NUMA node when memory policy is not permitted (#43756)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 21:09:41 +08:00
Артем Савкин d1357bf87b [NPU] Diffusion CI fixes (#43210) 2026-10-12 14:53:51 +03:00
Sasha SidorovandShangming Cai 7953751bfa [PD][Spec] Give a PD decode's first EAGLE draft step a proposal distribution (#41783)
Co-authored-by: Shangming Cai <csmthu@gmail.com>
2026-10-12 18:49:52 +08:00
CandCheng Wan 28621739fc [Fix] Make tokenizer request-state initialization atomic (#41911)
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
2026-10-12 03:10:57 -07:00
Jinyan Yi f006fb49ec [NPU] Stabilize HiCache and registered CI tests (#43308) 2026-10-12 17:10:26 +08:00
BingjiaWang 62ff466ac0 [DSV4.1][PD][1/N] Support prefill node use cp parallel strategy (#40323) 2026-10-12 01:05:21 -07:00
BingjiaWang 54409c2fa0 [BCG][DSV4-Flash][CP] Support DeepSeek V4 prefill with interleave CP and breakable CUDA graphs (#38603) 2026-10-12 01:03:10 -07:00
Anupa SajikumarandSiju Samuel 8d491fbc3e [XPU] Support decode context parallelism (DCP) on Intel XPU (#34355)
Co-authored-by: Siju Samuel <siju.samuel@intel.com>
2026-10-12 15:50:33 +08:00
Ke Bao a70a83bcd2 Keep NIXL Mamba checkpoint keys separate by TP rank (#43594) 2026-10-12 15:40:59 +08:00
Ke Bao b6ba148384 Capture KV host transfer buffers (#43711) 2026-10-12 15:40:06 +08:00
Mohammad Miadh AngkadandMohammad Angkad 06887827c7 [CI] Move tests that do not need GB300 off the per-PR GB300 suite (#43784)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-10-12 00:38:10 -07:00
5ca0695aa6 [HiCache] Preserve all MTP KV and recurrent state (#40223)
Co-authored-by: alphabetc1 <2508695655@qq.com>
Co-authored-by: Shuwen Wang <47200617+alphabetc1@users.noreply.github.com>
2026-10-12 15:37:59 +08:00
Shuwen Wang f5f5003323 [HiCache] Support Mamba in buffer-only host memory mode (#36345) 2026-10-12 00:35:16 -07:00
Shangming CaiandClaude Fable 5.1 4195980f5a [PD] Poison the deferred abort ack when a Mooncake send fails (#43719)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-12 15:33:09 +08:00
ashwini rathi 8ce76b88df [XPU CI] Optimize PR CI per-file timeouts and fail hanging tests fast (#42048) 2026-10-12 14:24:26 +08:00
Liangsheng Yin 9e0b001db4 [CI] Skip eval retries in the PD transfer-failure test (#43769) 2026-10-11 22:37:40 -07:00
a297cd3da1 [Fix] scheduler: stop latching batch_is_full when nothing can run (#41872)
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
2026-10-11 21:56:19 -07:00
Mohammad Miadh AngkadandMohammad Angkad bf703013f2 [Test] Re-anchor the VLM flashinfer throughput floor (#43762)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-10-11 21:34:50 -07:00
d434ea444d [DSV4.1] Reduce eager Engram extend preparation (#42159)
Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 21:31:55 -07:00
Liangsheng Yin a1001bcf3a [PD] Retract transferred decode requests on a retract pause (#43757) 2026-10-11 21:29:39 -07:00
ChengYM 0d0be8fc98 [AMD][MORI] Add MORI EP V2 support (#40666) 2026-10-11 20:54:13 -07:00
Shunkangz 20c49f12c9 Support kv cache sharding for DSA indexer (#40925) 2026-10-12 11:50:23 +08:00
37d2e5c34f [AMD][V4.1][*/N] Fix DP attention + TP MoE decode NaN and support bounded replay (#43077)
Co-authored-by: wunhuang <wunhuang@amd.com>
Co-authored-by: HAI <hixiao@gmail.com>
2026-10-11 20:47:13 -07:00
adenzhou1350 cd8ed1bb55 [Bugfix] Preserve available host memory for integrated MUSA devices (#43398)
Signed-off-by: Xucheng Zhou <aden1350@outlook.com>
2026-10-11 20:34:58 -07:00
Po-Han Huang (NVIDIA)andClaude Opus 5.5 6ac8602e8d [CI] Fix TestQwenImageModulationCache after Qwen-Image modulation sharding (#43747)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 20:31:29 -07:00
77a15cf4e2 [MemCache] Add shared component size and path APIs to the tree core interface (#43722)
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: Jialin <Jialin@users.noreply.github.com>
2026-10-11 20:14:07 -07:00
e4cfc0c42a [MemCache] Split the SWA mapping move out of locked-full recovery as RecoverEvictedSWA (#43721)
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: Jialin <Jialin@users.noreply.github.com>
2026-10-11 20:11:52 -07:00
WangHemeng 0c52cb1519 [MoE] Skip block-padding rows in the TMA down activation and quant (#42212) 2026-10-12 11:10:30 +08:00
Liangsheng Yin a65b776963 [DSpark] Add SGLANG_DSPARK_USE_GREEDY_DRAFT, an argmax draft accepted as one-hot (#43670) 2026-10-11 15:18:59 -07:00
Liangsheng Yin 3cc23b3b86 [DSpark] Merge the draft TP sync sites into dspark-draft (#43669) 2026-10-11 15:17:47 -07:00
Liangsheng Yin 00ba90fd1a [DSpark] Run the same markov block in the draft graph tail; drop SGLANG_DSPARK_FOLDED_SAMPLING (#43668) 2026-10-11 15:15:59 -07:00
Liangsheng Yin 48abbdb845 [DSpark] Draft the eager markov block through MarkovSampleStep (#43622) 2026-10-11 15:13:02 -07:00
Liangsheng Yin 467209e838 [DSpark] Add MarkovSampleStep, one op for a markov draft step's add and token pick (#43621) 2026-10-11 15:10:58 -07:00
Banruo LiuandShangming Cai d62b9c2480 [KV Events] Publish LoRA adapter names and namespaced block hashes (#42455)
Signed-off-by: Banruo Liu <lichangxinlivingshade@gmail.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
2026-10-11 23:23:30 +08:00
Ke Bao 46a97e5540 Preserve serialized page dtype in NIXL copy fallback (#43593) 2026-10-11 22:39:14 +08:00
Ke Bao b0a75c8196 Share MLA host capacity initialization (#43595) 2026-10-11 22:36:29 +08:00
Ke Bao c938b6e454 Build Mamba host pools from device buffer infos (#43596) 2026-10-11 22:35:52 +08:00
Shuwen Wang ef3e5c83c5 [HiCache] Keep storage workers alive on backend exceptions (#43550) 2026-10-11 22:26:56 +08:00
Shuwen Wang 0923bcd258 [Unified Radix] Invalidate lazy Mamba checkpoints before in-place tracking (#43567) 2026-10-11 21:12:51 +08:00
FeiHong Guo cf88ed4352 [BugFix][DP] Prevent health checks from perturbing user request routing (#35721)
Signed-off-by: Phoenix3334 <Phoenix3334@users.noreply.github.com>
2026-10-11 21:08:55 +08:00
trebladevandMick Qian 586d98cc15 [Diffusion] Add ModelOpt FP8 checkpoint quantization for Pi0.5 (#43062)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-10-11 20:55:59 +08:00
MickandMick Qian 04f740580c [VLM] fix: repair multimodal VMM lease cleanup and source-stream ordering (#43475)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-10-11 19:13:39 +08:00
Sage 53295c6972 [kvevents] fix missing topics in replayed kv events (#43386)
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com>
2026-10-11 17:25:43 +08:00
Liangsheng Yin 0a21a3b798 [Scheduler] Keep a DP-converted decode batch as the running batch (#43283) 2026-10-11 01:23:51 -07:00
jiaqiang-dot-liu 41652225d0 [Sampler] Sort only the max(top_k) leading columns in the torch sampler (#43346) 2026-10-11 01:00:59 -07:00
Liangsheng Yin dab108b9f3 [Scheduler] Hold back decode rows on a mixed step when they do not fit the KV pool (#43290) 2026-10-11 00:18:27 -07:00
Liangsheng Yin 89e24e7a20 [KV canary] Expect sweep counters on the target runner only (#43286) 2026-10-10 23:52:17 -07:00
Liangsheng Yin 448759b235 [Scheduler] Requeue preempted requests when admission admits nothing (#43298) 2026-10-10 22:46:55 -07:00
Qiaolin Yu 047405f534 [qwen 3.8 next] Fuse QSA prefill KV gathering and dtype conversion (#41933) 2026-10-10 22:40:23 -07:00