Commit Graph
14078 Commits
Author SHA1 Message Date
e0608d890e fix: use dp_size instead of attn_dp_size for msprobe rank id (#43041)
Co-authored-by: l30044004 <louyujing@huawei.com>
Co-authored-by: Claude Code <noreply@anthropic.com>
2026-10-09 09:40:43 +03:00
Gabriel Wu 217980566a fix(fa4): isolate sheared-bias workspaces by CUDA stream (#33000) 2026-10-08 23:37:02 -07:00
areneau-cfandJimmy Shong bd2d73daa5 [model] support Clef and Clef-Flash decision models on /v1/systemone (#42721)
Co-authored-by: Jimmy Shong <69131491+Jiminator@users.noreply.github.com>
2026-10-09 14:25:52 +08:00
Michael 51f7ea84c7 [AMD] Keep MiniMax decode-score tile at 128 tokens off gfx950 (#42772) 2026-10-08 23:22:19 -07:00
08fc290f48 [AMD] gfx950 small-batch MoE: expert-count gate for the small sort and FP8 block-scale small-M kernel (#41982)
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: jacky.cheng <yichiche@amd.com>
2026-10-08 23:05:50 -07:00
cba1d07a94 [Memory] Share one full prefill graph output buffer across shapes for tuple and list outputs (#43176)
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: 842974287 <842974287@users.noreply.github.com>
2026-10-08 23:05:29 -07:00
e14ef4b658 [Scheduler] Skip the decode for a request whose last token is already in flight (#42720)
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
2026-10-08 23:00:46 -07:00
e46aaaa789 [diffusion] feat: accept strided row views in the packed QKV all-to-all (#40588)
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-10-09 13:55:31 +08:00
Colin Z 3c1703d2ff [AMD] Support Quark DeepSeek-V4-MXFP4 checkpoints on the DSV4 stack, enabling shared-expert fusion and MTP (#31054) 2026-10-08 22:54:49 -07:00
Yuan Luoandluoyuan.luo 9a1aeacc3b [GLM-5.3-Flash] Capture the KDA extend in breakable prefill CUDA graphs (#41105)
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
2026-10-09 13:37:20 +08:00
33e7b4cc4b [diffusion] perf: decode plain base64 MiniMax-H3 materials without a per-character loop (#43117)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 13:25:23 +08:00
Chetan Kumar VermaandClaude Fable 5.1 5d8e98b7fd [XPU][Diffusion] Plain-torch scale-shift for inductor (#41885)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-09 13:07:14 +08:00
saatwiknagpal 6e7beace14 [diffusion] fix: fix int32 offset overflow in the Ulysses packed-QKV pack kernel (#43140) 2026-10-09 11:51:46 +08:00
Baizhou Zhang 33bffcd490 [CP] GLM-5.3-Flash Prefill CP implemented on layer boundaries (#42044) 2026-10-08 20:42:11 -07:00
metamergebotandjoshuadeng 7859eff6ca [srt] Inject "__req__" into custom_params only for custom logit processors (#43181)
Co-authored-by: joshuadeng <joshuadeng@users.noreply.github.com>
2026-10-08 20:40:50 -07:00
4ee3f0f370 [Session] Skip the KV free when page alignment empties a streaming-session rewind (#43183)
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: Joshua Deng <91448271+joshuadeng@users.noreply.github.com>
2026-10-08 20:40:25 -07:00
4e3eaba619 [HiCache] Re-admit a split node's queued buffer-mode write instead of dropping it (#43174)
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: xiezhq-hermann <xiezhq-hermann@users.noreply.github.com>
2026-10-08 20:39:41 -07:00
HEJIAN SANGandClaude Opus 5.5 3e13f1b2c7 [Perf] Serialize OpenAI chat/completion responses in one pass (#42889)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 11:17:25 +08:00
ormandjandkpham-sgl 438df9a2a4 [Spec] Fix wrong tokens from compact-verify CUDA graphs when a token tier is captured more than once (#42709)
Co-authored-by: kpham-sgl <khoa.pham@radixark.ai>
2026-10-08 19:49:23 -07:00
Khoa Pham 8e19db19b0 [KDA] Default safe-gate Blackwell prefill to FlashInfer (#43246) 2026-10-08 19:46:28 -07:00
Chetan Kumar VermaandClaude Opus 5.5 fedab5537b [XPU] Enable ring attention on XPUAttentionBackend (#41406)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-09 10:41:11 +08:00
MickandMick Qian df1e4f3f2b [diffusion] refactor: share SANA-WM refiner block computation (#42980)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-10-09 10:38:36 +08:00
Liangsheng Yin f6569bce4d [MemCache] Skip KV events and eviction config on a disabled radix cache (#43253) 2026-10-08 19:27:39 -07:00
Polisetty V R K Jyothendra Varma e54f08c860 [XPU] Misc changes to support XPU (#39817)
Signed-off-by: P V R K Jyothendra Varma <polisetty.v.r.k.jyothendra.varma@intel.com>
2026-10-09 10:26:46 +08:00
Liangsheng Yin e0ab91e3ca [MemCache] Rename SWAChunkCapPoolConfigurator to SWARequestCapPoolConfigurator (#43202) 2026-10-08 17:50:51 -07:00
b2786a670e [MemCache] Count reclaimable tokens in the unified tree ledgers (#43185)
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com>
Co-authored-by: xiezhq-hermann <xiezhq-hermann@users.noreply.github.com>
2026-10-08 17:48:11 -07:00
Liangsheng Yin b4fe282836 [MemCache] Delete ChunkCache (#43201) 2026-10-08 17:31:16 -07:00
Liangsheng Yin 508ba79688 [MemCache] Serve disabled hybrid-SWA models with UnifiedRadixCache (#43200) 2026-10-08 17:30:01 -07:00
Liangsheng Yin 265efe5e01 [MemCache] Serve disabled full-attention models with UnifiedRadixCache (#43199) 2026-10-08 17:27:24 -07:00
Liangsheng Yin 977e2719c7 [MemCache] Serve disabled pure-SWA models with PureSWARadixCache (#43198) 2026-10-08 17:25:54 -07:00
Liangsheng Yin 57ce47862d [MemCache] Report no prefix sharing from a disabled RadixCache (#43197) 2026-10-08 17:23:11 -07:00
Khoa Pham fc3385d530 feat(kda): integrate FlashInfer prefill checkpoints (#41400) 2026-10-08 17:16:33 -07:00
Khoa Pham 1e77db3215 [Perf] Avoid gate copies in gated RMSNorm (#43227) 2026-10-08 17:11:58 -07:00
20d1f3f56a [Scheduler] Refresh waiting requests' cached prefixes under FCFS-like policies with LRU eviction (#41593)
Signed-off-by: Pei Li <peili@nvidia.com>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
Co-authored-by: Po-Han Huang <pohanh@nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-10-08 17:05:04 -07:00
bb43927404 [PD] Charge decode admission the page-rounded allocation when the radix cache is disabled (#43173)
Co-authored-by: hanming-lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: xiezhq-hermann <xiezhq-hermann@users.noreply.github.com>
2026-10-08 16:48:31 -07:00
Liangsheng Yin 2e64740442 [CI] Avoid Hugging Face Hub API calls when the local model cache is complete (#43229) 2026-10-08 16:27:31 -07:00
Liangsheng Yin ae4da8cd78 [MemCache] Skip SWA request-cap pool sizing when chunked prefill is off (#43203) 2026-10-08 16:19:35 -07:00
2abb761639 [metrics] Count per-rank DP attention pairs and their imbalance (#43180)
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: hanming-lu <hanming-lu@users.noreply.github.com>
2026-10-08 15:01:37 -07:00
metamergebotandHanming Lu 0b658c12ff [Bench][MoE] Deal the simulated round-robin experts evenly across EP ranks (#43188)
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
2026-10-08 15:01:16 -07:00
Siyuan Chen b0e06359ac [DSV4.1][PD][5/N] Decode node support DP attention (#40177) 2026-10-08 14:42:16 -07:00
Xuanteng Huang d20bdc1443 Add per-token NVFP4 MoE support for ReLU2 activation (#39199)
Signed-off-by: Xuanteng Huang <xuanteng.huang@outlook.com>
2026-10-08 14:28:22 -07:00
ShunkangzandClaude Opus 5 3d8d44a4a9 [Qwen3.8 CP 3/4] Collocated prefill CP integration (#39723)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-10-08 14:20:27 -07:00
Sage 5a5ea81036 [rust-server] warm up each rust listener in-process (#43004)
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com>
2026-10-08 14:09:46 -07:00
e902207a4e [Speculative] Support block verification for the DFlash family (#43012)
Co-authored-by: maocheng <mao.cheng@MacBook-Pro-MD663G60VV.local>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 13:59:52 -07:00
d1ba943016 [Model] Add support for EmbeddingGemma 2 (#42782)
Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com>
Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 13:54:45 -07:00
metamergebotandhanming-lu 93f2acf614 [Metrics] Use widely supported bucket boundaries for the DP attention imbalance ratio (#43187)
Co-authored-by: hanming-lu <hanming-lu@users.noreply.github.com>
2026-10-08 12:58:58 -07:00
Ming Yang 3fef129ff8 [PD] Allow state-only KV checksum inputs (#42904) 2026-10-08 12:54:03 -07:00
075fd8fa0b [Mamba] Describe the track snapshot dtype by its kernel instead of fp32 (#43192)
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: jiayisuse <jiayisuse@users.noreply.github.com>
2026-10-08 12:47:46 -07:00
e3c5719713 [DFlash] Build the attention QKV projection through an overridable hook (#43179)
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: 842974287 <842974287@users.noreply.github.com>
2026-10-08 12:46:28 -07:00
kunchengit 00b3a48753 [dLLM] Add JointThresholdInDel algorithm for insertion & deletion decoding (#31773) 2026-10-08 12:44:14 -07:00