Commit Graph
14123 Commits
Author SHA1 Message Date
Zhang Yiyang (SII)andMick Qian 23c2a026db [diffusion] model: support MOSS-Image (#44325)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-10-15 19:23:56 +08:00
Cheng Wan f7ad4345b8 [Refactor] deepseek-v4: Use explicit stage boundaries and row transport (#43607) 2026-10-15 02:05:28 -07:00
Cheng Wan 3a835cf417 [Refactor] inkling: Use short-convolution stage boundaries (#43606) 2026-10-15 02:04:46 -07:00
Cheng Wan e1f9fce7b3 [Refactor] layer-boundary: Bind residual completion to producer updates (#43605) 2026-10-15 02:04:02 -07:00
Cheng Wan d59c3448b0 [Fix] kimi-k3: Preserve pipeline graph bank inputs and bound chunk profiling (#43604) 2026-10-15 02:03:27 -07:00
93dbe88183 Fuse MiMo verification RoPE and KV cache writes (#43106)
Co-authored-by: lianghougong <lianghouhong@xiaohongshu.com>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
2026-10-15 16:15:24 +08:00
86f4ede4fb [Fix] dsa: Make the fused top-k exact on an overflowing threshold bin (#37941)
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: DarkSharpness <2040703891@qq.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-15 15:52:48 +08:00
9ea5472833 [HiCache] Add the page_unified host KV layout and the unified L3 key (#39727)
Co-authored-by: huangtingwei <141888744+huangtingwei9988@users.noreply.github.com>
Co-authored-by: Zhangheng <hzh0425@apache.org>
2026-10-15 15:46:13 +08:00
33618d4f00 [Feat] runtime: Include FlashInfer’s default tactic in startup tuning. (#43927)
Co-authored-by: Po-Han Huang <pohanh@nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 15:30:45 +08:00
Liangsheng Yin a776851c97 [Refactor] scheduler: Check the prefill budget at the admission loop head; drop a redundant PD gate (#44420) 2026-10-15 00:16:56 -07:00
fa66e7d816 [NPU] [Bugfix] Fix rope sin cos pre-calculation for Qwen3.5 during Pipeline Parallelism (#29753)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Артем Савкин <58187114+OrangeRedeng@users.noreply.github.com>
2026-10-15 09:58:47 +03:00
Dayuxiaoshui a50c17e979 [diffusion] perf: fuse the Wan VAE conv bias epilogue and tile the RMSNorm+SiLU kernel (#38650) 2026-10-15 13:39:27 +08:00
676c038aa5 [diffusion] fix: fix the load duration missing from subclassed server tests (#40657)
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Michael <michaelzhang-ai@users.noreply.github.com>
Co-authored-by: quitenode <quitenode@users.noreply.github.com>
Co-authored-by: jacky.cheng <yichiche@amd.com>
2026-10-15 13:38:23 +08:00
Liangsheng Yin f9bdb74a5a [Fix] scheduler: Count a request's held req_to_token row once at admission (#44401) 2026-10-14 22:29:10 -07:00
Yuxingwang-intel 93842f52fe [CPU]: Add group-aware CPU SHM collective kernels (#35435) 2026-10-15 13:22:20 +08:00
1d809bbbbd [Fix] DSv4.1: Size the encoder-replay SWA pool by the DSpark draft's stages, and keep more prefix tails (#43502)
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
Co-authored-by: Po-Han Huang <pohanh@nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Yuwei An <ayw.sirius19@gmail.com>
2026-10-14 22:04:58 -07:00
Khoa Pham d4daf52369 [Fix] dsv41: Bound MoE routing to live SWA graph-tail rows (#44347) 2026-10-14 21:14:28 -07:00
WenhaoZhangandMick c92108bb93 [diffusion] comfyui: grouped SGLDOptions, native model loader and model-aware preflight (#44058)
Co-authored-by: Mick <mickjagger19@icloud.com>
2026-10-15 11:59:03 +08:00
Liangsheng Yin 3b5ae4ea86 [Fix] scheduler: Drop finished requests from the running batch before every admission (#44351) 2026-10-14 20:40:05 -07:00
2528cd2b91 [PD] Send DSA indexer state only past decode_prefix_len (#44140)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
2026-10-15 11:25:18 +08:00
Avaya Aggarwalandniehen6174 0cf683ea8d [diffusion] comfyui: return several server-mode images as one IMAGE batch (#44088)
Co-authored-by: niehen6174 <1639206518@qq.com>
2026-10-15 10:38:37 +08:00
xiaofei-zheng a12814c158 [Perf] [AMD] dsa: Enable the gfx950 fused DSA indexer for block-FP8 wq_b/wk (#43643) 2026-10-15 09:36:16 +08:00
9d68ecce2e [DeepSeek-V4.1] DeepSelect cluster top-k for long rows and bf16 index logits (#44051)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
2026-10-15 09:00:19 +08:00
xiaofei-zheng c5c4bbbd25 [Perf] [AMD] dsa: Run DSA prefill under breakable CUDA graphs on HIP (#43642) 2026-10-15 08:59:34 +08:00
Michaelandjacky.cheng 8333ccfccb [Fix] [AMD] diffusion: Compare ReduceOp by identity so the compiled TP all-reduce stays one graph (#43770)
Co-authored-by: jacky.cheng <yichiche@amd.com>
2026-10-14 17:40:07 -07:00
Yuwei An 64a0a08f0a [DSv4.1] Support TRT-LLM decoder bounded replay with BCG (#44326) 2026-10-14 17:39:52 -07:00
YAMYandYuwei An c891c5c7ca [DSv4.1] Fold encoder-SWA replay into the extend forward (#43166)
Co-authored-by: Yuwei An <ayw.sirius19@gmail.com>
2026-10-14 17:27:07 -07:00
Shuwen Wang 3c28614f12 [Fix] hicache: Defer query beliefs until the rank-synchronized hit drain (#43108) 2026-10-14 17:25:08 -07:00
Aditya Singhandhnyls2002 a6a8c56015 [Fix] detokenizer router: Route by rid without tokenizer worker ipcs and broadcast logging config (#33715)
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-10-14 17:20:21 -07:00
0536cec73d [Feat] metrics: Expose multi-detokenizer router CPU usage (#43323)
Co-authored-by: raghul chandramouli <raghul@c1xind0180-0243.local>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-10-14 17:17:22 -07:00
Junnan Liandhnyls2002 0f36c8015c [Bug] Fix multi-tokenizer BatchEmbeddingOutput repack dropping fields (#31892)
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-10-14 16:56:56 -07:00
209f8220d6 [Fix] sglang-server: Honour extra_key on the Rust frontend's /generate (#44207)
Co-authored-by: fei-xx <fei-xx@users.noreply.github.com>
Co-authored-by: shodoco <shodoco@users.noreply.github.com>
2026-10-14 16:44:31 -07:00
e7d6a88bfc [HiCache] Buffer-only L3 for an auxiliary SWA window; load back only missing window rows (#43932)
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: houseroad <houseroad@users.noreply.github.com>
Co-authored-by: xiezhq-hermann <xiezhq-hermann@users.noreply.github.com>
2026-10-14 16:43:24 -07:00
aa8793f33b [Rust] Serve scheduler metrics on the Rust frontend's /metrics (#43942)
Co-authored-by: hanminglu <hanminglu@users.noreply.github.com>
Co-authored-by: shodoco <shodoco@users.noreply.github.com>
2026-10-14 16:43:06 -07:00
2a9f6a25a7 [Core] Add subclass hooks for the generation request class and optimistic prefill eligibility (#44113)
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: CuiCoco <CuiCoco@users.noreply.github.com>
2026-10-14 16:42:30 -07:00
5f2861bf59 [Perf] session: Cache multimodal prompt token counts across turns (#44112)
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: joshuadeng <joshuadeng@users.noreply.github.com>
2026-10-14 16:41:34 -07:00
FREETRUMPandhnyls2002 97d3150a6a [Perf] Cache msgspec struct fields in the batch splitter (#43520)
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-10-14 14:51:26 -07:00
4273814771 [Perf] detokenizer: Group return IPC sends by tokenizer worker (#44288)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
2026-10-14 14:26:51 -07:00
Charles Chenandalphabetc1 d3dc14eb38 [mem_cache][PD] Fix decode radix cache ownership and allow EAGLE/EAGLE3 with decode radix cache (#40857)
Co-authored-by: alphabetc1 <2508695655@qq.com>
2026-10-14 14:19:51 -07:00
Yoray Zack 87ed1b9526 [4/N] elastic-ep: Scale at DP-attention replica granularity (#33728) 2026-10-15 01:17:01 +08:00
Michael 1737251730 [Test] [AMD] diffusion: Run the IPC all-to-all transport tests only on CUDA (#44096) 2026-10-14 23:04:23 +08:00
010776e94f [KV events] Publish opaque LoRA block IDs and adapter names (#42817)
Co-authored-by: Kan Wu <kan.wu@radixark.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
2026-10-14 22:56:28 +08:00
triple-mu 5e90312eb1 [diffusion] perf: support bit-exact norm/bias fusions and channels_last decode for Qwen-Image 2.1(#41340) 2026-10-14 20:47:46 +08:00
bab9c81fea fix: FA4 paged-KV zero-size page-table registers on SM90 (#35757)
Co-authored-by: razorback16 <razorback16@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: BBuf <1182563586@qq.com>
2026-10-14 19:10:15 +08:00
Mohammad Miadh AngkadandMohammad Angkad c82e8bd5fe [Fix] DeepSeek-V4.1: Align paged indexer score rows for DeepSelect decode (#44210)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-10-14 17:31:19 +08:00
df1a089cd3 [Spec] Fix DFLASH verify mask policy for wrapped attention backends (#43054)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
2026-10-14 17:15:36 +08:00
c54ba66cd4 [Perf] Enable native MXFP4 routing and CuTeDSL fusion for MiMo (#43639)
Co-authored-by: lianghougong <lianghouhong@xiaohongshu.com>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
2026-10-14 15:39:44 +08:00
Mohammad Miadh AngkadandMohammad Angkad 6daae9c98e [Perf] Bind scheduler CPUs to the GPU's NUMA node when memory policy is not permitted (#44023)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-10-14 00:01:32 -07:00
Ma Mingfei 68c18a5701 Revert "[XPU] Support Gemma-3 bidirectional image attention in XPUAttentionBackend" (#44216) 2026-10-14 14:27:46 +08:00
desmond-intel b87d62e581 [XPU] Support Gemma-3 bidirectional image attention in XPUAttentionBackend (#39805) 2026-10-14 14:24:04 +08:00