Commit Graph
20367 Commits
Author SHA1 Message Date
amote-i abf92826ee [NPU] [DOC] update NPU best practices and model deployment docs to CANN 9.1.0 (#44432) 2026-10-15 20:22:54 +08:00
Zhang Yiyang (SII)andMick Qian 23c2a026db [diffusion] model: support MOSS-Image (#44325)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-10-15 19:23:56 +08:00
Cheng Wan f7ad4345b8 [Refactor] deepseek-v4: Use explicit stage boundaries and row transport (#43607) 2026-10-15 02:05:28 -07:00
Cheng Wan 3a835cf417 [Refactor] inkling: Use short-convolution stage boundaries (#43606) 2026-10-15 02:04:46 -07:00
Cheng Wan e1f9fce7b3 [Refactor] layer-boundary: Bind residual completion to producer updates (#43605) 2026-10-15 02:04:02 -07:00
Cheng Wan d59c3448b0 [Fix] kimi-k3: Preserve pipeline graph bank inputs and bound chunk profiling (#43604) 2026-10-15 02:03:27 -07:00
Eddie Zhou 71a17dbc24 [AMD][CI] setup: Skip redundant pip-cache chown walk and MORI rebuild (#42148) 2026-10-15 16:21:46 +08:00
93dbe88183 Fuse MiMo verification RoPE and KV cache writes (#43106)
Co-authored-by: lianghougong <lianghouhong@xiaohongshu.com>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
2026-10-15 16:15:24 +08:00
kangwangamd 28703575f9 [AMD] Update ROCm AITER pin to 2bb549102 (#44379) 2026-10-15 16:10:41 +08:00
GeorgeSherifandXiaoyu Zhang 8f853b0041 [Doc] cookbook: Add Jais 3 deployment recipes (#42766)
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
2026-10-15 16:09:44 +08:00
86f4ede4fb [Fix] dsa: Make the fused top-k exact on an overflowing threshold bin (#37941)
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: DarkSharpness <2040703891@qq.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-15 15:52:48 +08:00
9ea5472833 [HiCache] Add the page_unified host KV layout and the unified L3 key (#39727)
Co-authored-by: huangtingwei <141888744+huangtingwei9988@users.noreply.github.com>
Co-authored-by: Zhangheng <hzh0425@apache.org>
2026-10-15 15:46:13 +08:00
33618d4f00 [Feat] runtime: Include FlashInfer’s default tactic in startup tuning. (#43927)
Co-authored-by: Po-Han Huang <pohanh@nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 15:30:45 +08:00
Liangsheng Yin a776851c97 [Refactor] scheduler: Check the prefill budget at the admission loop head; drop a redundant PD gate (#44420) 2026-10-15 00:16:56 -07:00
fa66e7d816 [NPU] [Bugfix] Fix rope sin cos pre-calculation for Qwen3.5 during Pipeline Parallelism (#29753)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Артем Савкин <58187114+OrangeRedeng@users.noreply.github.com>
2026-10-15 09:58:47 +03:00
ashwini rathiandClaude Opus 5.5 01ac2cf36d [CI] [XPU] nightly: Split Intel Main and Wheel workflows (#44108)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 14:56:27 +08:00
MickandMick Qian f2b73d4f9b [diffusion] doc: document reference resizing, LoRA auto mode and image profiling (#44377)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-10-15 13:41:44 +08:00
Dayuxiaoshui a50c17e979 [diffusion] perf: fuse the Wan VAE conv bias epilogue and tile the RMSNorm+SiLU kernel (#38650) 2026-10-15 13:39:27 +08:00
676c038aa5 [diffusion] fix: fix the load duration missing from subclassed server tests (#40657)
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Michael <michaelzhang-ai@users.noreply.github.com>
Co-authored-by: quitenode <quitenode@users.noreply.github.com>
Co-authored-by: jacky.cheng <yichiche@amd.com>
2026-10-15 13:38:23 +08:00
Liangsheng Yin f9bdb74a5a [Fix] scheduler: Count a request's held req_to_token row once at admission (#44401) 2026-10-14 22:29:10 -07:00
Yuxingwang-intel 93842f52fe [CPU]: Add group-aware CPU SHM collective kernels (#35435) 2026-10-15 13:22:20 +08:00
Michael 4dc9b5c68c [Test] [AMD] diffusion: Expect CUDA-only fast paths to be off on ROCm in kernel tests (#44311) 2026-10-15 13:18:01 +08:00
1d809bbbbd [Fix] DSv4.1: Size the encoder-replay SWA pool by the DSpark draft's stages, and keep more prefix tails (#43502)
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
Co-authored-by: Po-Han Huang <pohanh@nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Yuwei An <ayw.sirius19@gmail.com>
2026-10-14 22:04:58 -07:00
kkandroot e573d91275 [CI] tests: Fix stale fakes in the CP strategy and request-class override unit tests (#44380)
Co-authored-by: root <root@crsuse2-m2m-223.us-east2-a.compute.internal>
2026-10-14 21:35:20 -07:00
Khoa Pham d4daf52369 [Fix] dsv41: Bound MoE routing to live SWA graph-tail rows (#44347) 2026-10-14 21:14:28 -07:00
AMD-yanfeiwang 08f15e76bb [Feat] sgl-router: Add --dp-rank-policy round_robin for --dp-aware (#43511) 2026-10-15 12:03:01 +08:00
WenhaoZhangandMick c92108bb93 [diffusion] comfyui: grouped SGLDOptions, native model loader and model-aware preflight (#44058)
Co-authored-by: Mick <mickjagger19@icloud.com>
2026-10-15 11:59:03 +08:00
Liangsheng Yin 3b5ae4ea86 [Fix] scheduler: Drop finished requests from the running batch before every admission (#44351) 2026-10-14 20:40:05 -07:00
2528cd2b91 [PD] Send DSA indexer state only past decode_prefix_len (#44140)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
2026-10-15 11:25:18 +08:00
Avaya Aggarwalandniehen6174 0cf683ea8d [diffusion] comfyui: return several server-mode images as one IMAGE batch (#44088)
Co-authored-by: niehen6174 <1639206518@qq.com>
2026-10-15 10:38:37 +08:00
Thomas Wang 4851d182a1 [Doc][AMD] Update v41 amd cookbook 1015 (#44368) 2026-10-14 19:33:08 -07:00
xiaofei-zheng a12814c158 [Perf] [AMD] dsa: Enable the gfx950 fused DSA indexer for block-FP8 wq_b/wk (#43643) 2026-10-15 09:36:16 +08:00
Yuhao YangandClaude Opus 5.5 9fa9308e95 [Doc] cookbook: Add Step 5 Preview cookbook (#44357)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-15 09:21:06 +08:00
9d68ecce2e [DeepSeek-V4.1] DeepSelect cluster top-k for long rows and bf16 index logits (#44051)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
2026-10-15 09:00:19 +08:00
xiaofei-zheng c5c4bbbd25 [Perf] [AMD] dsa: Run DSA prefill under breakable CUDA graphs on HIP (#43642) 2026-10-15 08:59:34 +08:00
Michaelandjacky.cheng 8333ccfccb [Fix] [AMD] diffusion: Compare ReduceOp by identity so the compiled TP all-reduce stays one graph (#43770)
Co-authored-by: jacky.cheng <yichiche@amd.com>
2026-10-14 17:40:07 -07:00
Yuwei An 64a0a08f0a [DSv4.1] Support TRT-LLM decoder bounded replay with BCG (#44326) 2026-10-14 17:39:52 -07:00
YAMYandYuwei An c891c5c7ca [DSv4.1] Fold encoder-SWA replay into the extend forward (#43166)
Co-authored-by: Yuwei An <ayw.sirius19@gmail.com>
2026-10-14 17:27:07 -07:00
Shuwen Wang 3c28614f12 [Fix] hicache: Defer query beliefs until the rank-synchronized hit drain (#43108) 2026-10-14 17:25:08 -07:00
Aditya Singhandhnyls2002 a6a8c56015 [Fix] detokenizer router: Route by rid without tokenizer worker ipcs and broadcast logging config (#33715)
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-10-14 17:20:21 -07:00
0536cec73d [Feat] metrics: Expose multi-detokenizer router CPU usage (#43323)
Co-authored-by: raghul chandramouli <raghul@c1xind0180-0243.local>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-10-14 17:17:22 -07:00
Junnan Liandhnyls2002 0f36c8015c [Bug] Fix multi-tokenizer BatchEmbeddingOutput repack dropping fields (#31892)
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-10-14 16:56:56 -07:00
209f8220d6 [Fix] sglang-server: Honour extra_key on the Rust frontend's /generate (#44207)
Co-authored-by: fei-xx <fei-xx@users.noreply.github.com>
Co-authored-by: shodoco <shodoco@users.noreply.github.com>
2026-10-14 16:44:31 -07:00
e7d6a88bfc [HiCache] Buffer-only L3 for an auxiliary SWA window; load back only missing window rows (#43932)
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: houseroad <houseroad@users.noreply.github.com>
Co-authored-by: xiezhq-hermann <xiezhq-hermann@users.noreply.github.com>
2026-10-14 16:43:24 -07:00
aa8793f33b [Rust] Serve scheduler metrics on the Rust frontend's /metrics (#43942)
Co-authored-by: hanminglu <hanminglu@users.noreply.github.com>
Co-authored-by: shodoco <shodoco@users.noreply.github.com>
2026-10-14 16:43:06 -07:00
2a9f6a25a7 [Core] Add subclass hooks for the generation request class and optimistic prefill eligibility (#44113)
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: CuiCoco <CuiCoco@users.noreply.github.com>
2026-10-14 16:42:30 -07:00
5f2861bf59 [Perf] session: Cache multimodal prompt token counts across turns (#44112)
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: joshuadeng <joshuadeng@users.noreply.github.com>
2026-10-14 16:41:34 -07:00
9732188a58 [Rust] Serve /weights_checker from the Rust frontend (#43926)
Co-authored-by: hanminglu <hanminglu@users.noreply.github.com>
Co-authored-by: shodoco <shodoco@users.noreply.github.com>
2026-10-14 16:41:10 -07:00
FREETRUMPandhnyls2002 97d3150a6a [Perf] Cache msgspec struct fields in the batch splitter (#43520)
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-10-14 14:51:26 -07:00
4273814771 [Perf] detokenizer: Group return IPC sends by tokenizer worker (#44288)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
2026-10-14 14:26:51 -07:00