Commit Graph
14495 Commits
Author SHA1 Message Date
Aditya Singhandhnyls2002 a6a8c56015 [Fix] detokenizer router: Route by rid without tokenizer worker ipcs and broadcast logging config (#33715)
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-10-14 17:20:21 -07:00
0536cec73d [Feat] metrics: Expose multi-detokenizer router CPU usage (#43323)
Co-authored-by: raghul chandramouli <raghul@c1xind0180-0243.local>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-10-14 17:17:22 -07:00
Junnan Liandhnyls2002 0f36c8015c [Bug] Fix multi-tokenizer BatchEmbeddingOutput repack dropping fields (#31892)
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-10-14 16:56:56 -07:00
209f8220d6 [Fix] sglang-server: Honour extra_key on the Rust frontend's /generate (#44207)
Co-authored-by: fei-xx <fei-xx@users.noreply.github.com>
Co-authored-by: shodoco <shodoco@users.noreply.github.com>
2026-10-14 16:44:31 -07:00
e7d6a88bfc [HiCache] Buffer-only L3 for an auxiliary SWA window; load back only missing window rows (#43932)
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: houseroad <houseroad@users.noreply.github.com>
Co-authored-by: xiezhq-hermann <xiezhq-hermann@users.noreply.github.com>
2026-10-14 16:43:24 -07:00
aa8793f33b [Rust] Serve scheduler metrics on the Rust frontend's /metrics (#43942)
Co-authored-by: hanminglu <hanminglu@users.noreply.github.com>
Co-authored-by: shodoco <shodoco@users.noreply.github.com>
2026-10-14 16:43:06 -07:00
2a9f6a25a7 [Core] Add subclass hooks for the generation request class and optimistic prefill eligibility (#44113)
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: CuiCoco <CuiCoco@users.noreply.github.com>
2026-10-14 16:42:30 -07:00
5f2861bf59 [Perf] session: Cache multimodal prompt token counts across turns (#44112)
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: joshuadeng <joshuadeng@users.noreply.github.com>
2026-10-14 16:41:34 -07:00
FREETRUMPandhnyls2002 97d3150a6a [Perf] Cache msgspec struct fields in the batch splitter (#43520)
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-10-14 14:51:26 -07:00
4273814771 [Perf] detokenizer: Group return IPC sends by tokenizer worker (#44288)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
2026-10-14 14:26:51 -07:00
Charles Chenandalphabetc1 d3dc14eb38 [mem_cache][PD] Fix decode radix cache ownership and allow EAGLE/EAGLE3 with decode radix cache (#40857)
Co-authored-by: alphabetc1 <2508695655@qq.com>
2026-10-14 14:19:51 -07:00
Yoray Zack 87ed1b9526 [4/N] elastic-ep: Scale at DP-attention replica granularity (#33728) 2026-10-15 01:17:01 +08:00
Michael 1737251730 [Test] [AMD] diffusion: Run the IPC all-to-all transport tests only on CUDA (#44096) 2026-10-14 23:04:23 +08:00
010776e94f [KV events] Publish opaque LoRA block IDs and adapter names (#42817)
Co-authored-by: Kan Wu <kan.wu@radixark.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
2026-10-14 22:56:28 +08:00
triple-mu 5e90312eb1 [diffusion] perf: support bit-exact norm/bias fusions and channels_last decode for Qwen-Image 2.1(#41340) 2026-10-14 20:47:46 +08:00
bab9c81fea fix: FA4 paged-KV zero-size page-table registers on SM90 (#35757)
Co-authored-by: razorback16 <razorback16@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: BBuf <1182563586@qq.com>
2026-10-14 19:10:15 +08:00
Mohammad Miadh AngkadandMohammad Angkad c82e8bd5fe [Fix] DeepSeek-V4.1: Align paged indexer score rows for DeepSelect decode (#44210)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-10-14 17:31:19 +08:00
df1a089cd3 [Spec] Fix DFLASH verify mask policy for wrapped attention backends (#43054)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
2026-10-14 17:15:36 +08:00
c54ba66cd4 [Perf] Enable native MXFP4 routing and CuTeDSL fusion for MiMo (#43639)
Co-authored-by: lianghougong <lianghouhong@xiaohongshu.com>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
2026-10-14 15:39:44 +08:00
Mohammad Miadh AngkadandMohammad Angkad 6daae9c98e [Perf] Bind scheduler CPUs to the GPU's NUMA node when memory policy is not permitted (#44023)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-10-14 00:01:32 -07:00
Ma Mingfei 68c18a5701 Revert "[XPU] Support Gemma-3 bidirectional image attention in XPUAttentionBackend" (#44216) 2026-10-14 14:27:46 +08:00
desmond-intel b87d62e581 [XPU] Support Gemma-3 bidirectional image attention in XPUAttentionBackend (#39805) 2026-10-14 14:24:04 +08:00
Shunkangz 0d130f1915 [Qwen3.8 CP 4/4] Prefill context parallel E2E support of Qwen3.8 (#39724) 2026-10-13 23:19:46 -07:00
7ca4cbfcef [Disagg] Return prompt routed experts from the PD prefill server (#44002)
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
2026-10-13 23:13:44 -07:00
AndyLi429andAndyLi429 05bb5afb62 [NPU][Bugfix] use CANN libdevice for DSpark Markov sampling (#43849)
Co-authored-by: AndyLi429 <AndyLi429@noreply.gitcode.com>
2026-10-14 14:12:22 +08:00
ANSHUMAN TRIPATHY 30bdd09aed DSPARK enable for XPU (#33739) 2026-10-14 13:10:24 +08:00
Khoa Pham e00754bab1 [DSV4.1] Derive the late-layer DP geometry from the captured shape under the breakable prefill graph (#44144) 2026-10-13 21:54:35 -07:00
Chetan Kumar VermaandClaude Opus 5.5 2c1e85964e [diffusion] cache-dit: respect CFG parallel for dual-transformer models (#43704)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 12:25:07 +08:00
xinchen9andMa Mingfei 2428d0f531 [XPU] Use the shared fused decode-metadata kernel in the intel_xpu ba… (#42122)
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
2026-10-14 12:10:12 +08:00
16176278d0 [diffusion] perf: replay small Triton wrappers' launches per operand signature (#43859)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 11:18:19 +08:00
Mohammad Miadh AngkadandMohammad Angkad 4f751da975 [Agents] Add a rule to prefer sgl-eval for accuracy evals (#44040)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-10-13 20:17:04 -07:00
53fe21cdfd [diffusion] perf: replace padded rows without a host sync for Z-Image (#44069)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 11:15:28 +08:00
xiaofei-zhengandXiaoyu Zhang 7e226b1b71 [AMD] Use the preallocated EAGLE topk=1 draft chain on HIP (#43640)
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
2026-10-14 11:06:07 +08:00
5efb7384c1 [diffusion] comfyui: consolidate 6 unrelated plugin fixes (#43493)
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: niehen6174 <1639206518@qq.com>
2026-10-14 10:56:09 +08:00
Ziyi XuandClaude Opus 5.5 4f5fcbfef5 [JIT] Clean up kernel headers: move runtime helpers, add once.h (#41147)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 10:46:51 +08:00
yuyu5333andClaude Code b50fdeb0ef [DeepSeek V4.1] Use DeepSelect for Hopper full top-k decode. (#41561)
Co-authored-by: Claude Code <noreply@anthropic.com>
2026-10-14 10:46:12 +08:00
13a6dee99b [Fix] moe_sum_reduce: gate the bf16 vectorized path on hidden % 16 (#42837)
Signed-off-by: Monishver Chandrasekaran <monishverchandrasekaran@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
2026-10-14 10:32:50 +08:00
Mohammad Miadh AngkadandMohammad Angkad 7f0434e4af [GLM-5.3-Flash] Drop the NextN layer from the KDA state pool (#44098)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-10-13 19:31:02 -07:00
Avaya Aggarwal ac187c0cd5 [diffusion] Bound frame_interpolation_exp to RIFE's supported range (#43014)
Signed-off-by: onepunchmonk <aggarwal.avaya@yahoo.com>
2026-10-14 10:29:28 +08:00
JinandXinyuan Tong 44b44774c3 [Bugfix] Keep rope_theta when --json-model-override-args overrides rope_scaling (#43067)
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
2026-10-14 10:16:35 +08:00
Baizhou Zhang 536d065370 [CP] Automatically enable BCG for supported prefill CP configurations (#44022) 2026-10-13 19:00:42 -07:00
shyeh25andClaude Opus 5.5 684fea8af3 [DSV4.1] Skip Mega mHC fusion on 0-row (DP-idle) batches (#43993)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 16:36:15 -07:00
Liangsheng Yin 2ebdf9360b [Fix] Work around a CPython fork deadlock in the CPU preprocess pool (#44143) 2026-10-13 16:31:57 -07:00
b3991b53da GLM5.2 DSA Indexer Fallback (#32947)
Co-authored-by: Shu Wang <shuw@login-hecate03.hecate.clusters.nvidia.com>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
2026-10-13 16:27:33 -07:00
Khoa Pham 5917604a95 [DSV4.1] Support decoder SWA bounded replay with CUDA DP attention (#43812) 2026-10-13 16:10:57 -07:00
HEJIAN SANGandClaude Opus 5.5 a832609b92 [Feature] Return output top logprobs as flat base64 arrays on request (#42890)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 15:58:31 -07:00
baa632b049 [Score] Send stacked GPU embedding overrides through a CUDA IPC pool (#39730)
Co-authored-by: Sundara Raman Ramachandran <sundar24295@gmail.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-10-13 15:46:50 -07:00
Andrew Briand 08550acc28 Mark FP32 redux asm volatile (#44137) 2026-10-13 15:08:52 -07:00
Jiaqi LiandZijie Xia 5569b14d26 fix(benchmark): request token IDs when counting OpenAI prompts (#34511)
Co-authored-by: Zijie Xia <zijie.xia@radixark.ai>
2026-10-13 14:31:45 -07:00
Yizheng Jiao 2341e647d3 [DeepSeek-V4.1] Run decoder SWA bounded replay inside the breakable prefill CUDA graph (#43736) 2026-10-13 13:55:09 -07:00