Commit Graph
14541 Commits
Author SHA1 Message Date
Mohammad Miadh AngkadandMohammad Angkad 99c59a2ea6 Drop the GLM-5.3-Flash breakable prefill chunk-size pin (#41837)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-09-30 19:26:58 -07:00
Shuwen Wang bb62f125aa [HiSparse] fix: return HiSparse slots on PD decode waiting abort and DeepSeek V4 free (#41346) 2026-10-01 10:24:51 +08:00
Shuwen Wang c9d6c966d0 [HiSparse] fix: reject speculative decoding and mixed chunk at startup (#41347) 2026-10-01 10:24:15 +08:00
stanleys12andClaude Opus 5 3fb721917d [Fix] Require admin auth for /load_lora_adapter_from_tensors and /set_trace_level (#41322)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-10-01 10:23:12 +08:00
Shuwen Wang b62b21e7f4 [HiCache][PD] fix: keep decode load-back polling out of rank-local collectives (#41393) 2026-10-01 10:22:30 +08:00
zijieandZijie Chen 0148d9a2fd [AMD] Fix MiniMax-M3 EAGLE3 verification on ROCm (#41497)
Co-authored-by: Zijie Chen <300606707+zijiecode@users.noreply.github.com>
2026-09-30 19:05:49 -07:00
MickandMick Qian 2ab1b537f8 [diffusion] docs: recommend docker for gpu deployments (#41966)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-10-01 09:52:31 +08:00
Shafeeq IbraheemandYanbin Jiang b4df214948 Fix LoRA usage-counter accounting across request lifecycle paths (#31808)
Co-authored-by: Yanbin Jiang <jybsuper@gmail.com>
2026-09-30 18:37:53 -07:00
Yu-Yun Changandkk 6af651ea0c [AMD][Quark] Serve the Kimi-K3 MXFP4 checkpoint on ROCm (#40811)
Co-authored-by: kk <43161300+kkHuang-amd@users.noreply.github.com>
2026-09-30 18:14:15 -07:00
leonardHONGandlianghougong a702970ecb [Perf] Use FA4 by default for MiMo on SM100 (#41886)
Co-authored-by: lianghougong <lianghouhong@xiaohongshu.com>
2026-10-01 09:05:12 +08:00
Liangsheng Yin 63321e3693 [Scheduler] Move internal-state readback and updates into a collaborator (#41950) 2026-09-30 16:40:57 -07:00
Jan Bernlöhrandjbernloehr 488869c2d0 [Fix] Keep MiMo-V2 processor available without TorchCodec (#41667)
Co-authored-by: jbernloehr <janbernloehr@users.noreply.github.com>
2026-09-30 15:28:16 -07:00
826f15bf9c [Fix] Preserve inline instructions for Responses and Kimi K3 Messages (#40077)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
2026-09-30 15:20:52 -07:00
f050ab194a fix(spec): enable Qwen3.5 EAGLE3 capture and streaming overlap (#41154)
Co-authored-by: Yuhao Wang <59914901+jin-chuan@users.noreply.github.com>
Co-authored-by: Liangsheng Yin <95566987+hnyls2002@users.noreply.github.com>
2026-09-30 14:41:46 -07:00
Cheng Wan a9c97c9f69 [Feature] Add --attn-dp-size and deprecate --enable-dp-attention (#41818) 2026-09-30 13:34:18 -07:00
Jason Mancusoandhnyls2002 201c4b9b2a [RL, Spec] Introduce top-p mask capture for spec and add DFlash/DSpark impl (#34201)
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-09-30 13:32:26 -07:00
Cheng Wan 986cb8268b [Refactor] Enter draft TP scopes by attention ownership and drop ModelRunner.tp_group (#41817) 2026-09-30 13:28:50 -07:00
Cheng Wan e7f6a99313 [Refactor] Add make_pp_layers so models stop handing their PP position down (#41816) 2026-09-30 13:28:34 -07:00
Cheng Wan 00659ac099 [Refactor] Stop passing models' layers the placement they already read (#41815) 2026-09-30 13:28:17 -07:00
Cheng Wan c7fa37a5ca [Refactor] Let FusedMoE's weight-loading helpers read the layer's MoE-TP rank (#41814) 2026-09-30 13:27:58 -07:00
Cheng Wan 499e86db98 [Refactor] Keep only the TP and PP groups on the model runner (#41813) 2026-09-30 13:27:31 -07:00
Cheng Wan a09aedf1ca [Refactor] Drop model placement attributes and parameters nothing reads (#41812) 2026-09-30 13:27:11 -07:00
Cheng Wan b0cc246ff8 [Refactor] Drop placement values nothing reads (#41811) 2026-09-30 13:26:51 -07:00
Cheng Wan 4f3ee94d25 [Fix] Make the Solar model constructible and runnable (#41810) 2026-09-30 13:26:32 -07:00
Cheng Wan a659e6461f [Fix] Size the prefill delayer's gather buffer to its TP group (#41809) 2026-09-30 13:26:12 -07:00
Cheng Wan ecc0f0dec2 [Fix] Pass the draft's attention ownership to DFLASH's eager LiLiCorr scope (#41808) 2026-09-30 13:25:54 -07:00
Cheng Wan e289989933 [Fix] Shard MoE WNA16 and Quark INT4-FP8 weights by the MoE placement (#41807) 2026-09-30 13:25:38 -07:00
Cheng Wan 3f20738846 [Fix] State the configured MoE-DP width in the weight-cache fingerprint (#41806) 2026-09-30 13:25:19 -07:00
Cheng Wan e27b6e0971 [Fix] Read TensorCast's WORLD placement under its current names (#41805) 2026-09-30 13:25:03 -07:00
YAMY 4407a21f3c [Spec][PP] Fix hybrid recurrent-state commit and micro-batch pairing under PP x speculative decoding (#40001) 2026-09-30 13:05:39 -07:00
547286bb5a [Deps] Bump transformers to 5.17.0 (#39012)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
2026-09-30 12:52:02 -07:00
Mohammad Miadh AngkadandMohammad Angkad eae808903f [Deps] Bump FlashInfer to 0.7.0.post1 (#40709)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-09-30 12:42:19 -07:00
4dfc0ac131 [BugFix] Pass token-major Q/K tensors from Gemma-3 to RadixAttention (#37984)
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
Co-authored-by: Po-Han Huang <pohanh@nvidia.com>
2026-09-30 12:05:41 -07:00
Kevin MiandKevin Mi 3a398442bf dsv4.1-amd: serve DeepSeek-V4.1 on gfx950 (#41308)
Co-authored-by: Kevin Mi <kevin.mi@radixark.ai>
2026-09-30 11:47:14 -07:00
9f9f9e702a [HiCache] Attribute buffer-mode storage hits against the joint device match (#41758)
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com>
Co-authored-by: xiezhq-hermann <xiezhq-hermann@users.noreply.github.com>
2026-09-30 11:46:30 -07:00
Shuwen WangandShangming Cai aec4b799e6 [PD][Mamba] fix: free the COW mamba slot of decode requests dropped before preallocation (#41451)
Co-authored-by: Shangming Cai <csmthu@gmail.com>
2026-09-30 17:50:44 +00:00
jain-ria 21ed9fbf44 [Rust] Extract shared runtime.v1 protobuf bindings (#41726)
Signed-off-by: jain-ria <riajain@NVIDIA.com>
2026-09-30 10:13:08 -07:00
Sohom chakrabortyandShuwen Wang 28c5e7f5cb [Fix] Stop PD-decode queue_time from counting decode time before a retraction (#41380)
Signed-off-by: Sohom Chakraborty <sohomchakraborty.iitkgp@gmail.com>
Co-authored-by: Shuwen Wang <47200617+alphabetc1@users.noreply.github.com>
2026-09-30 23:17:03 +08:00
bd66ce343e [diffusion] nightly: measure every framework with one client end-to-end methodology (#41689)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 19:52:20 +08:00
Yuan Luoandluoyuan.luo 57c81258d4 [KDA] Enable the ptx_kda prefill backend on SM100 (B200 / GB200) (#41445)
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
2026-09-30 17:24:51 +08:00
Yuan Luoandluoyuan.luo c0296186b7 [KDA] Fix ptx_kda prefill NaN without a gate lower bound and workspace growth (#41572)
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
2026-09-30 17:23:54 +08:00
paulzhang-tmandKe Bao 8055ccd2cd [Linear Attention] Expose GDN/KDA prefill hooks and auxiliary cache accounting (#40227)
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
2026-09-30 17:19:02 +08:00
jiaryang e9b0d0c0a3 [AMD] Gate flashinfer and TRT-LLM DSA paths on CUDA (#40911) 2026-09-30 02:10:02 -07:00
Michael d7cce54885 [AMD] Resolve QSA packed-varlen decode to aiter on HIP (#41513) 2026-09-30 02:01:15 -07:00
45c8ddddd3 [diffusion] feat: check the final MP4 in-process instead of spawning ffprobe (#41825)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 16:26:39 +08:00
WenhaoZhang 0931a72eb2 [diffusion] feat: support MiniMax H3 RL (#34365) 2026-09-30 16:23:56 +08:00
682f4ef32f [ROCm] Select DSA indexer top-k wave size by arch (wave32 on gfx1250) (#41687)
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Thomas Wang <thomawan@amd.com>
2026-09-30 00:47:04 -07:00
Shuwen Wang 51cae5f303 [HiCache][PD] fix: clamp the decode restore to the prefix promised to prefill (#41450) 2026-09-30 15:26:15 +08:00
heziiopandsglang-npu-bot ec0006673a [NPU] Add decode context parallel support for dsa models (#37787)
Co-authored-by: sglang-npu-bot <sglangnpu@163.com>
2026-09-30 15:22:16 +08:00
eb9c9ee99d [Qwen4-Exp] Optional fp8 (e4m3) storage for the compressed QSA indexer cache (#39614)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: YAMY1234 <74099316+YAMY1234@users.noreply.github.com>
2026-09-29 22:11:46 -07:00