Commit Graph
13510 Commits
Author SHA1 Message Date
71dc292b97 [AMD] Use AITER ASM prefill for MiniMax-M3 HD128 attention (#41707)
Co-authored-by: Zijie Chen <300606707+zijiecode@users.noreply.github.com>
Co-authored-by: Thomas Wang <thomawan@amd.com>
2026-10-03 17:54:49 -07:00
4ab720e655 [HiCache] Fix cgroup page-cache accounting and the sizing fallback when cgroup discovery fails (#42420)
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com>
Co-authored-by: xiezhq-hermann <xiezhq-hermann@users.noreply.github.com>
Co-authored-by: nvjullin <nvjullin@users.noreply.github.com>
2026-10-03 16:01:32 -07:00
metamergebotandmerrymercy 4301be7bdb Pick the default nccl_port below the kernel ephemeral range (#42405)
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com>
2026-10-03 16:00:53 -07:00
Cheng Wan 10edb05e72 [Refactor] Let the remaining placement consumers read the parallel context (#42348) 2026-10-03 14:59:32 -07:00
Cheng Wan b2aa993cb8 [Fix] Skip a draft's decode recapture when it owns no decode graph (#42347) 2026-10-03 14:54:12 -07:00
Cheng Wan 1a18de4b29 [Fix] Load a draft's tensor weight update from its deployment TP rank (#42346) 2026-10-03 14:52:25 -07:00
sglang-botandsglang-bot 061e712bab chore: bump sgl-kernel version to 0.4.9 (#42427)
Co-authored-by: sglang-bot <sglang-bot@users.noreply.github.com>
2026-10-03 14:29:04 -07:00
Mohammad Miadh AngkadandMohammad Angkad 3122b9c995 Update AOT kernels for Torch 2.14 (#42366)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-10-03 14:26:34 -07:00
Liangsheng Yin df53c978f1 [mem_cache] Run mamba models on UnifiedRadixCache when the radix cache is disabled (#42354) 2026-10-03 14:11:13 -07:00
Liangsheng Yin 3e9a120add [Fix] Make Hf3fsMockClient reads and writes thread-safe with pread/pwrite (#42425) 2026-10-03 14:02:24 -07:00
8bf780dda5 feat(metrics): expose deferred decode KV release metrics (#41128)
Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-10-04 04:52:21 +08:00
Yanbin Jiang 749a89abd6 [LoRA] Reorganize kernels and add CODEOWNERS (#42299) 2026-10-03 13:43:40 -07:00
aaf7ebe077 Advertise the KV-event replay endpoint in /server_info (#42274)
Co-authored-by: Kan Wu <kan.wu@radixark.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 04:42:33 +08:00
AMD-yanfeiwang 3dc7f4b816 fix(cuda-graph): remove unnecessary CP batch alignment (#40543) 2026-10-03 12:14:28 -07:00
AMD-yanfeiwang e83c95b650 [AMD] Fix AITER weight slicing for CP decode TP (#39950) 2026-10-03 12:09:08 -07:00
AMD-yanfeiwang f327f92424 [AMD] Port CP V2 to the DeepSeek-V4 HIP backend (#34200) 2026-10-03 12:08:17 -07:00
6acf2f4736 [PD] Keep ingesting requests while a prefill forward result is pending (#42035)
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com>
Co-authored-by: Jialin <Jialin@users.noreply.github.com>
2026-10-03 12:02:21 -07:00
b016ca406f Support post-capture KV sizing for the unified hybrid-SWA pool (#41961)
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com>
Co-authored-by: ZYHowell <ZYHowell@users.noreply.github.com>
2026-10-03 09:54:20 -07:00
2bae12b9b3 [HiCache] Fix write-back SWA insert backups tripping the write-through pending-ack assert (#42264)
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com>
Co-authored-by: 842974287 <842974287@users.noreply.github.com>
2026-10-03 09:45:56 -07:00
5b5d721239 [PD] Add opt-in prefill-complete decode KV allocation (#40703)
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
Co-authored-by: Po-Han Huang <pohanh@nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
2026-10-03 21:03:09 +08:00
5916999afb [DSv4.1] Fused c1/c2 compress for eager extend, faster c2 decode (#41660)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: BBuf <1182563586@qq.com>
2026-10-03 19:31:05 +08:00
d75d5b33f7 [DSv4.1] Faster fp4 index-K gather and combine_topk_swa_indices (#41658)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: BBuf <1182563586@qq.com>
2026-10-03 18:25:22 +08:00
DarkSharpnessandClaude Opus 5 379ec90fb2 [DSv4.1] Fold q_rope_store into fused_q_norm_rope (#41657)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-10-03 18:24:41 +08:00
Cheng Wan 7be5e3473c [Refactor] Drop the reduction-skip mechanisms stage boundaries no longer use (#42312) 2026-10-03 02:21:55 -07:00
Cheng Wan 71b04e02cf [Refactor] Build the ZAYA1, IQuest-Q1 and Gemma 4 decoders from stage boundaries (#42311) 2026-10-03 02:21:34 -07:00
Cheng Wan f85c2c4923 [Fix] GigaChat 3.5: apply the sandwich norms to the complete sums (#42310) 2026-10-03 02:21:10 -07:00
Cheng Wan 6faccf3e84 [Refactor] Build the GLM-4, GLM-Image and Granite MoE hybrid decoders from stage boundaries (#42309) 2026-10-03 02:20:40 -07:00
Cheng Wan e4554fd5e5 [Refactor] Build the Llama and Nemotron-NAS decoders from stage boundaries (#42308) 2026-10-03 02:20:18 -07:00
Cheng Wan 740a4d5955 [Refactor] Build the Qwen2 decoders from stage boundaries (#42307) 2026-10-03 02:19:56 -07:00
Cheng Wan 2c739b225b [Fix] Jet-Nemotron build and Granite MoE hybrid final norm (#42306) 2026-10-03 02:19:33 -07:00
Cheng Wan b7fc516986 [Fix] EXAONE MoE under DP attention and DeepEP (#42305) 2026-10-03 02:19:19 -07:00
Cheng Wan 2fa17a6467 [Refactor] Build the ERNIE 4.5 VL MoE and EXAONE MoE decoders from stage boundaries (#42304) 2026-10-03 02:18:56 -07:00
Cheng Wan f89ead192f [Fix] EXAONE and ERNIE 4.5 VL MoE architecture, backend and PP issues (#42303) 2026-10-03 02:18:33 -07:00
Cheng Wan be20e7453c [Fix] Step-3.5 DeepEP routed scaling and Sarvam shared expert under dense TP1 (#42302) 2026-10-03 02:18:11 -07:00
Cheng Wan b157d548bb [Refactor] Let stage boundaries complete every stage-output sum (#42301) 2026-10-03 02:17:48 -07:00
Cheng Wan d5a8d76bf5 [Refactor] Drop TBO op methods that no strategy ever schedules (#42300) 2026-10-03 02:17:26 -07:00
Liangsheng Yin 122b59152c [mem_cache] Replace is_chunk_cache / is_tree_cache with supports_prefix_sharing (#42362) 2026-10-03 02:07:55 -07:00
Banruo Liu 8293f9af54 [PD] Centralize drain-aware abort acknowledgements (#37077) 2026-10-03 16:51:11 +08:00
saatwiknagpal 1093c501df [diffusion] optimization: route subBlock sparse attention in head chunks (#41985) 2026-10-03 15:54:54 +08:00
Yanbin Jiang 254aa41f78 [Fix] Guard DeepSeek NVFP4 shared-expert fusion for LoRA and FP4 backends (#42203) 2026-10-03 00:37:57 -07:00
Tokha233 ce617d410b [diffusion] fix: fix native FP8 format handling for FLUX 3 row-wise linears (#42255) 2026-10-03 15:23:41 +08:00
Mohammad Miadh AngkadandMohammad Angkad 00bcc25f6c [HiCache] Fix DeepSeek-V4 storage backend crash from missing storage_format_tag (#42215)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-10-02 22:27:57 -07:00
Jimmy Shong 0072e2cc08 [Fix] Validate per-layer rope_parameters on transformers 5.17 (Laguna-S/XS-2.1 nightly) (#42214) 2026-10-02 21:49:45 -07:00
Jacob0226andThomas Wang af1bef3eaf [AMD] GLM-5.3-Flash: fuse shared expert and KDA projections on Quark MXFP4 (#41870)
Co-authored-by: Thomas Wang <thomawan@amd.com>
2026-10-02 20:47:47 -07:00
Richard WangandRichard Wang 482b062d28 [Feature] serve pplx-decider decision checkpoints on /v1/systemone (#42183)
Co-authored-by: Richard Wang <11150595+rwang5203@users.noreply.github.com>
2026-10-03 10:18:53 +08:00
WenhaoZhangandXiaoyu Zhang 41c6ffea89 [diffusion] fix: keep residual_gate_add on the JIT CUDA path for cont… (#42240)
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
2026-10-03 10:12:11 +08:00
Xiaoyu Zhang 391e665dab [Perf] Optimize DeepSeek V4.1 Flash Hopper paths and Blackwell prefill selection (#41251) 2026-10-03 10:04:48 +08:00
MickandMick Qian 65c7425a56 [diffusion] fix: respect explicit residency over pipeline preload hints (#41622)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-10-03 09:54:21 +08:00
Liangsheng Yin a8b54f138e [Session] Run streaming sessions only on UnifiedRadixCache; reject unverified tree caches (#42295) 2026-10-02 18:15:59 -07:00
8ff02f5820 [QSA] Enable breakable prefill CUDA graphs for text-only Qwen3.8 Flash-Next (capture-safe metadata, MTP side-channel padding) (#41729)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Qiaolin Yu <liin1211@outlook.com>
2026-10-02 17:57:42 -07:00