Commit Graph
14351 Commits
Author SHA1 Message Date
Khoa Pham 5917604a95 [DSV4.1] Support decoder SWA bounded replay with CUDA DP attention (#43812) 2026-10-13 16:10:57 -07:00
HEJIAN SANGandClaude Opus 5.5 a832609b92 [Feature] Return output top logprobs as flat base64 arrays on request (#42890)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 15:58:31 -07:00
baa632b049 [Score] Send stacked GPU embedding overrides through a CUDA IPC pool (#39730)
Co-authored-by: Sundara Raman Ramachandran <sundar24295@gmail.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-10-13 15:46:50 -07:00
Andrew Briand 08550acc28 Mark FP32 redux asm volatile (#44137) 2026-10-13 15:08:52 -07:00
Jiaqi LiandZijie Xia 5569b14d26 fix(benchmark): request token IDs when counting OpenAI prompts (#34511)
Co-authored-by: Zijie Xia <zijie.xia@radixark.ai>
2026-10-13 14:31:45 -07:00
Yizheng Jiao 2341e647d3 [DeepSeek-V4.1] Run decoder SWA bounded replay inside the breakable prefill CUDA graph (#43736) 2026-10-13 13:55:09 -07:00
79af3df346 [gRPC] Move runtime.v1 bindings to tonic 0.14, report OpenAI status codes, return text on request (#42673)
Co-authored-by: Kan Wu <kan.wu@radixark.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 13:46:15 -07:00
huangtingwei 18210ed849 [HiSparse] Add speculative decoding support with multi-step KV cache swapping (#37771) 2026-10-13 13:38:31 -07:00
Xinyuan TongandNihar Patel 7dc2930585 [Feature] Add Aaronson-Gumbel text watermarking (#37577)
Co-authored-by: Nihar Patel <nihpat95@gmail.com>
2026-10-13 12:47:50 -07:00
01f3ed18a5 [unified memory pool] Prefer SWA reclaim quotas with a HiCache storage backend or on PD decode (#43919)
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: ZYHowell <ZYHowell@users.noreply.github.com>
2026-10-13 11:58:24 -07:00
7e3da1b784 [HiCache] Extract the buffer-mode prefetch and load-back lifecycle into pipeline operations (#43870)
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: xiezhq-hermann <xiezhq-hermann@users.noreply.github.com>
2026-10-13 11:51:44 -07:00
5bcaf0f8ed [Core] Add request-level customized_info keys and mark Full prefill capture batches (#43772)
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: jiayisuse <jiayisuse@users.noreply.github.com>
2026-10-13 11:45:52 -07:00
a8381d73be [HiCache] Block-group pool hits, caller-owned load destinations, allocation-target FULL eviction (#43868)
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: xiezhq-hermann <xiezhq-hermann@users.noreply.github.com>
2026-10-13 11:39:48 -07:00
Khoa Phamandvincentzed 8bf145d119 [DCP] Decode context parallelism for DSA (GLM-5.2 / GLM-5.3) with MTP (#43735)
Co-authored-by: vincentzed <207368749+vincentzed@users.noreply.github.com>
2026-10-13 11:35:34 -07:00
6ade23696d [Core] Keep request priority within the signed 64-bit range in radix caches (#43920)
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: qandrew <qandrew@users.noreply.github.com>
2026-10-13 11:33:26 -07:00
7f9f742535 [Rust] Refuse an unsupported multimodal model before the Rust listener binds (#43925)
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: shodoco <shodoco@users.noreply.github.com>
2026-10-13 11:30:43 -07:00
2a8e9db3b0 [lora] Support LoRA for the NemotronH_Nano_VL_V2 model family (#39798)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Po-Han Huang <pohanh@nvidia.com>
Co-authored-by: Yanbin Jiang <jybsuper@gmail.com>
2026-10-13 11:29:48 -07:00
ff3f8c348c [Observability] Report per-request storage prefetch and staging waits in meta_info (#43946)
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: xiezhq-hermann <xiezhq-hermann@users.noreply.github.com>
2026-10-13 11:13:01 -07:00
Rain Jiang e98c6d7b25 Rainj me/rust bench (#43950) 2026-10-13 11:07:42 -07:00
Kailash RanganathanandYanbin Jiang c0584cf9af [perf] [multi-lora] remove empty CTAs from fused LoRA expand and add split-K to shrink (#41951)
Co-authored-by: Yanbin Jiang <jybsuper@gmail.com>
2026-10-13 10:16:14 -07:00
11cd2701a8 fix(openai): preserve zero cached tokens when cache reporting is enabled (#43005)
Co-authored-by: ai-jz <ai-jz@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
2026-10-13 23:29:19 +08:00
3f1e185f6e fix(openai): keep chat stream created timestamp stable (#43026)
Co-authored-by: ai-jz <ai-jz@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
2026-10-13 22:43:33 +08:00
jiaqiang-dot-liuandCursor 4c5ed0597e Skip fully-masked KV tiles in sliding-window extend attention (#39769)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-13 22:40:53 +08:00
27b68303d0 fix(openai): omit absent tool_calls from chat responses (#43027)
Co-authored-by: ai-jz <ai-jz@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
2026-10-13 20:56:58 +08:00
Ho-Ren (Jack) ChuangandXiaoyu Zhang 5100f8d11d [DSA] Q8KV8 FP8 Sparse Prefill on GLM-5.3-Flash with BF16 KV Cache (#41790)
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
2026-10-13 20:30:13 +08:00
jlqibmandClaude Opus 5 74e0a47863 Fix/compressed tensors nvfp4 hopper (#40022)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-10-13 20:10:31 +08:00
LittleNyimaandMick Qian 5727f08932 [diffusion] feat: add configurable reference image resizing for MiniMax H3 Ref2VA (#35839)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-10-13 19:48:18 +08:00
WenhaoZhang 85254b5a8f [diffusion] comfyui: run Qwen-Image 2.1 in SGLD integrated mode (#43581) 2026-10-13 19:47:02 +08:00
Yunhao Zhouand周云昊 991a29d0ad fix(multimodal): convert grayscale tensors to RGB in PIL backend (#42127)
Co-authored-by: 周云昊 <zhouyunhao@zhouyunhaodeMacBook-Pro.local>
2026-10-13 19:03:48 +08:00
4b5b727145 Fix EVS for NemotronH_Omni_Reasoning_V3 (#42933)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
2026-10-13 10:59:00 +00:00
Xiaoyu Zhang 4e0fa52530 [Perf] Compact GDN causal-conv grids for skewed prefill batches (#43570) 2026-10-13 17:45:06 +08:00
Ziang Li 01f6840693 [FlashInfer] Add startup autotuning for MegaMoE (#43789) 2026-10-13 02:30:31 -07:00
Mohammad Miadh AngkadandMohammad Angkad 195cb7afb4 [CI] Use sgl-eval for GLM-5.3-Flash GSM8K tests (#43975)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-10-13 02:26:50 -07:00
Shu WangandAnish Shanbhag e0c957e222 [HiCache][PD] Allow DSA index-K elision under HiCache and disaggregation (#41769)
Co-authored-by: Anish Shanbhag <ashanbhag@nvidia.com>
2026-10-13 02:07:54 -07:00
Shu WangandPo-Han Huang 27be99ae73 [NVIDIA][MegaMoE] Fix the default spec decoding moe backend when the target model uses flashinfer megamoe (#40772)
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
2026-10-13 02:00:24 -07:00
Chunan ZengandClaude Opus 5.5 7ff3a11284 [Perf] Enable fast-math for JIT activations on Blackwell (#43236)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 15:18:05 +08:00
Michal ShalevandPo-Han Huang 35a014c7e8 [PD][NIXL] Fix staging buffer startup by accepting slot_layer_ids (#43723)
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
2026-10-12 23:48:13 -07:00
50a144057c [AMD][PD] enable mori transfer for PD decode DCP (#37358)
Signed-off-by: inkcherry <mingzhi.liu@amd.com>
Co-authored-by: Duyi-Wang <duyi.wang@amd.com>
Co-authored-by: HAI <hixiao@gmail.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
2026-10-13 14:35:09 +08:00
Xiaoyu Zhang 46b7357779 [Diffusion] Speed up LTX-2 guidance and video output (#43667) 2026-10-13 14:23:33 +08:00
Jimmy Shong 3eebf987ed [PD] Support heterogeneous TP for hybrid SWA KV transfer (#40909) 2026-10-12 23:21:54 -07:00
603a05e6c0 [ROCm][Spec] Route NGRAM tree reconstruction to the JIT kernel on HIP (#43328)
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Thomas Wang <thomawan@amd.com>
2026-10-12 23:17:49 -07:00
Xinyi Song a69e893b1a [AMD] Fuse prefill index-Q RoPE fake-quant and fp4 quantize kernels in DSV4.1 (#42911) 2026-10-12 22:33:53 -07:00
jacky.chengandMick Qian 42e43cf33f [AMD] Fix Flux2 kv-gather abort on non-contiguous K/V (#43793)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-10-12 22:20:55 -07:00
kkandwunhuang 43356304b4 [AMD][Kimi-K3] Use public FlyDSL APIs in KDA decode kernels (#43968)
Co-authored-by: wunhuang <wunhuang@amd.com>
2026-10-12 22:09:55 -07:00
WenhaoZhangandMick Qian ea13359da9 [diffusion] perf: implement paint_use_remesh (decimate to 40k faces before UV unwrap) for Hunyuan3D (#42770)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-10-13 12:28:32 +08:00
6a89181614 tokenizer perf optimization (#39771)
Co-authored-by: BBuf <1182563586@qq.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
2026-10-13 12:27:21 +08:00
WenhaoZhangandMick Qian feba04fa49 [diffusion] fix: avoid redundant LoRA scaling while preserving exact numerics (#43028)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-10-13 12:27:04 +08:00
WenhaoZhang cda267ab0f [diffusion] fix: keep MiniMax-H3 VAE fused norms off ROCm (#43350) 2026-10-13 12:26:16 +08:00
WenhaoZhang b4822bae03 [diffusion] comfyui: run MiniMax H3, FastH3 and VDN in SGLD integrated mode (#43504) 2026-10-13 12:01:48 +08:00
4d5692b6cd fix(openai): honor tool_choice=none in DeepSeek encoders (#43006)
Co-authored-by: ai-jz <ai-jz@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
2026-10-13 11:41:50 +08:00