Commit Graph
13735 Commits
Author SHA1 Message Date
7a4d6dfd2f [diffusion] feat: name the quality tiers after what they guarantee (#42370)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-10-07 09:49:30 +08:00
660b17e8dc [AMD] Let AITER unified verify run without host seq_lens (#42635)
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com>
Co-authored-by: zx3xyy <zx3xyy@users.noreply.github.com>
2026-10-06 18:27:27 -07:00
WenhaoZhang 6da08fd7cf [diffusion] CI: fix qwen cache-dit step 10/49 perf baselines (#42495) 2026-10-07 08:51:12 +08:00
MickandMick Qian 1351f32bb4 [diffusion] refactor: simplify Kandinsky 6 models, SR sampling and tests (#42768)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-10-07 08:48:16 +08:00
f1065423d3 [PD] Stop reserving unallocated mamba spec-verify scratch in prefill KV sizing (#42649)
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com>
Co-authored-by: 842974287 <842974287@users.noreply.github.com>
2026-10-06 17:16:22 -07:00
08aad0c196 [HiCache] Manage buffer-mode backups per storage pool (#42633)
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com>
Co-authored-by: xiezhq-hermann <xiezhq-hermann@users.noreply.github.com>
2026-10-06 16:45:05 -07:00
jain-riaandishandhanani fd62ee84f7 feat(grpc): expose follower metadata and node-local KV sources (#39659)
Signed-off-by: jain-ria <riajain@NVIDIA.com>
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
2026-10-06 16:39:01 -07:00
Chunan ZengandCheng Wan 90aa1cb0fb [Scheduler] Gather prefill-delayer queue timeout across ranks (#42625)
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
2026-10-06 16:15:34 -07:00
Jan BernlöhrandPo-Han Huang 0a0fe001f1 fix: route Thor SM110 FP8 and ModelOpt NVFP4 auto backends (#41765)
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
2026-10-06 16:02:45 -07:00
6b737fd4c6 Bind the MessageQueue remote socket atomically (#42695)
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com>
Co-authored-by: charlotte12l <charlotte12l@users.noreply.github.com>
2026-10-06 15:50:59 -07:00
264d1c2015 Back the DFlash-family draft KV pool to the post-capture token count (#42651)
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com>
Co-authored-by: 842974287 <842974287@users.noreply.github.com>
2026-10-06 14:23:18 -07:00
4506927635 [Metrics] Add per-rank DP attention imbalance metrics (#42755)
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com>
Co-authored-by: hanming-lu <hanming-lu@users.noreply.github.com>
2026-10-06 14:20:41 -07:00
Po-Han Huang (NVIDIA)andClaude Opus 5.5 3eec178e48 [Spec][MegaMoE] Let the speculative draft choose its own W4A4 MXFP4 MegaMoE MMA type (#42022)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 14:09:57 -07:00
Liangsheng Yin 12004043b8 [Metrics] Read scheduled prefill KV tokens from batch prefix_lens (#42797) 2026-10-06 13:11:29 -07:00
a34cba3c62 [HiCache] Resolve write_back to write_through under buffer_only host memory (#42754)
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com>
Co-authored-by: xiezhq-hermann <xiezhq-hermann@users.noreply.github.com>
2026-10-06 11:02:23 -07:00
Byron Hsu abed685de1 [PD] Fix crash for allocation when mamba + pd + decode radix cache (#41927) 2026-10-06 09:36:31 -07:00
Emil BogomolovandEmil Bogomolov e17935820a [diffusion] fix: fix width/height silently dropped on multipart /v1/videos requests (#35108)
Co-authored-by: Emil Bogomolov <zetyquickly@googlemail.com>
2026-10-06 23:31:31 +08:00
WenhaoZhangandMick 2579875b5d [diffusion] feat: support loading community ComfyUI-GGUF Qwen-Image-2.1 transformers (#42556)
Co-authored-by: Mick <mickjagger19@icloud.com>
2026-10-06 23:30:48 +08:00
49bafb58cd perf(mla): use native SATFINITE FP8 conversion in MLA KV/Q preparation (#42659)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
2026-10-06 22:47:47 +08:00
Xiaoshuai Zhangandjet 679d2851ba fix(mimo-vl): keep multimodal features when capturing aux hidden states and correct vision preprocessing (#40888)
Co-authored-by: jet <dev@jetd.one>
2026-10-06 22:15:14 +08:00
16a23a672b [diffusion] model: support Kandinsky 6 (#42743)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
Co-authored-by: Lev Novitskiy <57654885+leffff@users.noreply.github.com>
Co-authored-by: Kirill <kykozlov@gmail.com>
Co-authored-by: Artem Borisov <divotionchannel@gmail.com>
2026-10-06 22:13:29 +08:00
730f1f3e5b [DeepSeek-V4.1] Unify mHC into one state machine and drop medium-batch fusion variants (#42245)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: BBuf <1182563586@qq.com>
2026-10-06 21:01:42 +08:00
WenhaoZhang 2a5f45d3d4 [diffusion] fix: skip zero blocks when hashing large conditioning inputs (#42748) 2026-10-06 20:43:08 +08:00
WMCandjacky.cheng ba94488b42 [diffusion] feat: default Qwen-Image 2.1 to VAE tiling on gfx1151 (#41655)
Co-authored-by: jacky.cheng <yichiche@amd.com>
2026-10-06 20:20:58 +08:00
Mohammad Miadh AngkadandMohammad Angkad 45db430530 [DSA] Fix pooled page table refresh for target verify CUDA graph replay (#42750)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-10-06 18:57:52 +08:00
Gawr Gura 0721 b563bbd597 [Fix] Don't let a failed deep_ep import-time check kill servers that never use DeepEP (#40671) 2026-10-06 01:58:02 -07:00
e49e971131 [mem_cache] Keep prefix_indices current for chunked requests kept out of the tree (#42467)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
2026-10-06 01:40:35 -07:00
Jzz1943 1be3013118 [diffusion] feat: support Ideogram TurboTime LoRA (#30487) 2026-10-06 16:15:21 +08:00
Max Kong 01345197af [diffusion] chore: harden OpenAI upload filename handling (#17270)
=
2026-10-06 16:14:39 +08:00
Jinbao ChenandMick Qian d0494af08b [diffusion] security: pin multimodal-gen ZMQ scheduler ingress to loopback on wildcard --host (#36854)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-10-06 16:05:17 +08:00
Liangsheng Yin c7f3ae1dd5 [mem_cache] Hold a request's tree lock as one TreeLock (#42686) 2026-10-06 00:59:38 -07:00
c683190b40 [PD] Wait for prefill completion before Mooncake early KV transfer (#41395)
Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-06 15:17:12 +08:00
Rumit Desai 33ce0943e9 [Bugfix] Guard merge_state sgl_kernel import on non-CUDA platforms (#42682) 2026-10-06 14:19:20 +08:00
Joe 40fb5935c0 Allow platforms to declare Mamba extra-buffer support (#36410) 2026-10-06 14:18:49 +08:00
Liangsheng Yin 9a09b48da7 [Fix] Add maybe_hand_to_session to the one-batch benchmark TreeCacheNamespace (#42723) 2026-10-05 23:09:41 -07:00
b524de2de6 [diffusion] optimization: release the allocator cache when the server idles not per request (#41718)
Co-authored-by: Yihao Wang <42559837+AgainstEntropy@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-10-06 12:27:08 +08:00
e446698cdd [SM120&90] Add CUDA fused Triton sparse-MLA prefill backend for DSA (#32779)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
Co-authored-by: Po-Han Huang <pohanh@nvidia.com>
2026-10-05 21:08:05 -07:00
1e1bdeedbb [Perf] Improve GLM-5.3-Flash DFlash decoding on B300 (#42489)
Co-authored-by: lianghougong <lianghouhong@xiaohongshu.com>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
2026-10-06 12:05:37 +08:00
Mohammad Miadh AngkadandMohammad Angkad fcbbefa82a [GLM-5.3-Flash] Autotune MoE kernels at prefill sizes (#42693)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-10-05 21:04:54 -07:00
Yuwei An 620c76f25d [DSV4.1] Add opt-in TRT-LLM sparse attention support (#41603) 2026-10-06 12:02:10 +08:00
Ke Bao ed75fe67f1 Declare hybrid Mamba indexer host pool (#40915) 2026-10-06 10:18:36 +08:00
CuzMi 67959f80ab [diffusion] optimization: fuse Anima FP32 split-half Q/K RoPE (#42609) 2026-10-06 10:18:09 +08:00
Thomas Wang 1abd356595 [AMD] Add engram host/device address mapping (#42552) 2026-10-05 19:12:12 -07:00
MickandMick Qian 21c9bbdf2c [diffusion] fix: fix concurrent server perf dump writes across ranks (#41541)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-10-06 09:29:51 +08:00
5d385007b3 [diffusion] kernels: take the Wan VAE decoder's remaining first-decode work off the request path (#41830)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 09:05:29 +08:00
3120bb87a2 [diffusion] fix: fix the OOM hint's placement advice and add a 24 GB Wan2.1 recipe (#41723)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 09:01:02 +08:00
Liangsheng Yin ad7a9b53cc [mem_cache] Make release_kv_cache's final checkpoint an explicit checkpoint flag (#42270) 2026-10-05 17:32:07 -07:00
Liangsheng Yin c01d59eb90 [Session] Let a streaming session own its KV record and tree lock from the first turn (#42537) 2026-10-05 17:28:09 -07:00
Liangsheng Yin f3fe753475 [misc] Rename SGLANG_OPT_SWA_RELEASE_LEAF_LOCK_AFTER_WINDOW to SGLANG_OPT_RELEASE_PREFILL_SWA (#42657) 2026-10-05 17:24:15 -07:00
Yuwei An 18ec2ecce3 [Refactor][TCPCG] Relocate shared graph tensor and DSA head-gate helpers (1/9) (#42285) 2026-10-05 17:06:52 -07:00