Commit Graph
6055 Commits
Author SHA1 Message Date
26a2f18c02 [Feat] sglang-server: serve /v1/loads from the Rust frontend (#44532)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: shodoco <shodoco@users.noreply.github.com>
2026-10-16 10:35:37 -07:00
Xinyuan Tong a038a0dca0 [Fix] dsv41: Preserve tool-result image order and current-turn thinking (#44751) 2026-10-16 23:39:50 +08:00
Benjamin TruongandClaude Opus 5.5 7c99ace127 [Fix] dsa: Fix ragged seed indices before MTP draft decode (#42565)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 19:37:41 +08:00
Xinyi SongandHAI 4afae5568b [Perf] [AMD] dsv4: Add a separate MegaMoE decode instance (#44657)
Co-authored-by: HAI <hixiao@gmail.com>
2026-10-16 03:11:02 -07:00
kkandDuyi-Wang baf451fcd6 [Perf] [AMD] dsv4: carry aiter DSv4 GEMM tuned rows and mHC tile_m=16 until aiter ships them (7/8) (#44008)
Co-authored-by: Duyi-Wang <duyi.wang@amd.com>
2026-10-16 02:47:41 -07:00
f05da43d44 [Feat] [AMD] disagg: Support DSV4 decode radix cache on ROCm unified KV (bf16/fp8, DSpark) (#30929)
Co-authored-by: HAI <hixiao@gmail.com>
Co-authored-by: Duyi-Wang <duyi.wang@amd.com>
2026-10-16 02:39:27 -07:00
42eb556e66 [Perf] [AMD] dsv4: optional e8m0 mxscale GEMM for dense 128x128 fp8 linears (6/8) (#44007)
Co-authored-by: Duyi-Wang <duyi.wang@amd.com>
Co-authored-by: HAI <hixiao@gmail.com>
2026-10-16 02:28:31 -07:00
caihuali95 8d229d96e0 [Fix] unified-memory: enabling trtllm_mha with decode graphs and speculative decoding (#42930) 2026-10-16 02:24:06 -07:00
kkandDuyi-Wang 6efc96abf9 [Perf] [AMD] dsv4: FP4 indexer on aiter's row-group kernel for decode and target-verify (5/8) (#44006)
Co-authored-by: Duyi-Wang <duyi.wang@amd.com>
2026-10-16 02:16:06 -07:00
kkandDuyi-Wang 9fde8bcc46 [Perf] [AMD] moe: small-batch sort with histogram + bitonic sort for 32 < P <= 256 (3/8) (#44004)
Co-authored-by: Duyi-Wang <duyi.wang@amd.com>
2026-10-16 02:09:32 -07:00
kkandDuyi-Wang ab5858ac8a [Perf] [AMD] dsv4: fp8 unified_kv decode/verify on FlyDSL v4 nm decode with a latency-model split plan (1/8) (#44016)
Co-authored-by: Duyi-Wang <duyi.wang@amd.com>
2026-10-16 01:50:16 -07:00
Liangsheng Yin fad7d95a7f [Fix] deepseek-v4.1: Keep the DSpark draft SWA out of the radix tree (#44673) 2026-10-16 01:18:58 -07:00
jiaqiang-dot-liu a448fdf316 [Perf] [AMD] qwen3: Fuse QK-norm + RoPE + KV cache write for dense Qwen3 decode (#44267) 2026-10-16 00:51:55 -07:00
Ziang Li 7098ec387c [Feat] FlashInfer: Add NVFP4 W4A16 and SiTU support to MegaMoE (#44036) 2026-10-16 00:38:50 -07:00
Liangsheng YinandJingshu Chen 3a5d46e946 [Perf] pd: Stage decode-side KV preallocation lengths through pinned memory (#38955)
Co-authored-by: Jingshu Chen <96519051+elmo2019@users.noreply.github.com>
2026-10-16 00:34:56 -07:00
Shu Wang 83940471b4 [Fix] [NVIDIA] allreduce-fusion: Decline MNNVL fusion across NVLink cliques (#44387) 2026-10-15 22:50:33 -07:00
Yuwei An 06934575b9 [Fix] DSv4.1: Fix BCG image handling and add multimodal bounded replay coverage (#44131) 2026-10-15 22:32:55 -07:00
2a08e261ec [Feat] MiniCPM-V-4.7: add model support (#41993)
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
2026-10-16 13:24:48 +08:00
Kolade FajimiandXiaoyu Zhang 5117bfbaeb [Fix] compressed-tensors: Serve NVFP4 W4A4 linears via FP4 Marlin when --fp4-gemm-backend marlin is set on SM100 (#44213)
Signed-off-by: kolade fajimi <fajimikolade23@gmail.com>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
2026-10-16 12:36:02 +08:00
Mohammad Miadh AngkadandMohammad Angkad 6231a617d3 [Test] diffusion: Load NVFP4 block scales through the weight loader in tests (#44595)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-10-16 10:56:27 +08:00
ashwini rathiandClaude Opus 5.5 75e25a21bd [Fix] [XPU] nightly: Fix rerank, zimage, flux2 and EAGLE parity (#44100)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 10:27:51 +08:00
Shuwen Wang f9df33494a [Fix] scheduler: Keep PD decode non-idle while deferred KV releases are held (#44386) 2026-10-15 19:17:51 -07:00
4e85e8ed37 [Fix] imports: Bind FlashInfer lazily at four import sites so the server starts without it (#44430)
Co-authored-by: fei-xx <fei-xx@users.noreply.github.com>
Co-authored-by: yeqcharlotte <yeqcharlotte@users.noreply.github.com>
2026-10-15 18:52:33 -07:00
e3961eb6fa [Fix] utils: Allowlist torch.bcomplex32 in SafeUnpickler (#44440)
Co-authored-by: fei-xx <fei-xx@users.noreply.github.com>
Co-authored-by: zx3xyy <zx3xyy@users.noreply.github.com>
2026-10-15 17:50:43 -07:00
f110b2f801 [Feat] rust-server: Report cached_tokens and reasoning_tokens in meta_info (#44531)
Co-authored-by: fei-xx <fei-xx@users.noreply.github.com>
Co-authored-by: shodoco <shodoco@users.noreply.github.com>
2026-10-15 17:50:27 -07:00
d393298736 [Fix] tokenizer_manager: re-report only emitted customized_info keys on cumulative streams (#44537)
Co-authored-by: fei-xx <fei-xx@users.noreply.github.com>
Co-authored-by: shodoco <shodoco@users.noreply.github.com>
2026-10-15 17:49:49 -07:00
metamergebotandYe Qi bda6ef7e01 [Perf] [NVIDIA] scheduler: Pageable host staging and inline hidden-state/DSpark readback under Confidential Computing (#44568)
Co-authored-by: Ye (Charlotte) Qi <7269575+yeqcharlotte@users.noreply.github.com>
2026-10-15 17:48:50 -07:00
ed8c3139f1 [Fix] deepseek-v4.1: Keep the full SWA window for short bounded-replay extends (#41303)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-10-15 17:32:50 -07:00
Khoa Pham a93f16f0cf [Perf] deepseek-v4.1: Integrate FlashMLA MegaAttention with NVFP4 KV (#43922) 2026-10-15 17:21:43 -07:00
ec7ab9d0c2 [Feat] [AMD] monokernels: one runtime switch for vendor monokernel plugins (#44353)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: wunhuang <wunhuang@amd.com>
2026-10-15 15:50:25 -07:00
0b1a73599a [Feat] rust-server: Apply the x-override-* request headers (#44312)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: shodoco <shodoco@users.noreply.github.com>
2026-10-15 15:50:04 -07:00
William Hu 4e0609ea27 [Fix] moe: Preserve negative routing sentinel in EP mapping (#40498) 2026-10-15 15:05:08 -07:00
Yuwei An 91f6ef2322 [DSv4.1] Fuse request-window KV copies (#44555) 2026-10-15 14:11:51 -07:00
snadampal aa5c7e887c [Feat] docker: Add AWS EFA image variants built on published images (#41006)
Signed-off-by: Sunita Nadampalli <nadampal@amazon.com>
2026-10-15 13:19:22 -07:00
87211088c9 perf: fix overlap scheduling for NVIDIA Confidential Computing(CC) on Blackwell (#36810)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Po-Han Huang <pohanh@nvidia.com>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
2026-10-15 12:09:09 -07:00
02a43eeef6 [Feat] rust-server: Attach configured metadata headers to a healthy /health_generate (#44315)
Co-authored-by: fei-xx <fei-xx@users.noreply.github.com>
Co-authored-by: shodoco <shodoco@users.noreply.github.com>
2026-10-15 12:04:23 -07:00
173e0f2e9b [Feat] hicache: Add opt-in tail-only SWA backups in buffer mode (#44173)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: xiezhq-hermann <xiezhq-hermann@users.noreply.github.com>
2026-10-15 12:02:59 -07:00
c5a5bf0d0b [Fix] pp: Order output send/recv by rank parity to fix cross-node hang (#44453)
Co-authored-by: chuixue <chuixue@minimaxi.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
2026-10-16 01:46:37 +08:00
elvischenvandClaude Opus 5.5 6fe5596ebb Support EPLB for Nemotron-H (#43685)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 13:50:09 +00:00
ormandjandBenjamin Oldenburg c5bf3aafbc Fix overflow in legacy and v1 DSA top-k selectors (#37625)
Co-authored-by: Benjamin Oldenburg <benjamin.oldenburg@ordis.co.th>
2026-10-15 21:44:58 +08:00
jiaqiang-dot-liuandCursor 2f5407da28 [Fix] [AMD] qwen3: Fix KV cache write in the fused QK-norm + mRoPE decode path (#44266)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-15 21:22:46 +08:00
Zhang Yiyang (SII)andMick Qian 23c2a026db [diffusion] model: support MOSS-Image (#44325)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-10-15 19:23:56 +08:00
Cheng Wan f7ad4345b8 [Refactor] deepseek-v4: Use explicit stage boundaries and row transport (#43607) 2026-10-15 02:05:28 -07:00
Cheng Wan 3a835cf417 [Refactor] inkling: Use short-convolution stage boundaries (#43606) 2026-10-15 02:04:46 -07:00
Cheng Wan e1f9fce7b3 [Refactor] layer-boundary: Bind residual completion to producer updates (#43605) 2026-10-15 02:04:02 -07:00
Cheng Wan d59c3448b0 [Fix] kimi-k3: Preserve pipeline graph bank inputs and bound chunk profiling (#43604) 2026-10-15 02:03:27 -07:00
86f4ede4fb [Fix] dsa: Make the fused top-k exact on an overflowing threshold bin (#37941)
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: DarkSharpness <2040703891@qq.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-15 15:52:48 +08:00
9ea5472833 [HiCache] Add the page_unified host KV layout and the unified L3 key (#39727)
Co-authored-by: huangtingwei <141888744+huangtingwei9988@users.noreply.github.com>
Co-authored-by: Zhangheng <hzh0425@apache.org>
2026-10-15 15:46:13 +08:00
33618d4f00 [Feat] runtime: Include FlashInfer’s default tactic in startup tuning. (#43927)
Co-authored-by: Po-Han Huang <pohanh@nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 15:30:45 +08:00
Liangsheng Yin a776851c97 [Refactor] scheduler: Check the prefill budget at the admission loop head; drop a redundant PD gate (#44420) 2026-10-15 00:16:56 -07:00