Commit Graph
18660 Commits
Author SHA1 Message Date
jthomson04 15ba54bd5d perf(engine): avoid timed waits for Engine responses (#39486)
Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
2026-09-21 20:42:03 -07:00
Jan BernlöhrandPo-Han Huang 56fee88e23 fix(moe): support Llama4 NVFP4 router input weights on SM120 (#35504)
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
2026-09-21 20:18:11 -07:00
Kan Wu 59a723ef1e [sgl-router] refactor - SLO ordering for bucket selection (#40292) 2026-09-22 11:16:10 +08:00
Kan Wu 27f796ca6c [sgl-router] Fix readiness, IPv6 discovery, logging, and model validation (#40604) 2026-09-22 11:13:06 +08:00
Cheng Wan 1d025491f3 [Test] Set DP size in the mocked Metal profiler test (#40667) 2026-09-21 19:57:52 -07:00
90cf471723 [AMD] [GLM-5.3-Flash Day 0] Support non-2048 top-k widths in the DSA page-table transform (#39340)
Co-authored-by: Thomas Wang <thomawan@amd.com>
Co-authored-by: Kevin Mi <45493463+kevin-mii@users.noreply.github.com>
Co-authored-by: Kevin Mi <mikevin920@yahoo.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-21 19:50:17 -07:00
5f9c6b9eb0 [diffusion] fix: separate a use-scoped layerwise release from release_all (#40590)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-22 10:38:44 +08:00
MickandMick Qian a1b2b976fe [diffusion] CI: restore public Qwen-Image 2.1 TP2 E2E coverage (#40507)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-09-22 10:38:00 +08:00
Yuhan Zhou 15eba3b464 Feat: Add TensorCast storage as a new HiCache backend (#27265) 2026-09-22 10:19:04 +08:00
jacky.cheng bc30fa1759 [AMD][Fix] AgentX HIP TPOT regression when SGLANG_SIMULATE_ACC_LEN is set (#40598) 2026-09-21 19:16:15 -07:00
Cheng Wan 9eda772a21 [Test] Handle tied top-k indices in graph-pool logprob regression (#40661) 2026-09-21 19:02:43 -07:00
Mohammad Miadh Angkadandmmangkad e332e1b84e [Fix] Don't write conv state from the fused KDA verify kernel (#39524)
Co-authored-by: mmangkad <mohammad.angkad@radixark.ai>
2026-09-21 18:33:03 -07:00
Dayananda VandClaude Opus 5 35eb7cf8d6 [Intel][XPU][KVCanary] Enable KV Canary on Intel XPU (#33520)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-22 09:19:01 +08:00
Ma Mingfei 046cd6f4ea [XPU][ci]: disable XPU NIXL disaggregation test (#40540) 2026-09-22 09:02:22 +08:00
Khoa Pham c4d3770a68 [Kimi K3] Fix CUDA graph stream explosion (#40640) 2026-09-21 17:49:50 -07:00
jacky.cheng 31b577bb08 [AMD] Pad QSA MQA decode Q-heads to 16 for ROCm MFMA (#38875) 2026-09-21 17:43:46 -07:00
Cheng Wan 98c8dee23b Fix lint failure from draft-decode window test location (#40654) 2026-09-21 17:41:00 -07:00
042b6a488f [AMD] [GLM-5.3-Flash Day 0] Enable zero-RoPE MHA prefill on ROCm (#39338)
Co-authored-by: Thomas Wang <thomawan@amd.com>
Co-authored-by: Kevin Mi <45493463+kevin-mii@users.noreply.github.com>
2026-09-21 17:38:07 -07:00
Cheng Wan 582389cec5 [Fix] Keep diffusion encoder TP context bindings consistent (#40646) 2026-09-21 17:23:15 -07:00
Tao LiandXiaoyu Zhang c53cc8e1eb [NPU][BugFix] Avoid M-RoPE recompilation for variable sequence lengths (#40371)
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
2026-09-22 08:19:25 +08:00
avalliappan-nvidia 61d0cf2074 [Spec] Windowed draft-decode attention for built-in EAGLE / MTP drafts (#32673) 2026-09-22 08:17:57 +08:00
Vedant V JhaveriandCopilot 9fdb71732a Avoid materializing GDN QKV tensors during target verification (#33778)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-09-21 17:04:18 -07:00
Cheng Wan 506698761d [unified-memory] Hierarchical cache for every unified pool shape (#37507) 2026-09-21 16:50:37 -07:00
Cheng Wan 22587fb15c [Fix] Run KV canary hooks for context-parallel prefill (#40642) 2026-09-21 16:46:22 -07:00
Yuxuan ZhangandXinyuan Tong 00986c81be Support GLM-5.3-Flash hybrid attention CPU offload and PD index mapping (#40310)
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
2026-09-21 16:03:03 -07:00
YAMY 0229025127 [Spec][PP] Launch extend microbatches before the spec output exchange (#40499) 2026-09-21 15:47:03 -07:00
RuibinCheung 0c53fec476 [ROCm] fix: remove extra bf16 -> fp32 cast in jit grouped topk kernel path (#39775) 2026-09-21 15:43:21 -07:00
Kan Wu d47b8c454c [sgl-router] Release cancelled circuit-breaker probes (#40603) 2026-09-21 15:39:50 -07:00
Liangsheng Yin a5c2cc517c [CI] Split the CI control labels into four axes and resolve them live (#40527) 2026-09-21 15:37:28 -07:00
Zhang, Jiejing 66f19f5c46 [AMD] Enable HiCache for GLM-5.2 MI355X throughput recipe (#40570) 2026-09-21 15:28:27 -07:00
8bde82c0ad [AMD] [GLM-5.3-Flash Day 0] Build the fused DSA k-pool top-k JIT kernel on HIP (#39339)
Co-authored-by: Raiden-Makoto <Raiden-Makoto@users.noreply.github.com>
Co-authored-by: Thomas Wang <thomawan@amd.com>
Co-authored-by: Kevin Mi <mikevin920@yahoo.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-21 15:24:44 -07:00
2261c2e618 Add MiMo-V2.6 cookbook (#40622)
Co-authored-by: zijiexia <37504505+zijiexia@users.noreply.github.com>
Co-authored-by: Zijie Xia <zijie.xia@radixark.ai>
2026-09-21 14:58:53 -07:00
Cheng Wan acac4dd9d9 [Refactor] Clean up parallel runtime comments (#40632) 2026-09-21 14:32:22 -07:00
William Hu f532ad1f9a Fix GLM-5.3 forget-gate shape for nvCUTEDSL verify (#40607) 2026-09-21 13:52:26 -07:00
jacky.cheng e0c2e8dc4d [AMD] Tune Qwen3.5 TP4 GDN recurrent launch on gfx950 (#39987) 2026-09-21 13:20:54 -07:00
Liangsheng Yin 1ed6822039 [Test] Anchor basic_perf thresholds to each metric's measured spread (#40617) 2026-09-21 13:02:10 -07:00
Liangsheng Yin b18ca9ca44 [CI] Bump sgl-eval to 0.1.2 (#40620) 2026-09-21 13:00:54 -07:00
11e661fd45 [Fix] Don't free the multi-CTAs KV counter the decode graphs captured (#39175)
Co-authored-by: mmangkad <mohammad.angkad@radixark.ai>
Co-authored-by: kpham-sgl <khoa.pham@radixark.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-21 12:58:48 -07:00
Cheng Wan 44bdf225d8 Fix lint failure from MXFP8 reserved-slot test location (#40618) 2026-09-21 12:31:11 -07:00
Cheng Wan bccf691b22 Bringing the parallel runtime up becomes a phase, not a side effect (#40345) 2026-09-21 12:29:50 -07:00
Cheng Wan 1d3243d05f Take the parallel getters off the package's public surface (#40344) 2026-09-21 12:27:50 -07:00
Cheng Wan 970e946e4f Retire the per-runner parallel record (#40343) 2026-09-21 12:26:40 -07:00
Cheng Wan 73f071db52 Deprecate the parallel getters the context answers, and ratchet them shut (#40342) 2026-09-21 12:25:32 -07:00
Cheng Wan 65be3fa71a A runner and the objects it builds freeze the placement they describe (#40341) 2026-09-21 12:24:17 -07:00
Cheng Wan 2d0e94e3a3 Check the topology identities where the layout is written, and build at the published widths (#40340) 2026-09-21 12:22:59 -07:00
ishandhananiandKangyan-Zhou d5fdab7022 chore: add NIXL owners and CI access (#40602)
Co-authored-by: Kangyan-Zhou <zky314343421@gmail.com>
2026-09-21 12:21:30 -07:00
Cheng Wan 0db1a93adb State the draft's whole topology in its scope, and read the rest from the context (#40339) 2026-09-21 12:19:38 -07:00
ae7a516ba7 feat: use XGrammar V4.1 DSML parameter constraints (#39026)
Co-authored-by: yuchuan <yuchuan.7streams@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
2026-09-21 12:12:28 -07:00
Faradawn Yang f0940fe3a6 Update DeepSeek-V4 Pro for B200 FP4 agentic PD disaggregation (#40610)
Signed-off-by: Faradawn Yang <73060648+faradawn@users.noreply.github.com>
2026-09-21 12:10:03 -07:00
jacky.cheng 90b3f8544c [AMD] Use Triton softmax routing for Qwen3.5 on gfx950 (#39986) 2026-09-21 12:02:29 -07:00