Commit Graph
109 Commits
Author SHA1 Message Date
YAMYandOasis-Git c82f7141ad [Perf] [NVIDIA] dspark: Move the host work between DSv4.1 decode graphs into the graphs for the megakernel path (#45450)
Co-authored-by: Oasis-Git <ayw.sirius19@gmail.com>
2026-10-22 19:08:06 -07:00
bb9e9d5632 [Perf] mlp: Fuse dense SwiGLU with block-FP8 quantization on decode (#45219)
Co-authored-by: Jayden Yu <jaydenyu@fb.com>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
2026-10-23 07:49:39 +08:00
Jimmy Shong 9301ca37a0 [Fix] modelopt: Correct NVFP4 TRT-LLM clamps for untied gate/up scales (#45386) 2026-10-21 17:37:07 -07:00
Liangsheng Yin 3438ed660d [Test] cpu: prune mirror and tautology cases from the per-commit CPU suite (#45672) 2026-10-21 16:02:47 -07:00
Khoa Pham 028de2c5d4 [Perf] dsv41: Use tuned MXFP8 tactics at decode sizes (#45630) 2026-10-21 13:35:56 -07:00
vorapolsiloai 6e98c84c60 [Perf] [AMD] quark: Add opt-in AITER ASM FP4 GEMM for Quark dense linear (#39598) 2026-10-21 09:37:47 -07:00
Junlin Wu 844d2b5dec [Fix] [NPU] qwen3.5: Fix dense serving on Ascend 950 (#32745) 2026-10-21 09:50:38 +03:00
Baizhou Zhang 5b1dbc2c80 [Fix] [NVIDIA] modelopt-nvfp4: Keep MoE global scales in gate/up order (#45289) 2026-10-20 16:49:13 -07:00
Cheng WanandAditya Suresh 299a75ac5d [Refactor] moe: Derive routing inputs from the layer and active forward (#44994)
Co-authored-by: Aditya Suresh <152752311+adi-suresh01@users.noreply.github.com>
2026-10-20 16:26:15 -07:00
Cheng WanandAditya Suresh 4f8f06faf3 [Refactor] moe: Derive layer contracts from the selected quant method (#44993)
Co-authored-by: Aditya Suresh <152752311+adi-suresh01@users.noreply.github.com>
2026-10-20 16:18:45 -07:00
Ho-Ren (Jack) Chuang f8fd41c36e [Perf] moe: Extend FlashInfer CUTLASS MoE autotune coverage to prefill chunks (#43363) 2026-10-20 01:41:16 -07:00
Mohammad Miadh AngkadandMohammad Angkad 78185f4b3f [Fix] [NVIDIA] modelopt-nvfp4: Read w13 global scales in the TRT-LLM [up, gate] order (#44858)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-10-20 07:38:45 +00:00
Gabriel Wuandnumber-eleven11 67d9ee156c [Fix] [SM120] quantization: Requantize block-FP8 MoE weights to UE8M0 for DeepGEMM (#43556)
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: number-eleven11 <1635219023@qq.com>
2026-10-19 19:06:21 -07:00
Qiaolin Yu 5385866108 [Fix] moe: gated NVFP4 MoE intermediate alignment for FlashInfer TRTLLM (#44138) 2026-10-19 12:52:36 -07:00
Sam Shleifer 417606527d [Fix] nvfp4: Own streamed tensors while waiting for paired weights (#44500) 2026-10-19 22:41:53 +08:00
Kolade FajimiandXiaoyu Zhang 5117bfbaeb [Fix] compressed-tensors: Serve NVFP4 W4A4 linears via FP4 Marlin when --fp4-gemm-backend marlin is set on SM100 (#44213)
Signed-off-by: kolade fajimi <fajimikolade23@gmail.com>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
2026-10-16 12:36:02 +08:00
jlqibmandClaude Opus 5 74e0a47863 Fix/compressed tensors nvfp4 hopper (#40022)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-10-13 20:10:31 +08:00
Liangsheng Yin c55790d38b [CI] Drop CPU tests with no bug-catching signal and run Rust workspace tests in their own workflow (#43503) 2026-10-10 02:13:53 -07:00
Vladislav NosivskoyandBBuf 2ff3e6dce1 [SM90] Use native block32 DeepGEMM for DeepSeek-V4.1 (#42604)
Co-authored-by: BBuf <1182563586@qq.com>
2026-10-10 10:29:42 +08:00
Colin Z 3c1703d2ff [AMD] Support Quark DeepSeek-V4-MXFP4 checkpoints on the DSV4 stack, enabling shared-expert fusion and MTP (#31054) 2026-10-08 22:54:49 -07:00
Liangsheng Yin 71584b62c0 [CI] Fix stale fakes in the NVFP4 MoE dispatch and CP strategy unit tests (#43226) 2026-10-08 15:51:39 -07:00
Xuanteng Huang d20bdc1443 Add per-token NVFP4 MoE support for ReLU2 activation (#39199)
Signed-off-by: Xuanteng Huang <xuanteng.huang@outlook.com>
2026-10-08 14:28:22 -07:00
Cheng Wan 5e12831cec [Refactor] Load auxiliary and expert checkpoints using owner partitions (#42589) 2026-10-07 16:24:03 -07:00
Cheng Wan 64c7f65eac [Refactor] Retain process groups in parallel linear layers (#42588) 2026-10-07 16:23:40 -07:00
Tuan Nguyen GiaandCursor c0df52be2d [AMD] Fix Quark W4A4 MXFP4 MoE clamped-SwiGLU on the AITER path (#34540)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-07 03:20:40 +00:00
Zhang, Jiejingandkyle-256 aa0ce20201 [ROCm] GLM-5.2 decode path: decode-shaped MoE/MLA tiles, split speculative softmax, and bf16 GEMM routing (#41725)
Co-authored-by: kyle-256 <Kyle.Zhao@amd.com>
2026-10-04 11:19:37 -07:00
Cheng Wan 10edb05e72 [Refactor] Let the remaining placement consumers read the parallel context (#42348) 2026-10-03 14:59:32 -07:00
Yanbin Jiang 254aa41f78 [Fix] Guard DeepSeek NVFP4 shared-expert fusion for LoRA and FP4 backends (#42203) 2026-10-03 00:37:57 -07:00
Yu-Yun Changandkk 6af651ea0c [AMD][Quark] Serve the Kimi-K3 MXFP4 checkpoint on ROCm (#40811)
Co-authored-by: kk <43161300+kkHuang-amd@users.noreply.github.com>
2026-09-30 18:14:15 -07:00
Cheng Wan e289989933 [Fix] Shard MoE WNA16 and Quark INT4-FP8 weights by the MoE placement (#41807) 2026-09-30 13:25:38 -07:00
Colin Z 875dd41e6f [AMD] Fix Load and Inference of MLA models with Quark PTPC FP8 attention on ROCm (#28734) 2026-09-29 11:20:10 -07:00
Xiaoyu Zhang 38ec649048 [Refactor] Restore logical kernel groups and test organization (#41243) 2026-09-26 12:30:44 +08:00
Liangsheng Yin f35ec18d8b [Test] Remove more unit tests that mirror implementation or never run in CI (#41297) 2026-09-25 19:22:14 -07:00
f4d9d2e670 [RL] Add RL weight-update sessions and support updating spec draft runners (#40777)
Co-authored-by: Jiajun Li <guapisolo@gmail.com>
Co-authored-by: Nan Jiang <59716405+nanjiangwill@users.noreply.github.com>
Co-authored-by: maocheng23 <maocheng@berkeley.edu>
Co-authored-by: Byron Hsu <byron+per@periodiclabs.ai>
Co-authored-by: JD-ETH <jaedon.guo@gmail.com>
Co-authored-by: Letian Ruan <ruanlt420@gmail.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-09-23 20:04:57 -07:00
Liangsheng Yin 78980a3b0b [misc] Remove deprecated endpoints, env vars and aliases past two releases (#40795) 2026-09-22 18:14:45 -07:00
c19dc43cc7 [AMD] [GLM-5.3-Flash Day 0] Load the MXFP4 MTP draft layer (#39779)
Co-authored-by: Thomas Wang <thomawan@amd.com>
Co-authored-by: CHANG <jacchang@crsuse2-slog-003.us-east2-a.compute.internal>
Co-authored-by: Kevin Mi <mikevin920@yahoo.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Kevin Mi <45493463+kevin-mii@users.noreply.github.com>
2026-09-22 15:07:51 -07:00
YAMY 9b59fc5db5 [ModelOpt][PP] Keep BF16 shared experts out of the NVFP4 fusion so TP1 pipeline stages can load (#40628) 2026-09-21 21:45:58 -07:00
e1daf68304 [AMD] [GLM-5.3-Flash Day 0] Honor fused and per-expert names in quark exclude (#39317)
Co-authored-by: Yikai Zhang <ykzhang12@gmail.com>
Co-authored-by: Thomas Wang <thomawan@amd.com>
Co-authored-by: Kevin Mi <45493463+kevin-mii@users.noreply.github.com>
Co-authored-by: Kevin Mi <mikevin920@yahoo.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-21 21:26:42 -07:00
b44e248682 [AMD] [GLM-5.3-Flash Day 0] Enable FP8 and Quark MXFP4 MoE on gfx950 (#38546)
Co-authored-by: Raiden-Makoto <Raiden-Makoto@users.noreply.github.com>
Co-authored-by: Thomas Wang <thomawan@amd.com>
Co-authored-by: andyluo7 <andy.luo@amd.com>
Co-authored-by: Kevin Mi <mikevin920@yahoo.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-21 21:07:37 -07:00
Cheng Wan acac4dd9d9 [Refactor] Clean up parallel runtime comments (#40632) 2026-09-21 14:32:22 -07:00
Liangsheng Yin 76a9065bef [Fix] Raise on undelivered embeddings in send_with_url, fix broken tests (#40502) 2026-09-20 17:35:27 -07:00
42875bcd2a fix(modelopt): dispatch NVFP4 MoE on the cached backend, not the live global (#38932)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
2026-09-20 19:28:54 -04:00
DarkSharpnessandBBuf d1acbe0746 [DSV4.1] Big fused wo_a quant (#39957)
Co-authored-by: BBuf <1182563586@qq.com>
2026-09-19 19:46:53 +08:00
Cheng Wan 567d5925fe Fix mxfp4 padding test stubbing an accessor the module no longer imports (#40308) 2026-09-19 00:43:56 -07:00
Liangsheng Yin d507accadc [Test] Drop dead and strictly-subsumed CI test registrations (#40264) 2026-09-18 17:49:13 -07:00
Cheng Wan afe71f4b9e Read process groups through the runtime context (#40068) 2026-09-18 17:40:32 -07:00
Sam (Kesen Li) d346b214fb feat(kv-cache): support SM100 NVFP4 GenMHA and speculative decoding (#36340) 2026-09-18 14:50:46 -07:00
a6cf05817f dsv4.1: remaining model and runtime integration (#38798)
Co-authored-by: BBuf <1182563586@qq.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Xiaoyu Zhang <xiaoyu.zhang@radixark.ai>
Co-authored-by: Yuwei An <ayw.sirius19@gmail.com>
Co-authored-by: Khoa Pham <khoa.pham@radixark.ai>
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: Zhichen Zeng <zczeng@uw.edu>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
2026-09-18 02:55:30 -07:00
Liangsheng Yin 1b200ffaaa [Quant] Serve 32-wide-K ue8m0 block-FP8 linears through the FlashInfer MXFP8 GEMMs (#40039) 2026-09-18 02:51:12 -07:00
cebca698e2 [Qwen3.8] Enable NVIDIA NVFP4 on DGX Spark with file-backed PLE and PDL router fix (#39126)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: rdxa <rdxa@rdxa-int-spark-01.yvb.moe>
Co-authored-by: Yangmin Li <yangminl@nvidia.com>
Co-authored-by: Manrique <nanomlm@gmail.com>
Co-authored-by: yhyang201 <yhyang201@gmail.com>
2026-09-13 16:23:41 +08:00