Commit Graph
12534 Commits
Author SHA1 Message Date
Liangsheng Yin dc5f59c3a2 [Refactor] Clarify DeepSeek V4 metadata names for V4.1 (#38947) 2026-09-10 15:42:52 -07:00
Pengyun Lin d076eec427 [SM120] Use exact query-head widths for DeepSeek-V4 sparse MLA decode (#36655) 2026-09-10 15:40:17 -07:00
paulzhang-tmandQiaolin-Yu bb15be6d79 [Spec] Stage Inkling MTP draft metadata before verify (#38169)
Co-authored-by: Qiaolin-Yu <liin1211@outlook.com>
2026-09-10 15:32:19 -07:00
paulzhang-tm a63efd9056 [Spec] Support large MTP batches in short-convolution metadata (#38558) 2026-09-10 15:13:17 -07:00
fd7743e0e1 [Sampling] Support sampling masks with overlap scheduling (#36631)
Co-authored-by: ByronHsu <ByronHsu@users.noreply.github.com>
Co-authored-by: root <root@slurm-h200-208-179.slurm-compute.tenant-slurm.svc.cluster.local>
Co-authored-by: Byron Hsu <byron+per@periodiclabs.ai>
2026-09-10 14:52:44 -07:00
YAMY 55b45cb45a fix(qsa): dequantize FP8 cached prefixes in the sparse prefill kernels (#38855) 2026-09-10 14:07:39 -07:00
06dfe05d65 [Deps] Raise smg-grpc-servicer floor to >=0.9.0 to unbreak SMG E2E CI (#38801)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 13:55:36 -07:00
a26273d668 [AMD] Quantize the bf16 MTP draft experts online to MXFP4 for Qwen3.5 (#38748)
Co-authored-by: Zijie Chen <300606707+zijiecode@users.noreply.github.com>
Co-authored-by: jacky.cheng <yichiche@amd.com>
2026-09-10 13:00:41 -07:00
James 209654c420 [NPU][Hicache] Optimize HiCache L2 IO with Memfabric acc_offload (#38826) 2026-09-11 00:23:36 +08:00
Shuwen Wang 19b83b4b9d [Radix Cache] Fix PureSWA tail release without insertion (#38349) 2026-09-10 23:55:48 +08:00
Shuwen Wang cc7e43ad22 [HiCache] Account for newly pinned ancestors in load-back quota (#38481) 2026-09-10 23:54:08 +08:00
pllimax 92dffebe16 [NPU] Set DEEPEP_HYBRID_DEPLOYMENT for new DeepEP tests; switch glm5_2 to w8a8; tune nightly timeouts (#38775) 2026-09-10 23:32:54 +08:00
Chengze Fan 12771786f2 [AMD] Restore AITER verify runtime sizing reverted by #34647 (#38575) 2026-09-10 05:57:47 -07:00
Jialin Ouyang 908226fea2 [Rust TreeCore] Support external cache linker (#37306) 2026-09-10 19:22:24 +08:00
liupeng374 334e94d8ac [NPU] fix pp 2 hang on npu (#38249) 2026-09-10 19:21:37 +08:00
Mohammad Miadh AngkadandMohammad Angkad dc2157dcd6 [JIT] Port the expert-pack MXFP4 kernels to load_jit and fix their launch limits (#38830)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-09-10 17:50:51 +08:00
MickandMick Qian c9c26d56b2 [diffusion] docs: sync snapshot and minimax-h3 subblock features (#38784)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-09-10 17:23:43 +08:00
8a6ab89bf0 [AMD] Enable Fast Triton Sparse MLA backend (#30575)
Co-authored-by: clintg6 <7388379+clintg6@users.noreply.github.com>
Co-authored-by: HAI <hixiao@gmail.com>
2026-09-10 01:58:00 -07:00
ashwini rathi b23db06769 pyproject(xpu): drop human-eval git dep to unblock image build (#38824) 2026-09-10 16:24:43 +08:00
1b77f498a0 [NVIDIA] Support flashinfer Mega Moe (#31470)
Co-authored-by: djns99 <40156487+djns99@users.noreply.github.com>
Co-authored-by: 云挚 <ningyunxiao.nyx@antgroup.com>
Co-authored-by: Yangmin Li <yangminl@nvidia.com>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
2026-09-10 00:22:47 -07:00
Brayden ZhongandBrayden Zhong c0b790cf7f Delete cutlass_mla, non-Marlin GPTQ, AWQ AOT kernel, and Dual Chunk Flash Attention (#32114)
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
2026-09-10 15:12:01 +08:00
Brayden ZhongandBrayden Zhong 7152c14384 Fix DeepSeek-V4 routing: sqrtsoftplus underflow and unfloored renorm (#34459)
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
2026-09-10 15:10:50 +08:00
Shuwen Wang 058cca0192 [HiCache] Fix side pools to use resolved host allocator (#38350) 2026-09-10 14:28:20 +08:00
DAI0818andPo-Han Huang 03e4c06589 Fix Qwen3.5 GDN multi-item scoring (#33922)
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
2026-09-09 22:30:39 -07:00
ashwini rathiandarathi-hlab 6ab6df54b9 Add missing test dependencies to pyproject.toml variants (#36098)
Co-authored-by: arathi-hlab <arathi.rajendran@intel.com>
2026-09-10 12:49:16 +08:00
Cheng Wan f1a512c51c [Config] msgspec.Struct for the config tier (#38753) 2026-09-09 19:41:19 -07:00
Cheng Wan 53dc77ff4e [Config] One writer for the declaration stash; no exception to the write seal (#38752) 2026-09-09 19:22:06 -07:00
fd596a474c [XPU][Fix] Pack device-pointer tables as uint64 to avoid 64-bit address overflow (#35051)
Co-authored-by: roopaksrivastav <roopak.srivastava@intel.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 10:12:18 +08:00
Mick 7f0285093e [diffusion] feat: spill large tensors over shared memory like numpy arrays (#38656) 2026-09-10 09:14:06 +08:00
MickandMick Qian ce555ed82a [diffusion] refactor: refactor utility ownership and document helper placement (#38699)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-09-10 09:11:01 +08:00
LinyuanLi 3ff226ba8f [NPU]Support GLM5.2 and FP8 DSA&Indexer kvcache for 950 (#38250) 2026-09-10 09:10:10 +08:00
92d831d3d7 [AMD] Parallelize aiter spec-decode KV index building over token blocks (#37659)
Co-authored-by: Zijie Chen <300606707+zijiecode@users.noreply.github.com>
Co-authored-by: jacky.cheng <yichiche@amd.com>
2026-09-09 17:53:53 -07:00
880d6fa64d [DSv4] Integrate TRT-LLM DSv4 Attention for SM100/103 (#30805)
Co-authored-by: Yangmin Li <yangminl@nvidia.com>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
2026-09-09 17:40:07 -07:00
William Hu 0084030179 Add Opt-In for GLM-5.3 Flash breakable prefill CUDA graphs (#38522) 2026-09-09 17:02:08 -07:00
Po-Han Huang (NVIDIA) 51c8581a26 [Rust] Gate health on startup warmup completion (#37994) 2026-09-09 15:55:13 -07:00
Zhiqiang Xie beaf3d9252 [HiCache] Replace skip_lock_node_ids with a segment lock protocol (#36848) 2026-09-09 14:55:51 -07:00
FrancisandQiaolin-Yu a84ffd1326 feat: add optimized Domino rollout to DFlash V2 (#36899)
Co-authored-by: Qiaolin-Yu <liin1211@outlook.com>
2026-09-09 14:12:15 -07:00
Po-Han Huang (NVIDIA)andYAMY 96d91ef926 [Model] Support GLM-5.3 Flash NVFP4 loading (#38621)
Co-authored-by: YAMY <74099316+YAMY1234@users.noreply.github.com>
2026-09-09 13:51:08 -07:00
bede776c2a fix(unified-memory): evict Full KV for Mamba byte shortfalls (#36713)
Co-authored-by: Yangmin Li <yangminl@nvidia.com>
Co-authored-by: YAMY <74099316+YAMY1234@users.noreply.github.com>
2026-09-09 13:46:51 -07:00
paulzhang-tm 6c1d0b1b29 Revert "[Spec] Publish the final multi-layer EAGLE shared-read event" (#38041) 2026-09-09 13:01:16 -07:00
Alison ShaoandClaude Opus 5 95a88bfd69 Relax GSM8K thresholds for the GLM-5.2 DSA-MTP variants (#38725)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-09 12:46:10 -07:00
cctry 354ee46c26 Keep VMM capability votes on CPU (#38722) 2026-09-09 11:56:30 -07:00
Sage 8733da8cf4 [rust-server] Use node-local HTTP ports for DP attention (#34430)
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com>
2026-09-10 02:22:44 +08:00
7b791c9534 [Bugfix] Keep a shared MAX_LEN prefill CUDA graph bucket when the graph captures a DP gather (MegaMoE sparse-DP hang) (#37933)
Co-authored-by: shyeh25 <206795756+shyeh25@users.noreply.github.com>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
2026-09-09 10:52:16 -07:00
0027af2eac [diffusion] [NPU] Optimize LTX-2/2.3 inference performance for NPU (#34722)
Co-authored-by: Elizaveta Martirosian <you@example.com>
Co-authored-by: ronnie_zheng <zl19940307@163.com>
Co-authored-by: root <root@localhost.localdomain>
Co-authored-by: Makcum888e <79456407+Makcum888e@users.noreply.github.com>
Co-authored-by: mickqian <mickqian@users.noreply.github.com>
2026-09-09 19:36:18 +03:00
HuangJi ffe98a4279 [Diffusion][MiniMax-H3] Add SM90 Sage compute for SubBlock sparse attention (#37982) 2026-09-09 20:00:03 +08:00
Even Zhou dba34cc964 [NPU] Bump memfabric and sgl-kernel-npu versions in docs and pyproject_npu.toml (#38437) 2026-09-09 19:53:49 +08:00
Mohammad Miadh AngkadandMohammad Angkad 72d5c5bb73 [Kimi-K3] Accept fp32 routing weights in the fused MoE finalize (#38612)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-09-09 01:33:39 -07:00
ashwini rathiandarathi-hlab d10ebdd0cb docker(xpu): unblock nightly build (setvars.sh + sgl-kernel rename) (#38617)
Co-authored-by: arathi-hlab <arathi-hlab@users.noreply.github.com>
2026-09-09 16:32:16 +08:00
YAMY 1ad3eb09a9 [Attention] Size FlashInfer MLA indptr buffers to the padded max batch (#38590) 2026-09-09 01:25:29 -07:00