Commit Graph
9903 Commits
Author SHA1 Message Date
Mick 25b6051c70 [diffusion] warmup: default to model sampling resolution (declare Z-Image default) (#29519) 2026-06-30 10:32:11 +08:00
Ho-Ren (Jack) Chuang bc8b3ab1f5 [Kernel] Add SM90 Q8KV8 FP8 Sparse MLA Prefill JIT Kernel with Tests and Benchmark (#25751) 2026-06-30 09:00:11 +08:00
Ho-Ren (Jack) Chuang a6bc432fd8 [Perf][Kernel] Fuse SiLU+Mul into NVFP4 Expert Quantization for CUTLASS MoE (#18612) 2026-06-29 16:51:01 -07:00
eeechoandAliceChenyy f0bf96390b [SM120] Add FlashInfer sparse MLA decode for DSv4-Flash (#27455)
Co-authored-by: AliceChenyy <alicechenyy@users.noreply.github.com>
2026-06-29 16:27:16 -07:00
Trevor Morris b8c25bfaa7 [NVIDIA] Support flashinfer a2a with flashinfer_trtllm_routed moe (#22394) 2026-06-29 16:23:58 -07:00
Khoa PhamandClaude Opus 4.7 6bdecb8206 [DeepSeek V3] Reland: run routed experts on main stream in dual-stream MoE (#29463)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-29 15:05:01 -07:00
Xinyuan Tong 45314a9fcb [spec] Fix index_share_for_mtp_iteration being a no-op in EAGLE MTP draft (#29654) 2026-06-29 14:52:23 -07:00
Cheng Wanandlch1475369 fc96edd297 feat(mem_cache): page-major (layer-major within a page) KV/state layout (#29533)
Co-authored-by: lch1475369 <lch1475369@gmail.com>
2026-06-29 14:49:54 -07:00
Mohammad Miadh Angkad 6c018eb4d1 Fix bounded checkpoint prefetching and buffered drop-cache handling (#29156) 2026-06-29 21:49:16 +00:00
5556631789 [Speculative Decoding] Validate vocabulary compatibility in STANDALONE mode (#23838)
Co-authored-by: kpham-sgl <khoa.pham@radixark.ai>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 14:45:12 -07:00
Brayden Zhong 98d0e702c3 [GLM-5] Tune the threshold of router GEMM (#29470) 2026-06-29 14:19:24 -07:00
cctryandcctry a5c45a12bb CUDA graph executable dedup via cudaGraphExecUpdate (#29625)
Co-authored-by: cctry <cctry@fb.com>
2026-06-29 13:56:33 -07:00
f480c5f1f9 [Spec] Frozen-KV MTP: delay target KV binding to pool init + reset stale draft out_cache_loc (#29616)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: bef0rewind <490285+ronhuafeng@users.noreply.github.com>
Co-authored-by: AlejandroParedesLT <99832715+AlejandroParedesLT@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-29 11:57:10 -07:00
Mick b0be644133 [diffusion] feat: keep image-model auxiliary components resident under auto memory policy (#29649) 2026-06-30 01:25:37 +08:00
Feng Yao e6c15f76f3 [optimize] fix swa eviction boundary for unfinished inserts (#29350) 2026-06-30 00:19:15 +08:00
Jyothirmai KottuandXinyuan Tong 473a278dd1 model: support nvidia/LocateAnything-3B (#28958)
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
2026-06-30 00:16:42 +08:00
McZyWu bb7d3440b5 bugfix revise interface get cpu copy for npu mem pool to align with gpu (#29146) 2026-06-29 19:30:14 +08:00
danielafrimiandDaniel Afrimi a2b5ce2ed1 Add stochastic rounding for FP16 Mamba SSM cache (#26929)
Signed-off-by: Daniel Afrimi <dafrimi@login-lyris01.lyris.clusters.nvidia.com>
Co-authored-by: Daniel Afrimi <dafrimi@login-lyris01.lyris.clusters.nvidia.com>
2026-06-29 01:47:09 -07:00
iridiumineandiridiumine d5133e925b [NPU][Bugfix] Add scoring_func for mimo_v2 (#29493)
Co-authored-by: iridiumine <iridiumine@users.noreply.github.com>
2026-06-29 16:34:54 +08:00
bef0rewind 3b1b512a9a Fix disaggregation receiver ZMQ cleanup (#29570) 2026-06-29 16:22:29 +08:00
SSSunzt 91cf159696 [PP] bugfix: include CP size in PP rank offset (#29571) 2026-06-29 16:01:41 +08:00
Liangsheng Yin c0d45cda29 [Spec] Add DFLASH basic sanity CI test (#29338) 2026-06-28 23:45:05 -07:00
AndyLi429 62d7929b3a [NPU][Bugfix] Accept in_capture in Ascend replay metadata (#29598) 2026-06-29 14:37:45 +08:00
Yueming Yuan cd91dd0757 fix test_weight_checker_comparator assertion and ue8m0 scale unpack (#29623) 2026-06-28 23:36:37 -07:00
silencejade f85cc94d82 [NPU] Qwen3-VL-30B use split_qkv_rmsnorm_rope for extend (#29505) 2026-06-29 14:35:19 +08:00
Lianmin Zheng bb74ed4a8d Replace hasattr with isinstance in SHM feature helpers (#29549) 2026-06-28 21:31:28 -07:00
Lianmin Zheng f76e707f59 Clean up follow-ups for eagle hidden dim clean up (#29546) 2026-06-28 21:30:53 -07:00
Peng Xingchen 06fd2efedd [NPU][Bugfix] Fix a ModelSlim loading failure (#29029) 2026-06-29 09:53:01 +08:00
Liangsheng Yin be17475a1b [Spec] Publish DFLASH verify read-done event for fine-grained WAR barrier (#29541) 2026-06-28 18:51:38 -07:00
Yueming Yuan e1ca92a7fd [weight checker] refactor: add precision branch; allow ULP quant err; used chunked compare (#28974) 2026-06-28 18:44:05 -07:00
Liangsheng Yin 909123ddb8 [misc] Use --cuda-graph-max-bs-decode in tests, examples, and docs (#29591) 2026-06-28 18:38:28 -07:00
Ma Mingfei 3217410cf6 [CPU] enable fused_sigmoid_mul on CPU device (#29378) 2026-06-29 09:38:04 +08:00
Polisetty V R K Jyothendra Varma d5abafcc1c [Intel GPU] add pytorch profiling support for XPU in bench offline throughput and enhance num steps (#28308)
Signed-off-by: P V R K Jyothendra Varma <polisettyvarma@gmail.com>
2026-06-29 09:10:15 +08:00
Liangsheng Yin abaee4664e [dflash] fa3/fa4: device-side page table; drop seq_lens_cpu D2H sync (#29343) 2026-06-28 18:06:33 -07:00
Chao Shi bd2a5db987 [HiCache] Detect for double-free in HostKVCache (#29310) 2026-06-29 09:02:21 +08:00
Brayden ZhongandBrayden Zhong b9b860652e Fix DSA indexer fusion bug causing excessive memory consumption. (#29576)
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
2026-06-28 14:11:07 -07:00
Aurick Qiao ad30a9958e Fix dummy weight init for tensor subclasses (#29229) 2026-06-29 01:17:00 +08:00
Yinghai Lu c5b9388721 [scheduler] Add scheduler metrics reporter init hook (#29535) 2026-06-28 09:59:34 -07:00
ishandhanani aaa31eb0a1 feat: first-class session identity in SGLang (#29436) 2026-06-28 07:16:38 -07:00
Lianmin Zheng 828411e6f1 Fix FlashInfer A2A dispatcher during CUDA graph capture (#29461) 2026-06-28 02:21:26 -07:00
Mick 4cb6d81bda [diffusion] CI: make consistency GT probe robust to transient CDN failures (#29545) 2026-06-28 17:13:53 +08:00
Liangsheng Yin 6582fc1e5d dflash: drop verify_done barrier; rely on scheduler WAR fallback (#29556) 2026-06-28 01:47:17 -07:00
Liangsheng Yin 593f5ba68f [Spec] Replace shared-infra dflash special-cases with capabilities (WAR barrier + seq_lens_cpu) (#29232) 2026-06-28 00:02:32 -07:00
Mick 643e1cc779 fix: fix prefill-aware SWA floor tracking (#29520) 2026-06-28 14:18:28 +08:00
Xinyuan Tong ddc389cf09 [minimax-m3] Split 3/4: disagg K-only index-K transfer (#28714) 2026-06-28 13:54:42 +08:00
Lianmin Zheng 6eedc8f376 Fix EAGLE draft hidden dim extraction and centralize spec helpers (#29464) 2026-06-27 21:48:26 -07:00
Lianmin ZhengandYinghai Lu 5747ed3b19 Fix DP-attention SHM feature finalization race (#29543)
Co-authored-by: Yinghai Lu <yinghai@meta.com>
2026-06-27 21:36:58 -07:00
Mick 4a76699dfc [diffusion] fix: fix --warmup silently downgrading server-based warmup to request mode (#29514) 2026-06-28 11:46:07 +08:00
Lianmin Zheng e6cbc8f5fe Fix SWA cache loc slicing for all attention backends (#29460) 2026-06-27 20:37:58 -07:00
Liangsheng Yin 4e6ce8e4ab [Spec] Capture DFLASH draft greedy sampling inside the draft decode cuda graph (#29395) 2026-06-27 19:34:26 -07:00