Commit Graph
6055 Commits
Author SHA1 Message Date
50dae2d99d Amd/dsv4 shared experts fusion top6 (#32340)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: kk <43161300+kkHuang-amd@users.noreply.github.com>
Co-authored-by: Thomas Wang <thomawan@amd.com>
2026-08-19 22:57:58 -07:00
Michael 02b93e7e01 [AMD] Add GLM-5.2 MI35x nightly accuracy and perf benchmark (#32570) 2026-08-19 22:49:43 -07:00
Alex NailsandClaude Opus 5 b6dcd393d6 [Fix] Support 128-aligned hidden sizes in the W4AFP8 DeepEP low-latency requant kernel (#35593)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 22:18:17 -07:00
9db4ba8da1 [DeepSeek-V4] Add Q8KV8 sparse MLA prefill runtime backend (#32327)
Co-authored-by: Ho-Ren (Jack) Chuang <horenchuang@bytedance.com>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
2026-08-20 10:23:49 +08:00
dc175b3ad2 [CI][AMD] Run the profiling suite without CUDA graphs on ROCm (#34452)
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: quitenode <quitenode@users.noreply.github.com>
2026-08-20 09:41:28 +08:00
Mick ab203663c4 [diffusion] fix: reject unsupported modelopt checkpoint algorithms (#35182) 2026-08-20 09:13:24 +08:00
Jimmy ShongandLING ZHI 1cf2b8c54d [Spec] Support quantized target lm_head in the DFlash2 selector (#35496)
Co-authored-by: LING ZHI <1747985437lz@gmail.com>
2026-08-19 18:06:41 -07:00
MichaelandCursor Agent e805a8f98e [AMD] Keep the PTX-inline-asm diffusion norm fusions off on ROCm (fix FLUX warmup crash) (#34481)
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2026-08-20 08:55:28 +08:00
Jimmy Shong 5375babbac [Quant] Load compressed-tensors quantized lm_head instead of value-casting it (#35228) 2026-08-19 15:37:45 -07:00
c7e2c08d14 fix(constrained): reject NUL bytes in grammar specs to stop an xgrammar segfault (#34679)
Signed-off-by: Junhao Shen <junshen@nvidia.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
Co-authored-by: kpham-sgl <khoa.pham@radixark.ai>
2026-08-19 15:31:14 -07:00
milesial 082aac8fce [Bugfix] Fix min-new-token EOS handling (#31378)
Signed-off-by: Alexandre Milesi <milesial@users.noreply.github.com>
2026-08-19 15:29:22 -07:00
Liangsheng Yin 1270204d2c Revert "[Feature] Add DeepEPv2 (ElasticBuffer) MoE A2A backend" (#35568) 2026-08-19 14:14:38 -07:00
elvischenvandBrayden Zhong a6bc0532c9 [Fix] Fix Nemotron-H Mamba illegal memory access under DP attention with CUDA graph (#34561)
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
2026-08-19 14:13:40 -07:00
milesial ed12d6827d fix(disagg): allow fake transfer with decode DCP (#35409)
Signed-off-by: Alexandre Milesi <milesial@users.noreply.github.com>
2026-08-19 13:54:37 -07:00
Jason Wiemels defb2a3100 feat(openai): Accept the input_audio content part in chat completions (#33606) 2026-08-19 13:37:50 -07:00
DarkSharpnessandClaude Opus 5 746418a1ec [DSA] Trim top-k v2 output modes and tighten its PDL waits (#35041)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 13:09:21 -07:00
cctry 1c82955861 [HiCache] Split the host-memory budget across co-located ranks (#35540) 2026-08-19 13:01:58 -07:00
Po-Han Huang (NVIDIA) 6f69f927da [Scheduler] Add configurable decode interval after prefill (#35017) 2026-08-19 12:01:36 -07:00
MengYuandmenyu 4f8ecf6ae9 [Feature] Add DeepEPv2 (ElasticBuffer) MoE A2A backend (#29525)
Co-authored-by: menyu <menyu@nvidia.com>
2026-08-19 11:52:45 -07:00
YAMY 5f12839591 [Fix] Support Kimi-K3 ModelOpt mixed NVFP4/FP8 checkpoint (#35077) 2026-08-19 08:13:45 -07:00
Shuwen Wangandhzh0425 41c018a9ec [UnifiedTree] feat: support runtime attach/detach (#35269)
Co-authored-by: hzh0425 <hzh0425@apache.org>
2026-08-19 22:49:08 +08:00
Rohit Kumar SinghandSingh 3e5ce26c2d fix: fix transcription & audio-understanding for ASR/audio/speech models (#32611)
Co-authored-by: Singh <rohitsi2@iil-login.iind.intel.com>
2026-08-19 18:44:23 +08:00
f446e853e7 [AMD] DeepSeek-V4: route decode wo_a bf16 batched matmul to aiter batched_gemm_bf16 (#33313)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Thomas Wang <thomawan@amd.com>
2026-08-19 03:04:48 -07:00
karverma-amd ce1830c59b [AMD] DeepSeek-V4 MI355X: eliminate bpreshuffle fp8-scale relayout copy in dense w8a8 linear (#33165) 2026-08-19 03:02:40 -07:00
HeYaoandHeYao f22442d3a4 Add three new test cases (#35502)
Co-authored-by: HeYao <heyao@example.com>
2026-08-19 17:59:45 +08:00
Shangming Cai adca19c497 [PD] Deferred decode-side KV release for the NIXL backend (#35360) 2026-08-19 17:29:45 +08:00
YAMY aa215e5523 [PD] Overlap prefill DP-rank bootstrap queries (#35071) 2026-08-19 17:25:52 +08:00
Khoa PhamandCursor 0e4a09480c [HiCache] Support DCP with DSpark (#35221)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-19 01:42:19 -07:00
8a1e6e4e46 Qwen3.8-27B Model Support (#34859)
Co-authored-by: Jimmy Shong <69131491+Jiminator@users.noreply.github.com>
Co-authored-by: Brayden Zhong <brayden.zhong@radixark.ai>
Co-authored-by: BBuf <1182563586@qq.com>
Co-authored-by: Zijie Xia <zijie.xia@radixark.ai>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Qiaolin Yu <liin1211@outlook.com>
2026-08-19 16:31:43 +08:00
Liangsheng Yin ccbe380028 [CI] Trim the base-c 4-gpu-h100 stage from 5 shards to 4 (#35407) 2026-08-19 00:48:07 -07:00
MickandClaude Opus 5 c0c87e0547 VLM: feed the packed qkv projection output to vision backends uncopied (#35336)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-19 14:39:52 +08:00
Alison ShaoandXinyuan Tong 3391ab3712 [Constrained] Support MistralCommon tokenizers in the XGrammar backend (#35215)
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
2026-08-19 12:34:48 +08:00
Shuwen Wang a72d8d29d3 Fix HiCache PP sync test fixture (#35446) 2026-08-19 12:25:17 +08:00
Jimmy ShongandClaude Fable 5 c863760ae1 [Fix] DCP: advertise the logical KV-event block size (#35298)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 20:55:11 -07:00
Muqi LiandCodex 88f6074392 feat(api): add sglext_spec (#33518)
Signed-off-by: Muqi Li <muqi1029@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
2026-08-19 11:54:18 +08:00
luoroger37 3b065a56b0 [HiCache] Batch PP write and load completion sync (#33473) 2026-08-19 11:27:27 +08:00
HuangJi ee1f2e8dfd [diffusion][Minimax H3]support subblock sparse attention on SM90 (#34680) 2026-08-19 10:31:45 +08:00
Zhiqiang Xie 977412ae61 [HiCache] Buffer-only mode for HiCache host memory layer (#34798) 2026-08-18 19:21:24 -07:00
Mick 4cef72faee [diffusion] refactor: reuse srt qwen vision and text modules (#35006) 2026-08-19 10:12:44 +08:00
Chunyuan WU 58c5bee3ac Fix DP attention on CPU (#12961) 2026-08-19 09:56:21 +08:00
Mick ef490853bb quant: extract shared checkpoint quant metadata resolver (#35172) 2026-08-19 08:26:41 +08:00
MickandClaude Opus 5 77fc5c128e [perf] overlap page preprocessing, pack the vit, enable prefill CUDA graph for paddle-ocr (#35318)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-19 08:20:55 +08:00
c14312a664 [Spec] DFlash2: local convolution + candidate selector (#35371)
Co-authored-by: Jian Chen <jianchen0311@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-08-18 17:07:28 -07:00
Baizhou Zhang cfc6dfb364 Apply latest DeepEP branch (#34923) 2026-08-18 16:24:50 -07:00
cctryandcctry 37c09ff3d8 [Memory] Borrow CUDA graph pool storage for EAGLE sampling (#35375)
Co-authored-by: cctry <cctry@fb.com>
2026-08-18 16:19:41 -07:00
Liangsheng Yin 7ebaa98f81 [Fix] Assert the page-aligned SWA evict floor on both PD decode prealloc paths (#35396) 2026-08-18 15:48:55 -07:00
Liangsheng Yin 79dfef390b [Spec] Page-align the DFLASH decode KV reservation (#35265) 2026-08-18 13:36:04 -07:00
Yuwei AnandClaude Opus 5 955704544c [Fix] Skip padded state slots in the chunked GDN kernel (#33431)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 13:07:19 -07:00
307a90f6d3 Stop losing Kimi-K3 tool calls to reasoning, constraint conflicts, and truncation (#34881)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
2026-08-18 12:42:56 -07:00
Wei YangandWei Yang 8bb106cee9 Fix NIXL cleaner grouping for hybrid cache keys (#35130)
Co-authored-by: Wei Yang <yawei@microsoft.com>
2026-08-18 13:55:51 -05:00