Commit Graph
6055 Commits
Author SHA1 Message Date
bea282cede [DeepSeek-V4] Fuse UE8M0 scale rounding into FP8 group quantization (#26766)
Co-authored-by: liqichao <liqichao@baidu.com>
Co-authored-by: yhyang201 <yhyang201@gmail.com>
2026-06-18 14:41:56 -07:00
Zhaoyi Li 27a374eaef [AMD][CI] Use non-gated Qwen3-8B for MI35x disaggregation tests (#28678) 2026-06-18 14:29:35 -07:00
Liangsheng Yin 8f6d9ef9a5 [misc] Drop redundant req_pool_indices_cpu guards; fold hisparse into GLM-5.1 e2e (#28607) 2026-06-18 12:51:23 -07:00
Zhanghengandispobock 867707f1f2 [UnifiedTree]: move some kl test from base to extra stage (#28627)
Co-authored-by: ispobock <ispobaoke@gmail.com>
2026-06-18 01:39:46 -07:00
Vladislav NosivskoyandZhangheng b7ae7149e8 [HiCache] Fix SWA L3 cache miss due to a prefetch/hit len mismatch (#27291)
Co-authored-by: Zhangheng <hzh0425@apache.org>
2026-06-18 14:55:00 +08:00
DovLin 0188c54fbe [FA3] Add unit test for only_qv (NoPE) KV decode path (#28595)
Signed-off-by: Shijin Zhang <dovis.zhang02@gmail.com>
2026-06-17 23:41:44 -07:00
Michael 5d1949152d [AMD] ci: add extra-a 1-gpu-large tier (fp8kv-triton, streaming-session, spec-standalone) (#28458) 2026-06-17 23:31:32 -07:00
Michaelandmichaelzhang-ai 0e5a66dca4 [AMD] Register 3 JIT kernel unit tests for AMD CI (#27837)
Co-authored-by: michaelzhang-ai <michaelzhang@example.com>
2026-06-17 23:28:56 -07:00
cctry 7976928c57 Abort during chunked prefill + PD peer-liveness abort (#28086) 2026-06-17 23:13:28 -07:00
Yuan Luoandluoyuan.luo 3340f4e3da [GDN][KDA][mem_cache] int8 checkpoint pool for the linear-attn prefix cache (#28185)
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
2026-06-17 20:41:46 -07:00
Cheng Wan 53318911ca Add get_parallel(): a structured accessor for parallel-topology state (#28567) 2026-06-17 20:23:43 -07:00
Dongjun Na 462c01ea6b [observability] add Ray metric backend wrappers (#26252)
Signed-off-by: Dongjun Na <kmu5544616@gmail.com>
2026-06-17 19:41:53 -07:00
luoroger37 c208a96a7d Fix ScheduleBatch req pool CPU metadata (#28514) 2026-06-17 19:25:47 -07:00
Liangsheng Yin 9888b7b42b [misc] Trim dead code in trtllm_mha page-table backend; reuse eager page-table buffer (#28578) 2026-06-17 18:36:22 -07:00
zijiexia 74e2e48c82 Introduce CpuDeviceMixin and CpuSRTPlatform (#26385) 2026-06-17 17:41:17 -07:00
cctry cfa4aa988f Revert "revert the head_dim assignment from PR 23862" (#28583) 2026-06-17 17:11:05 -07:00
Liangsheng Yin a663500ea9 [Test] Fold EAGLE return_hidden_states regression into spec triton suite (#28577) 2026-06-17 16:42:02 -07:00
5ea0d1d093 add dflash gemma4 support (#27471)
Co-authored-by: kpham-sgl <khoa.pham@radixark.ai>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 16:39:28 -07:00
Jonny Kongandjonnykong cd60c4edd0 [attn backend] Make seq_lens_cpu optional in trtllm_mha backend (#28106)
Co-authored-by: jonnykong <jonnykong@fb.com>
2026-06-17 16:24:19 -07:00
Khoa PhamandClaude Opus 4.7 e4fd613def [Spec] Fix return_hidden_states under spec V2 (issue #26163) (#28496)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-17 16:21:36 -07:00
cctry 5d6b35eabb revert the head_dim assignment from PR 23862 (#28571) 2026-06-17 15:46:28 -07:00
Lijuan Tang 0a28a929dc [MLX] Add Metal profiling hooks to server profiler (#28122) 2026-06-17 13:06:20 -07:00
Thomas WangandBingxu Chen f5b041622b [AMD] Fix deepseek-v4 mtp accept length issue (#28520)
Co-authored-by: Bingxu Chen <bingxche@amd.com>
2026-06-17 11:13:55 -07:00
Ziang Li 3fb65ebabd [RL] Fix FlashInfer TRTLLM MXFP8 dense weight layout (#28459) 2026-06-17 10:33:35 +00:00
Ryan Zzzandzhujunyu 8fd1694dd2 Deepseek v4: support mixed dtype compression states (#27277)
Co-authored-by: zhujunyu <zhujunyu.666@bytedance.com>
2026-06-17 00:56:41 -07:00
7256ee9871 [AMD] Update test_aiter_allgather_amd.py data types alignment between benchmark aiter and custom all-reduce kernel (#27815)
Co-authored-by: YC Yen-Ching Tseng <yctseng@amd.com>
Co-authored-by: Hubert Lu <55214931+hubertlu-tw@users.noreply.github.com>
2026-06-17 00:48:15 -07:00
Michael a2aa51c818 [AMD] register 4 2-gpu tests to stage-b-test-2-gpu-large-amd (#28344) 2026-06-16 21:58:34 -07:00
Yanbin Jiang 093908d4c0 [LoRA] Fix chunked SGMV (csgmv) CUDA graph segment replay (#28371) 2026-06-16 19:19:07 -07:00
37ef295c78 [AMD] Feat/dp moe reduce scatter (#28216)
Co-authored-by: wunhuang <wunhuang@amd.com>
Co-authored-by: Wang, FangYuan <39615225+At1a8@users.noreply.github.com>
2026-06-16 18:12:39 -07:00
jasonjk-park d86a7e7018 Custom spec algorithm can handle server args (#28162) 2026-06-16 17:13:52 -07:00
Qiaolin Yu 2ad00faae1 [ci] add kimi nvfp4 nightly tests (#28467) 2026-06-16 16:49:51 -07:00
huangtingwei 9b4432fe18 [HiCache]Asymmetric pool support direct backend (#28446) 2026-06-16 13:17:57 -07:00
13537f8e20 Unskip Marlin NVFP4 tests (#27589)
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
Co-authored-by: shaunkotek <shaunkotek@users.noreply.github.com>
2026-06-16 11:58:22 -07:00
feliang-gitandxutizhou 92b42c8d8a LPLB: linear-programming load balancer for MoE expert parallelism (#24515)
Co-authored-by: xutizhou <xutingz@nvidia.com>
2026-06-16 10:19:42 -07:00
265202cda2 fix(openai): validate assistant tool call arguments before chat template (#28035)
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
2026-06-16 12:21:23 +00:00
Yinghai LuandLianmin Zheng fcca4611fa [CAR] Let custom allreduce support VMM based allocation (#27593)
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
2026-06-16 04:49:05 -07:00
Brayden ZhongandBrayden Zhong 25e696aa8d Fix Stage B CUDA CI (#28367)
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
2026-06-16 02:39:17 -07:00
Wang, FangYuanandThomas Wang a362ba9da3 [AMD] Feat: Add prefill context parallel support for deepseek v4 unified kv attention (#27928)
Co-authored-by: Thomas Wang <thomawan@amd.com>
2026-06-16 02:00:51 -07:00
jacky.cheng 149fabcca7 [AMD] Fuse sigmoid + mul into single Triton kernel for shared expert gating (#27636) 2026-06-16 01:16:58 -07:00
Bingxu Chen 0fc2bc4a8b [AMD] Test DeepSeek V4 FlashMLA backend variants nightly (#28290) 2026-06-16 01:03:26 -07:00
Baizhou Zhang 77f327cb6e [2/n] [CP] Add context parallel strategy abstractions (#27313) 2026-06-16 00:20:04 -07:00
Michael 72d962be88 [AMD] Fix jit-kernel-unit-test-amd: activation.cuh ROCm build + per_token CUDA-only (R165) (#27947) 2026-06-15 23:44:26 -07:00
Qiaolin Yuandshuwenn e068355831 [spec decoding] supports step 0 in adaptive spec decoding (updating draft kv cache without draft decoding) (#23994)
Co-authored-by: shuwenn <2508695655@qq.com>
2026-06-15 22:21:26 -07:00
Shu Wang 32685874f3 Reenable MNNVL backend for FlashInfer allreduce fusion (#23402) 2026-06-15 20:19:15 -07:00
zhangxiaolei 063ab89ac1 DeepSeek-V4 Online Compress support MTP (#26471) 2026-06-15 19:56:07 -07:00
huangtingwei b5bcd76a41 [HiCache & JIT Kernel] Refactoring HiCache Write-Back Kernel (#21631) 2026-06-15 19:44:27 -07:00
Michael c6d9d73fd6 [Spec][test] fix(kv_canary): assert draft-extend-v2 oracle tokens in token_oracle test (#28325) 2026-06-15 19:14:09 -07:00
Kangyan-ZhouandClaude Fable 5 cad43d3212 [CI] Reclaim leaked /dev/shm segments on server startup (#28089)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-06-15 16:14:08 -07:00
Jia Guo 4ed698a491 fix(fa3): no NaN embeddings with fa_skip_kv_cache under piecewise CUDA graph (#27343) 2026-06-15 13:46:22 -07:00
33719cfb31 [PD] Optimize SWA allocation (#28085)
Co-authored-by: cctry <cctry@fb.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
2026-06-15 11:01:55 -07:00