Commit Graph
6055 Commits
Author SHA1 Message Date
Hanming Lu 526af15845 [Metrics] Discount queued prefill load by recent cache hits when waiting-queue matching is off (#35248) 2026-08-18 11:27:21 -07:00
Xingyu Liuandxingyuliu 7dcaf11987 [Fix] Select custom all-reduce v2 by topology capability (#35061)
Co-authored-by: xingyuliu <xingyuliu@fb.com>
2026-08-18 10:30:10 -07:00
Ke Bao 480033def0 Refactor kv cache event mixin into a recorder (#35164) 2026-08-19 01:20:17 +08:00
jain-ria 83d7d45330 fix: preserve output logprobs without input logprobs (#34627)
Signed-off-by: jain-ria <riajain@NVIDIA.com>
2026-08-18 12:07:13 -05:00
Shangming CaiandClaude Opus 4.8 97dedd1ce9 [PD] Deferred decode-side KV release for aborts mid-transfer (#35049)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-08-18 23:56:33 +08:00
pllimax 499e90a125 [NPU CI] Reorganize test output/log directory structure with workflow context (#33685) 2026-08-18 23:46:00 +08:00
paulzhang-tm 0065fbfae1 [Scheduler] Cap prefill-delayer queue target by admission capacity (#35191) 2026-08-18 21:40:13 +08:00
Xiaoyu ZhangandClaude Opus 5 ae6945e112 [kernels] Reorganize ops/diffusion by operator domain behind a lazy facade (#35114)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 20:37:43 +08:00
HeYaoandHeYao 7605529bdf Add deepseek_v4_flash_w8a8_8p_in32k_out1k_50ms (#35162)
Co-authored-by: HeYao <heyao@example.com>
2026-08-18 19:41:25 +08:00
karverma-amd 24d625698d [AMD] feat(moe): fold padded-topk_ids fill into fused shared-experts append+remap (#31370) 2026-08-18 02:32:57 -07:00
vijay-kodamalla 880ab72f34 test: extend NVFP4 Marlin tests to SM120 (#34327) 2026-08-18 16:24:00 +08:00
mohbasit fc0b95e7ba Profiling Enhancements [2/3]: detailed execution step annotations (#24911) 2026-08-18 01:09:07 -07:00
Lianmin Zheng a779a2a2a5 [Chore] Move version tag helper to release scripts (#35196) 2026-08-18 00:11:12 -07:00
Jiajun Li e6df23f3c2 refactor: rename chat response token IDs (#35225) 2026-08-17 23:07:43 -07:00
Shuwen Wang 0077f84d37 [mem_cache][8/N] refactor: move MambaPoolHost to pool_host.mamba (#31180) 2026-08-18 05:22:45 +00:00
Colin Z ea27e3ddab [AMD] Fix Quark Shared Experts Fusion Gate after load-time-override Removal (#35200) 2026-08-17 22:10:01 -07:00
8ea5229d42 [AMD] Add the Kimi-K3 MI35x perf benchmarks in nightly (#34985)
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Michael <michaelzhang-ai@users.noreply.github.com>
2026-08-17 21:23:51 -07:00
Ke Bao d528192bf9 Skip inkling sheared bias under batch invariance (#35161) 2026-08-18 12:22:58 +08:00
amd-danli103andThomas Wang d01812d89e [AMD] Optimize KIMI-K3 with Triton MLA decode kernel by tuning the stage-1 geometry for gfx950 (#34580)
Co-authored-by: Thomas Wang <thomawan@amd.com>
2026-08-17 21:16:14 -07:00
Khoa PhamandClaude Opus 5 f44a130c5e [DCP] Drop the prefill index-selection syncs by taking each rank's rows by stride (#35084)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 19:55:51 -07:00
Siyuan Chen fcdaaf8a5d [Feature] Optimize TP LMHead with All-to-All (#32313) 2026-08-17 19:55:27 -07:00
Ma Mingfei 7f51e6bba0 [CPU] Explicitly import sgl_kernel in CPU kernel tests (#35119) 2026-08-18 10:20:28 +08:00
Baizhou Zhang c2c1c4d3eb [CI] Move DSA PD+MTP+CP Layersplit test to basic B300 test suite (#35220) 2026-08-17 17:44:31 -07:00
Cheng Wan b3c8f0d923 config: one control-plane log for the process (#35028) 2026-08-17 16:19:00 -07:00
Cheng Wan c70c7d72a8 config: the readback and the resolving view say what they are (#35027) 2026-08-17 16:18:19 -07:00
Cheng Wan cba3c5d5ac config: the per-instance families read the bags (#35026) 2026-08-17 16:17:53 -07:00
Cheng Wan a97bc8db32 config: the DP/EP topology reads come from the parallel bag (#35025) 2026-08-17 16:17:26 -07:00
Cheng Wan d2bc697396 spec: size the speculative buffers from the bags, not the startup record (#35024) 2026-08-17 16:16:56 -07:00
Cheng Wan 3d7ec00179 config: publish before a process reads configuration (#35023) 2026-08-17 16:16:20 -07:00
Cheng Wan 2b278b4ac4 config: retire the multi-engine accommodation in the runtime context (#35022) 2026-08-17 16:15:33 -07:00
Baizhou Zhang bc312d185d Clean deprecated DeepSeek V4 Environs (#34926) 2026-08-17 16:07:00 -07:00
Lennox Fu b42abbb1ba [metrics] Fix prefill FLOPs estimate to count prefix and per-request causal pairs (#34316) 2026-08-17 15:18:54 -07:00
816ea65058 [AMD] Add Kimi-K3 8-GPU MI35x nightly accuracy CI (#32568)
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Michael <michaelzhang-ai@users.noreply.github.com>
2026-08-17 14:48:29 -07:00
cctryandcctry 2e7c85da68 [PD] Preserve decode KV across retraction in HiCache (#34801)
Co-authored-by: cctry <cctry@fb.com>
2026-08-17 08:49:11 -07:00
Lianmin Zheng af743371cc Clean up environ.py: remove dead env vars, unify deprecation handling, move examples to a unit test (#35060) 2026-08-17 06:53:34 -07:00
Ke Bao 6e8a4abb57 Add bit-exact class for MTP (#35143) 2026-08-17 19:51:39 +08:00
Mohammad Miadh Angkad 82995a001b Stabilize GB300 nightly tests (#35044) 2026-08-17 03:30:16 -07:00
744740dbea [XPU] upgrade sglang xpu backend to PyTorch 2.13 (#31751)
Co-authored-by: MingxuZh <109504044+MingxuZh@users.noreply.github.com>
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
2026-08-17 18:29:15 +08:00
8cc112d486 [DSA] Skip indexer KV cache for skip-topk layers (#30531)
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
Co-authored-by: mmangkad <mohammad.angkad@radixark.ai>
2026-08-17 02:02:23 -07:00
YAMY 7c423cfd41 [PD] Avoid unused PREBUILT prompt tensor transfer (#35070) 2026-08-17 16:48:12 +08:00
Liangsheng Yin 711bdacb82 [Spec] Resolve shared-read ends from the backend declaration alone (#35059) 2026-08-17 01:35:29 -07:00
b83d507cd7 [NPU] Support DeepSeek-V4 DSpark and refactor DSV4 cache management (#33676)
Co-authored-by: JiaruiChang5268 <jc5268@columbia.edu>
Co-authored-by: Kelon <kelonlu@163.com>
Co-authored-by: unknown <z8ruev42yk@gmail.com>
Co-authored-by: Talantan1102 <545811257@qq.com>
Co-authored-by: Talantan1102 <44429302+Talantan1102@users.noreply.github.com>
2026-08-17 16:27:44 +08:00
Mohammad Miadh AngkadandMohammad Angkad 5769b6d637 [JIT Kernel] Migrate causal_conv1d_fwd and causal_conv1d_update from AOT to JIT (#35031)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-08-17 15:22:37 +08:00
Mohammad Miadh Angkad 721e359ca7 Suppress expected FlashInfer TRT-LLM workspace warnings (#34921) 2026-08-17 15:16:16 +08:00
vikram singh shekhawatandClaude Sonnet 4.5 f7a404e9c3 Fix rope config compatibility and VL/transformers-fallback weight loading (#31575)
Co-authored-by: Claude Sonnet 4.5 (1M context) <noreply@anthropic.com>
2026-08-17 14:53:59 +08:00
ziang663andZhangheng 43226af812 fix(hicache): limit load-back pending to write-back (#34519)
Co-authored-by: Zhangheng <hzh0425@apache.org>
2026-08-17 14:23:54 +08:00
Wang, FangYuan eb61cb2823 [AMD] Support prefill context parallel two batch overlap for DeepSeek V4 (#33480) 2026-08-16 22:40:29 -07:00
Kaixi f3225bceb3 [DSV4] Emit TMA-aligned UE8M0 scales for FP8 einsum (#34277) 2026-08-16 22:06:41 -07:00
Khoa PhamandClaude Opus 5 0fb040cbeb [DCP]Localize HiCache DCP indices once per transfer, not per layer (#34889)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 21:54:51 -07:00
Enrique Shockwave 92b1d382c7 [Fix] Correct dense FP8 Marlin bias ordering (#35020) 2026-08-17 03:43:36 +00:00