Commit Graph
6055 Commits
Author SHA1 Message Date
Liangsheng Yin d20d010574 [Fix] deepseek-v4.1: Exchange late-layer DP row counts before the forward (#44948) 2026-10-17 21:46:46 -07:00
Liangsheng Yin 6fe630ae14 [CI] deepseek-v4: Set the KV pool in the BCG replay contract test (#44946) 2026-10-17 20:59:03 -07:00
140c57d5bf [Perf] diffusion: classify residual_gate_add operands once per call (#44478)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 11:51:52 +08:00
Liangsheng YinandVincent Zhong ff73d63121 [Perf] deepseek-v4.1: Run encoder SWA bounded replay in the breakable prefill graph (#44930)
Co-authored-by: Vincent Zhong <207368749+vincentzed@users.noreply.github.com>
2026-10-17 20:51:18 -07:00
Richard Wang 01f3cba70e [Fix] dLLM: Size decode graphs to the default max running requests (#40941) 2026-10-17 20:22:39 -07:00
14ad64c411 [Perf] custom_op: call register_custom_op ops directly in plain eager execution (#44158)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 10:10:10 +08:00
8d02b69c11 [Perf] diffusion: Replay Triton launches without the runner's per-launch hook metadata (#44471)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 09:58:54 +08:00
949e1a2080 [Perf] diffusion: Cache the fused linear-GELU verdict per linear (#44643)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 09:55:29 +08:00
f53bd436c1 [Perf] distributed: cache the eager launch of custom all-reduce v2 per size (#44642)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 09:54:58 +08:00
Yuwei An b234b5c32e [Refactor] TCPCG: Consolidate attention eager regions and isolate Inkling policy (4/9) (#42288) 2026-10-17 16:06:24 -07:00
Liangsheng Yin 563cd60345 [Fix] [AMD] deepseek-v4.1: Reject attention DP with encoder SWA bounded replay on ROCm (#44934) 2026-10-17 15:47:41 -07:00
05c1539098 [Feat] unified_cache: Support hybrid SWA + Mamba models in the Mooncake direct linker (#41343)
Co-authored-by: Zhangheng <hzh0425@apache.org>
Co-authored-by: Teng Ma <sima.mt@alibaba-inc.com>
2026-10-17 13:24:08 -07:00
YAMYandClaude Opus 5.5 3b34d9f8b6 [Perf] [NVIDIA] qsa: Keep sparse decode scratch in FP8 for FP8 KV pools (#44114)
With an FP8 KV pool on SM10x, keep the packed sparse-decode K/V scratch in FP8 instead of dequantizing to BF16 (BF16-q + FP8-KV TRTLLM-gen decode). Tests cover FP8 pool -> FP8 scratch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 11:50:38 -07:00
MickandMick Qian e01dfd3cbc [Refactor] multimodal: Replace padding strategies with helpers (#44845)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-10-17 20:43:02 +08:00
Shuwen Wang 86e67f7d93 [Fix] unified_cache: Prevent misaligned Mamba checkpoints after speculative truncation (#43387) 2026-10-17 20:20:49 +08:00
SuperSong 025655068c [Fix] unified-memory: Avoid gather scratch during Mamba and Conv compaction (#44097) 2026-10-17 04:56:41 -07:00
SuperSong ee30dc4d40 [Fix] unified-memory: Bound joint reclaim and preserve cached prefixes (#39294)
Bound joint FULL/SWA/Mamba reclaim so feasible allocations recover capacity without evicting unnecessary cached prefixes, and impossible demands preserve the cache. Use exact unsharded paged-extend demand and update allocation regression fixtures.
2026-10-17 04:46:48 -07:00
Shuwen Wang ae28306a74 [Fix] cuda_graph: Wait for in-flight layer-wise cache loads before prefill graph replay (#44220) 2026-10-17 19:13:57 +08:00
Shuwen WangandZhangheng 5fcb07da34 [Fix] hicache: Key MLA Mamba state per TP rank in the HF3FS backend (#43817)
Co-authored-by: Zhangheng <hzh0425@apache.org>
2026-10-17 11:08:17 +00:00
56a20d7e34 [Perf] kernels: Enter a JIT kernel's device guard only when the device differs (#44644)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 16:43:27 +08:00
f75232cb42 [Fix] hicache: Order write-back host copies after the forward that produced the KV (#44800)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: xiezhq-hermann <xiezhq-hermann@users.noreply.github.com>
2026-10-17 01:31:40 -07:00
0d13efb2b8 [Feat] rust-server: Serve /v1/tokenize and /v1/detokenize from the Rust frontend (#44767)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: shodoco <shodoco@users.noreply.github.com>
2026-10-17 01:30:50 -07:00
a2953d7b7f [Fix] deepseek-v4.1: Serve prompt-logprob requests under decoder SWA bounded replay (#44850)
Co-authored-by: Pedram Ghazi <pedram.ghazi@amd.com>
Co-authored-by: David Orman <ormandj@corenode.com>
2026-10-17 01:25:33 -07:00
Liangsheng Yin ef26f6ad49 [Fix] deepseek-v4.1: Keep the cached KV of folded encoder replay rows (#44857) 2026-10-17 00:56:18 -07:00
01dc08e23f [Refactor] dsv41: Shard Engram tables on hash-column boundaries (#42787)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-10-17 00:15:49 -07:00
Mohammad Miadh AngkadandMohammad Angkad 2d8a05e719 [Test] attention: Fix replicated attention test missing full_attention_interval (#44602)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-10-16 22:28:37 -07:00
Mohammad Miadh AngkadandMohammad Angkad 4fc86f73c2 [CI] glm5.2: Keep EOS in HiSparse needle test output (#44597)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-10-17 12:20:51 +08:00
Shuwen Wang 6a39ddf1f2 [Fix] unified_cache: Resize test allocator before allocating backed pages (#44843) 2026-10-16 20:21:49 -07:00
3603824541 [Perf] DSv4.1: Enable decoder bounded replay by default (#44403)
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
2026-10-16 19:52:02 -07:00
McZyWu 2d3dd3168d [Fix] [NPU] attention: Use FIA TND for MLA prefix attention and add Gemma norm fallback (#40131) 2026-10-17 10:12:41 +08:00
Khoa Pham ff979e0816 [Fix] megamoe: Respect request-pool limits in decode validation (#44816) 2026-10-16 18:56:38 -07:00
5cbe2371a5 [Feat] rust-server: Report Python's launch-time /server_info fields from the Rust frontend (#44764)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: shodoco <shodoco@users.noreply.github.com>
2026-10-16 18:14:50 -07:00
Shuwen Wang 33b5d26fa6 [CI] lora: Use the canonical Llama model ID in eviction tests (#44771) 2026-10-16 18:13:37 -07:00
b62a48ee15 [Feat] sglang-server: Serve /flush_cache from the Rust frontend (#44766)
Co-authored-by: fei-xx <fei-xx@users.noreply.github.com>
Co-authored-by: shodoco <shodoco@users.noreply.github.com>
2026-10-16 17:24:01 -07:00
d8dd49d21d [Fix] srt: Apply --gc-threshold in every server process and freeze every tokenizer worker (#44785)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: joshuadeng <joshuadeng@users.noreply.github.com>
2026-10-16 16:55:54 -07:00
d230f2c35b [Feat] profiler: Add the forward pass id to detailed step span labels (#44775)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: charlotte12l <charlotte12l@users.noreply.github.com>
2026-10-16 16:51:20 -07:00
36cf028346 [Perf] unified-memory: Prewarm the compaction move at startup and copy whole page envelopes by page id (#44592)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: ZYHowell <ZYHowell@users.noreply.github.com>
2026-10-16 16:47:50 -07:00
Khoa PhamandClaude Fable 5.1 aaf5f397a5 [Feat] dsv4.1: Support encoder SWA bounded replay with CUDA DP attention (#43813)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-16 15:38:33 -07:00
Yuwei An 8e3f11b219 [Perf] DSv4.1: Skip discarded KV rows during window commit (#44783) 2026-10-16 15:05:45 -07:00
a46fe83c70 [CI] tests: Fork the gloo exchange test's ranks from a preloaded forkserver (#44679)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: fei-xx <fei-xx@users.noreply.github.com>
2026-10-16 14:53:21 -07:00
AaravandAarav d5d87899f9 [Fix] DSA: skip-topk pipeline handoff for owed outputs (#43832)
Co-authored-by: Aarav <211792885+AaravMehta-07@users.noreply.github.com>
2026-10-16 14:35:23 -07:00
2948662026 [Perf] dp-attention: Exchange the scheduler's MLP sync rows with gather + broadcast on gloo (#44622)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: hanming-lu <hanming-lu@users.noreply.github.com>
2026-10-16 13:53:41 -07:00
Chunan Zeng 07950be745 [Perf] [NVIDIA] custom-all-reduce: Add 2shot_push all-reduce and retune SM100 TP4/TP8 (#42811) 2026-10-16 13:43:18 -07:00
Mohammad Miadh AngkadandMohammad Angkad 21791e7023 [CI] [NVIDIA] DSv4.1-Flash: Split GB300 accuracy test into one file per config (#44696)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-10-16 13:40:03 -07:00
metamergebotandmetamergebot a31a8c2bbd [CI] tests: Let test_hicache_storage size its host cache to the runner (#44729)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
2026-10-16 13:38:39 -07:00
Liangsheng Yin a18a908066 [Test] mem_cache: Free SWA peers below the eviction floor in the SWA repair test (#44777) 2026-10-16 12:41:39 -07:00
ormandjandhnyls2002 80c54e6ac5 [Fix] [AMD] hicache: Serve encoder-replay prefix hits with a paged DSpark draft SWA (#42716)
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-10-16 10:45:16 -07:00
553068af25 [Fix] rust-server: Honour cache_salt and require_reasoning on /generate (#44533)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: shodoco <shodoco@users.noreply.github.com>
2026-10-16 10:38:09 -07:00
453b251dbd [Fix] hicache: Bound the storage prefetch timeout from the first blocked admission poll (#44614)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: xiezhq-hermann <xiezhq-hermann@users.noreply.github.com>
2026-10-16 10:37:11 -07:00
metamergebotandjiayisuse 989ccf4863 [Feat] disaggregation: Let a KV transfer backend skip the checksum for an incomparable peer (#44605)
Co-authored-by: jiayisuse <jiayisuse@users.noreply.github.com>
2026-10-16 10:36:46 -07:00