Commit Graph
6225 Commits
Author SHA1 Message Date
786cdfc040 [Fix] nccl: Disable graph buffer registration whenever a captured graph can hold a collective (#43216)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Jimmy Shong <69131491+Jiminator@users.noreply.github.com>
2026-10-22 13:56:19 -07:00
Yanbin Jiang 13999673ef [Feat] lora: Add plan-driven dense and MoE LoRA engines (#45892) 2026-10-22 13:40:56 -07:00
Shunkangz 154e29c1ec [Perf] CP: QSA Context Parallel Optimization (#43792) 2026-10-22 13:33:04 -07:00
aae2145eb2 [Feat] radix-tree: Report the last visited node and a shorter FULL device match in match results (#45703)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: Jialin <Jialin@users.noreply.github.com>
2026-10-22 13:11:33 -07:00
Liangsheng Yin 4f96807ea8 [CI] radix-cache: Build unified radix cache test states through production paths (#45334) 2026-10-22 13:04:23 -07:00
YAMY d67d1ae02c [Fix] logits: Preserve lm_head precision with shared buffers (#45855) 2026-10-22 12:46:27 -07:00
Zhangheng 913e91ed51 [Fix] CI: add missing entry point to Muse Spark router test (#45845) 2026-10-23 00:45:41 +08:00
Aarne Talman c80285ca9a [Fix] [AMD] deterministic: Refuse RCCL all-reduce in deterministic mode above TP2 (#43809)
Signed-off-by: Aarne Talman <aarne.talman@amd.com>
2026-10-22 09:31:25 -07:00
ShunkangzandClaude Opus 5.5 cc811ff0c3 [Feat] KVShard: Add attention and indexer support for page-sharded DSA caches (#44280)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 00:16:35 +08:00
Shuwen Wang 8cd3c494a3 [Fix] mem_cache: Skip linker L3 lookup on PD decode (#45538) 2026-10-22 22:36:32 +08:00
trebladev 698fa949ec [Perf] diffusion: fuse FLUX3 SwiGLU with rowwise FP8 quantization (#42758) 2026-10-22 21:17:56 +08:00
gjsheu a9902f47e3 [Fix] [NPU] qwen: adapt pcp-v2 (#41564) 2026-10-22 20:54:11 +08:00
Song Bianandkernel-design-agents 851419717a [Perf] [KDA] muse_spark: Run the one-token router GEMV on CuTe DSL (#45508)
Co-authored-by: kernel-design-agents <kernel-design-agents@users.noreply.github.com>
2026-10-22 20:49:29 +08:00
MonishverandXiaoyu Zhang 178febef5d [Perf] qknorm: run head_dim-128 rows two per warp with 16B lanes, bit-identical (#45214)
Signed-off-by: Monishver Chandrasekaran <monishverchandrasekaran@gmail.com>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
2026-10-22 20:34:16 +08:00
81841d6bdc [Fix] hicache: Fail fast on a null pointer from the Mooncake host allocator (#40565)
Co-authored-by: huangtingwei <141888744+huangtingwei9988@users.noreply.github.com>
Co-authored-by: Teng Ma <sima.mt@alibaba-inc.com>
2026-10-22 20:12:25 +08:00
Jzz1943 8a6770d566 [Fix] NIXL: Preserve explicit UCX network device selection (#45722) 2026-10-22 20:07:27 +08:00
ashwini rathi 5c2cb60790 [Fix] [Intel] diffusion: Restore copy offload for FLUX.2-dev nightly (#45725) 2026-10-22 16:24:41 +08:00
Michael 0a87f163dc [Test] [AMD] deepseek-v4.1: Fix the HIP breakable-graph unit tests for the default prefill context limit and the DSpark draft ring (#45507) 2026-10-22 01:14:14 -07:00
kangwangamd 1c967f5043 [Fix] [AMD] test: Recalibrate MiniMax-M2.7 GSM8K threshold 0.93->0.91 for MI300 runners (#34241) 2026-10-22 01:08:54 -07:00
jacky.cheng b5137693a5 [Fix] [AMD] diffusion: Enable --quantization fp8 on gfx1250 (torch.compile lowering + per-channel weight scales) (#45304) 2026-10-22 01:05:36 -07:00
WenhaoZhangandMick Qian 24e45961c6 [Perf] diffusion: fuse QKV norm + RoPE, cached RoPE tables for FLUX 3 video VAE (#41894)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-10-22 15:44:54 +08:00
f68e82e7bc [Fix] epd: Accept tokens-in requests on scheduler-side transfer backends (#41376)
Signed-off-by: Krishnan Prashanth <kprashanth@nvidia.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 14:45:38 +08:00
Shangming Cai 9569a36561 [Feat] scheduler: Gate --prefill-decode-interval on running batch size (#44001) 2026-10-22 14:30:12 +08:00
Richard Wang ce66b3ab59 [Feat] systemone: serve Kev pointer head decision models (#45413) 2026-10-22 14:16:52 +08:00
Shangming Cai a6a6beffac [Refactor] disaggregation: Share the flag all-reduce and host-destination predicate in the decode transfer queue (#45641) 2026-10-22 13:42:55 +08:00
ApeiriaandRain Jiang 8d7df07a56 [Feat] rust-server: Add HiCache storage management endpoints (#44726)
Signed-off-by: 0130w <1157374677@qq.com>
Co-authored-by: Rain Jiang <96632942+rainj-me@users.noreply.github.com>
2026-10-22 04:01:35 +00:00
Khoa Pham f1e26dc658 [Perf] logits: Write lm_head logits straight into the fp32 logits buffer (#45631) 2026-10-21 20:32:16 -07:00
Shuwen Wang 6929afb528 [Fix] hisparse: Keep PD destination host pages until the decode KV release (#45108) 2026-10-22 11:19:21 +08:00
8dc7f98005 [Perf] pd: Send prefill KV chunks without a host sync on the scheduler thread (#45492)
Co-authored-by: alphabetc1 <2508695655@qq.com>
Co-authored-by: Shuwen Wang <47200617+alphabetc1@users.noreply.github.com>
2026-10-21 20:11:16 -07:00
Tamir BaydasovandClaude Opus 5.5 b057d5b3f1 [Fix] [NPU] mla-pool: DCP allocator page and index-K bound, short-pool warning (#42976)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 05:24:14 +03:00
em-dreyerandClaude Opus 5.5 3d508d614f [CI] [NVIDIA] gb300: Run DSV4.1-Flash tests on 2 GPUs with engram host tables (#45294)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 18:53:24 -07:00
Liangsheng Yin d61f618276 [Perf] spec: Let DSPARK prefill publish read-done by placing its draft KV inject before the target forward (#45682) 2026-10-21 18:40:44 -07:00
Khoa PhamandClaude Opus 5.5 aa10927271 [Perf] dsv41: Mega-mHC and the wo_a einsum write MXFP8 for wqkv_a and wo_b (#45632)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-21 18:27:21 -07:00
Baizhou Zhang 59149553b9 [Refactor] attention: centralize linear and draft TP group selection (#45684) 2026-10-21 18:25:40 -07:00
Jimmy Shong 9301ca37a0 [Fix] modelopt: Correct NVFP4 TRT-LLM clamps for untied gate/up scales (#45386) 2026-10-21 17:37:07 -07:00
eacd0a07d7 [Feat] bench: support --tokenize-prompt for generated-shared-prefix (#40900)
Co-authored-by: Yihao Wang <42559837+AgainstEntropy@users.noreply.github.com>
Co-authored-by: chaoyujiang <chaoyujiang@hygon.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
2026-10-22 08:17:15 +08:00
Liangsheng Yin da7b97a737 [Perf] dsv4: Publish prefill read-done under the request window and on V4.1 (#45582) 2026-10-21 16:29:28 -07:00
Liangsheng Yin 3438ed660d [Test] cpu: prune mirror and tautology cases from the per-commit CPU suite (#45672) 2026-10-21 16:02:47 -07:00
ed1f56fead [Fix] [Apple] mlx: Give the stub model a shared-expert layout (#45419)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: R0CKSTAR <yeahdongcn@gmail.com>
2026-10-21 15:54:49 -07:00
Baizhou Zhang f4035cab0b [Feat] cp: Enable GLM-5.3 Flash interleave BCG with batched prefill and DFlash2 (#43182) 2026-10-21 15:51:28 -07:00
991b846227 [Perf] speculative: Parallelize EAGLE spec-decode KV index building over token blocks (flashinfer draft decode; verify/draft-extend for flashinfer, flashinfer_mla, aiter) (#42803)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: YAMY1234 <74099316+YAMY1234@users.noreply.github.com>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
Co-authored-by: Po-Han Huang <pohanh@nvidia.com>
2026-10-21 15:26:35 -07:00
Michael 51baa1f6e9 [Test] [AMD] debug-utils: Opt TestCrashDump into expect_server_exit (#45408) 2026-10-21 14:49:53 -07:00
Liangsheng Yin f5eca05b05 [Fix] [SM100] quantization: Honor deep_gemm for 32x32 block-FP8 linears (#45585) 2026-10-21 14:42:50 -07:00
Liangsheng Yin 68623cae92 [Fix] test: Gather Mamba track indices from track_position in the mixed-batch track test (#45670) 2026-10-21 14:33:48 -07:00
Liangsheng Yin f7d4f06242 [Perf] dsv4: Compute request-window history rows once per batch (#45579) 2026-10-21 14:01:19 -07:00
Liangsheng Yinandlixiufei-leo 2c7a9e3b50 [Fix] radix-cache: Order linker offloads behind the forward that writes the pages (#45559)
Co-authored-by: lixiufei-leo <294029388+lixiufei-leo@users.noreply.github.com>
2026-10-21 13:53:37 -07:00
Khoa Pham 028de2c5d4 [Perf] dsv41: Use tuned MXFP8 tactics at decode sizes (#45630) 2026-10-21 13:35:56 -07:00
Yoni GozlanandJimmy Shong a2e4ffbcea [Fix] parser: Accept whole-number floats for integer tool arguments (#43941)
Co-authored-by: Jimmy Shong <69131491+Jiminator@users.noreply.github.com>
2026-10-21 13:04:15 -07:00
a3d00eff58 [Perf] srt: Record on the host whether an extend batch completes a prompt (#45487)
Co-authored-by: Lu Fang <116399278+luccafong@users.noreply.github.com>
Co-authored-by: shodoco <shodoco@users.noreply.github.com>
2026-10-21 11:58:25 -07:00
7e5256d1ba [Feat] srt: Balance uvicorn tokenizer workers with per-worker SO_REUSEPORT listeners (#45482)
Co-authored-by: luccafong <116399278+luccafong@users.noreply.github.com>
Co-authored-by: joshuadeng <joshuadeng@users.noreply.github.com>
2026-10-21 11:57:26 -07:00