Commit Graph
18170 Commits
Author SHA1 Message Date
cctryandcctry 206034e520 Keep graph-pool borrows on their allocation stream (#39180)
Co-authored-by: cctry <17473714+cctry@users.noreply.github.com>
2026-09-12 19:26:09 -07:00
cctryandcctry 18cc55dc0b Expose a capacity check for graph-pool borrows (#39178)
Co-authored-by: cctry <17473714+cctry@users.noreply.github.com>
2026-09-12 19:24:57 -07:00
Xiaoyu ZhangandMick Qian 23bc4c6ed9 [Diffusion] Return Qwen-Image-Layered outputs and preserve CFG2 rounding (#38549)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-09-13 09:15:47 +08:00
Xiaoyu ZhangandMick Qian 3e035a3513 [Diffusion] Optimize Qwen-Image-Edit attention on Hopper (#38584)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-09-13 09:15:01 +08:00
Cheng Wan fa663e7297 config: delete the redundant full stamp in initialize_model_parallel (#39202) 2026-09-12 17:27:21 -07:00
Cheng Wan fa260f26da config: delete dead ensure_model_parallel_initialized (#39137) 2026-09-12 17:27:12 -07:00
Cheng Wan 6804eeaabe config: an out-of-tree replacement point for every resolution-pipeline step (#39134) 2026-09-12 17:26:54 -07:00
Baizhou Zhang a8b5616303 Fix DeepGEMM release dependencies and bound GPU validation (#39241) 2026-09-12 17:00:58 -07:00
6657f7d844 [MoE][ROCm] Admit the unified Triton router on ROCm, including single-group routing (#38328)
Co-authored-by: JohnQinAMD <yanyuan.qin@amd.com>
Co-authored-by: RuibinCheung <ruibzhan@amd.com>
Co-authored-by: Zhang, Jiejing <jiejing.zhang@amd.com>
2026-09-12 16:21:28 -07:00
a66451c058 [GLM-5.3 Flash] Restore and enable KPool metadata fusion (#38845)
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: zRzRzRzRzRzRzR <Yuxuan.Zhang2@liverpool.ac.uk>
Co-authored-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com>
Co-authored-by: zanes-ops <zanes@nvidia.com>
2026-09-12 16:01:11 -07:00
Zhang, Jiejing 288627e400 [AMD] Document GLM-5.2 MXFP4 recipe update on MI355X (#39230) 2026-09-12 15:57:15 -07:00
Xinyi Song 21289cfd50 [AMD] Fix Dspark accept length and reduce host bubble on DSV4 (#39116) 2026-09-12 15:21:43 -07:00
Xinyuan Tong b5a2aebc7e [Docs] GLM-5.3-Flash cookbook: fixed MTP 5/1/6, EP1 + flashinfer_trtllm on Blackwell (#39213) 2026-09-12 12:56:57 -07:00
cctryandcctry 7ae4af8187 Scope graph-pool borrowing to the runtime and reduce fragmentation (#39177)
Co-authored-by: cctry <17473714+cctry@users.noreply.github.com>
2026-09-12 11:45:17 -07:00
cctryandcctry 2784a86062 Reuse live CUDA graph executables during dedup registration (#39176)
Co-authored-by: cctry <17473714+cctry@users.noreply.github.com>
2026-09-12 11:44:17 -07:00
25a5641cf2 [Session + MM] Fix text positions in session continuations (#39144)
Co-authored-by: Byron Hsu <byron+per@periodiclabs.ai>
Co-authored-by: Manik Singhal <3400497+Manikvsin@users.noreply.github.com>
2026-09-12 10:31:55 -07:00
ae1acf822d [GraniteMoE] Load split per-expert quantized MoE weights (#37679)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
2026-09-12 23:10:20 +08:00
7b89b95168 [diffusion] feat: pick the attention backend by measuring it (#38689)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-12 21:52:16 +08:00
MickandMick Qian dc3171c322 [diffusion] CI: remove mova ulysses two-gpu CI case (#39097)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-09-12 21:48:04 +08:00
925e684a88 [Feat][Responses API] Support custom tools, encrypted reasoning replay, developer tier and model validation (#38690)
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
2026-09-12 21:29:12 +08:00
desmond-intel 6dc7b3421b Inference Support Mamba 2 and 1 (#34556) 2026-09-12 20:51:18 +08:00
KunalandClaude Fable 5.1 fd32226706 Fix gpt-oss RunAI streamer weight ownership (#38908)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-12 20:21:59 +08:00
Shuwen Wang bd45cd50ca [Unified Tree] Port SWA Branching-Point Caching to the Rust TreeCore (#37584) 2026-09-12 18:37:57 +08:00
Yanbin JiangandShuwen Wang 0b415fa573 [HiCache][LoRA] Isolate storage pages by extra key (#38577)
Co-authored-by: Shuwen Wang <47200617+alphabetc1@users.noreply.github.com>
2026-09-12 18:18:14 +08:00
Shuwen Wang b9cb96496d [Unified Tree] Preserve aux LRU recency when splitting nodes (#38482) 2026-09-12 18:17:38 +08:00
Qiaolin Yu 6953dae005 [Qwen 3.8 Next] Remove unused tokenwise QSA implementation and tests (#38960) 2026-09-12 01:32:01 -07:00
ChangLiu0709 7bc4eb3740 [AMD] Update MI355X MXFP4 HiCache defaults and quick-reduce quantization for Qwen3.5 cookbook (#39104) 2026-09-12 00:59:32 -07:00
Tuan Nguyen Gia 7c195b9151 [AMD] Fix Quark load of MiniMax-M3 MXFP4 index_qkv_proj (#37254) 2026-09-12 00:50:41 -07:00
MickandClaude Opus 5 bf3305b65e [diffusion] feat: read mapped layers directly when the host cannot cache them (#39022)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-12 15:48:03 +08:00
Xiaoyu ZhangandCursor 0a57403468 [Perf] Optimize Qwen3-VL unique-image serving on H100 (#36411)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-12 15:16:59 +08:00
Khoa PhamandClaude Fable 5.1 6ba96d329f [DCP] Resolve --dcp-comm-backend to fi_a2a/a2a by default for every model (#39165)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-11 23:48:42 -07:00
Shangming CaiandClaude Fable 5 981b947568 [Fix] Seed raw tokenizer_path for smg-grpc-servicer in gRPC mode (#39105)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-09-12 14:41:15 +08:00
Baizhou Zhang 56a4f47ca6 [CI] Fix base-a wait for skipped CPU matrix (#39195) 2026-09-11 23:39:53 -07:00
Baizhou Zhang 3bb2a5231a [CI] Extend DeepGEMM SM90 test timeout to four hours (#39194) 2026-09-11 23:36:54 -07:00
ehuaa 1cdc5bca5e fix(qsa): clamp the compress gather to the rows this forward has (#38346) 2026-09-11 23:24:17 -07:00
Khoa PhamandClaude Fable 5.1 0d08668821 [Cookbook] Kimi-K3: keep DCP under HiCache L1+L2 with DSPARK (#39190)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-11 23:14:24 -07:00
YAMY 55e5e21c88 [Qwen3.8-Next] Add PD state transfer for Flash Next (#36651) 2026-09-11 22:31:45 -07:00
Ziang Li dbd4302bbb Keep NVFP4 blockscale swizzle padding on the input device (#39141) 2026-09-11 22:18:02 -07:00
Aurick Qiao c389dd0863 Fix RunAI object-storage checkpoint index filtering (#38988) 2026-09-12 12:25:57 +08:00
iridiumine a984c78330 [NPU] Support DFlash speculative decoding for MiMo-V2.5-Pro (mxfp4) (#37565) 2026-09-12 11:59:23 +08:00
Baizhou Zhang 9c365e97b1 [CI] Fix DeepGEMM sanitizer setup and Blackwell test timeouts (#39163) 2026-09-11 20:54:52 -07:00
ff1ce11348 [diffusion] model: support VDN-H3 with a hybrid_window_attn_h3 backend (#37903)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Haocheng Xi <xihc@berkeley.edu>
Co-authored-by: Mick <mickjagger19@icloud.com>
2026-09-12 11:36:32 +08:00
FrancisandQiaolin-Yu e91c948057 feat: support TP>1 Domino rollout for DFlash V2 (#37069)
Co-authored-by: Qiaolin-Yu <liin1211@outlook.com>
2026-09-11 18:51:11 -07:00
Vignesh Sethuraman 0d1bea77da [AMD] Allow aiter attention backend for Gemma-4 (#38758) 2026-09-11 18:48:26 -07:00
Vignesh Sethuraman e1d364dd1f [AMD] aiter: route head_dim>256 prefill through Triton unified_attention (#38757) 2026-09-11 18:40:11 -07:00
b805cc5014 [Bugfix] Track DFlash Mamba state at checkpoint boundaries (#37818)
Co-authored-by: lvweiv <vwei.lv@gmail.com>
Co-authored-by: kpham-sgl <khoa.pham@radixark.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
2026-09-11 18:38:09 -07:00
Baizhou Zhang 29cc9d2dc9 [CI] Install elfutils headers for DeepGEMM wheel builds (#39152) 2026-09-11 17:29:40 -07:00
a207786205 [PD] Transfer the DCP-replicated DSPARK draft KV in DCP1->DCP-N relayouts (#37709)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-11 17:15:18 -07:00
7d9c57da6e [Session] Fix session idle timeout after rejected requests (#39035)
Co-authored-by: Byron Hsu <byron+per@periodiclabs.ai>
Co-authored-by: Manik Singhal <3400497+Manikvsin@users.noreply.github.com>
2026-09-11 16:59:43 -07:00
Kevin Mi 5f3606c7b2 [Cookbook][AMD] Kimi-K3 MI350X/MI355X: pin a ROCm image with the DSPARK graph-capture fix, add measured cell numbers (#39029) 2026-09-11 16:10:44 -07:00