Commit Graph
14440 Commits
Author SHA1 Message Date
Jzz1943 a361f0c410 [Fix] kda: Align graph tracking branch dtype with the recurrent state pool (#44886) 2026-10-21 15:58:34 +08:00
f7b8c6fe40 [Fix] glm-5.2: Record side-stream work right before its join to keep CUDA-graph replay on one stream (#45349)
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
2026-10-21 15:36:54 +08:00
Hongkuan Zhouandishandhanani 8acfabbd67 [Fix] observability: Correct speculative FPM timing and decode lengths (#38711)
Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
2026-10-21 15:31:18 +08:00
Junlin Wu 844d2b5dec [Fix] [NPU] qwen3.5: Fix dense serving on Ascend 950 (#32745) 2026-10-21 09:50:38 +03:00
Jimmy Shong 8599b7251f [Chore] deps: Bump the OpenAI SDK to 2.8.0 for Responses shell tools (2/8) (#45291) 2026-10-20 23:49:30 -07:00
wutao fb6e6a09ce [Fix] [NPU] disaggregation: Fix host-pool backup for decode retraction (#43814) 2026-10-21 14:48:01 +08:00
Khoa PhamandAMD-yanfeiwang da7e3f1c12 [Fix] cache: Accept packed draft layer mappings in the UMBP linker (#44929)
Co-authored-by: AMD-yanfeiwang <yanfei.wang@amd.com>
2026-10-20 23:27:22 -07:00
Thomas Wang bd5a445bf7 [Fix] [AMD] dsv41: Set moe act to fp8 by default (#45480) 2026-10-20 23:09:27 -07:00
956b9a24ee [Perf] [AMD] qwen3.5: Fuse activation quant into small-M FP8 block-scale GEMM on gfx950 (#42229)
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: jacky.cheng <yichiche@amd.com>
2026-10-20 22:32:30 -07:00
Aarne Talman 4b5c9c4b5e [Perf] [AMD] attention: Restore the gfx950 extend tile for head_dim <= 128 (#45099) 2026-10-20 22:02:50 -07:00
32c26788f3 [Fix] mamba: Keep prefill rows' checkpoint tracking through mixed chunked prefill (#45435)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: YavorGIvanov <YavorGIvanov@users.noreply.github.com>
2026-10-20 21:56:50 -07:00
Liangsheng Yin 573b5716f5 [Fix] spec: Take post-verify KV move targets from the verify window (#45491) 2026-10-20 21:12:41 -07:00
1c2f42470c [Fix] disaggregation: Keep a requeued optimistic prefill request's multimodal inputs (#45440)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: libertyeagle <30348494+libertyeagle@users.noreply.github.com>
Co-authored-by: fei-xx <135589532+fei-xx@users.noreply.github.com>
2026-10-20 20:27:34 -07:00
metamergebotandshodoco 6bd1c76735 [Fix] rust-server: Drop the fixed 64-item multimodal cap in favor of --limit-mm-data-per-request (#45432)
Co-authored-by: shodoco <shodoco@users.noreply.github.com>
2026-10-20 20:25:22 -07:00
eb91a41f38 [Feat] unified_cache: Support FULL + MAMBA MLA models (Kimi-K3) in the direct external linker (#44977)
Co-authored-by: Yutong Wu <yutong.wu@amd.com>
Co-authored-by: Zhangheng <hzh0425@apache.org>
2026-10-21 10:47:17 +08:00
Chunan Zeng 93088901fb [Chore] attention: Remove the tokenspeed_mla backend and dependency (#44803) 2026-10-20 18:50:05 -07:00
unclezhou c0d18c0254 [Fix] [NPU] dspark: Fix attn_tp > 1 draft wo_b all_reduce group (#40399) 2026-10-21 09:20:15 +08:00
b326e8349d [Perf] mamba: Copy tracked decode states at HBM bandwidth in one persistent launch (#45213)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: YavorGIvanov <YavorGIvanov@users.noreply.github.com>
2026-10-20 16:52:15 -07:00
Baizhou Zhang 5b1dbc2c80 [Fix] [NVIDIA] modelopt-nvfp4: Keep MoE global scales in gate/up order (#45289) 2026-10-20 16:49:13 -07:00
kkandwunhuang 9686edcf80 [Perf] [AMD] dsv41: faster mHC boundary Triton tile for 96+ rows, bitwise unchanged (#45124)
Co-authored-by: wunhuang <wunhuang@amd.com>
2026-10-20 16:39:34 -07:00
Cheng Wan 027208ae02 [Doc] layer-boundary/moe: Correct guides and comments that drifted from the code (#44896) 2026-10-20 16:26:40 -07:00
Cheng WanandAditya Suresh 299a75ac5d [Refactor] moe: Derive routing inputs from the layer and active forward (#44994)
Co-authored-by: Aditya Suresh <152752311+adi-suresh01@users.noreply.github.com>
2026-10-20 16:26:15 -07:00
kkandwunhuang 533020b500 [Perf] [AMD] dsv41: run gfx950 MXFP8 dense GEMMs on aiter's preshuffled FlyDSL kernel (#45123)
Co-authored-by: wunhuang <wunhuang@amd.com>
2026-10-20 16:19:32 -07:00
Cheng WanandAditya Suresh 4f8f06faf3 [Refactor] moe: Derive layer contracts from the selected quant method (#44993)
Co-authored-by: Aditya Suresh <152752311+adi-suresh01@users.noreply.github.com>
2026-10-20 16:18:45 -07:00
zydoandCheng Wan 7b1d5cd02d [Doc] agent-skills: Fix stale paths and deleted guard tests in skills and rules (#43666)
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
2026-10-20 16:15:10 -07:00
bf145750a6 [Perf] prefill-cuda-graph: Make prefill shared-read completion explicit and default-on (#37958)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
Co-authored-by: Po-Han Huang <pohanh@nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-10-20 16:12:55 -07:00
Aarne Talman d5e83ea4f6 [Perf] [AMD] deterministic: Faster extend attention launch on gfx950 (#45098) 2026-10-20 16:11:01 -07:00
Cheng WanandAditya Suresh f1eec1232d [Fix] config: Preserve ServerArgs positional constructor slots (#44991)
Co-authored-by: Aditya Suresh <152752311+adi-suresh01@users.noreply.github.com>
2026-10-20 16:08:28 -07:00
Aditya SureshandCheng Wan e2e1a97fc3 [Fix] moe: Map expert ids for layers whose runner needs local ids under EP (#44480)
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
2026-10-20 16:07:48 -07:00
Baizhou Zhang 583cfc4d55 [CI] muse_spark: Use complete local checkpoints in model tests (#45428) 2026-10-20 14:57:01 -07:00
Yuwei An f0b6590722 [Fix] DSpark: Disable simulated acceptance shortcuts by default (#45433) 2026-10-20 14:10:59 -07:00
YAMY 7d9bc10493 [Perf] [NVIDIA] DSpark: Enable the simulated-acceptance fast paths on CUDA (#45421) 2026-10-20 13:36:33 -07:00
cc7ae372ac [Refactor] radix-cache: Route pending-backup split bookkeeping through one component action (#45242)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: Jialin <Jialin@users.noreply.github.com>
2026-10-20 13:24:49 -07:00
0b536ca112 [Fix] radix-cache: Separate logical and reclaimable value lengths in tree components (#45238)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: Jialin <Jialin@users.noreply.github.com>
2026-10-20 13:24:20 -07:00
metamergebotandjoshuadeng 468e26f253 [Perf] detokenizer: Add SGLANG_BATCH_TOKENIZER_OUTPUTS to opt out of grouped return sends (#45280)
Co-authored-by: joshuadeng <joshuadeng@users.noreply.github.com>
2026-10-20 13:24:14 -07:00
bab4a808d8 [Perf] unified-memory: Allocate free pages from each sub-pool's own end first (#45222)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: ZYHowell <ZYHowell@users.noreply.github.com>
2026-10-20 13:24:09 -07:00
0d7638b035 [Fix] scheduler: Exclude accepted storage-tier hits from the waiting prefill load (#45158)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: Jialin <Jialin@users.noreply.github.com>
2026-10-20 13:24:04 -07:00
dc62c3effd [Feat] radix-tree: Report each path node's child count in collect_path_info (#45241)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: Jialin <Jialin@users.noreply.github.com>
2026-10-20 13:23:59 -07:00
metamergebotanddaniel-salib 194876bea0 [Fix] srt: Admit only allowlisted peers on TCP-bound ZMQ IPC sockets (#45250)
Co-authored-by: daniel-salib <daniel-salib@users.noreply.github.com>
2026-10-20 13:23:53 -07:00
Baizhou Zhang cfec1b91bb [Fix] muse_spark: support unified KV memory with TRT-LLM (#45356) 2026-10-20 12:54:53 -07:00
Aarne Talman 29274b41c6 [Fix] [AMD] activation: Run ReLU2 natively on ROCm (#44198)
Signed-off-by: Aarne Talman <aarne.talman@amd.com>
2026-10-20 12:05:14 -07:00
Thomas Wang 82ac83a8d0 [Perf] [AMD] dsv41: Skip unneeded layer 20 computation under decoder-bounded-replay (#45339) 2026-10-20 11:49:05 -07:00
Khoa Pham c2e5742464 [Perf] deepseek-v4: DSpark draft stages take the TF32 mHC statistics path from 32 rows on SM100 (#45260) 2026-10-20 10:12:06 -07:00
sogalin_codegen 89860fbd2e [Fix] [AMD] gpt-oss: support EP for gfx1250 W4A8 MXFP4 MoE (#45365) 2026-10-20 10:08:39 -07:00
Michael 7591118794 [Fix] [AMD] grok: accept allow_patterns_overrides in presharded _prepare_weights patch (#45217) 2026-10-20 16:12:06 +00:00
eb4b167d27 [Feat] KVShard: PD transfer and e2e test (#38473)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Zhangheng <hzh0425@apache.org>
2026-10-20 22:48:43 +08:00
7656071309 [Feat] platform: Add initial T-HEAD SAIL platform support [2/N] (#43826)
Co-authored-by: Zhangheng <hzh0425@apache.org>
Co-authored-by: yzshi <163985193+Yunzshi310@users.noreply.github.com>
2026-10-20 22:19:07 +08:00
Jensenandronnie_zheng 10d5293546 [Perf] [NPU] glm5.2: Support dynamic INT8 MLAProlog for attention (#43469)
Co-authored-by: ronnie_zheng <zl19940307@163.com>
2026-10-20 16:33:08 +03:00
Jensen 3bf4cbc6cf [Perf] [NPU] mem_cache: Increase GLM-5.2-W4A8C8 KV cache capacity with compact DSA indexer allocation (#44436) 2026-10-20 16:30:01 +03:00
gjsheu aa91108f64 [Fix] [NPU] glm5.2: support pcp v2 with interleave & zigzag (#40165) 2026-10-20 21:26:30 +08:00