Commit Graph
12265 Commits
Author SHA1 Message Date
Estrella-xx 4c2c169e6b [NPU]Strip padding before FIA kernel for vision encoder padded sequences (#36329) 2026-09-01 19:28:19 +08:00
Eric.Chin.AMDandKingRei ed122ea984 [AMD] Enable topk v2 GLM ROCm (#36851)
Co-authored-by: KingRei <hiroki1139@gmail.com>
2026-09-01 03:48:50 -07:00
YC Yen-Ching Tseng b425897366 [AMD] Gate the aiter memory-reserve exemption behind an env var (#37242) 2026-09-01 03:46:15 -07:00
Jialin Ouyang a77283fb02 [Rust] Rename mem-cache to sglang-radix-tree (#37290) 2026-09-01 03:03:44 -07:00
chx96642264 c16a8fc899 [NPU] [bugfix] Fix NPU MLA HiCache backup accessing missing data_ptrs. (#36813) 2026-09-01 17:31:11 +08:00
eeecho b68702be99 [DSV4] hc-prenorm: fuse the combine step into a Triton kernel (#35118) 2026-09-01 02:04:05 -07:00
Ziang Li 5edcd0a445 [FlashInfer V0.6.18] feat(dsv4): support --dsa-topk-backend flashinfer with fused top-k (#33237) 2026-09-01 01:18:10 -07:00
Liangsheng Yin 3484f7f836 [mem_cache] Add free_kv_row to release a request's kv row by row range (#36721) 2026-09-01 01:14:43 -07:00
Xiaoyu ZhangandCursor 1c3ad92438 [Diffusion] Fuse FLUX.2 ModelOpt FP8 producers and QKV packing (#37162)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-01 16:14:29 +08:00
chien-an-chen 8a191554e3 [AMD] Enable 12-head MLA aiter fp8 Gluon decode (batched bh16bn128). (#34647) 2026-08-31 23:47:45 -07:00
Liangsheng Yin 959ca033eb refactor(hicache): simplify decode offload state bookkeeping (#37299) 2026-08-31 23:09:41 -07:00
Mick ae2bd5728b [vlm] fix: contain multimodal feature transport failures (#37047) 2026-09-01 13:46:38 +08:00
huangtingweiandhzh0425 b21000aef1 [Unified Cache][4/N]: Add Mooncake backend for external linker (#37205)
Co-authored-by: hzh0425 <hzh0425@apache.org>
2026-09-01 11:40:06 +08:00
hhhh1252023 60f881b40c [CI/NPU] Isolate multi-node tests by run_id to prevent concurrent-run… (#35500) 2026-09-01 11:08:27 +08:00
Yuan Luoandluoyuan.luo 5b04408784 [MoE] Add FlashInfer SM90 MXFP4 W4A8 CUTLASS MoE (#34967)
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
2026-08-31 20:04:41 -07:00
22337e9c56 fix(unified-memory): forward the KV-index translator through every wrapper backend (#37307)
Co-authored-by: Caihua Li <caihua.li@bytedance.com>
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
2026-08-31 19:55:21 -07:00
Mohammad Miadh Angkad e6f21cdadc [Cohere Command-A-Plus] Optimize decode and BCG capture on SM10X (#36624) 2026-08-31 19:37:47 -07:00
Yiqi Yang 1591dcd91a [diffusion] fix: fix loading a block-FP8 quantized MiniMax-H3 DiT (#35703) 2026-09-01 10:33:58 +08:00
71cee04ebe [Diffusion] Optimize Qwen-Image TP collectives and attention (#36680)
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-01 10:28:37 +08:00
Martin Hua 562b661e0e [Feature] Megatron LayerNorm sequence parallelism (--enable-layernorm-sp) (#30915) 2026-08-31 19:27:28 -07:00
ashwini rathiandarathi-hlab 783af667fb xpu: record per-model metrics to jsonl for nightly dashboard (#36699)
Co-authored-by: arathi-hlab <arathi-hlab@users.noreply.github.com>
2026-09-01 10:20:56 +08:00
f50b4ad7ae [DCP] Support decode context parallelism on the trtllm_mla decode path (#33926)
Co-authored-by: kpham-sgl <khoa.pham@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-31 19:19:44 -07:00
Khoa Pham 97744189b8 [Kernel] Raise shape limits in shared FLA and MoE kernels (ported from #36507) (#37317) 2026-08-31 18:53:46 -07:00
Ziang Li 9a85473a89 [FlashInfer v0.6.18] add FlashInfer CuTe DSL NVFP4 W4A16 mode (#35120) 2026-08-31 18:47:30 -07:00
Xiaoyu ZhangandCursor 175973d834 [Diffusion] Fuse Qwen-Image residual norm and NVFP4 quantization (#37129)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-01 08:54:10 +08:00
Xiaoyu ZhangandCursor 079afaffb1 [Diffusion] Fuse FLUX.2 gated residual normalization on Blackwell (#37112)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-01 08:38:33 +08:00
Xiaoyu ZhangandCursor 6715debb2a [Diffusion] Fuse Qwen-Image FP8 QKV projection and Blackwell epilogue (#37123)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-01 08:34:41 +08:00
Baizhou Zhang 07c8f7294d Bump sgl-deep-gemm to 0.1.7 (#37279) 2026-08-31 17:16:34 -07:00
ef9e58fd6d feat(unified-memory): three sub-pools for mamba + hybrid-SWA models (#35177)
Co-authored-by: Caihua Li <caihua.li@bytedance.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
2026-08-31 15:10:12 -07:00
98cb3535b7 feat(unified-memory): byte-budget sizing, feasibility floor, and a conservation verifier (#35158)
Co-authored-by: Caihua Li <caihua.li@bytedance.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
2026-08-31 15:09:28 -07:00
961beee9e5 fix(unified-memory): four boot/correctness fixes on the hybrid model paths (#35154)
Co-authored-by: Caihua Li <caihua.li@bytedance.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
2026-08-31 15:08:43 -07:00
Liangsheng Yin 2530204502 [mem_cache] Make release, row-reuse asserts, and presence checks read the KV record (#37167) 2026-08-31 12:46:15 -07:00
Liangsheng YinandBaizhou Zhang 95f0f41021 [CI] Move tests onto the right CI stages (#34074)
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
2026-08-31 12:39:59 -07:00
Lianmin Zheng 48098b5f23 [Rust] Derive server address and accept signed env values (#37221) 2026-08-31 12:30:40 -07:00
Lianmin Zheng 1da86b9801 [Rust] Split and rename embedded server components (#37220) 2026-08-31 12:28:43 -07:00
ishandhanani cf51650335 fix(config): retain pre-engine resolution declarations (#37195)
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
2026-08-31 12:24:29 -07:00
Shuwen Wang 9e9d26a4af Fix Mooncake serving benchmark trace rows (#37201) 2026-08-31 11:43:59 -07:00
Yuwei AnandClaude Fable 5 07d84ebd6d [2/N][Mixed] Mixed chunk prefill with spec enabled (#36933)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-31 10:48:07 -07:00
9cf157c252 [Radix Cache] Add Rust TreeCore backend with shared parity tests (#32710)
Co-authored-by: alphabetc1 <2508695655@qq.com>
Co-authored-by: ispobock <ispobaoke@gmail.com>
2026-09-01 00:26:20 +08:00
Xiaoyu Zhang 52e1c24744 [Diffusion] Fuse FLUX.2 token concatenation and NVFP4 quantization (#37141) 2026-08-31 21:33:02 +08:00
Xiaoyu ZhangandSong Bian d60d658f5f [Kernel] Add GB300 Triton MoE configs for GLM-4.5 FP8 (#37159)
Co-authored-by: Song Bian <biansonghz@gmail.com>
2026-08-31 21:31:04 +08:00
Xiaoyu Zhang 771e613d96 [Diffusion] Fuse Qwen-Image final adaptive LayerNorm (#37144) 2026-08-31 18:22:20 +08:00
Xiaoyu ZhangandCursor 1ed9bfac2c [Diffusion] Fuse Qwen-Image FP8 norm and activation quantization (#37156)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-31 18:19:42 +08:00
pllimax 63b2adbeac [NPU] Fix evalscope accuracy parsing and add glm5_1 aime26 request timeout (#36459) 2026-08-31 17:19:03 +08:00
YAMY a53718e88c fix(ci): update attention backend test fixtures (#37219) 2026-08-31 02:15:17 -07:00
6580d5cd9a weight cache: key daemon paths by GPU UUID (#36101)
Co-authored-by: siyu <liusy58@linux.alibaba.com>
Co-authored-by: Alex Nails <alex.nails@radixark.ai>
2026-08-31 01:44:45 -07:00
+1 3865efc9f7 [AMD] support gfx1250 on ROCM 10 (#36871)
Co-authored-by: HAI <hixiao@gmail.com>
Co-authored-by: Kao <akao@amd.com>
Co-authored-by: wunhuang <wunhuang@amd.com>
Co-authored-by: Thomas Wang <1am9trash@gmail.com>
Co-authored-by: Xinyi Song <86638975+RolaoDenthu@users.noreply.github.com>
Co-authored-by: Lin, Soga <soga.lin@amd.com>
Co-authored-by: kk <43161300+kkHuang-amd@users.noreply.github.com>
Co-authored-by: Bingxu Chen <bingxche@amd.com>
Co-authored-by: sogalin_codegen <39478626+sogalin@users.noreply.github.com>
Co-authored-by: Thomas Wang <thomawan@amd.com>
2026-08-31 01:19:11 -07:00
Yuan Luoandluoyuan.luo 712a720c8a [KDA] Fused-accept state advance for FlashInfer KDA MTP verify (#33722)
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
2026-08-31 16:11:49 +08:00
Cheng Wan f61bb7b40a [unified-memory] Drop the vacated 'dense' qualifier and the restating comments (#37170) 2026-08-31 00:54:13 -07:00
8bb776dc48 feat(unified-memory): read unified pool from attention backends fa3/flashinfer/trtllm_mha/flashmla (#34613)
Co-authored-by: Caihua Li <caihua.li@bytedance.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
2026-08-30 23:58:24 -07:00