Commit Graph
11714 Commits
Author SHA1 Message Date
Baizhou Zhang 60ff1e33a5 [DeepSeek V4] Default FP4 checkpoints to FlashInfer MXFP4 MoE (#35919) 2026-08-21 16:48:58 -07:00
Rain Jiang 7d7ab4b5c6 Rainj me/rust server refactor2 (#35239) 2026-08-21 16:37:02 -07:00
YAMY 834400705f perf: overlap Qwen shared expert with DeepEP routed experts (#34938) 2026-08-21 15:39:44 -07:00
Yanbin Jiang 7d893255c3 [Spec][LoRA] Support multi-adapter LoRA with EAGLE/NEXTN/DFLASH/DSPARK speculative decoding (#34337) 2026-08-21 14:21:53 -07:00
Hank HanandTRAE CLI 590b11a5ef [Runtime] Don't override CUDA_MODULE_LOADING (#35711)
Co-authored-by: TRAE CLI <traecli@bytedance.com>
2026-08-21 14:09:27 -07:00
Shangming Cai 729a050ea3 refactor(disagg): extract _all_reduce_polls helper (#35886) 2026-08-22 02:30:33 +08:00
Connor Carpenter c3735625de fix(grpc): derive choice count before normalization (#35778)
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
2026-08-21 11:29:52 -07:00
Mohammad Miadh AngkadandMohammad Angkad 61c2da42bb [Fix] Pass Anthropic thinking history as reasoning_content for custom chat encoders (#35480)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-08-22 02:22:11 +08:00
Davis Wertheimer 70983bd7db Add SGLang Granite SWA support via existing Granite models (#35794)
Signed-off-by: Davis Wertheimer <davis.wertheimer@ibm.com>
2026-08-21 11:13:15 -07:00
Shuwen Wangandispobock 0bdd28d487 [mem_cache] docs: add a layer map and placement rules (#35643)
Co-authored-by: ispobock <ispobaoke@gmail.com>
2026-08-22 00:35:07 +08:00
Mick 8658d00764 [diffusion] feat: support loading peft lora (#35868) 2026-08-21 22:58:59 +08:00
li_maxandMick 0447ade326 [diffusion] fix: fall back to a component's default attention backend (#35796)
Co-authored-by: Mick <mickjagger19@icloud.com>
2026-08-21 22:56:44 +08:00
王鹤男 932f632158 [diffusion] fix: do not warn that the recommended short edge is unverified (#35745) 2026-08-21 22:56:03 +08:00
Shangming Cai a41da991c8 refactor(disagg): collapse duplicated branches in get_kv_class (#35847) 2026-08-21 22:11:31 +08:00
王鹤男 5ecd6d794d [diffusion] fix: fix quantized qkv scales and missing-param policy for minimax-h3 (#35740) 2026-08-21 22:10:50 +08:00
Shangming Cai 4f343abc13 refactor(disagg): remove unreferenced dead code (#35838) 2026-08-21 22:09:25 +08:00
Mick 5a46d657b7 [diffusion] refactor: resolve lora weight sources deterministically (#35774) 2026-08-21 21:05:40 +08:00
Mick 5206f11543 [diffusion] fix: stop the mapped-weight store from holding the parameter itself (#35813) 2026-08-21 21:02:21 +08:00
Shangming CaiandClaude Opus 5 dad6fd0f04 refactor(disagg): remove dead get_embedding_port (#35844)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-21 18:12:23 +08:00
Xiaoyu Zhang 39d4d65a51 [diffusion] Accelerate SANA-Video linear attention in quality=high (#35728) 2026-08-21 18:05:43 +08:00
Xiaoyu Zhang a5c52a9358 [diffusion] Enable LongCat breakable CUDA graphs (#35724) 2026-08-21 17:59:16 +08:00
Shangming Cai e7a37c8550 refactor(disagg): remove dead build_and_send_encode_request (#35843) 2026-08-21 17:58:35 +08:00
jasonjk-park 0db2c53dec Fix overlap prebuilt row reuse race (#35748) 2026-08-21 02:00:13 -07:00
Liangsheng Yin 896acc8860 [Fix] Clear full-to-SWA mapping with index_fill_ to avoid a blocking H2D copy (#35773) 2026-08-21 01:23:26 -07:00
4c98759c73 [AMD] fix(rocm): support flydsl 0.3.0 in the FlyDSL fused norm kernel (#34536)
Co-authored-by: Bingxu Chen <195740905+bingxche@users.noreply.github.com>
Co-authored-by: thomawan <thomawan@amd.com>
2026-08-21 01:13:21 -07:00
Shuwen Wang 8ff9c2b227 [mem_cache][9/N] refactor: move DSAIndexerPoolHost to pool_host.dsa (#35306) 2026-08-21 15:59:15 +08:00
8a123cbd0e [Refactor] New EPD (#30398)
Co-authored-by: Yuang Chen <1131578721@qq.com>
Co-authored-by: Yuang Chen <cya539102@antgroup.com>
Co-authored-by: ZhengWG <zwg0606@gmail.com>
2026-08-21 15:22:48 +08:00
44c90c6282 [AMD] DSv4: fuse the qk-norm-rope pair on the MTP target-verify path (#34973)
Co-authored-by: HAI <hixiao@gmail.com>
Co-authored-by: Thomas Wang <thomawan@amd.com>
2026-08-20 23:08:22 -07:00
Ke Bao 73a2c117c6 Support mxfp8 KV cache in PD transfer (#35718) 2026-08-21 13:05:06 +08:00
Thomas Wang 34180a0d35 [AMD] Improve K3 dspark draft attn kernel perf (#35499) 2026-08-20 21:48:59 -07:00
bda9952377 [AMD] DeepSeek-V4 MI355X: eliminate bpreshuffle fp8-scale copies at producer sites (MoE down, MLA o_proj bmm) (#33166)
Co-authored-by: kk <43161300+kkHuang-amd@users.noreply.github.com>
Co-authored-by: Thomas Wang <thomawan@amd.com>
2026-08-20 21:46:23 -07:00
Mick 6127d1daee [diffusion] feat: allow offloaded weights stay on the checkpoint mapping (#35701) 2026-08-21 10:54:28 +08:00
Zhangheng 44806dc507 Using unified radix tree by default for all case (#35081) 2026-08-21 10:45:46 +08:00
Xiaoyu Zhang 7e80e889a2 [diffusion] Fuse LTX-2.5 decoder 3D RoPE (#35698) 2026-08-21 10:13:09 +08:00
ishandhananiandShangming Cai 978244d671 [P/D disagg] Decode-side radix cache for SWA hybrid models (unified radix tree) (#27770)
Co-authored-by: Shangming Cai <csmthu@gmail.com>
2026-08-21 09:25:06 +08:00
Mick 5bb981dce1 [diffusion] chore: read the cgroup this process is actually in (#35707) 2026-08-21 08:48:50 +08:00
Nan Jiang f825d72936 [Sampling] Restore finite top-k requirement for sampling masks (#35205) 2026-08-20 16:55:25 -07:00
ethcheandEthan Che 67f6ad61d9 fix(kernel) Fix Helion small-token prefill bug (#35197)
Co-authored-by: Ethan Che <eche@meta.com>
2026-08-20 16:41:59 -07:00
Shenxiu LiuandQiaolin Yu 779e593bd1 Fix _GenerationStreamAccumulator logprob_end off-by-one under retract (#26510)
Co-authored-by: Qiaolin Yu <liin1211@outlook.com>
2026-08-20 16:03:13 -07:00
Wenkai DuandHubert Lu 5a7b26c636 [AMD] [sgl-kernel] Bypass caches for peer traffic in ROCm custom all-reduce (#32832)
Co-authored-by: Hubert Lu <Hubert.Lu@amd.com>
2026-08-20 15:24:18 -07:00
Jiajun Li a4ef828207 fix(openai): avoid duplicate routed expert in response when return_meta_info = True (#35323) 2026-08-20 15:21:58 -07:00
ishandhananiandAlex Nails 0f744b6848 feat: make mm_inputs msgpack-native (#29656)
Co-authored-by: Alex Nails <alex.nails@radixark.ai>
2026-08-20 14:30:07 -07:00
Liangsheng Yin 5a100d9086 [misc] Trim restating comments and docstrings in srt/managers (#35622) 2026-08-20 14:18:40 -07:00
Lee Nau ad367d72b0 [Kimi K3] Select FlashInfer MXFP4 for SM107 auto MoE (#35554) 2026-08-20 14:09:31 -07:00
eac91ac362 [Fix] Land the decode mamba checkpoint depth on the tree page under DCP (#35412)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
2026-08-20 12:15:42 -07:00
Junlin Wuandronnie_zheng 308bc1228b 📝 [NPU] Clean up quantization comments (#34829)
Co-authored-by: ronnie_zheng <zl19940307@163.com>
2026-08-20 22:03:58 +03:00
Connor Carpenter 61fa64ae7e feat(grpc): expose KV event discovery metadata (#35714)
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
2026-08-20 13:39:24 -05:00
Chao Shi 2ef0fe4669 TP/PP Consensus checker (#34406) 2026-08-21 01:36:03 +08:00
Shangming Cai 23cb04093c fix(multimodal): keep LLaVA image fetch off the CPU-preprocess timeout budget (flaky test_mixed_batch) (#35700) 2026-08-21 01:26:30 +08:00
Ke Bao ba97cc6397 Skip empty linear-attention state buffers in PD transfer (#35689) 2026-08-21 01:00:50 +08:00