Commit Graph
97 Commits
Author SHA1 Message Date
Chunan Zeng 93088901fb [Chore] attention: Remove the tokenspeed_mla backend and dependency (#44803) 2026-10-20 18:50:05 -07:00
Cheng Wan 027208ae02 [Doc] layer-boundary/moe: Correct guides and comments that drifted from the code (#44896) 2026-10-20 16:26:40 -07:00
lpdinkandZhangheng 4a651146de [Feat] hicache: Notify storage backends when the initial host-pool registration burst completes (#44030)
Co-authored-by: Zhangheng <hzh0425@apache.org>
2026-10-20 14:09:13 +08:00
YAMY 722669e606 [Feat] [NVIDIA] megakernels: Add --enable-nvidia-megakernel and a layer for NVIDIA megakernel plugins (#44945) 2026-10-19 11:54:44 -07:00
AMD-yanfeiwang 678913a0f0 [Feat] sgl-model-gateway: Add --decode-dispatch-limit to pick PD decode workers near KV arrival (#44689) 2026-10-20 02:16:21 +08:00
Jan SchalkampandXiaoyu Zhang 8e7601ae4a [Feat] paged-experts: Serve MoE models whose weights do not fit VRAM (#42557)
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
2026-10-19 23:10:04 +08:00
Kotthagattu Meher Sai 7f96dc6874 [Perf] [XPU] server_args: Enable fused QK-norm + RoPE by default (#43345) 2026-10-19 10:19:52 +08:00
Liangsheng Yin 6708426728 [Fix] spec: Remove --speculative-skip-dp-mlp-sync (#45043) 2026-10-18 16:41:21 -07:00
+14 d33dd0566a [Feat] srt: Add muse spark runtime, multimodal support and tests (3/5) (#44666)
Co-authored-by: 1am9trash <1am9trash@gmail.com>
Co-authored-by: Alex Nails <alex.nails@radixark.ai>
Co-authored-by: Chunan Zeng <zcnrex@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Ethan Che <eche@meta.com>
Co-authored-by: HAI <hixiao@gmail.com>
Co-authored-by: Jiajun Li <guapisolo@gmail.com>
Co-authored-by: Jiajun Li <jiajun.li@radixark.ai>
Co-authored-by: Jimmy Shiu <jimmysh341@gmail.com>
Co-authored-by: Jimmy Shong <69131491+Jiminator@users.noreply.github.com>
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
Co-authored-by: Wookje Han <52435620+wookjeHan@users.noreply.github.com>
Co-authored-by: Wookje Han <wookjeh@nvl72d079-T17.cm.cluster>
Co-authored-by: Wookje Han <wookjeh@nvl72d094-T17.cm.cluster>
Co-authored-by: Wookje Han <wookjeh@nvl72d157-T16.cm.cluster>
Co-authored-by: Wookje Han <wookjeh@nvl72d170-T04.cm.cluster>
Co-authored-by: Wookje Han <wookjeh@nvl72d214-T17.cm.cluster>
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: Yuhao Yang <yhyang201@users.noreply.github.com>
Co-authored-by: Zijie Chen <300606707+zijiecode@users.noreply.github.com>
Co-authored-by: ethche <ethche@gmail.com>
Co-authored-by: jacky.cheng <yichiche@amd.com>
Co-authored-by: jianan-gu <jianan.gu@intel.com>
2026-10-18 01:00:24 -07:00
Mohammad Miadh AngkadandMohammad Angkad 3fc08b939a [Feat] server-args: Add --enable-hicache alias for --enable-hierarchical-cache (#44849)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-10-17 13:22:34 +08:00
feixiangpengandzijiexia 9590dc80cc [Doc] hicache: Explain HiCache host pool sizing with a storage backend (#44790)
Co-authored-by: zijiexia <37504505+zijiexia@users.noreply.github.com>
2026-10-17 11:28:37 +08:00
f05da43d44 [Feat] [AMD] disagg: Support DSV4 decode radix cache on ROCm unified KV (bf16/fp8, DSpark) (#30929)
Co-authored-by: HAI <hixiao@gmail.com>
Co-authored-by: Duyi-Wang <duyi.wang@amd.com>
2026-10-16 02:39:27 -07:00
ec7ab9d0c2 [Feat] [AMD] monokernels: one runtime switch for vendor monokernel plugins (#44353)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: wunhuang <wunhuang@amd.com>
2026-10-15 15:50:25 -07:00
snadampal aa5c7e887c [Feat] docker: Add AWS EFA image variants built on published images (#41006)
Signed-off-by: Sunita Nadampalli <nadampal@amazon.com>
2026-10-15 13:19:22 -07:00
Xinyuan TongandNihar Patel 7dc2930585 [Feature] Add Aaronson-Gumbel text watermarking (#37577)
Co-authored-by: Nihar Patel <nihpat95@gmail.com>
2026-10-13 12:47:50 -07:00
Khoa Phamandvincentzed 8bf145d119 [DCP] Decode context parallelism for DSA (GLM-5.2 / GLM-5.3) with MTP (#43735)
Co-authored-by: vincentzed <207368749+vincentzed@users.noreply.github.com>
2026-10-13 11:35:34 -07:00
Ho-Ren (Jack) ChuangandXiaoyu Zhang 5100f8d11d [DSA] Q8KV8 FP8 Sparse Prefill on GLM-5.3-Flash with BF16 KV Cache (#41790)
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
2026-10-13 20:30:13 +08:00
Jimmy Shong 3eebf987ed [PD] Support heterogeneous TP for hybrid SWA KV transfer (#40909) 2026-10-12 23:21:54 -07:00
Anupa SajikumarandSiju Samuel 8d491fbc3e [XPU] Support decode context parallelism (DCP) on Intel XPU (#34355)
Co-authored-by: Siju Samuel <siju.samuel@intel.com>
2026-10-12 15:50:33 +08:00
Baizhou Zhang 62c853b654 docs: clarify prefill CP terminology and simplify wording (#43787) 2026-10-12 00:24:57 -07:00
Shunkangz de487f8039 [kv-shard 3/4] Enable Control Plane C (#40102) 2026-10-10 22:53:58 +08:00
tao 6e8f3e09dd [Model Loader] Add InstantTensor loader support (#40453) 2026-10-09 23:01:59 -07:00
Yuwei An 8b2aaf7060 [Refactor][TCPCG] Retire backend selection and configuration (3/9) (#42287) 2026-10-09 22:42:35 -07:00
Xinyuan Tong 91b3692a4c docs: add thinking budget guide (#41601) 2026-10-09 17:50:34 +08:00
Liangsheng Yin b4fe282836 [MemCache] Delete ChunkCache (#43201) 2026-10-08 17:31:16 -07:00
e902207a4e [Speculative] Support block verification for the DFlash family (#43012)
Co-authored-by: maocheng <mao.cheng@MacBook-Pro-MD663G60VV.local>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 13:59:52 -07:00
f795fe352d [Test] Add a docs drift check for server_arguments.mdx (#38520)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
2026-10-08 00:37:22 -07:00
Po-Han Huang (NVIDIA)andClaude Opus 5.5 3eec178e48 [Spec][MegaMoE] Let the speculative draft choose its own W4A4 MXFP4 MegaMoE MMA type (#42022)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 14:09:57 -07:00
e446698cdd [SM120&90] Add CUDA fused Triton sparse-MLA prefill backend for DSA (#32779)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
Co-authored-by: Po-Han Huang <pohanh@nvidia.com>
2026-10-05 21:08:05 -07:00
maocheng23 57826a62ea [Speculative] Add opt-in block verification (#42297) 2026-10-05 11:49:23 -07:00
Cheng Wan 1f299e9f2f [Refactor] Build layer stacks in order with append_stages (#42481) 2026-10-04 16:40:24 -07:00
Cheng Wan 10edb05e72 [Refactor] Let the remaining placement consumers read the parallel context (#42348) 2026-10-03 14:59:32 -07:00
aaf7ebe077 Advertise the KV-event replay endpoint in /server_info (#42274)
Co-authored-by: Kan Wu <kan.wu@radixark.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 04:42:33 +08:00
5b5d721239 [PD] Add opt-in prefill-complete decode KV allocation (#40703)
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
Co-authored-by: Po-Han Huang <pohanh@nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
2026-10-03 21:03:09 +08:00
cdadb1c79a [unified-memory] Token-major dense views for the unified memory pool (3/7) (#38592)
Co-authored-by: Caihua Li <caihua.li@bytedance.com>
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
2026-10-01 17:09:38 -07:00
Cheng Wan a9c97c9f69 [Feature] Add --attn-dp-size and deprecate --enable-dp-attention (#41818) 2026-09-30 13:34:18 -07:00
Cheng Wan 6dad152490 [Refactor] Document layer boundary contracts and integration (#41557) 2026-09-28 20:18:27 -07:00
Cheng Wan 9b89b96230 [Refactor] Migrate specialized decoder and overlap boundaries (#41553) 2026-09-28 20:17:17 -07:00
Cheng Wan 49b5e76e45 [Refactor] Construct independent decoder stage boundaries (#41552) 2026-09-28 20:17:00 -07:00
77091cea68 [Radix Cache] Sync Rust TreeCore and make it the default (#39627)
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
Co-authored-by: Zhangheng <hzh0425@apache.org>
Co-authored-by: alphabetc1 <alphabetc1@users.noreply.github.com>
2026-09-29 10:18:57 +08:00
Baizhou Zhang 7ee7bef79d docs: add prefill context parallelism guide and design draft (#39354) 2026-09-28 00:26:02 -07:00
78eee88113 [Spec] Add LiLiCorr: a candidate-lattice reranker for DFlash drafts (#37462)
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
Co-authored-by: kpham-sgl <khoa.pham@radixark.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 15:36:48 -07:00
Erik WijmansandEthan Su c421d16563 [sglang][lora] Support DP attention in LoRA backends (#36389)
Co-authored-by: Ethan (Yusheng) Su <yushengsu@radixark.ai>
2026-09-25 12:03:47 -10:00
f4d9d2e670 [RL] Add RL weight-update sessions and support updating spec draft runners (#40777)
Co-authored-by: Jiajun Li <guapisolo@gmail.com>
Co-authored-by: Nan Jiang <59716405+nanjiangwill@users.noreply.github.com>
Co-authored-by: maocheng23 <maocheng@berkeley.edu>
Co-authored-by: Byron Hsu <byron+per@periodiclabs.ai>
Co-authored-by: JD-ETH <jaedon.guo@gmail.com>
Co-authored-by: Letian Ruan <ruanlt420@gmail.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-09-23 20:04:57 -07:00
Liangsheng Yin 78980a3b0b [misc] Remove deprecated endpoints, env vars and aliases past two releases (#40795) 2026-09-22 18:14:45 -07:00
Liangsheng Yin 4ce23542bf [HiCache] Remove the unused HiRadixCache (#40787) 2026-09-22 16:17:12 -07:00
Liangsheng Yin d34f7b2326 [mem_cache] Clean up SWA/Mamba radix cache leftovers and drop SGLANG_ENABLE_UNIFIED_RADIX_TREE (#40780) 2026-09-22 15:40:57 -07:00
Po-Han Huang (NVIDIA) 771c9d782d [PD] Validate Mooncake EFA allocator compatibility (#39973) 2026-09-22 15:12:29 +08:00
Guangda LiuandGuangda Liu 04c0913434 [HiSparse] Add MHA hisparse support for MiniMax M3 (#31446)
Co-authored-by: Guangda Liu <bingps@users.noreply.github.com>
2026-09-22 13:28:03 +08:00
b63f8416b3 [Feature] Gigachat 3.5 support (#29189)
Co-authored-by: Stanislav Petrov <stapetrov@sberbank.ru>
Co-authored-by: Viacheslav Barinov <vvadbarinov@sberbank.ru>
Co-authored-by: Viacheslav <viacheslav.teh@gmail.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
2026-09-21 14:37:55 +08:00