Commit Graph
72 Commits
Author SHA1 Message Date
e902207a4e [Speculative] Support block verification for the DFlash family (#43012)
Co-authored-by: maocheng <mao.cheng@MacBook-Pro-MD663G60VV.local>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 13:59:52 -07:00
f795fe352d [Test] Add a docs drift check for server_arguments.mdx (#38520)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
2026-10-08 00:37:22 -07:00
Po-Han Huang (NVIDIA)andClaude Opus 5.5 3eec178e48 [Spec][MegaMoE] Let the speculative draft choose its own W4A4 MXFP4 MegaMoE MMA type (#42022)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 14:09:57 -07:00
e446698cdd [SM120&90] Add CUDA fused Triton sparse-MLA prefill backend for DSA (#32779)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
Co-authored-by: Po-Han Huang <pohanh@nvidia.com>
2026-10-05 21:08:05 -07:00
maocheng23 57826a62ea [Speculative] Add opt-in block verification (#42297) 2026-10-05 11:49:23 -07:00
Cheng Wan 1f299e9f2f [Refactor] Build layer stacks in order with append_stages (#42481) 2026-10-04 16:40:24 -07:00
Cheng Wan 10edb05e72 [Refactor] Let the remaining placement consumers read the parallel context (#42348) 2026-10-03 14:59:32 -07:00
aaf7ebe077 Advertise the KV-event replay endpoint in /server_info (#42274)
Co-authored-by: Kan Wu <kan.wu@radixark.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 04:42:33 +08:00
5b5d721239 [PD] Add opt-in prefill-complete decode KV allocation (#40703)
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
Co-authored-by: Po-Han Huang <pohanh@nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
2026-10-03 21:03:09 +08:00
cdadb1c79a [unified-memory] Token-major dense views for the unified memory pool (3/7) (#38592)
Co-authored-by: Caihua Li <caihua.li@bytedance.com>
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
2026-10-01 17:09:38 -07:00
Cheng Wan a9c97c9f69 [Feature] Add --attn-dp-size and deprecate --enable-dp-attention (#41818) 2026-09-30 13:34:18 -07:00
Cheng Wan 6dad152490 [Refactor] Document layer boundary contracts and integration (#41557) 2026-09-28 20:18:27 -07:00
Cheng Wan 9b89b96230 [Refactor] Migrate specialized decoder and overlap boundaries (#41553) 2026-09-28 20:17:17 -07:00
Cheng Wan 49b5e76e45 [Refactor] Construct independent decoder stage boundaries (#41552) 2026-09-28 20:17:00 -07:00
77091cea68 [Radix Cache] Sync Rust TreeCore and make it the default (#39627)
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
Co-authored-by: Zhangheng <hzh0425@apache.org>
Co-authored-by: alphabetc1 <alphabetc1@users.noreply.github.com>
2026-09-29 10:18:57 +08:00
Baizhou Zhang 7ee7bef79d docs: add prefill context parallelism guide and design draft (#39354) 2026-09-28 00:26:02 -07:00
78eee88113 [Spec] Add LiLiCorr: a candidate-lattice reranker for DFlash drafts (#37462)
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
Co-authored-by: kpham-sgl <khoa.pham@radixark.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 15:36:48 -07:00
Erik WijmansandEthan Su c421d16563 [sglang][lora] Support DP attention in LoRA backends (#36389)
Co-authored-by: Ethan (Yusheng) Su <yushengsu@radixark.ai>
2026-09-25 12:03:47 -10:00
f4d9d2e670 [RL] Add RL weight-update sessions and support updating spec draft runners (#40777)
Co-authored-by: Jiajun Li <guapisolo@gmail.com>
Co-authored-by: Nan Jiang <59716405+nanjiangwill@users.noreply.github.com>
Co-authored-by: maocheng23 <maocheng@berkeley.edu>
Co-authored-by: Byron Hsu <byron+per@periodiclabs.ai>
Co-authored-by: JD-ETH <jaedon.guo@gmail.com>
Co-authored-by: Letian Ruan <ruanlt420@gmail.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-09-23 20:04:57 -07:00
Liangsheng Yin 78980a3b0b [misc] Remove deprecated endpoints, env vars and aliases past two releases (#40795) 2026-09-22 18:14:45 -07:00
Liangsheng Yin 4ce23542bf [HiCache] Remove the unused HiRadixCache (#40787) 2026-09-22 16:17:12 -07:00
Liangsheng Yin d34f7b2326 [mem_cache] Clean up SWA/Mamba radix cache leftovers and drop SGLANG_ENABLE_UNIFIED_RADIX_TREE (#40780) 2026-09-22 15:40:57 -07:00
Po-Han Huang (NVIDIA) 771c9d782d [PD] Validate Mooncake EFA allocator compatibility (#39973) 2026-09-22 15:12:29 +08:00
Guangda LiuandGuangda Liu 04c0913434 [HiSparse] Add MHA hisparse support for MiniMax M3 (#31446)
Co-authored-by: Guangda Liu <bingps@users.noreply.github.com>
2026-09-22 13:28:03 +08:00
b63f8416b3 [Feature] Gigachat 3.5 support (#29189)
Co-authored-by: Stanislav Petrov <stapetrov@sberbank.ru>
Co-authored-by: Viacheslav Barinov <vvadbarinov@sberbank.ru>
Co-authored-by: Viacheslav <viacheslav.teh@gmail.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
2026-09-21 14:37:55 +08:00
Sam (Kesen Li) d346b214fb feat(kv-cache): support SM100 NVFP4 GenMHA and speculative decoding (#36340) 2026-09-18 14:50:46 -07:00
a6cf05817f dsv4.1: remaining model and runtime integration (#38798)
Co-authored-by: BBuf <1182563586@qq.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Xiaoyu Zhang <xiaoyu.zhang@radixark.ai>
Co-authored-by: Yuwei An <ayw.sirius19@gmail.com>
Co-authored-by: Khoa Pham <khoa.pham@radixark.ai>
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: Zhichen Zeng <zczeng@uw.edu>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
2026-09-18 02:55:30 -07:00
yl3469andShuwen Wang 1a90ae6727 Add Agentic-Aware Tail-Optimized LRU eviction to the unified radix cache (#34012)
Co-authored-by: Shuwen Wang <47200617+alphabetc1@users.noreply.github.com>
2026-09-17 17:52:59 +08:00
amote-i bdf8886ad3 [NPU] [DOC] Rename NPU hardware to Ascend A2/A3 Series product (#39389) 2026-09-15 10:33:14 +08:00
6220f45d8e [PD] Add /v1/responses support to the HTTP PD router (#36141)
Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
2026-09-13 22:48:13 +08:00
Shangming CaiandXinyuan Tong 14a131ad5b [PD][OpenAI] Gate /v1/responses persistence behind --enable-response-store, default off (#39122)
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
2026-09-13 21:20:47 +08:00
cebca698e2 [Qwen3.8] Enable NVIDIA NVFP4 on DGX Spark with file-backed PLE and PDL router fix (#39126)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: rdxa <rdxa@rdxa-int-spark-01.yvb.moe>
Co-authored-by: Yangmin Li <yangminl@nvidia.com>
Co-authored-by: Manrique <nanomlm@gmail.com>
Co-authored-by: yhyang201 <yhyang201@gmail.com>
2026-09-13 16:23:41 +08:00
Khoa PhamandClaude Fable 5.1 6ba96d329f [DCP] Resolve --dcp-comm-backend to fi_a2a/a2a by default for every model (#39165)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-11 23:48:42 -07:00
Vignesh Sethuraman 0d1bea77da [AMD] Allow aiter attention backend for Gemma-4 (#38758) 2026-09-11 18:48:26 -07:00
Siju Samuel 67d3a2ea57 [XPU] Make checkpoint_engine worker device-agnostic (#32382) 2026-09-11 09:56:39 +08:00
1b77f498a0 [NVIDIA] Support flashinfer Mega Moe (#31470)
Co-authored-by: djns99 <40156487+djns99@users.noreply.github.com>
Co-authored-by: 云挚 <ningyunxiao.nyx@antgroup.com>
Co-authored-by: Yangmin Li <yangminl@nvidia.com>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
2026-09-10 00:22:47 -07:00
Brayden ZhongandBrayden Zhong c0b790cf7f Delete cutlass_mla, non-Marlin GPTQ, AWQ AOT kernel, and Dual Chunk Flash Attention (#32114)
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
2026-09-10 15:12:01 +08:00
Baizhou Zhang e54ff1efb9 [CP V1 Deprecation 5/5] Update prefill CP documentation (#36230) 2026-09-08 19:27:49 -07:00
Cheng Wan db272201a2 [Config] Retire get_global_server_args, and clear the deprecated flags that have a replacement (#38375) 2026-09-08 16:42:12 -07:00
HZY a6b542813f fix(glm-5.2-nvfp4): bound Mooncake synchronous transfer batches (#32758) 2026-09-08 22:14:33 +08:00
Shuwen WangandClaude Opus 5 4b44a1cde2 [Refactor] Let eviction policies take construction parameters (#37795)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-05 13:48:30 +08:00
320bdd1ee2 [Docs] Document --retraction-policy, --return-hidden-states-mode, --language-model-only (#37989)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: mottopanikeiku <fcetin@hawk.iit.edu>
Co-authored-by: alp <falpercetin@gmail.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
2026-09-05 05:26:28 +08:00
alpandXinyuan Tong 3e873c2110 [Docs] Clarify OpenAI chat template defaults (#32172)
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
2026-09-05 05:22:05 +08:00
72078cd7f5 [XPU] Support GPT-OSS MXFP4 checkpoints on Intel XPU (#35751)
Co-authored-by: Meng, Hengyu <hengyu.meng@intel.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-04 14:20:28 +08:00
59799a3687 [Simulator] Add high-fidelity CPU-based inference simulator (#33824)
Co-authored-by: zhouhaizhu.zhz <zhouhaizhu.zhz@alibaba-inc.com>
Co-authored-by: LinSiyuan814 <linsiyuan.lsy@alibaba-inc.com>
Co-authored-by: hzh0425 <hzh0425@apache.org>
2026-09-04 11:12:11 +08:00
Martin Hickey 3ffacf949b [Docs] [BugFix] Sync --tool-call-parser and --reasoning-parser lists with the code (#37788)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
2026-09-03 13:17:33 -07:00
2bb25dc18b [Speculative Decoding] Add native UNO serving support (#37667)
Co-authored-by: drproduck <drproduck@MacBook-Air-2.local>
Co-authored-by: BBuf <1182563586@qq.com>
2026-09-03 20:08:41 +08:00
Yuan Luoandluoyuan.luo 5b04408784 [MoE] Add FlashInfer SM90 MXFP4 W4A8 CUTLASS MoE (#34967)
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
2026-08-31 20:04:41 -07:00
Shuwen WangandClaude Opus 5 000c636342 docs: state that HiCache L2 is instance-private and only L3 is shared (#37050)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-29 23:47:50 +08:00
97ba99067d Publish per-scheduler load on a dedicated socket for load-aware routers (#34608)
Co-authored-by: Kangyan Zhou <zky314343421@gmail.com>
Co-authored-by: Zhangheng <hzh0425@apache.org>
2026-08-27 19:33:18 +08:00