Commit Graph
20441 Commits
Author SHA1 Message Date
72d886efb8 [Perf] DeepSeek-V4.1: Run the small-batch decode fusions at TP2 (#43838)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
Co-authored-by: nbommi19 <227267412+nbommi19@users.noreply.github.com>
Co-authored-by: Po-Han Huang <pohanh@nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
2026-10-19 21:17:05 -07:00
Mohammad Miadh AngkadandMohammad Angkad fdd53418a0 [CI] [NVIDIA] gb300: Drop Kimi-K2.5 and Qwen3.5 nightly tests (#45308)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-10-19 21:13:00 -07:00
Shuwen Wang c53b98d1fe [Fix] test-utils: Allow expected server exits in canary tests (#45172) 2026-10-19 21:07:03 -07:00
Mohammad Miadh AngkadandMohammad Angkad 5b683d6b72 [Fix] test: Add logical_forward_mode to the KDA prefill-graph stand-in batch (#45218)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-10-19 21:06:10 -07:00
Mingyi 4a0c6f2fe0 docs: promote SGLang Summit 2026 in README (#45271) 2026-10-20 03:36:56 +00:00
wxzhoucsandXiaoyu Zhang e3205080ce [Fix] qwen3.8: Fix batch-invariant kernels for Qwen3.8 Flash-Next (#42087)
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
2026-10-20 11:32:40 +08:00
1140a8dfd9 [Fix] [AMD] aiter: Fix garbage decode output for non-sliding-window models on the vectorized_5d KV layout (#43335)
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
2026-10-20 11:31:20 +08:00
Cao EandMa Mingfei 3c07ff2654 [Perf] [XPU] fp8: Optimize FP8 scale layouts (#40052)
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
2026-10-20 11:24:16 +08:00
huangtingwei 3eb4bd598f [Perf] hicache: Unify PP prefetch hit queries and buffer allocation (#43344) 2026-10-20 11:05:00 +08:00
Trevor Morris ec1fdc22a9 [Fix] [NVIDIA] docker: Fix CUDA 13.4 container by updating torch nightly (#45252) 2026-10-20 10:52:42 +08:00
kpjeeja 8683f8ad4a [Feat] [Intel] hicache: Enable L1<->L2 support on XPU (#32503) 2026-10-20 10:42:21 +08:00
Liangsheng Yin 5e630fa571 [Perf] deepseek-v4.1: Drop host syncs from encoder-replay folding and tail metadata (#45275) 2026-10-19 19:39:39 -07:00
Mohammad Miadh AngkadandMohammad Angkad af856717c2 [Fix] mamba: Skip zero-width conv pools in the MTP verify scatter (#45212)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-10-19 19:23:17 -07:00
Vladimir Serovandronnie_zheng 55b2aef7ad [Fix] [NPU] LoRA: Update for follow-up with changes in sgl-kernel-npu (#40908)
Co-authored-by: ronnie_zheng <zl19940307@163.com>
2026-10-20 05:16:05 +03:00
wutao 2c00b5d129 [Test] constrained: Cover JSON whitespace limits (#43981) 2026-10-20 10:12:22 +08:00
Gabriel Wuandnumber-eleven11 67d9ee156c [Fix] [SM120] quantization: Requantize block-FP8 MoE weights to UE8M0 for DeepGEMM (#43556)
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: number-eleven11 <1635219023@qq.com>
2026-10-19 19:06:21 -07:00
Liangsheng Yin ce329c85fa [Perf] deepseek-v4.1: Skip the request-window history check when invariant checks are off (#45276) 2026-10-19 18:38:44 -07:00
79ed13fc2c [Fix] glm47: Disable tool grammar for tool_choice="none" (#41233)
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: Po-Han Huang <pohanh@nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 09:36:06 +08:00
Lee Nau 1897575ce1 [Fix] runtime: Version FlashInfer caches for default-tactic tuning. (#44514) 2026-10-19 18:33:49 -07:00
Anihcandsunguozhong ebc71e650e [Perf] [NPU] mem_cache: Add optional ScatterPa KV cache writes (#40535)
Co-authored-by: sunguozhong <sun_gz2772@163.com>
2026-10-20 09:33:34 +08:00
71ee2dc950 [Fix] inkling: Return 400 instead of 500 for unfetchable/undecodable multimodal input (#40899)
Co-authored-by: apex-mochen <265981728+apex-mochen@users.noreply.github.com>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
Co-authored-by: apex-mochen <2756823972@qq.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Po-Han Huang <pohanh@nvidia.com>
2026-10-20 09:18:18 +08:00
Xiaoyu ZhangandClaude Opus 5.5 693d333306 [Refactor] kernels: Regroup operators by computation and dissolve ops/muse_spark (#45112)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 08:47:57 +08:00
Liangsheng Yin ab580b2e06 [CI] radix-cache: Split unified radix cache unit tests into topic files and a shared kit (#45236) 2026-10-19 17:00:42 -07:00
Cheng Wan 6b90e5716d [Refactor] moe: Scope shared-expert configuration to each model (#44913) 2026-10-19 16:59:37 -07:00
Cheng Wan 964ab2b280 [Refactor] moe: Split the two meanings of num_fused_shared_experts (#44912) 2026-10-19 16:57:15 -07:00
Cheng Wan 48ea486928 [Fix] moe: Renormalize per-rank shared-slot routing over routed experts only (#44904) 2026-10-19 16:56:21 -07:00
Cheng Wan 3a059de94c [Fix] moe: Count routed_scaling_factor once when top-k folds it (#44901) 2026-10-19 16:54:27 -07:00
Qiaolin Yuandzijiexia 28448a86d4 [Doc] cookbook: Default Qwen3.8-Flash-Next datacenter recipes to FP8 KV cache and QSA indexer (#45245)
Co-authored-by: zijiexia <37504505+zijiexia@users.noreply.github.com>
2026-10-19 15:57:22 -07:00
Liangsheng Yin 1238edd954 [Perf] deepseek-v4.1: Reset request-window tags without a host sync (#45247) 2026-10-19 15:45:17 -07:00
Mingyi 9a949403f1 [Doc] tpu: Introduce SGLang-TorchTPU in README and TPU guide (#45246) 2026-10-19 15:34:47 -07:00
ab258b6453 [Fix] symm-mem: Name the window registration in the scheduler watchdog dump (#44681)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-10-19 15:29:35 -07:00
Gabriel Wu 05d7f7a166 [Fix] attention: Honor selected FlashAttention implementation (#33007) 2026-10-19 15:27:52 -07:00
Jiuqiang Tang 49766a74f9 [Fix] attn_residual: int64 program ids in the score/combine kernels (int32 overflow at 37,451 tokens) (#43438) 2026-10-19 15:12:12 -07:00
0z5aandRichard Wang 80ea5ee638 [Fix] dLLM: Preserve prompt tails and literal mask tokens across prefix reuse (#39592)
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Co-authored-by: Richard Wang <wangrichard08@gmail.com>
2026-10-19 14:48:37 -07:00
Liangsheng Yin fdc2dd0d05 [Perf] dspark: Make the prefill SWA valid-from update sync-free (#45234) 2026-10-19 14:44:01 -07:00
Liu Keandhnyls2002 81ac15391c [Fix] metrics: record HTTP request metrics in tokenizer workers (#44663)
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-10-19 14:36:16 -07:00
Mohammad Miadh AngkadandMohammad Angkad 7656ea0e74 [CI] python: Move the remaining CUDA build paths off Python 3.10 (#44393)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-10-19 14:29:00 -07:00
Liangsheng Yin b20cb3665d [CI] radix-cache: Drop redundant unified radix cache unit tests (#45232) 2026-10-19 14:20:32 -07:00
Baizhou Zhang 642a671c30 [CI] tests: Re-enable muse spark B300 PD coverage (#44975) 2026-10-19 13:40:02 -07:00
Liangsheng Yin 8b1d1e0d74 [Perf] deepseek-v4.1: Check request-window SWA history without per-layer host syncs (#45223) 2026-10-19 13:34:52 -07:00
Rain Jiang aaa9388c91 [Fix] kda: Keep FlashInfer prefill off its grouped kernel's bf16/flat-dt_bias contract (#45018) 2026-10-19 13:16:54 -07:00
Liangsheng Yin f1a2ee763d [Perf] deepseek-v4.1: Run decoder replay tail graphs in prefill-graph steps and under DP attention (#45107) 2026-10-19 13:02:07 -07:00
Qiaolin Yu 5385866108 [Fix] moe: gated NVFP4 MoE intermediate alignment for FlashInfer TRTLLM (#44138) 2026-10-19 12:52:36 -07:00
Qiaolin Yu 9ab74b8517 [Fix] Qwen3.8next: Stop retaining per-size PLE outputs in breakable prefill CUDA graphs (#44150) 2026-10-19 12:41:39 -07:00
0ce41c5a06 [Fix] PD: retraction restores racing the in-flight decode forward (#42702)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
2026-10-19 11:55:04 -07:00
YAMY 722669e606 [Feat] [NVIDIA] megakernels: Add --enable-nvidia-megakernel and a layer for NVIDIA megakernel plugins (#44945) 2026-10-19 11:54:44 -07:00
AMD-yanfeiwang 678913a0f0 [Feat] sgl-model-gateway: Add --decode-dispatch-limit to pick PD decode workers near KV arrival (#44689) 2026-10-20 02:16:21 +08:00
jacky.cheng 80b86dbecb [Fix] [AMD] diffusion: Accept residual and skip_bias in the ROCm Conv3d-to-Conv2d VAE patch (#44672) 2026-10-19 11:07:37 -07:00
jacky.cheng 5ed306bd69 [Fix] [AMD] diffusion: Keep gfx1250 FlyDSL BF16 attention out of torch.compile (#44671) 2026-10-19 10:36:28 -07:00
lixiufei-leoandXiufei Li 4ef9765e76 [Feat] sgl-router: Add --pd-stream-after-prefill (#45102)
Co-authored-by: Xiufei Li <294029388+lixiufei-leo@users.noreply.github.com>
2026-10-20 01:13:25 +08:00