This website requires JavaScript.
52a88fb212
Avoid mutating ScheduleBatch fields in place (#30672 )
fzyzcjy
2026-07-15 14:27:08 +08:00
e77d95c3d5
Pass per-forward overrides to ForwardBatch.init_new as explicit arguments (#30670 )
fzyzcjy
2026-07-15 14:25:59 +08:00
861d97d24d
Remove dead ScheduleBatch fields and avoid inplace seq_lens bump (#30669 )
fzyzcjy
2026-07-15 14:23:31 +08:00
a3194d3585
[AMD] Remove ROCm page_first+kernel -> layer_first HiCache fallback (follow-up to #28534 ) (#30622 )
AMD-yanfeiwang
2026-07-15 13:48:16 +08:00
0832d856ca
[Bugfix] fix quickreduce acc error in cudagraph mode (#29508 )
haoyangli0109
2026-07-15 13:16:02 +08:00
4aadf94146
[Kernel] Relocate vendored fla and mamba kernel trees to sglang.kernels (RFC #29630 , Phase 2.5, 7/7) (#30795 )
Xiaoyu Zhang and Claude Fable 5
2026-07-15 12:52:15 +08:00
23f2b77d82
Make UTs compatible for XPU (#27106 )
ANSHUMAN TRIPATHY
2026-07-15 10:05:56 +05:30
c00131ebaa
[Kernel] Migrate linear-attention, MiniMax-sparse and diffusion kernels to sglang.kernels (RFC #29630 , Phase 2.5, 6/7) (#30793 )
Xiaoyu Zhang and Claude Fable 5
2026-07-15 11:21:36 +08:00
ba5be86d42
[Kernel] Migrate DSA + DSV4 attention kernels to sglang.kernels (RFC #29630 , Phase 2.5, 5/7) (#30792 )
Xiaoyu Zhang and Claude Fable 5
2026-07-15 11:11:22 +08:00
4ae9cc3c81
Fix gate stride for 4D decode layouts (#31231 )
2026-07-14 20:06:50 -07:00
b4fdce3b63
Fix post-capture KV sizing for SWA pools (#31092 )
Lianmin Zheng
2026-07-14 20:06:15 -07:00
532cd337ed
[Intel GPU] DeepSeek V4 12/N: use sgl-kernel implementation of silu_and_mul_clamp to run on XPU (#28428 )
Polisetty V R K Jyothendra Varma
2026-07-15 08:33:53 +05:30
46b675ce70
[Intel GPU] DeepSeek V4 11/N: support fp8_paged_mqa_logits_triton from sgl-kernel to run on XPU (#28059 )
Polisetty V R K Jyothendra Varma and Ma Mingfei
2026-07-15 08:33:35 +05:30
aafa706f8f
[AMD] Update qwen3.5 cookbook (#31258 )
Thomas Wang
2026-07-15 10:59:12 +08:00
43124cdd90
fix: fix image benchmark backend parity (#30867 )
Mick
2026-07-15 10:11:22 +08:00
b22f20b660
[CI] Fix CUDA 12 NVIDIA wheel cleanup (#31035 )
Hank Han
2026-07-15 10:06:11 +08:00
76dc427806
[Spec] Single-source num_tokens_per_req derivation and access (#31013 )
Liangsheng Yin
2026-07-14 18:41:08 -07:00
ca0ee3f1a8
[Spec] Consolidate spec-worker weight updates into BaseSpecWorker via draft_runners (#31078 )
Liangsheng Yin
2026-07-14 18:39:46 -07:00
a9cf5e68e6
[DSV4] Remove per-step seqlen D2H from speculative to make overlap scheduler work (#30365 )
weireweire and weireweire
2026-07-15 09:01:22 +08:00
b8a00e2ec8
docs: sync LMSYS SGLang blog cards (#31242 )
sglang-bot and sglang-bot
2026-07-14 17:49:30 -07:00
90f10cbe26
[diffusion] post_training: Add LoRA IPC weight sync via lora_merge mode (#31029 )
WenhaoZhang
2026-07-15 08:42:54 +08:00
50d1edaa7f
[misc] Move SchedulerRecvSkipper into scheduler_components (#31222 )
Liangsheng Yin
2026-07-14 16:35:04 -07:00
21a6d08557
ci: strip invisible Unicode format chars from slash-command input (#31234 )
Alison Shao
2026-07-14 16:33:45 -07:00
771e386332
Disable flaky DSV4-Flash FP4 BCG determinism test (nondeterminism from #30898 idle-rank dummy extend) (#31125 )
Alison Shao
2026-07-14 16:21:10 -07:00
463a3f4248
[Mamba] Support configurable conv-window layouts (#31059 )
paulzhang-tm
2026-07-14 14:41:10 -07:00
08c46e1f1a
Add dummy forward batch preparation hook (#31070 )
paulzhang-tm
2026-07-14 14:30:31 -07:00
0d89564d27
Support scheduler_recv_interval (recv skipper) under DP-attention (#30457 )
2026-07-14 16:02:58 -05:00
bdc9848c25
[Doc]Standardize the names of PyTorch NPU-related software throughout the documentation by replacing them all with TorchNPU. (#29886 )
2026-07-15 02:56:08 +08:00
cb47a68717
[PD] Stride KV token->page indices on device before D2H copy (#31173 )
cctry and cctry
2026-07-14 10:08:30 -07:00
a5c3e0283f
[Bug fix] Account for KV replication fan-out in transfer-byte metrics (#30351 )
hunhokim and Hun-ho Kim
2026-07-15 02:08:24 +09:00
271e5ef5c3
[CI] Fix Flash MLA SM120 test import path (#31199 )
Mohammad Miadh Angkad
2026-07-15 00:01:23 +08:00
04af94d150
fix: avoid tilelang cuda runtime pollution (#30870 )
Mick
2026-07-14 22:30:27 +08:00
f853293440
[NPU] Fix CPU device for node topology probe (#30619 )
Peng Xingchen
2026-07-14 22:17:54 +08:00
31548781e0
[CPU] bypass scoring_func argument in topk for cpu device (#31110 )
Ma Mingfei
2026-07-14 21:48:22 +08:00
bbb5702a3c
fix: avoid double KV release on disaggregated prefill grammar errors (#30937 )
Kevin Flansburg and Shangming Cai
2026-07-14 05:16:58 -07:00
43241b7f3f
[diffusion] model: support fal Ideogram V4 Fast and Instant (#31177 )
Mick
2026-07-14 19:51:35 +08:00
ee000f6734
[CI] Fix SGLANG_JIT_KERNEL_RUN_FULL_TESTS never activating the nightly full jit-kernel sweep (#31042 )
Xiaoyu Zhang and Claude Fable 5
2026-07-14 17:32:29 +08:00
1a35440c4a
[Kernel] Migrate generic attention kernels to sglang.kernels (RFC #29630 , Phase 2.5, 4/7) (#30789 )
Xiaoyu Zhang and Claude Fable 5
2026-07-14 16:53:46 +08:00
a5a71c6c26
Enhance mechanical-refactor-verify skill with a whole-chain verifier, new relocation primitives, and generator inference (#30585 )
fzyzcjy
2026-07-14 16:45:48 +08:00
0fe2dbd42c
Split initialize() into orchestration helpers (#31169 )
fzyzcjy
2026-07-14 16:09:00 +08:00
64a70c9097
[AMD] jit_kernel: complete utils.cuh HIP-compat (cudaDevAttr / cudaDeviceGetAttribute) (#31143 )
Ankith Averineni
2026-07-14 01:06:36 -07:00
bf04cc9b14
Extract cuda-graph setup into a module (#31168 )
fzyzcjy
2026-07-14 16:04:45 +08:00
ed2fcd3201
Extract attention-backend setup into a module (#31167 )
fzyzcjy
2026-07-14 16:03:48 +08:00
54f99a21d5
Narrow component dependencies to injected fields instead of ModelRunner (#31166 )
fzyzcjy
2026-07-14 16:03:07 +08:00
6999007a13
Drop ModelRunner's duplicated parallel-degree fields and read them via self.ps (#31165 )
fzyzcjy
2026-07-14 16:02:40 +08:00
cfd17301a8
Extract per-architecture KV-cache pool builders into KVCacheConfigurator (#31163 )
fzyzcjy
2026-07-14 16:02:09 +08:00
d6cf2908ce
Introduce KVCacheConfigurator and migrate KV-cache config logic (#31162 )
fzyzcjy
2026-07-14 16:01:45 +08:00
725920915f
Introduce ModelRunner.ps ParallelState (#31161 )
fzyzcjy
2026-07-14 16:01:14 +08:00
1dc48c2c3b
Absorb capturer setup and extract the shared-mooncake gate (#31160 )
fzyzcjy
2026-07-14 16:00:51 +08:00
08798dba0d
Extract MoE/EP setup into a moe_ep_setup module (#31159 )
fzyzcjy
2026-07-14 16:00:24 +08:00
440aebdfe0
Extract small single-function helpers into modules (#31158 )
fzyzcjy
2026-07-14 16:00:01 +08:00
17c04602c6
Extract spec aux-hidden-state resolution into a module (#31157 )
fzyzcjy
2026-07-14 15:59:35 +08:00
39e508b7fc
Extract layer-index setup into a module (#31156 )
fzyzcjy
2026-07-14 15:59:07 +08:00
5b540b16de
Extract load_model helpers into a load_model_utils module (#31155 )
fzyzcjy
2026-07-14 15:58:40 +08:00
d15f6a9ac3
Introduce NgramEmbeddingManager component (#31154 )
fzyzcjy
2026-07-14 15:58:08 +08:00
0f20f52e5e
Introduce RemoteInstanceWeightTransporter component (#31153 )
fzyzcjy
2026-07-14 15:57:43 +08:00
caa85ea022
Extract init_torch_distributed and refactor into functions (#31152 )
fzyzcjy
2026-07-14 15:56:57 +08:00
205a2f2de4
Move LoRA cuda-graph buffers and logging into LoRAManager (#31151 )
fzyzcjy
2026-07-14 15:56:18 +08:00
e20c346541
Extract hybrid-arch helpers into configs.hybrid_arch and ModelConfig (#31150 )
fzyzcjy
2026-07-14 15:55:32 +08:00
c9b4081016
Extract expert location updating into EPLBManager (#31149 )
fzyzcjy
2026-07-14 15:54:40 +08:00
6507d4a090
Introduce WeightUpdater and WeightExporter components (#31148 )
fzyzcjy
2026-07-14 15:53:33 +08:00
45dfa318fb
Extract kv cache dtype configuration into mem_cache (#31147 )
fzyzcjy
2026-07-14 15:52:39 +08:00
cad8fe7a66
Extract leaf helpers out of ModelRunner into utility modules (#31146 )
fzyzcjy
2026-07-14 15:52:03 +08:00
b1a60ad00d
Clean up ModelRunner by renaming effective-token property and remove dead code (#31145 )
fzyzcjy
2026-07-14 15:50:53 +08:00
2cf753c4fe
Clarify ModelRunner.dp_size into attn_dp_size (#31142 )
fzyzcjy
2026-07-14 15:49:05 +08:00
afa3c06d1f
Using UnifiedRadixTree by default for SWA, Mamba, and DSA models (#30468 )
Zhangheng and ispobock
2026-07-14 15:17:08 +08:00
7e0b29ac03
docs: add VLA card image to cookbook overview (#31132 )
zijiexia and Claude Fable 5
2026-07-14 00:08:49 -07:00
96c2ebc58b
[docs] Note the default dsa-topk-backend on all DSA-model cookbook pages (#31124 )
zijiexia and Claude Fable 5
2026-07-14 00:04:47 -07:00
2ced88238a
[NPU] [BUGFIX] Fix input parameters of swiglu_oai operator (#30458 )
gjsheu
2026-07-14 14:16:24 +08:00
702bddcee8
[Model] Add support for JetBrains' Mellum v2 code generation model (#27375 )
2026-07-14 05:54:38 +00:00
7e229e2a81
support GLM-5.2 MTP index sharing with prefill CP (#30992 )
Yuxuan Zhang and Baizhou Zhang
2026-07-14 12:11:27 +08:00
1b4176cc46
[PD] Fix optimistic prefill inflight-queue hangs on parked/aborted reqs (#31075 )
cctry
2026-07-13 20:34:44 -07:00
23b2c6f1ce
docs: fix diffusion cookbook overview cards (#31101 )
Mick
2026-07-14 10:40:42 +08:00
78dc581518
[bug-fix] Stabilize GLM-5.2 MTP IndexShare across PD and CUDA graph replay (#30839 )
2026-07-14 10:37:07 +08:00
41ad0d9c26
Allow prefill breakable CUDA graph for Qwen3.5 via multimodal opt-in allowlist (#30620 )
YAMY
2026-07-13 18:39:44 -07:00
7431f35fd8
Delete CUTLASS FP8 blockwise for SM90 and SM100, move SM120 to JIT and add SwapAB (#30438 )
2026-07-13 18:31:32 -07:00
c124bec99d
[CI] Disable gated Llama-2 EAGLE spec tests to unblock Xeon CPU CI (#30995 )
Haotong Zou
2026-07-14 09:28:47 +08:00
9756f768a6
Refactor FP4 quantization and remove deprecated JIT kernels (#30448 )
Brayden Zhong and root
2026-07-13 18:22:07 -07:00
0c01971eeb
[Intel GPU] DeepSeek V4 9/N: use sgl-kernel implementation of hadamard_transform on XPU (#28046 )
2026-07-14 06:51:31 +05:30
e9ef06c560
[Kernel] Migrate top-level srt/layers stray kernels to sglang.kernels (RFC #29630 , Phase 2.5, 3/7) (#30787 )
Xiaoyu Zhang and Claude Fable 5
2026-07-14 09:20:59 +08:00
e489685509
[Intel GPU] DeepSeek V4 5/N: Use sgl-kernel implementation of fused_q_indexer_rope_hadamard_quant to run on XPU (#27873 )
Polisetty V R K Jyothendra Varma
2026-07-14 06:50:50 +05:30
ee464fedc6
[Kernel] Migrate scattered MoE kernels to sglang.kernels (RFC #29630 , Phase 2.5, 2/7) (#30786 )
Xiaoyu Zhang and Claude Fable 5
2026-07-14 09:03:21 +08:00
464fe1b77c
[CI] Use torch.testing.assert_close in custom-all-reduce test (~1400x faster compare) (#31080 )
Alison Shao
2026-07-13 17:53:20 -07:00
33f83011e0
fix: fix Kimi-VL encoder parallelism (#30869 )
Mick
2026-07-14 08:44:06 +08:00
423b8485fb
[Quantization] add humming quantization kernel (#23754 )
2026-07-14 08:42:56 +08:00
4c997310f5
[Kernel] Hotfix: update sgl-kernel imports of relocated fp8_kernel (RFC #29630 #30784 ) (#31089 )
Xiaoyu Zhang and Claude Opus 4.8
2026-07-14 08:41:23 +08:00
cfe4eefabb
[diffusion] model: support LongLive 2.0 T2V and I2V inference (#27639 )
2026-07-13 17:39:30 -07:00
2f79d334f2
[Bugfix] Fix DeepSeek ForwardFlags across custom op boundary (#30987 )
Mohammad Miadh Angkad
2026-07-14 08:19:44 +08:00
cfc3d0555e
Fix ModelOpt NVFP4 scalar scales for merged linears (#29151 )
Po-Han Huang (NVIDIA)
2026-07-14 07:14:20 +08:00
a909077d22
Return top-p/top-k sampling mask/nucleas (#27408 )
hari-hm
2026-07-13 15:58:51 -07:00
0ee236ebdf
more fixes for Nemotron 3 parser for tool call and force nonempty content (#30533 )
Brayden Zhong
2026-07-13 15:50:27 -07:00
50ed4c011f
Remove legacy Sphinx docs/ and finish the Mintlify cutover (#28964 )
zijiexia
2026-07-13 15:06:08 -07:00
e2728ac504
[Spec] Remove dead padded_static_len and stale SGLANG_ENABLE_SPEC_V2 references (#30998 )
Liangsheng Yin
2026-07-13 15:30:31 -05:00
47030b28be
Fix MockDSV4ModelRunner missing spec_algorithm (#31056 )
Jialin Ouyang
2026-07-13 13:24:28 -07:00
2ab531cfcf
fa3/fa4: sync-free for all backends and phases (#29589 )
Liangsheng Yin and ronhuafeng
2026-07-13 15:09:57 -05:00
92fc692411
fix: include OpenSSL headers in runtime image (#31064 )
ishandhanani
2026-07-13 13:09:41 -07:00
f49cbbd67d
Fix GLM/DeepSeek NVFP4 + flashinfer_trtllm long-context "!!!!" collapse (NaN routing) (#31001 )
Khoa Pham
2026-07-13 12:54:38 -07:00
86c59ac1aa
Revert "[Tiny] Enable Full Cuda Graph with Page size = 1" (#31062 )
Yuwei An
2026-07-13 12:23:44 -07:00
805385414e
chore: bump docs install version to 0.5.15 (#31058 )
sglang-bot and sglang-bot
2026-07-13 11:58:48 -07:00
48fff1f2bd
[Spec] Deduplicate spec-v2 worker lifecycle boilerplate into BaseSpecWorker (#31008 )
Liangsheng Yin
2026-07-13 13:48:40 -05:00