This website requires JavaScript.
a5c3e0283f
[Bug fix] Account for KV replication fan-out in transfer-byte metrics (#30351 )
hunhokim and Hun-ho Kim
2026-07-15 02:08:24 +09:00
271e5ef5c3
[CI] Fix Flash MLA SM120 test import path (#31199 )
Mohammad Miadh Angkad
2026-07-15 00:01:23 +08:00
04af94d150
fix: avoid tilelang cuda runtime pollution (#30870 )
Mick
2026-07-14 22:30:27 +08:00
f853293440
[NPU] Fix CPU device for node topology probe (#30619 )
Peng Xingchen
2026-07-14 22:17:54 +08:00
31548781e0
[CPU] bypass scoring_func argument in topk for cpu device (#31110 )
Ma Mingfei
2026-07-14 21:48:22 +08:00
bbb5702a3c
fix: avoid double KV release on disaggregated prefill grammar errors (#30937 )
Kevin Flansburg and Shangming Cai
2026-07-14 05:16:58 -07:00
43241b7f3f
[diffusion] model: support fal Ideogram V4 Fast and Instant (#31177 )
Mick
2026-07-14 19:51:35 +08:00
ee000f6734
[CI] Fix SGLANG_JIT_KERNEL_RUN_FULL_TESTS never activating the nightly full jit-kernel sweep (#31042 )
Xiaoyu Zhang and Claude Fable 5
2026-07-14 17:32:29 +08:00
1a35440c4a
[Kernel] Migrate generic attention kernels to sglang.kernels (RFC #29630 , Phase 2.5, 4/7) (#30789 )
Xiaoyu Zhang and Claude Fable 5
2026-07-14 16:53:46 +08:00
a5a71c6c26
Enhance mechanical-refactor-verify skill with a whole-chain verifier, new relocation primitives, and generator inference (#30585 )
fzyzcjy
2026-07-14 16:45:48 +08:00
0fe2dbd42c
Split initialize() into orchestration helpers (#31169 )
fzyzcjy
2026-07-14 16:09:00 +08:00
64a70c9097
[AMD] jit_kernel: complete utils.cuh HIP-compat (cudaDevAttr / cudaDeviceGetAttribute) (#31143 )
Ankith Averineni
2026-07-14 01:06:36 -07:00
bf04cc9b14
Extract cuda-graph setup into a module (#31168 )
fzyzcjy
2026-07-14 16:04:45 +08:00
ed2fcd3201
Extract attention-backend setup into a module (#31167 )
fzyzcjy
2026-07-14 16:03:48 +08:00
54f99a21d5
Narrow component dependencies to injected fields instead of ModelRunner (#31166 )
fzyzcjy
2026-07-14 16:03:07 +08:00
6999007a13
Drop ModelRunner's duplicated parallel-degree fields and read them via self.ps (#31165 )
fzyzcjy
2026-07-14 16:02:40 +08:00
cfd17301a8
Extract per-architecture KV-cache pool builders into KVCacheConfigurator (#31163 )
fzyzcjy
2026-07-14 16:02:09 +08:00
d6cf2908ce
Introduce KVCacheConfigurator and migrate KV-cache config logic (#31162 )
fzyzcjy
2026-07-14 16:01:45 +08:00
725920915f
Introduce ModelRunner.ps ParallelState (#31161 )
fzyzcjy
2026-07-14 16:01:14 +08:00
1dc48c2c3b
Absorb capturer setup and extract the shared-mooncake gate (#31160 )
fzyzcjy
2026-07-14 16:00:51 +08:00
08798dba0d
Extract MoE/EP setup into a moe_ep_setup module (#31159 )
fzyzcjy
2026-07-14 16:00:24 +08:00
440aebdfe0
Extract small single-function helpers into modules (#31158 )
fzyzcjy
2026-07-14 16:00:01 +08:00
17c04602c6
Extract spec aux-hidden-state resolution into a module (#31157 )
fzyzcjy
2026-07-14 15:59:35 +08:00
39e508b7fc
Extract layer-index setup into a module (#31156 )
fzyzcjy
2026-07-14 15:59:07 +08:00
5b540b16de
Extract load_model helpers into a load_model_utils module (#31155 )
fzyzcjy
2026-07-14 15:58:40 +08:00
d15f6a9ac3
Introduce NgramEmbeddingManager component (#31154 )
fzyzcjy
2026-07-14 15:58:08 +08:00
0f20f52e5e
Introduce RemoteInstanceWeightTransporter component (#31153 )
fzyzcjy
2026-07-14 15:57:43 +08:00
caa85ea022
Extract init_torch_distributed and refactor into functions (#31152 )
fzyzcjy
2026-07-14 15:56:57 +08:00
205a2f2de4
Move LoRA cuda-graph buffers and logging into LoRAManager (#31151 )
fzyzcjy
2026-07-14 15:56:18 +08:00
e20c346541
Extract hybrid-arch helpers into configs.hybrid_arch and ModelConfig (#31150 )
fzyzcjy
2026-07-14 15:55:32 +08:00
c9b4081016
Extract expert location updating into EPLBManager (#31149 )
fzyzcjy
2026-07-14 15:54:40 +08:00
6507d4a090
Introduce WeightUpdater and WeightExporter components (#31148 )
fzyzcjy
2026-07-14 15:53:33 +08:00
45dfa318fb
Extract kv cache dtype configuration into mem_cache (#31147 )
fzyzcjy
2026-07-14 15:52:39 +08:00
cad8fe7a66
Extract leaf helpers out of ModelRunner into utility modules (#31146 )
fzyzcjy
2026-07-14 15:52:03 +08:00
b1a60ad00d
Clean up ModelRunner by renaming effective-token property and remove dead code (#31145 )
fzyzcjy
2026-07-14 15:50:53 +08:00
2cf753c4fe
Clarify ModelRunner.dp_size into attn_dp_size (#31142 )
fzyzcjy
2026-07-14 15:49:05 +08:00
afa3c06d1f
Using UnifiedRadixTree by default for SWA, Mamba, and DSA models (#30468 )
Zhangheng and ispobock
2026-07-14 15:17:08 +08:00
7e0b29ac03
docs: add VLA card image to cookbook overview (#31132 )
zijiexia and Claude Fable 5
2026-07-14 00:08:49 -07:00
96c2ebc58b
[docs] Note the default dsa-topk-backend on all DSA-model cookbook pages (#31124 )
zijiexia and Claude Fable 5
2026-07-14 00:04:47 -07:00
2ced88238a
[NPU] [BUGFIX] Fix input parameters of swiglu_oai operator (#30458 )
gjsheu
2026-07-14 14:16:24 +08:00
702bddcee8
[Model] Add support for JetBrains' Mellum v2 code generation model (#27375 )
2026-07-14 05:54:38 +00:00
7e229e2a81
support GLM-5.2 MTP index sharing with prefill CP (#30992 )
Yuxuan Zhang and Baizhou Zhang
2026-07-14 12:11:27 +08:00
1b4176cc46
[PD] Fix optimistic prefill inflight-queue hangs on parked/aborted reqs (#31075 )
cctry
2026-07-13 20:34:44 -07:00
23b2c6f1ce
docs: fix diffusion cookbook overview cards (#31101 )
Mick
2026-07-14 10:40:42 +08:00
78dc581518
[bug-fix] Stabilize GLM-5.2 MTP IndexShare across PD and CUDA graph replay (#30839 )
2026-07-14 10:37:07 +08:00
41ad0d9c26
Allow prefill breakable CUDA graph for Qwen3.5 via multimodal opt-in allowlist (#30620 )
YAMY
2026-07-13 18:39:44 -07:00
7431f35fd8
Delete CUTLASS FP8 blockwise for SM90 and SM100, move SM120 to JIT and add SwapAB (#30438 )
2026-07-13 18:31:32 -07:00
c124bec99d
[CI] Disable gated Llama-2 EAGLE spec tests to unblock Xeon CPU CI (#30995 )
Haotong Zou
2026-07-14 09:28:47 +08:00
9756f768a6
Refactor FP4 quantization and remove deprecated JIT kernels (#30448 )
Brayden Zhong and root
2026-07-13 18:22:07 -07:00
0c01971eeb
[Intel GPU] DeepSeek V4 9/N: use sgl-kernel implementation of hadamard_transform on XPU (#28046 )
2026-07-14 06:51:31 +05:30
e9ef06c560
[Kernel] Migrate top-level srt/layers stray kernels to sglang.kernels (RFC #29630 , Phase 2.5, 3/7) (#30787 )
Xiaoyu Zhang and Claude Fable 5
2026-07-14 09:20:59 +08:00
e489685509
[Intel GPU] DeepSeek V4 5/N: Use sgl-kernel implementation of fused_q_indexer_rope_hadamard_quant to run on XPU (#27873 )
Polisetty V R K Jyothendra Varma
2026-07-14 06:50:50 +05:30
ee464fedc6
[Kernel] Migrate scattered MoE kernels to sglang.kernels (RFC #29630 , Phase 2.5, 2/7) (#30786 )
Xiaoyu Zhang and Claude Fable 5
2026-07-14 09:03:21 +08:00
464fe1b77c
[CI] Use torch.testing.assert_close in custom-all-reduce test (~1400x faster compare) (#31080 )
Alison Shao
2026-07-13 17:53:20 -07:00
33f83011e0
fix: fix Kimi-VL encoder parallelism (#30869 )
Mick
2026-07-14 08:44:06 +08:00
423b8485fb
[Quantization] add humming quantization kernel (#23754 )
2026-07-14 08:42:56 +08:00
4c997310f5
[Kernel] Hotfix: update sgl-kernel imports of relocated fp8_kernel (RFC #29630 #30784 ) (#31089 )
Xiaoyu Zhang and Claude Opus 4.8
2026-07-14 08:41:23 +08:00
cfe4eefabb
[diffusion] model: support LongLive 2.0 T2V and I2V inference (#27639 )
2026-07-13 17:39:30 -07:00
2f79d334f2
[Bugfix] Fix DeepSeek ForwardFlags across custom op boundary (#30987 )
Mohammad Miadh Angkad
2026-07-14 08:19:44 +08:00
cfc3d0555e
Fix ModelOpt NVFP4 scalar scales for merged linears (#29151 )
Po-Han Huang (NVIDIA)
2026-07-14 07:14:20 +08:00
a909077d22
Return top-p/top-k sampling mask/nucleas (#27408 )
hari-hm
2026-07-13 15:58:51 -07:00
0ee236ebdf
more fixes for Nemotron 3 parser for tool call and force nonempty content (#30533 )
Brayden Zhong
2026-07-13 15:50:27 -07:00
50ed4c011f
Remove legacy Sphinx docs/ and finish the Mintlify cutover (#28964 )
zijiexia
2026-07-13 15:06:08 -07:00
e2728ac504
[Spec] Remove dead padded_static_len and stale SGLANG_ENABLE_SPEC_V2 references (#30998 )
Liangsheng Yin
2026-07-13 15:30:31 -05:00
47030b28be
Fix MockDSV4ModelRunner missing spec_algorithm (#31056 )
Jialin Ouyang
2026-07-13 13:24:28 -07:00
2ab531cfcf
fa3/fa4: sync-free for all backends and phases (#29589 )
Liangsheng Yin and ronhuafeng
2026-07-13 15:09:57 -05:00
92fc692411
fix: include OpenSSL headers in runtime image (#31064 )
ishandhanani
2026-07-13 13:09:41 -07:00
f49cbbd67d
Fix GLM/DeepSeek NVFP4 + flashinfer_trtllm long-context "!!!!" collapse (NaN routing) (#31001 )
Khoa Pham
2026-07-13 12:54:38 -07:00
86c59ac1aa
Revert "[Tiny] Enable Full Cuda Graph with Page size = 1" (#31062 )
Yuwei An
2026-07-13 12:23:44 -07:00
805385414e
chore: bump docs install version to 0.5.15 (#31058 )
sglang-bot and sglang-bot
2026-07-13 11:58:48 -07:00
48fff1f2bd
[Spec] Deduplicate spec-v2 worker lifecycle boilerplate into BaseSpecWorker (#31008 )
Liangsheng Yin
2026-07-13 13:48:40 -05:00
c0f1f7e062
[Spec] Rename num_tokens_per_bs to num_tokens_per_req (#30977 )
Liangsheng Yin
2026-07-13 13:47:53 -05:00
b677babc62
docs: sync LMSYS SGLang blog cards (#30571 )
sglang-bot and sglang-bot
2026-07-13 11:44:39 -07:00
11a82af5f8
[Platform] Route pin memory availability through current_platform (#28113 )
2026-07-14 02:37:59 +08:00
afaa17a7f2
[Feature] Add --default-chat-template-kwargs server arg (#29579 )
Xinyuan Tong
2026-07-14 02:34:39 +08:00
b44ac5d49a
[Tiny] Enable Full Cuda Graph with Page size = 1 (#30835 )
Yuwei An and Claude Fable 5
2026-07-13 10:07:02 -07:00
978bce2063
[HiCache & HybridModel] nixl hicache backend support hybrid models (#29191 )
ziruiliu
2026-07-14 00:09:37 +08:00
be9791071a
[NPU] [DOC] Fix Ascend NPU docs issues found by AIDD (#31036 )
amote-i
2026-07-13 23:45:35 +08:00
2cf2920d07
[FlashInfer v0.6.13] Use CuTe DSL backend for FlashInfer per-token NVFP4 quantization (#28220 )
Ziang Li
2026-07-13 07:37:46 -07:00
f391c71758
[NPU] [DOC] --pp-size can not be used witgh --tp-size (#31039 )
amote-i
2026-07-13 22:35:20 +08:00
9fec359a60
[Fix] Load HunyuanV3 NextN final_layernorm into the draft head's output norm (#30331 )
2026-07-13 05:37:36 -07:00
a74bee2261
[PD] Handle NIXL abort notifications (#30352 )
Yichao Cheng
2026-07-13 05:05:25 -07:00
eb31b5310c
Support Waterfill with MegaMoE backend (#27350 )
xutizhou
2026-07-13 18:56:46 +08:00
874fc07d9b
[Kernel] Migrate scattered quantization kernels to sglang.kernels (RFC #29630 , Phase 2.5, 1/7) (#30784 )
Xiaoyu Zhang and Claude Fable 5
2026-07-13 16:17:01 +08:00
2225817424
[NPU] [DOC] Optimize and fix docs issues on Ascend NPU (#30767 )
amote-i
2026-07-13 15:35:02 +08:00
08d6d297e5
[Bugfix][NPU] Fix Hunyuan3 model where MoE's routing_scaling_ratio is missing on NPU (#29909 )
2026-07-13 14:48:41 +08:00
82e7cdcff9
[Misc] Remove a few dead code paths in DSA (#30973 )
Baizhou Zhang
2026-07-12 23:25:03 -07:00
9dd57ef8c4
[mem_cache][7/N] refactor: move MLATokenToKVPoolHost to pool_host.mla (#30616 )
shuwenn
2026-07-13 14:23:38 +08:00
cbcbef6811
[Bugfix] Fix Nemotron ForwardFlags across custom op boundary (#30968 )
Mohammad Miadh Angkad
2026-07-13 13:10:23 +08:00
4cec9ef9d7
[Fix] Forward on_after_cuda_graph_warmup through HybridLinearAttnBackend (#30846 )
yz-wqf
2026-07-13 09:40:52 +08:00
80965db8d3
[AMD] Pin cmake==4.3.4 in ROCm Dockerfile to fix MoRI gtest_discover build break (#30942 )
YC Yen-Ching Tseng
2026-07-13 09:20:17 +08:00
22c08a9bee
Preserve RMSNorm shape in batch-invariant mode (#30956 )
Lianmin Zheng and Lu Fang
2026-07-12 17:51:53 -07:00
7da30f4e55
feat: enable piecewise prefill graph for Kimi K2.5/K2.7 (#30889 )
Mick
2026-07-13 08:37:30 +08:00
b94ac87e0c
Enable breakable prefill CUDA graph for DP attention (#30898 )
Lianmin Zheng
2026-07-12 17:10:04 -07:00
c616d5a55e
[PD] Improve optimistic prefill (#30951 )
cctry
2026-07-12 15:31:53 -07:00
6cc9352dfe
[Spec] Add DSpark: confidence-scheduled speculative decoding (#30261 )
2026-07-12 15:25:26 -07:00
24d59d8d74
Fix CUDA 12 Docker dependency resolution (#30858 )
Mohammad Miadh Angkad
2026-07-13 04:50:41 +08:00
7a82178277
[Fix] Disable FlashInfer allreduce fusion in Nemotron-3-Nano lm-eval test (#30945 )
Liangsheng Yin
2026-07-12 15:39:59 -05:00
5ba3c5147e
[Spec] Add kill-switch env for draft-extend CUDA graph capture (#30944 )
Liangsheng Yin
2026-07-12 14:54:36 -05:00
96a04cb13f
Fix DeepEP CI test registration (#30873 )
Mohammad Miadh Angkad
2026-07-12 20:54:50 +08:00
