sglang

minke.yu/sglang

Fork 0

5af1f949ca [diffusion] CI: prefer official diffusion consistency GT (#29831) Mick 2026-07-04 10:09:33 +08:00
5f623ad24e Revert "Fix wrong RMSNorm fallback to old Flashinfer CUDA kernel when in PCG" (#30083) Cheng Wan 2026-07-03 19:05:20 -07:00
c21f6f19cf [MoE] Fix moe_fused_gate out-of-range expert id on all-NaN rows (fixes eagle_dp_attention crash) (#30079) Cheng Wan 2026-07-03 18:48:51 -07:00
b28bc1060f fix(mimo-vl): pass padded_context_dim to Qwen2_5_VisionPatchMerger (#29994) Alison Shao 2026-07-03 17:08:25 -07:00
6ce02b95ad fix: reorder DSA indexer dual-stream ops to avoid CUDA graph stream explosion (#30025) Khoa PhamandKhoa Pham 2026-07-03 13:53:16 -07:00
1f0f353d92 [CI] Add GLM52 NVFP4 MTP B200 tests (#30021) Baizhou Zhang 2026-07-03 13:43:38 -07:00
0203c60fdf [CP] Consolidate decode-context-parallel (DCP) helpers into layers/dcp/ (#29365)
Thanhhao Hao Phan Claude Opus 4.8
2026-07-04 02:25:39 +07:00
7820dc60a7 [Apple Silicon] Add labeler config (#29908) R0CKSTAR 2026-07-04 00:43:00 +08:00
486bcb48e7 [diffusion] CI: fix AMD diffusion CI import (#30039) Mick 2026-07-03 22:26:08 +08:00
1058d00fe0 [diffusion] feat: support cache-dit for Ideogram 4 (#29631) Jzz1943 2026-07-03 19:58:48 +08:00
42acfd1550 [diffusion] feat: performance_mode=speed enables torch.compile by default (#30016) Mick 2026-07-03 19:12:17 +08:00
0ab095eb3f [AMD]: hot-patch transformers dynamic_module_utils symlink bug (#29986) Zhaoyi Li 2026-07-03 05:20:36 -05:00
430418e218 [NPU] bugfix for dsv4 memory pool (#30001) khalilzhk 2026-07-03 16:41:39 +08:00
e3258d3b54 [Fix] Turn off dsa indexer fusion by default (#30018) Baizhou Zhang 2026-07-03 01:08:06 -07:00
4dddb04325 [Intel GPU] DeepSeek V4 6/N: use sgl-kernel implemetation of flash_mla_with_kvcache on XPU (#27914) Polisetty V R K Jyothendra VarmaandMa Mingfei 2026-07-03 13:29:38 +05:30
e90fec4868 [Intel GPU] DeepSeek V4 10/N : Add sqrtsoftplus support to fused_topk_torch_native (#28048)
Polisetty V R K Jyothendra Varma Rahul Vijayaraghavan Ma Mingfei
2026-07-03 13:29:12 +05:30
9df16b5ba9 [XPU] Remove redundant xpu graph backend and make xpu graph opt-in by default (#29911) Cao E 2026-07-03 15:58:56 +08:00
67697fb891 [AMD] Temporarily disabled: every-6-hours rocm 7.2 test (#30014) YC Yen-Ching Tseng 2026-07-03 15:38:07 +08:00
2fe7182e75 [DOC] [NPU] update supported features on ascend npu (#30011) amote-i 2026-07-03 15:36:41 +08:00
8416544ab0 [NPU] bugfix for Base class add mamba_track_indices parameter (#29999) McZyWu 2026-07-03 15:10:41 +08:00
fe60764f54 [diffusion] fix: add profiling support and fix VBench dataset handling in bench_offline_throughput (#27704) Chandrakant Khandelwal 2026-07-03 12:38:38 +05:30
76f7f7c006 [Spec] Remove the ServerArgs clone + global save/restore hack from DFlashWorkerV2 (#29995) Cheng Wan 2026-07-02 23:37:04 -07:00
d364cd8ead Support DSV4 shared expert fusion for DeepEP and MegaMOE (#27349) xutizhou 2026-07-03 14:18:25 +08:00
e81f05cf4f [dsv4] Trigger MHC prenorm prewarm at weight-load time with rank sync (#29988) YAMY 2026-07-02 23:05:03 -07:00
a6ee64d237 [DeepSeek-V4] Add an opt-in non-paged indexer for long-context prefill (#29619) YAMY 2026-07-02 22:54:40 -07:00
70b6c06793 fix: avoid DSA indexer CPU seq lens fallback (#29798) san-tianandSan Tian 2026-07-03 13:50:24 +08:00
e878c6ebdd [diffusion] feat: shard qwen-image dit across tp ranks (#29774) Yiqi YangandMick 2026-07-03 13:17:53 +08:00
f011d8c2e5 [Anthropic] Fix missing cache_read_input_tokens in streaming responses (#29703) Yiqi Yang 2026-07-03 12:46:45 +08:00
860244d4b4 [CI] Add per-stage NVIDIA model inventory tool (#29447) Kangyan-ZhouandClaude Opus 4.8 2026-07-02 20:38:04 -07:00
372a893744 Move deferred mamba cow and clear (#29945) Ke Bao 2026-07-03 11:29:05 +08:00
a2d7eb303e [MoE] Consolidate ungrouped + grouped gate/topk onto one Triton router (#26771) — faster than AOT on B200/H100/H200, at parity with flashinfer (#29771) Xiaoyu ZhangandClaude Opus 4.8 2026-07-03 11:18:59 +08:00
d8a4f7a7aa add mimo-v2-flash model tutorial (#29932) ming_wang 2026-07-03 11:02:24 +08:00
bee0f34e68 [AMD] Accept ROCm tensors in JIT kernel TensorMatcher + register 4 kernel tests (#29822) Michael 2026-07-02 19:29:47 -07:00
a6bc7fef90 glm5.2 on ascend doc (new version) (#29828) Peng Xingchen 2026-07-03 10:19:44 +08:00
70a813493f [NPU] [DOC] add missing DEEP_NORMAL_MODE_USE_INT8_QUANT for w8a8+deepep scenarios (#29937) amote-i 2026-07-03 10:18:22 +08:00
05bc3f2aa7 [AMD] Gate broken CK block-FP8 GEMM shapes to aiter-triton-GEMM to fix ROCm 7.0 Qwen3.5 accuracy (#29918) YC Yen-Ching Tseng 2026-07-03 09:18:37 +08:00
91d7645aca docs: sync LMSYS SGLang blog cards (#29990) sglang-bot 2026-07-02 18:01:32 -07:00
75cdaf432a Add XPU CI job monitor workflow (#29807) ashwini rathiandClaude Opus 4.7 2026-07-03 06:00:17 +05:30
2b3223a953 [model-gateway] PD router: cancel paired decode when prefill fails (#29017) chenkaiyue 2026-07-03 08:03:14 +08:00
17cce6a85f Fix shared logits buffer for reduced-vocab draft models (#29943) Po-Han Huang (NVIDIA) 2026-07-03 07:08:56 +08:00
8519be82e8 [AMD][DeepSeek V4] Fix default FlashMLA sparse prefill off on ROCm/HIP (#29982) Michael 2026-07-02 16:00:09 -07:00
f3904f0293 [bugfix][AMD] Disable aiter allreduce+RMSNorm fusion under DP attention / EP (#27835) Oxana Korzh 2026-07-02 16:31:25 -06:00
caf2e5da2d [AMD] Fix MiniMax M3 state transfer in Mori PD (#29756) YukioZzzandDuyi-Wang 2026-07-03 06:02:10 +08:00
ae1f0c6d07 session_id dataclass field should not put in msgpack struct (#29977) Rain Jiang 2026-07-02 14:10:12 -07:00
cfb9c574d3 Fix UE8M0 scale rounding for DeepGEMM (#29956) Mohammad Miadh Angkad 2026-07-03 04:42:17 +08:00
bc25abb786 perf(triton): avoid per-step D2H .item() sync in cuda-graph loc translate (#29921) Cheng Wanandlch1475369 2026-07-02 13:20:00 -07:00
85e71b7e13 [Doc] Cookbook Laguna-XS-2.1: add AIME25 accuracy (B300 + GB300) (#29974) Jimmy Shong 2026-07-02 13:15:01 -07:00
f19246e59a [HiCache] write_back policy refinement (#29817) Zhiqiang Xie 2026-07-02 12:16:03 -07:00
cba3801f52 docs: add PD disaggregation to GLM-5.2 cookbook playground (#29544) zijiexiaandClaude Opus 4.8 2026-07-02 09:57:38 -07:00
1b6d1e9752 Fix wrong RMSNorm fallback to old Flashinfer CUDA kernel when in PCG (#29702) Brayden ZhongandBrayden Zhong 2026-07-02 09:49:49 -07:00
9588cacaa1 Remove transformers 5.12.1 dead-code workarounds (#29758) Xinyuan Tong 2026-07-03 00:03:06 +08:00
c05c48b35e bugfix for npu Grok2 model --detokenizer without all special ids (#29853) McZyWu 2026-07-02 22:12:48 +08:00
bdd3515389 NPU case rl update weights for tensor load_format == None and flatten bucket (#29503) McZyWu 2026-07-02 22:12:05 +08:00
a375e9f3da Fix SWA eviction tombstoning the last leaf (#29860) Ke Bao 2026-07-02 21:42:30 +08:00
3c1adddff9 [diffusion] refactor: refactor cuda attention backend resolver (#29852) Mick 2026-07-02 21:34:54 +08:00
476c946543 [Doc] Cookbook: Laguna-XS-2.1 (DFlash low-latency + high-throughput) (#29884) Jimmy Shong 2026-07-02 05:05:33 -07:00
119b76567d [diffusion] feat: add --offload-during-compile to fit max-autotune on tight-memory GPUs (#29862) Mick 2026-07-02 19:39:19 +08:00
26b15a0825 [LFM2-MoE] Support Transformers v5 packed MoE expert weights (#29659)
Piotr Mazurek Piotr Mazurek Changyi Yang
2026-07-02 11:13:02 +02:00
b276a9acee chore: cleanup garbage code (#29770) Xiaoyu Zhang 2026-07-02 16:14:01 +08:00
0ae76117ef [AMD] Register 2 CPU/ROCm-safe tests for AMD 1-GPU PR CI (#29680) Michael 2026-07-02 00:25:32 -07:00
1c75243f5e docs: add Qwen3.6-27B-NVFP4 variant to cookbook (#29905) zijiexiaandClaude Opus 4.8 2026-07-02 00:05:29 -07:00
697b400d70 Share one logits output buffer across prefill/decode/draft cuda-graph runners (#29779) cctry 2026-07-01 23:30:56 -07:00
926140d789 [XPU] Enable XPU graph support (decode full-graph + prefill tc_piecewise) (#29053) Cao E 2026-07-02 13:24:35 +08:00
4fffc6448b Speculative decoding support on XPU (#23180) ANSHUMAN TRIPATHY 2026-07-02 10:53:24 +05:30
790156c98f [Intel GPU] DeepSeek V4 7/N: Support fused_rope_inplace on XPU using triton (#27915)
Polisetty V R K Jyothendra Varma Rahul Vijayaraghavan Ma Mingfei
2026-07-02 10:52:20 +05:30
cb06c4e6ce [CPU] Fix model failures on Xeon (#29497) Zaili Wang 2026-07-02 13:20:18 +08:00
aff44d748d [CPU] Padding for dim divisibility in TP3/6 cases (#20072) Zaili WangandMa Mingfei 2026-07-02 13:14:39 +08:00
80ac11eda3 Fix capture-mode detection during breakable CUDA graph capture (#29866) cctry 2026-07-01 22:12:56 -07:00
e88645d65f [amd][diffusion] fix: fix causal Conv3D cat/pad fusion crashes for wan2.2 t2v (#29672) sushil Dubey 2026-07-02 10:41:30 +05:30
0543246184 Enable Breakable Cuda Graph as Default (#29458) Yuwei An 2026-07-01 22:04:44 -07:00
d15b79e96e [bug6] clear stale mamba cow source on rematch (#29354)
Feng Yao Zhangheng Hanming Lu Hanming Lu
2026-07-01 22:04:09 -07:00
307094dc7d [DeepSeek V4] Cover both dense and sparse prefill paths in the compress attention unittest (#29885) YAMYandYuwei An 2026-07-01 21:20:12 -07:00
9ba4b8f8ba sgl-kernel: bump sgl-attn for varlen num_splits OOM fix (#29551) Xinyuan Tong 2026-07-02 12:13:53 +08:00
e6f6a353bf pad customized_info for mixed output batches (#29842) Feng Yao 2026-07-01 21:07:58 -07:00
4605d4d94b [Apple Silicon] [CI] Add model-free unit-test workflow on macos-26 (#29691) Jae B. 2026-07-02 00:00:22 -04:00
eb23f9a86b [KDA] Add FlashKDA prefill backend for safe-gate KDA linear attention (#29472) Yuan Luoandluoyuan.luo 2026-07-02 11:37:04 +08:00
6eeb9871e5 Add Laguna XS.2.1 DFlash support to SGLang (#29446)
adamkbaranowski Jimmy Shong Claude Fable 5
2026-07-02 05:26:23 +02:00
bac351d617 [NPU] Fix block_table batch size mismatch in GLM-4.7-Flash DeepEP + MTP without CUDA Graphs (#29829) Estrella-xx 2026-07-02 11:09:01 +08:00
b558cc1abb feat(short-conv): shared ShortConvAttnBackend for ZAYA1 CCA + LFM2 short conv (#29867) Cheng Wan 2026-07-01 20:08:00 -07:00
70df09b833 Fix MambaPool.clear_slots OOM by replacing expand-based tensor allocation with scalar zeroing (#27923) iridiumine 2026-07-02 09:26:55 +08:00
0c1a0be3b2 fix(precision): do not promote failed runs to the comparison baseline (#28190) Xinyuan TongandAlison Shao 2026-07-02 09:19:15 +08:00
a3f6680874 [AMD]: docker(rocm) bump Mooncake to latest main + enable multi-protocol (#27730) Zhaoyi LiandMichael 2026-07-01 19:41:46 -05:00
cf7c6ac234 fix(nightly-precision): pin flashinfer allreduce-fusion backend for TP-partial capture contract (#28925) Xinyuan Tong 2026-07-02 08:26:19 +08:00
1a5977d41b [chore] Add no-getattr rule; refine no-dataclasses rule (#29871) Lianmin Zheng 2026-07-01 15:31:34 -07:00
76d828a4e8 Add pranjalssh to CI_PERMISSIONS.json (#29870) Lianmin Zheng 2026-07-01 14:52:02 -07:00
5ae214f18b Extract reusable VMM shareable-handle helpers from register_graph_inputs (#29621) Ming Yang 2026-07-01 14:02:00 -07:00
c865347b98 [DeepSeek V4] Enable FlashMLA sparse prefill by default (#29775) YAMYandBaizhou Zhang 2026-07-01 13:50:05 -07:00
8f0d320d31 [Spec] Enable FlashInfer autotune for spec draft (#29595) Mohammad Miadh Angkad 2026-07-02 04:36:07 +08:00
4a8e76805c feat(mem_cache): unified memory pool for hybrid Mamba / SWA models (#29678) Cheng Wanandlch1475369 2026-07-01 13:21:59 -07:00
3adfd0f34b [AMD] Register 3 unit mem_cache + utils tests for stage-b-test-1-gpu-small-amd (#29782) Michael 2026-07-01 12:57:48 -07:00
8361561c24 [AMD] Fix int8 per-token quant Triton portability + register test for AMD nightly CI (#29694) Michael 2026-07-01 12:56:33 -07:00
c312cdd3a7 Upgrading tvm-ffi/sgl-deep-gemm/tilelang (#29554) Baizhou Zhang 2026-07-01 12:32:18 -07:00
779ea4a9b5 [RL] fix deepseek v4 MXFP8 flashinfer_trtllm_routed MoE weight update (#28676) Zhihao Wang 2026-07-01 12:29:08 -07:00
eb75d990f7 [Bugfix] compressed-tensors WNA16 MoE: don't assume a "Linear" config group (#29761)
Joe Rowell Jiminator Jimmy Shong
2026-07-01 20:08:09 +02:00
30c9801b39 [disagg] Fix KV-event publisher port collision under pure data parallelism (#29211) Kangyan-ZhouandClaude Opus 4.8 2026-07-01 10:57:34 -07:00
03b9278da0 [diffusion] CI: tighten multimodal-gen consistency thresholds (#29824) Mick 2026-07-02 01:32:51 +08:00
9bb7de9258 [AMD][DI][CI] 2/N Add DSV4 DP8/EP8 and MTP MI355X 1P1D nightly recipes (#29784) Zhaoyi Liandbingxche 2026-07-01 10:46:48 -05:00
2a6e5c60fe [AMD] Rebalance stage-c-large-8-gpu-mi35x partitions to fix 60-min timeout (#29726) YC Yen-Ching Tseng 2026-07-01 23:17:00 +08:00
13dc5f2dc7 [HiCache][AMD] Add UMBP tiered DRAM + SSD L3 storage backend with hugepage host allocator (#25377)
inkcherryTianDi101 ditian12@amd.comNiko Ma nima@amd.comWu, Yutong yutong.wu@amd.comfigo fizhang@amd.comAMD-yanfeiwang Zhangheng
2026-07-01 22:21:37 +08:00
a0d9791810 [CI] Fix fused EH norm CI registration (#29845) Mohammad Miadh Angkad 2026-07-01 21:27:07 +08:00

... 38 39 40 41 42 ...

Commit Graph Select branches Hide Pull Requests dsv41-pd dsv41-pd-visioncp main Mono Color

Commit Graph

Select branches

Hide Pull Requests

dsv41-pd

dsv41-pd-visioncp

main