sglang

minke.yu/sglang

Fork 0

45604a0f4a [refactor] Unify CUDA graph runner input buffers behind CudaGraphBufferRegistry (#26742) Cheng WanandClaude Opus 4.8 2026-06-03 10:54:10 -07:00
b0f78bef97 [diffusion] improve: improve realtime webui playback pacing (#27148) Mick 2026-06-04 00:33:56 +08:00
45a66f4088 [Docs] Update unified Text/Vision/Audio model cookbook: install + sgl-eval accuracy (#27171) Xinyuan Tong 2026-06-03 17:32:08 +01:00
9d0e6a2df4 fix(mlx): set canary_manager and materialize overlap-loop inputs on Apple Silicon (#26882) Lijuan TangandXiaodong Ye 2026-06-03 09:03:46 -07:00
fa5c8a3101 [model] support encoder-free unified Text/Vision/Audio model (#27167) Xinyuan Tong 2026-06-03 16:58:06 +01:00
f65aae8493 [HiCache] fix: truncate prefetch key on degraded allocation (#25991) shuwenn 2026-06-03 22:06:18 +08:00
33f943fbf5 [diffusion] optimize: batch usp replicated kv prefix all-to-all (#27143) Mick 2026-06-03 21:22:39 +08:00
03c77dc33d [PD] Deduplicate PD logprob normalization (#27085) Ye (Charlotte) Qi 2026-06-03 04:08:15 -07:00
44d4a25a07 Type hicache transfer hook kwargs in unified cache (#27071) Ke Bao 2026-06-03 18:57:25 +08:00
d7013b6537 [AMD] [CI] Remove hardcoded model/cache paths from MI35x nightly tests (#27001) Bingxu Chen 2026-06-03 17:14:32 +08:00
e67810bea7 [SGLang Tracing] Add pd disaggregation mooncake backend tracing (#23755)
Feng Su Mu Huai Shangming Cai
2026-06-03 16:43:29 +08:00
73b53e7a87 Revert "Support NextN = 2/4 in DSV32" (#27138) Cheng Wan 2026-06-03 01:29:52 -07:00
63dc20ae6c [UnifiedTree] Add CP sync (#25395) Vladislav NosivskoyandZhangheng 2026-06-03 11:10:27 +03:00
93173b27e8 integrate flash_mla_sparse_fwd (#25418)
Chunan Zeng Yuan Luo Baizhou Zhang laixinn MeowGrange Claude Opus 4.8
2026-06-03 01:09:25 -07:00
f790674ad8 fix(moe): avoid unpacking None from masked deep_gemm without overlap when sbo enabled (#26839) ybyang 2026-06-03 15:44:28 +08:00
9450696aa5 [diffusion] CI: add cosmos3 nano t2v gpu test (#26963) Mick 2026-06-03 15:42:51 +08:00
ac16dbf412 docs: add DeepSeek-V4 EPLB Waterfill tips (#27049) xutizhou 2026-06-03 15:37:45 +08:00
8e77af1afc [AMD] fix(triton-mla): cap max_kv_splits at 256 on gfx942 (Kimi-K2.6 hang) (#24762) Bingxu Chen 2026-06-03 15:13:18 +08:00
e5b8e3a66a Optimize streaming detokenizer updates (#24659) inkcherryandAlex Nails 2026-06-03 14:44:21 +08:00
52f2fe456a fix(disagg): correct DSA/SWA state-page transfer mismatch in PD disaggregation (#27004) Kevin Flansburg 2026-06-03 00:33:41 -06:00
dae86f51f5 [diffusion] chore: polish realtime webui waiting state (#27068) Mick 2026-06-03 14:29:26 +08:00
b5560ffc36 Fix flashinfer autotune oom glm51 (#24195) Khoa Pham 2026-06-02 23:28:57 -07:00
202e618898 Revert "Fix hybrid linear attention misrouting plain-RadixAttention linear layers to the full backend (Ring-2.5-1T)" (#27116) Cheng Wan 2026-06-02 23:27:57 -07:00
0ef39784ef [Bugfix] Gate DP-attention even-token padding to CP-enabled configs (#26911) Jimmy Shong 2026-06-02 23:06:52 -07:00
ab7c4ab6bb [AMD] Fix correctness for AITER MLA backend with --page-size > 1 (#25556) Duyi-Wang 2026-06-03 14:01:19 +08:00
6d53615699 [Gemma4] Use hard GSM8K accuracy floor for 31B MTP test (#27101) Khoa PhamandClaude Opus 4.8 2026-06-02 22:57:12 -07:00
b8d7351a74 Feat/add w4a16 moe support to nemotron (#25655) Shaun Kotek 2026-06-03 08:42:26 +03:00
512bfbb1e1 [CPU] Explicitly enable AVX512 & AMX instruction set (#26145) Zaili WangandMa Mingfei 2026-06-03 13:40:41 +08:00
aa510bda45 Support specific pass of bias_grouped_topk for xpu (#26349) gaopengff 2026-06-03 13:13:48 +08:00
f4e7a98fe5 [HiCache] feat: support draft offload for mooncake (#24984)
shuwenn huangtingwei9988 stmatengss
2026-06-03 12:42:04 +08:00
c3aaafc5f2 [Bugfix] Clean up failed NIXL sender state (#27011) CrazyCoder 2026-06-03 12:15:15 +08:00
3e681d7fff Add per-rank staggered weight loading for improved TP I/O concurrency (#26937) zhaoshangandShangming Cai 2026-06-03 11:25:21 +08:00
1ebc7438ac model: support Command A plus (#26106) zijiexiaandYihao Wang 2026-06-02 20:23:04 -07:00
71a747cf15 [diffusion] fix: fix lingbot realtime consistency gt pin (#27080) Mick 2026-06-03 10:29:37 +08:00
3d54056391 [diffusion] optimize: optimize Cosmos3 i2v latent prep (#27084) Chi McIsaac 2026-06-02 19:23:37 -07:00
3715a07e87 [diffusion] improve: clamp wanvae decode output in place (#27086) Mick 2026-06-03 10:16:10 +08:00
f0e18be0ba [diffusion] improve: use conv2d width padding in wanvae (#27081) Mick 2026-06-03 10:14:50 +08:00
13852d3f31 Support NextN = 2/4 in DSV32 (#24870) Brayden Zhongandb8zhong 2026-06-02 19:06:06 -07:00
2d8cb87de7 docs: tag new diffusion cookbooks (#27094) Mick 2026-06-03 09:50:27 +08:00
b678448b8a ci(xeon): merge 2 partitions into 1 job to reduce runner contention (#26904) MingxuZh 2026-06-02 18:46:27 -07:00
eda21f6839 Add fused_rope and for xpu (#25773) gaopengff 2026-06-03 09:41:42 +08:00
83bc776612 [diffusion] optimize: preserve dtype in wanvae nearest upsample (#27077) Mick 2026-06-03 08:32:28 +08:00
c55548ba11 [perf] Replicate embed_tokens to drop the post-embed all-reduce (#26970) Qiaolin Yu 2026-06-02 16:48:18 -07:00
76c9899da7 Fix hybrid linear attention misrouting plain-RadixAttention linear layers to the full backend (Ring-2.5-1T) (#26623) Alison ShaoandCheng Wan 2026-06-02 16:24:49 -07:00
72929c7000 [AMD] Enable AITER custom all-gather on ROCm (#25093) Hubert Lu 2026-06-02 15:57:37 -07:00
a711c57a32 [Spec] Fix Gemma 4 MTP with trtllm_mha crash issue (#26966) Khoa Pham 2026-06-02 14:37:51 -07:00
365cc2ade5 jit_kernel tests: bump multiprocess_test timeout 90s -> 240s (cold JIT cache) (#26994) Alison Shao 2026-06-02 14:33:52 -07:00
22bb9a6421 test: disable test_gemma4_mtp_26b_a4b_extra from CI (#27082) Khoa Pham 2026-06-02 14:00:48 -07:00
68caf49154 Update sgl-deep-gemm to 0.1.2 (#26993) Baizhou Zhang 2026-06-02 13:51:54 -07:00
b603f08c0c [DP] Fix FlashInfer dispatcher workspace sizing and set_dp_buffer_len (#26643) Hanming Lu 2026-06-02 13:25:20 -07:00
9e717cae46 [sglang] Fix Mamba COW over-releasing SWA locks (cascade-evict assert crash) (#27038) Bi Xue 2026-06-02 10:42:04 -07:00
6ba31e33e6 feat(api): add require_reasoning field for engine's generate api (#27019) Muqi Li 2026-06-03 01:36:29 +08:00
99da43b900 [refactor] init_forward_metadata 3-method ABC + side-channel removal + ForwardMetadata type rename (#26735) Cheng WanandClaude Opus 4.7 2026-06-02 10:33:33 -07:00
6c69756fa8 NIXL: use prep+make API to improve performance (#26406) Ilia Yastrebov 2026-06-02 18:36:35 +02:00
28f9c1ff24 Relax mamba unified cache kl threshold (#27070) Ke Bao 2026-06-02 23:28:22 +08:00
22043b917b [diffusion] CI: ad lingbot case (#27055) Mick 2026-06-02 22:26:51 +08:00
b5e154dc73 Fix stale import after kl_nightly rename (#27064) Ke Bao 2026-06-02 21:46:21 +08:00
38d4c9ba88 Improve type annotations in unified radix cache (#26948) Ke Bao 2026-06-02 21:17:50 +08:00
7271318dc3 【docs】The remote weight download function has been adjusted to be unsupported until the PTA interface is fixed. (#27050) littleyellowbicycle 2026-06-02 20:55:23 +08:00
64a1dec8b6 [diffusion] feat: add realtime webui super resolution controls (#27026) Mick 2026-06-02 20:29:21 +08:00
ce7da7397a [diffusion] optimize: optimize cosmos3 (#27041) Mick 2026-06-02 19:47:42 +08:00
c2eea4d7b3 [Bugfix] Fix orphaned aborted prefill bootstrap requests in PP disaggregation (#27028) Librua 2026-06-02 19:03:46 +08:00
ee4bf0a9d3 [UnifiedTree]: Add HiCache Nightly CI For GLM5 (#26927) Zhangheng 2026-06-02 19:02:50 +08:00
3394931044 [diffusion] optimize: optimize lingbot performance (#27023) Mick 2026-06-02 18:33:06 +08:00
a777672939 [diffusion] feat: enable parallel decode for cosmos3(#27037) Mick 2026-06-02 18:18:18 +08:00
84e1108312 Optimize ngram decode id computation (#24757)
Xiaoyu Zhang Codex BBuf
2026-06-02 17:37:34 +08:00
f651b48764 Apply apply_group_norm_silu to LTX-2 latent upsampler (#26045) pengduriceandgithub-actions[bot] 2026-06-02 02:28:57 -07:00
3ea1ba5b15 [GDN] Optimize prefill QKV split dispatch (#26206) Xiaoyu ZhangandBBuf 2026-06-02 16:48:31 +08:00
559581b383 [codex] Centralize Triton utility kernels (#26000) Xiaoyu ZhangandBBuf 2026-06-02 16:47:45 +08:00
172bd8e6b9 [scheduler] Zero gen_throughput and flush KV events on pause (#24003) Bruce Changlong XuandKe Bao 2026-06-02 01:43:04 -07:00
b55570d38e [PD] Optimistic prefill (#26780) cctryandgemini-code-assist[bot] 2026-06-02 01:16:14 -07:00
5ae8d286d2 perf(gemma4): single-launch fused router (topk + softmax + scale) (#26502) Charles Chen 2026-06-02 01:00:17 -07:00
8cea0473ea Fix dp-attention token alignment in the dumper comparator e2e test (#26996) fzyzcjy 2026-06-02 15:50:45 +08:00
3e993f6140 [PD]: Support HiCache prefetching and pd-incremental transfer on decode side (#26227)
Zhangheng huangtingwei Shangming Cai 晟海
2026-06-02 15:40:10 +08:00
2582134a59 [AMD] Add amd ci mamba state scatter test (#26677) Hsiu-Chun, HungandHung 2026-06-02 15:24:59 +08:00
301bcf0872 Add FP4 Indexer for DeepSeek V4 (#26209) Jinyan ChenandJinyan Chen 2026-06-02 15:14:38 +08:00
547b886b3c ci: drop redundant multimodal-server jobs from nightly (Nvidia) (#26985) Alison Shao 2026-06-01 23:40:48 -07:00
a6985e1be0 [diffusion] doc: add cookbook for lingbot-world (#26958) Mick 2026-06-02 14:35:22 +08:00
1033d835ff [diffusion] optimize: reduce cosmos3 denoise overhead (#26973) Mick 2026-06-02 14:23:02 +08:00
3b26644bc4 [diffusion] misc: add realtime-webui (#26959) Mick 2026-06-02 14:13:02 +08:00
2fc548f250 [diffusion] model: support lingot-world (#26954) Mick 2026-06-02 13:52:49 +08:00
f531bd7ff3 [Bug] Fix circular import in forward_batch_info from runtime cp_utils import (#27014) Liangsheng Yin 2026-06-01 22:51:32 -07:00
d15a2dc72c [AMD] dpsk-v4 swa loc cache support (#26931) Thomas Wang 2026-06-02 13:37:07 +08:00
4226a6f13a [AMD] Fix GPT-OSS MXFP4 accuracy on ROCm AITER path (#26884)
kk wunhuang Bingxu Chen
2026-06-02 13:30:43 +08:00
08526c7fca [Spec] FrozenKVMTP fold assistant seed into captured draft graph (#25539) Khoa PhamandCursor 2026-06-01 22:27:25 -07:00
f27fa0da93 [NPU][Docs] Kimi-K2.5 best practice (#26774) Kurkur 2026-06-02 13:14:14 +08:00
f40b2ca9d3 chore(CODEOWNERS): add allocator/ owners and @alphabetc1 to mem_cache (#27003) shuwenn 2026-06-02 13:07:15 +08:00
594ec6335d [Bug Fix][HiCache] Drop @lru_cache on UnifiedTreeNode.get_prefix_hash_values (#26939) Ethan ZHUandZhangheng 2026-06-02 12:38:28 +08:00
1c0019da75 [Docs] GLM-4.7 cookbook: add NVIDIA Blackwell (B200, GB200) + NVFP4 sections (#26384)
Thanhhao Hao Phan Zijie Xia Claude Opus 4.8
2026-06-02 10:47:51 +07:00
951fa05a09 [MoE] Support BF16 standard A2A with DeepGEMM runner (#26473) popsiclexuandpopsiclexu 2026-06-02 11:40:38 +08:00
b562da0d9f [PD] docs: clarify disaggregation IB device formats (#25521) Teng MaandZijie Xia 2026-06-02 11:38:33 +08:00
9fe8b72912 Speed up DeepGEMM JIT warmup with per-PP-rank parallel compile (#26567) ybyang 2026-06-02 10:51:27 +08:00
0574d2b8a5 [NVIDIA] [GDN] Enable FlashInfer MTP verify on SM100+ (Blackwell) (#23273)
Shu Wang Yangmin Li Claude Opus 4.7
2026-06-01 20:56:42 -05:00
54143264bf ci: disable cross-job fast-fail for run_all_tests dispatch (#26990) Liangsheng Yin 2026-06-01 18:32:41 -07:00
98a1b58c47 docs(cookbook): port popular model usage guides into cookbook pages (#25813)
Prabhu Prakash Kagitha Claude Sonnet 4.6 zijiexia Zijie Xia
2026-06-01 20:41:49 -04:00
5e63200064 ci: full parallelism for run_all_tests dispatch (#26986) Liangsheng Yin 2026-06-01 17:40:43 -07:00
f6d0beaca8 Revert "Support spec v2 tree drafting (eagle topk>1) with page_size==1" (#26981) Liangsheng Yin 2026-06-01 17:16:44 -07:00
167272e785 [LoRA] add lora chunked req test and fix (#23179) Glen Liu 2026-06-01 19:25:27 -04:00
dff45411da [HiCache] Prevent KV cache data loss when radix tree node is split b… (#16946) chenkaiyueandZhiqiang Xie 2026-06-02 06:58:06 +08:00
4151a04d1a [Perf][Spec Decoding] Skip cat/topk/sort/gather in draft_forward for topk=1 (#26424) Qiaolin Yu 2026-06-01 15:37:47 -07:00

... 51 52 53 54 55 ...

Commit Graph Select branches Hide Pull Requests dsv41-pd dsv41-pd-visioncp main Mono Color

Commit Graph

Select branches

Hide Pull Requests

dsv41-pd

dsv41-pd-visioncp

main