sglang

minke.yu/sglang

Fork 0

0eded9e208 Add Laguna-M.1 cookbook (#28661) Jimmy Shong 2026-06-18 08:23:53 -07:00
97e3b8998d Pass quant_config to attention gate projection (#28649) Jimmy Shong 2026-06-18 05:04:25 -07:00
bb9d31f22d [NPU] Add head_dim=256 to _can_use_tnd whitelist (#28635) syy-hw 2026-06-18 17:55:50 +08:00
2a9cce5d27 [NPU] Add MTP support for GLM-4.7-Flash (#28516) Estrella-xx 2026-06-18 17:54:06 +08:00
b7d7dfb4ed [Bugfix] Fix Intern-S1 FP8 expert count lookup (#28629) Xiaoyu Zhang 2026-06-18 17:49:59 +08:00
67db2ac3e7 refactor(runner): unify pp_proxy_tensors forward kwarg into one helper (#28382) Cheng Wan 2026-06-18 02:23:47 -07:00
105e095e00 [Docker] Fix cu12 dev image build: pin torch reinstall + JIT-fallback for missing x86 cubins (#28632) Douglas Yang 2026-06-18 01:51:57 -07:00
867707f1f2 [UnifiedTree]: move some kl test from base to extra stage (#28627) Zhanghengandispobock 2026-06-18 16:39:46 +08:00
fa71064147 fix: speculative draft worker clobbering target attention backend (#28559) cctry 2026-06-18 01:21:14 -07:00
59001267c3 Fix bench serving base-url-only runs (#28617) Mohammad Miadh Angkadandshuwenn 2026-06-18 15:34:00 +08:00
9b10821c8e [NPU] Add Ascend NPU support for DeepSeek-V4 (#25144)
Talantan1102 khalil2ji3mp6 randgun t00937989
2026-06-18 15:30:25 +08:00
3f66873304 [Docs] DeepSeek-V4 cookbook: drop --disable-flashinfer-autotune from GB300 Flash low-latency (#28590) zijiexiaandClaude Opus 4.8 2026-06-18 00:05:21 -07:00
b7ae7149e8 [HiCache] Fix SWA L3 cache miss due to a prefetch/hit len mismatch (#27291) Vladislav NosivskoyandZhangheng 2026-06-18 09:55:00 +03:00
0188c54fbe [FA3] Add unit test for only_qv (NoPE) KV decode path (#28595) DovLin 2026-06-18 14:41:44 +08:00
6309fb9abb [AMD] Fix Always mask padded topk_ids on HIP to prevent garbage MoE routing (DeepSeek-R1-MXFP4 accuracy regression) (#28378) kangwangamdandkangwangamd 2026-06-18 14:39:44 +08:00
3b61dc32c9 [diffusion] fix: fix fp8 fused tp scale loading (#28546) Mick 2026-06-18 14:38:38 +08:00
5d1949152d [AMD] ci: add extra-a 1-gpu-large tier (fp8kv-triton, streaming-session, spec-standalone) (#28458) Michael 2026-06-17 23:31:32 -07:00
0e5a66dca4 [AMD] Register 3 JIT kernel unit tests for AMD CI (#27837) Michaelandmichaelzhang-ai 2026-06-17 23:28:56 -07:00
b55cf4382d docs: add DeepSeek-V4 compressed state dtype tip (#28613) Ryan Zzzandzhujunyu 2026-06-18 14:22:22 +08:00
7976928c57 Abort during chunked prefill + PD peer-liveness abort (#28086) cctry 2026-06-17 23:13:28 -07:00
d2539980b6 [Fix] don't force hybrid-SWA when sliding_window is disabled (#28604) Jimmy Shong 2026-06-17 22:11:27 -07:00
4ee7882a46 [XPU] fix(deps): upgrade diffusers to fix fresh installs (#28443) Chandrakant KhandelwalandChandrkant Khandelwal 2026-06-18 10:14:11 +05:30
8d4a22c5af [Docs] Add fp8 kv cache for tokenspeed mla docs (#28201) Mohammad Miadh Angkad 2026-06-18 12:42:00 +08:00
cf93712937 [misc] Share bench HTTP-client base-URL resolution with IPv6-compatible formatting (#28598) Liangsheng Yin 2026-06-17 20:50:49 -07:00
3b9db3a1f0 [Mamba][GDN] Deduplicate spec conv-window intermediate cache via sliding window layout (#28302) Yuan Luoandluoyuan.luo 2026-06-18 11:42:31 +08:00
3340f4e3da [GDN][KDA][mem_cache] int8 checkpoint pool for the linear-attn prefix cache (#28185) Yuan Luoandluoyuan.luo 2026-06-18 11:41:46 +08:00
53318911ca Add get_parallel(): a structured accessor for parallel-topology state (#28567) Cheng Wan 2026-06-17 20:23:43 -07:00
d27d8b24de fix (#28597) siyu 2026-06-18 11:03:23 +08:00
462c01ea6b [observability] add Ray metric backend wrappers (#26252) Dongjun Na 2026-06-18 11:41:53 +09:00
d2b5488392 [misc] Centralize bench launch-vs-connect into a reusable acquire_endpoint (#28592) Liangsheng Yin 2026-06-17 19:40:43 -07:00
c208a96a7d Fix ScheduleBatch req pool CPU metadata (#28514) luoroger37 2026-06-18 10:25:47 +08:00
05b3fd0f44 [diffusion] chore: remove ltx2 snapshot mode (#28533) Mick 2026-06-18 10:20:21 +08:00
9888b7b42b [misc] Trim dead code in trtllm_mha page-table backend; reuse eager page-table buffer (#28578) Liangsheng Yin 2026-06-17 18:36:22 -07:00
1981464ba4 docs: sync LMSYS SGLang blog cards (#28589) sglang-botandsglang-bot 2026-06-17 18:23:58 -07:00
74e2e48c82 Introduce CpuDeviceMixin and CpuSRTPlatform (#26385) zijiexia 2026-06-17 17:41:17 -07:00
cfa4aa988f Revert "revert the head_dim assignment from PR 23862" (#28583) cctry 2026-06-17 17:11:05 -07:00
343aeeef39 [Model] Laguna: support per-element output gating (#28400)
Joe Rowell Jimmy Shong Jimmy Shong
2026-06-18 02:09:04 +02:00
d773b49e5b Fix MXFP8 FlashInfer CUTLASS scale selection (#28553) Mohammad Miadh Angkad 2026-06-18 07:44:23 +08:00
a663500ea9 [Test] Fold EAGLE return_hidden_states regression into spec triton suite (#28577) Liangsheng Yin 2026-06-17 16:42:02 -07:00
b88bada64e [misc] Unify bench seed default to 42 and rename --profile-filename-prefix to --profile-prefix (#28576) Liangsheng Yin 2026-06-17 16:41:23 -07:00
5ea0d1d093 add dflash gemma4 support (#27471)
David Wang kpham-sgl Claude Opus 4.8
2026-06-17 19:39:28 -04:00
cd60c4edd0 [attn backend] Make seq_lens_cpu optional in trtllm_mha backend (#28106) Jonny Kongandjonnykong 2026-06-17 16:24:19 -07:00
e4fd613def [Spec] Fix return_hidden_states under spec V2 (issue #26163) (#28496) Khoa PhamandClaude Opus 4.7 2026-06-17 16:21:36 -07:00
b1d18d562b chore: bump sglang-kernel version to 0.4.4 (#28572) sglang-botandsglang-bot 2026-06-17 16:13:39 -07:00
bcf298c28c [HiCache & Bench] add cache hit breakdown in bench_serving (#22053) Vladislav NosivskoyandZhiqiang Xie 2026-06-18 02:02:30 +03:00
5d6b35eabb revert the head_dim assignment from PR 23862 (#28571) cctry 2026-06-17 15:46:28 -07:00
732b81d5b9 [Fix] DeepSeek-OCR-2 bench_serving: fix processor loading (#28483) Chetan Kumar Verma 2026-06-18 03:59:37 +05:30
e053890b6f [Fix] Reuse an already-running server in bench_one_batch_server instead of forking an orphan (#28563) Liangsheng Yin 2026-06-17 15:14:44 -07:00
7fd63f4cf2 Remove stale load collection from output streaming hot path (#28408) weireweireandweireweire 2026-06-18 06:02:20 +08:00
3b5aae278e Fix EagleDraftExtendInput missing kv_indptr crash with triton/DP attention (#28221) YC Yen-Ching Tseng 2026-06-18 05:54:33 +08:00
3e97c9239f chore: bump sgl-kernel version to 0.4.4 (#28556) sglang-botandsglang-bot 2026-06-17 13:38:00 -07:00
4b817f5d7f Upgrade fa3 hash (#28394) Baizhou ZhangandShijin 2026-06-17 13:32:11 -07:00
753aa89a83 [spec decoding] fix mrope_positions in draft extend (#28464) Qiaolin Yu 2026-06-17 13:25:31 -07:00
0a28a929dc [MLX] Add Metal profiling hooks to server profiler (#28122) Lijuan Tang 2026-06-17 13:06:20 -07:00
3c4130c741 [Kimi K2.5] Fix eagle3 aux capture for tp>1 when AR fusion is enabled (#28343) Khoa Pham 2026-06-17 12:59:11 -07:00
7cead0fb8f Add JonnyKong to CI_PERMISSIONS.json (#28550) Lianmin Zheng 2026-06-17 11:42:45 -07:00
f5b041622b [AMD] Fix deepseek-v4 mtp accept length issue (#28520) Thomas WangandBingxu Chen 2026-06-18 02:13:55 +08:00
8aaca72c21 [FIX]Fix Step3-VL multi-image embedding and local patch splitting (#24970) kousakawangandkousakawang 2026-06-18 01:31:32 +08:00
873196f7fa ♻️ [llm][npu][quant] Delegate MXFP8 dense scheme to kernel and use torch.ops.npu (#28505) Junlin Wu 2026-06-18 01:18:26 +08:00
735a256f98 [diffusion] feat: use LocalAttention for mistral3 encoder (#28176) Mick 2026-06-17 21:18:41 +08:00
dad890fff1 [diffusion] perf: shard text when using sp in flux.1/2 (#27066) Aleksi VesantoandMick 2026-06-17 16:17:39 +03:00
3fb65ebabd [RL] Fix FlashInfer TRTLLM MXFP8 dense weight layout (#28459) Ziang Li 2026-06-17 03:33:35 -07:00
2f1390fcb1 fix: preserve divisible FP8 block K configs on CUDA (#27553) lmyybh 2026-06-17 18:30:56 +08:00
9b8c41171a [AMD] Fall back to layer_first layout for kernel write-back on ROCm (#28473) YC Yen-Ching Tseng 2026-06-17 18:01:43 +08:00
21a95333d4 [AMD] Add transpose_scale arg for o_proj to fix GLM accuracy issue (#27798) Thomas Wang 2026-06-17 16:01:02 +08:00
f86e9b48e8 [Perf] Make spec-decode penalty H2D non-blocking and share decode cumulate path (#28500) Liangsheng Yin 2026-06-17 00:59:58 -07:00
8fd1694dd2 Deepseek v4: support mixed dtype compression states (#27277) Ryan Zzzandzhujunyu 2026-06-17 15:56:41 +08:00
7256ee9871 [AMD] Update test_aiter_allgather_amd.py data types alignment between benchmark aiter and custom all-reduce kernel (#27815)
kangwangamd YC Yen-Ching Tseng Hubert Lu
2026-06-17 15:48:15 +08:00
d0e974f40b [NPU] Use use_dsa to dispatch Ascend DSA attention (#28436) Peng Xingchen 2026-06-17 15:47:12 +08:00
b54f8432ad Batch EAGLE draft/draft-extend replay memcpys via grouped foreach copy (#28465)
Khoa Pham Cursor Qiaolin Yu
2026-06-17 00:34:21 -07:00
3bc618485a [Perf] Make latest_output_ids H2D non-blocking in prepare_for_decode (#28491) Liangsheng Yin 2026-06-16 23:46:35 -07:00
27291118b9 Upgrade sgl-deep-gemm to 0.1.3 (#28402) Baizhou Zhang 2026-06-16 23:06:36 -07:00
c01f62e341 [bugfix] guard NVIDIA SM-capability checks with is_cuda() for AMD/ROCm (#28486) Oxana Korzh 2026-06-16 23:51:42 -06:00
0f5e14e1d9 [diffusion] fix: use Megatron-style tp for native encoders and dits (#28318) Mick 2026-06-17 13:07:44 +08:00
9371062ef3 Fix deep seek ocr2 image processing (#27884) Rohit Kumar Singhandgithub-actions[bot] 2026-06-17 10:29:18 +05:30
a2aa51c818 [AMD] register 4 2-gpu tests to stage-b-test-2-gpu-large-amd (#28344) Michael 2026-06-16 21:58:34 -07:00
66ac385f52 fix(moe): MoRI EP init_mori_op missing BF16 dispatch branch (#28469) Chengze Fan 2026-06-16 21:02:06 -07:00
b8a73bfba0 Call Flashinfer mm_fp8 for per-tensor FP8 GEMMs on SM100 (#28333) Brayden ZhongandBrayden Zhong 2026-06-16 20:50:03 -07:00
72ccfec594 docs(cookbook): verify GLM-5.2 single-node B300 (FP8 + BF16) (#28460) Xinyuan TongandZijie Xia 2026-06-17 04:47:32 +01:00
c17190c059 update codeowners (#28478) Mick 2026-06-17 11:16:52 +08:00
6c8fdb5b62 [diffusion] fix: fix PicklingError with --backend diffusers on non-T2I models (#21472) Yufeng HeandYufeng He 2026-06-17 11:14:25 +08:00
827fc56e00 [router] Multi-arch experimental sgl-router image + fix distroless libpcre2 startup crash (#28474) Kangyan-ZhouandClaude Opus 4.8 2026-06-16 20:09:39 -07:00
f06e2d3d1f Support asymmetric compressed-tensors MoE (#27690) Zilin Zhu 2026-06-17 09:53:33 +07:00
224b1dc775 [NPU]Replace ascend vision attn operator (#25768) zhaozx-cnandgithub-actions[bot] 2026-06-17 10:33:37 +08:00
0ae4740bd1 fix: add missing clamp_limit for CompressedTensorsWNA16MoE (#27328) Qeeweew 2026-06-17 10:29:57 +08:00
9c53853ea3 Use pack topk ids triton kernel for flashinfer_trtllm_routed (#25702) Trevor MorrisandClaude Opus 4.7 2026-06-16 19:27:14 -07:00
093908d4c0 [LoRA] Fix chunked SGMV (csgmv) CUDA graph segment replay (#28371) Yanbin Jiang 2026-06-16 19:19:07 -07:00
71b090a8e7 [Ascend]GLM 5.2 deployment (#28433) chenxu214 2026-06-17 09:55:50 +08:00
74155d32bd [NPU] [DOC] Update Ascend NPU docs: HDK 25.5.2, Triton 3.2.1.dev20260530 (#28311) amote-i 2026-06-17 09:38:31 +08:00
0d651e653b [AMD] Update v4 amd cookbook (#28423) Thomas Wang 2026-06-17 09:15:16 +08:00
37ef295c78 [AMD] Feat/dp moe reduce scatter (#28216)
kk wunhuang Wang, FangYuan
2026-06-17 09:12:39 +08:00
d86a7e7018 Custom spec algorithm can handle server args (#28162) jasonjk-park 2026-06-16 17:13:52 -07:00
ca84d52b78 [Router] [Docs] Refresh policy-selection notes (#28321) Jia-Wei Jiang 2026-06-17 08:07:19 +08:00
2ad00faae1 [ci] add kimi nvfp4 nightly tests (#28467) Qiaolin Yu 2026-06-16 16:49:51 -07:00
4f9b12c5dd [XPU] Guard tvm_ffi import in dsv4 compress modules under TYPE_CHECKING (#28426) ashwini rathi 2026-06-17 04:22:12 +05:30
a10eee3d80 [Tokenizer] Fix abort racing server crash when large amount of aborts (#28341) Hanming Lu 2026-06-16 14:49:35 -07:00
b8b8992dde docs: add Amazon SageMaker AI deployment guide (#28338) Jyothirmai Kottu 2026-06-16 13:33:22 -07:00
9b4432fe18 [HiCache]Asymmetric pool support direct backend (#28446) huangtingwei 2026-06-17 04:17:57 +08:00
c0a6c3ce66 Fix circular import when sglang.srt.model_executor.runner_backend is imported first (#28002) Jason Mancuso 2026-06-16 15:04:34 -04:00
13537f8e20 Unskip Marlin NVFP4 tests (#27589)
Brayden Zhong Brayden Zhong shaunkotek
2026-06-16 11:58:22 -07:00

... 44 45 46 47 48 ...

Commit Graph Select branches Hide Pull Requests dsv41-pd dsv41-pd-visioncp main Mono Color

Commit Graph

Select branches

Hide Pull Requests

dsv41-pd

dsv41-pd-visioncp

main