sglang

minke.yu/sglang

Fork 0

e6efe10072 [AMD] Register 5 JIT kernel unit tests for AMD nightly CI (#29197) Michael 2026-06-25 13:12:58 -07:00
118d6b2e5e [Cleanup] Style and type annotation improvements extracted from #28688 (#29224) Lianmin Zheng 2026-06-25 12:51:15 -07:00
3344b73c80 Add DeepSeek V4 Pro GB300 nightly and expand Kimi K25 nightly test (#28103) Baizhou Zhang 2026-06-25 12:13:52 -07:00
f9a3720e2b Skip empty non-idle output batches (#28504) weireweire 2026-06-26 02:40:01 +08:00
9495737d82 Fix CI broken by #28450 (#29308) Brayden ZhongandBrayden Zhong 2026-06-25 10:35:34 -07:00
4d06d4c97f Sync Gemma4 hardware table with Blackwell recipes (#29266) Xiaoyu Zhang 2026-06-25 22:57:33 +08:00
52c32035eb [diffusion] Add Qwen-Image ModelOpt NVFP4 support (#28928)
Xiaoyu Zhang jingyu-ml Claude Opus 4.8
2026-06-25 22:56:09 +08:00
890b38c211 [diffusion] doc: fix diffusion docs and cookbook drift (#29302) Mick 2026-06-25 21:13:00 +08:00
bc150173b2 [AMD] Fix stage-b-test-1-gpu-small-amd-nondeterministic timeout after VLM model swap (#29234) YC Yen-Ching Tseng 2026-06-25 18:56:04 +08:00
0075c8f02b [AMD] [GLM5] GLM-5.1 MXFP4 (MI355X) + enable EAGLE for gfx950 in cookbook (#29194) Raiden MakotoandRaiden-Makoto 2026-06-25 03:54:48 -07:00
ddda4f9028 [Bugfix] Fix Ministral3 init argument forwarding (#29111) Xiaoyu Zhang 2026-06-25 18:26:45 +08:00
e4976683f4 [Docs] Fix broken links in cookbook (#29261) Mohammad Miadh Angkad 2026-06-25 17:31:55 +08:00
3d3a7ec031 [AMD] fix(moe): correct fused shared-expert scaling on aiter/DeepEP path (mori all-to-all) (#28237) Rita Brugarolas 2026-06-25 01:56:30 -07:00
ec12a28a87 [AMD] Register 7 JIT kernel unit tests for AMD nightly CI (#28967) Michael 2026-06-25 01:51:30 -07:00
4ba8634780 [AMD] Register scripted-core chunked-prefill test for AMD extra-a CI (#29058) Michael 2026-06-25 01:50:08 -07:00
de2d01c8da [AMD] Fuse shared-expert append + DeepEP remap into one Triton kernel (#28450) Rita Brugarolas 2026-06-25 01:22:57 -07:00
4a8200565e Fused QK GemmaRMSNorm + RoPE + gate kernel for Qwen3.5 (#28320) Yuhao Yang 2026-06-25 15:58:54 +08:00
2812a3c93a [Spec] Unify spec/non-spec decode result handling and overlap relay-payload gating (#29225) Liangsheng Yin 2026-06-25 00:44:38 -07:00
32bae911f6 Add CI permission for jiayisunx (#28899) jiayisunx 2026-06-25 14:49:07 +08:00
d717e926c1 fix(runner): prevent eager token buffer under-allocation (#28894) jiayisunx 2026-06-25 14:37:19 +08:00
9215da2515 [AMD] [GLM5] skip redundant -inf pre-fill of HIP indexer MQA-logits (#28757) Raiden MakotoandRaiden-Makoto 2026-06-24 23:21:39 -07:00
72cac88022 [NPU] perf: precompute mamba conv-state track indices once per batch (#29105) AndyLi429 2026-06-25 14:04:56 +08:00
7c9804ef21 Add MiMo V2.5 Blackwell vision FA4 recipe (#29253) Xiaoyu Zhang 2026-06-25 13:47:32 +08:00
9fd6d0ec99 [LoRA] BF16 support + EP cuda-graph crash fix for experimental_sgl_trtllm MoE-LoRA (#28953) Yanbin JiangandYusheng Su 2026-06-24 21:16:50 -07:00
a9270250c3 Dllm radix cache npu (#27144) wenxuewuhdandronnie_zheng 2026-06-25 11:44:03 +08:00
efbe67d237 Tune Gemma4 26B-A4B B200 memory recipe (#29252) Xiaoyu Zhang 2026-06-25 11:31:25 +08:00
d1cf09d4d4 [AMD-miles] Add a ROCm 7.0 MI35x miles nightly image (#29219) Zhiyao JiangandXinyu Jiang 2026-06-24 23:27:06 -04:00
c268a7edf5 [Bugfix] Fix inverted PP-disable condition for diffusion LLM inference (#27700) CrazyCoderandClaude Opus 4.8 2026-06-25 11:18:00 +08:00
8b7a1e908a Revert Gemma4 modelopt fp4 MoE backend change (#28347) Mohammad Miadh Angkad 2026-06-25 10:49:09 +08:00
40439acd0a Sync backend docs with #29063 (#29233) Mohammad Miadh Angkad 2026-06-25 10:48:34 +08:00
8314247d9d [Spec] Merge dflash triton kernels into a single dflash.py (#29228) Liangsheng Yin 2026-06-24 18:57:15 -07:00
ae6f787637 [diffusion] rl: add sleep/wake support for diffusion engine (#22659)
Fenglin Yu klhhhhh zhaochenyang20 shuwenn MikukuOvO
2026-06-24 18:37:27 -07:00
67b2a9ed0c [NPU] Fix the DeepSeek-V2-Coder model accuracy issue (#29042) 看海的人 2026-06-25 09:14:32 +08:00
4e155ecc31 [NPU] adaptation to support operator FA3 in deterministic inference on NPU. (#26724) chx96642264 2026-06-25 09:12:57 +08:00
c7734e6871 [Spec] Dissolve EagleDraftInputV2Mixin so spec-info dataclasses hold data only (#29220) Liangsheng Yin 2026-06-24 18:08:30 -07:00
2b64fc7a2c Remove Req.extend_logprob_start_len field and make it pure (#27625) fzyzcjy 2026-06-25 09:06:14 +08:00
2c3f007a65 [CPU] optimize GDN prefill performance (#29117) Ma Mingfei 2026-06-25 09:04:34 +08:00
7002a37ea1 Inline extend_range accessors and remove the extend_input_len/fill_len properties (#27611) fzyzcjy 2026-06-25 08:56:44 +08:00
9305d10099 [NPU] adapt_fused_rope_qk_mqa_optimize (#28872) cen121212 2026-06-25 08:55:52 +08:00
d0524d6433 Avoid scattered assignment of extend_input_len and fill_len by merging them into Req.extend_range (#27610) fzyzcjy 2026-06-25 08:33:28 +08:00
6c839368e0 [Cleanup] IPC struct renames, better typing, and SenderWrapper removal (#29214) Lianmin Zheng 2026-06-24 17:25:24 -07:00
563c3418a7 Avoid dual semantics of extend_input_len by computing the candidate on the fly (#27616) fzyzcjy 2026-06-25 08:17:37 +08:00
5d4e63d49e Sync the changes in #23402 (#29063) Brayden ZhongandBrayden Zhong 2026-06-24 17:01:02 -07:00
f82addd4a8 Support online MXFP8 quantization for ungated MoE (#27939) Brayden ZhongandBrayden Zhong 2026-06-24 16:58:48 -07:00
e4bf0043fe [fix] Skip routed expert capture for draft model under spec v2 (#26980) Jiajun LiandClaude Opus 4.8 2026-06-24 16:56:07 -07:00
7e63feee6f Add scheduler metrics extension hooks (#29207) Lianmin ZhengandYinghai Lu 2026-06-24 15:50:05 -07:00
e26bceb81e [perf] simplify _apply_cuda_graph_metadata for draft extend in trtllm_mla backend (#29077) Qiaolin Yu 2026-06-24 15:45:26 -07:00
85a3f1f75c [Spec] Unify the overlap stash relay behind a RelayPayload dataclass (#29124) Liangsheng Yin 2026-06-24 15:36:54 -07:00
c6822f81b1 [Spec] Make the overlap bonus-token relay unconditional (#29122) Liangsheng Yin 2026-06-24 15:34:02 -07:00
d3dd184e3e [Spec] Fold DFlash verified_id into the shared bonus_tokens relay channel (#29118) Liangsheng Yin 2026-06-24 15:30:29 -07:00
3c95a87b66 Fix the CuDNN failure on bmm_fp8 when two libcudart.so exists. (#29201) Brayden ZhongandBrayden Zhong 2026-06-24 15:26:46 -07:00
5f30fa258c Support DP-aware PD router dispatch (#26245) weireweireandweireweire 2026-06-25 06:02:51 +08:00
fd87a85388 Convert SamplingParams to msgspec Struct (#29198) Lianmin ZhengandRain Jiang 2026-06-24 13:42:45 -07:00
d5e9176f65 [BCG][GLM5] perf: BCG support and prefill enhancements (#27053) Kaixi 2026-06-24 22:13:35 +02:00
d46afbf8b4 Fix nightly CI test for GLM-4.6 + B200 (#28749) Brayden ZhongandBrayden Zhong 2026-06-24 12:34:11 -07:00
c01ad10d16 [perf] tiny optimize select_index op for draft extend (#29078) Qiaolin Yu 2026-06-24 12:27:13 -07:00
2c697daf5f [Cookbook] Nemotron3-Ultra: align MTP draft depth with NVIDIA reference (num_steps 5) (#29200) Brayden ZhongandBrayden Zhong 2026-06-24 12:12:34 -07:00
e97cc339e3 Add DeepSeek V4 Flash demo notebook (#28952) Anusha Pant 2026-06-24 11:42:33 -07:00
d5c566e59b Extract profile request cleanups (#29098) Lianmin Zheng 2026-06-24 11:22:58 -07:00
d6aacd2801 Handle input-embed-only batches in eager runner (#29121) Lianmin ZhengandCheng Wan 2026-06-24 10:33:52 -07:00
76db6c9d9e [Refactor] Share CUDA graph memory pool across prefill and decode (#28973) cctryandcctry 2026-06-24 10:18:20 -07:00
dd4caf9459 [Docs] fix SGLang-diffusion installation links (#29139) Yihao Wang 2026-06-24 09:46:06 -07:00
4a4f063b79 [diffusion] fix: paint multiview vae must follow unit dtype (#29041) Mick 2026-06-24 23:54:01 +08:00
03773ae35b [HiCache] Add NIXL FILE cache cleaner (#28258) qiaozpandishandhanani 2026-06-24 23:13:55 +08:00
09b808ab7e [diffusion] fix: fix Qwen-Image-Layered latent shape (#28832) Mick 2026-06-24 20:57:54 +08:00
84a7a84018 [PD] Fix data race in NixlKVManager for NIXL backend (#28897) nvjullin 2026-06-24 18:11:01 +08:00
7430c56b20 [Misc] Use logger instead of print() in utils/common.py (#29004) hirakunaramuka2andClaude Fable 5 2026-06-24 17:45:01 +08:00
10e0bcd622 [NPU] fix dcp break (#29140) zhaozx-cn 2026-06-24 17:33:02 +08:00
73d976e375 [NPU] [DOC] Fix TOC of Ascend NPU Docs (#29129) amote-i 2026-06-24 16:47:10 +08:00
4992f7a108 Fix TRTLLM MHA FP8 KV cache scale handling (#28144) Mohammad Miadh Angkad 2026-06-24 16:43:50 +08:00
0fc815aa2c [CI] Temporarily disable openbmb MiniCPM tests (#29095) Yuhao Yang 2026-06-24 16:39:44 +08:00
8e1988b746 [Perf] Overlap result D2H copy with the next forward step (#29075) Liangsheng Yin 2026-06-24 01:34:02 -07:00
e04ed05193 [CI] reduce CPU CI scope with base-c suite (#28623) jundu 2026-06-24 16:18:06 +08:00
4da23fdc35 [NPU] removes deprecated pr testing files (#27859) Even Zhou 2026-06-24 16:06:45 +08:00
dd2d919e21 [Docs] Fix mem-fraction-static default and document how it is computed (#29135) zijiexiaandClaude Opus 4.8 2026-06-24 01:05:06 -07:00
39f5de9ba1 [NPU] bump sgl-kernel-npu to 2026.6.2 (#29101) Even Zhou 2026-06-24 15:59:41 +08:00
00eac7ec9b [diffusion] fix: disable proxy env in warmup health probe (#28503) Tai An 2026-06-24 00:58:44 -07:00
a707b2054d [CI] Fix pre-commit failures in MLX backend tests (#29141) zijiexiaandClaude Opus 4.8 2026-06-24 00:56:52 -07:00
5f76736427 kimik2_detector fix the normal text detection before tool call. (#25071)
Rohit Harkhani gemini-code-assist[bot]Rohit Harkhani
2026-06-24 13:23:22 +05:30
b6a8000473 [bugfix][decode hicache] _storage_hit_query use HybridPrefetchOperation (#28422) weibingoandZhangheng 2026-06-24 15:13:48 +08:00
6c6fa19a90 [NPU] Support fsdp for rl_on_policy_target (#29128) silencejade 2026-06-24 14:57:19 +08:00
26e1d4d847 [KDA-Pilot] Add B200 diffusion norm-scale-shift CUDA fast path for Qwen-Image (#27392) Xiaoyu Zhang 2026-06-24 14:38:36 +08:00
0df796473b [VLM] Qwen3-VL / Moss-VL ViT preprocessing optimizations (#28940) Yuan Luoandluoyuan.luo 2026-06-24 14:36:29 +08:00
534ac98eb2 [codex] Optimize DMD Wan auto residency on high-memory GPUs (#28780) Xiaoyu Zhang 2026-06-24 14:03:23 +08:00
6842335fcf [MUSA][24/N] CI:Fix LLM server smoke test (#28934) Stella-17andxinyue.fan 2026-06-24 13:58:59 +08:00
c394f812d1 [MLX] Fix Apple Silicon server startup; align MLX tests with upstream (#28770) toufupiand“toufupi” 2026-06-24 13:58:49 +08:00
5beddc8afc [Diffusion] Add Krea 2 support (#29052) Yihao WangandClaude Opus 4.8 2026-06-23 22:56:51 -07:00
24bf8d91bb Refactor / simplify MultiLayerEagleDraftExtendCudaGraphRunner to use rotation (#28492) Tarushii Goel 2026-06-23 22:36:47 -07:00
bf231f01a3 [qwen3.5][XPU]Add XPU support for set_embed_and_head and fused QK RMSNorm kernel (#27870) jiayisunxandMa Mingfei 2026-06-24 13:03:49 +08:00
e3f1fa9d8e [misc] Unify benchmark deprecation shims and one_batch_server CLI entrypoint (#29108) Liangsheng Yin 2026-06-23 21:10:25 -07:00
52b89c4948 [misc] Add bench_serving compatibility shim (#28997) Alex Tumanov 2026-06-23 21:49:53 -05:00
33373cbb12 [misc] Move bench_serving into sglang.benchmark (#28996) Alex Tumanov 2026-06-23 21:34:11 -05:00
f76c6c95be [misc] Add sglang.bench_offline_throughput deprecation shim (#29085) Liangsheng Yin 2026-06-23 18:39:58 -07:00
b448b08401 [misc] Move bench_offline_throughput into sglang/benchmark/ with a back-compat shim (#28747) Liangsheng Yin 2026-06-23 18:37:52 -07:00
20b2817bdf [AMD] Enable BCG on ROCm + route aiter prefill via MHA during PCG/BCG capture for Kimi-2.5 (#27833) karverma-amdandsogalin_codegen 2026-06-23 20:22:41 -05:00
5338e44483 [Intel GPU] fix triton-mla attention on XPU by limiting max_kv_splits to 8 which is default (#28646) Polisetty V R K Jyothendra VarmaandMa Mingfei 2026-06-24 06:36:13 +05:30
7454735be9 [AMD] [GLM5] Add opt-in Triton fp8 sparse-MLA prefill kernel for gfx950 (#28975) Raiden MakotoandRaiden-Makoto 2026-06-23 18:04:07 -07:00
5e6d7c1615 [AMD] Fix DeepSeek-V4 fp8 KV path on gfx942 (e4m3fnuz) (#28455) Raiden MakotoandRaiden-Makoto 2026-06-23 17:45:13 -07:00
c07811bfc6 [AMD] Register MI355X 2N 1P1D disagg nightly workflow for testing (#29091) Michael 2026-06-23 17:41:55 -07:00
b2c8f7a22e [AMD] Support triton backend decode context parallel for Qwen3.5 (#25090)
Wang, FangYuan Baizhou Zhang Kangyan-Zhou Khoa Pham Hubert Lu zhengyao
2026-06-24 08:35:56 +08:00

... 41 42 43 44 45 ...

Commit Graph Select branches Hide Pull Requests dsv41-pd dsv41-pd-visioncp main Mono Color

Commit Graph

Select branches

Hide Pull Requests

dsv41-pd

dsv41-pd-visioncp

main