minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
12,689 Commits 2 Branches 0 Tags
33f1d3915f4446887aae016934ae608faf16c74e
Commit Graph
8435 Commits
Author SHA1 Message Date
Alex Nails 4df42da658 Introduce CudaDeviceMixin and CudaSRTPlatform (#24096) 2026-05-15 10:59:02 -07:00
Cheng WanandClaude Opus 4.7 ee93795476 perf(mla): hybrid Triton fused cat+FP8-quantize for MLA chunked-prefill K/V (#25333) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-15 10:51:00 -07:00
Kai-Hsun ChenandClaude Opus 4.7 17c8a2fa53 [Llama4] Use strided in-place fused QK RMSNorm to drop a redundant copy (#25089) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-15 10:33:08 -07:00
Zhangheng 21b3ac52b4 [UnifiedTree]: Fix UnifiedRadixCache device match semantics with HiCache (#25277) 2026-05-16 00:40:55 +08:00
Cheng Wan 3f7e538b2f Add NPU condition for cosine and sine caching (#25399) 2026-05-15 05:21:53 -07:00
Chi McIsaac 20123e0b16 [diffusion] fix: mount Cache-DiT before torch.compile in native denoising (#25328) 2026-05-15 18:08:39 +08:00
zhaochen zhang 12408ec668 [Disagg] Add retry with exponential backoff for prefill bootstrap reg… (#25125) 2026-05-15 16:38:51 +08:00
Baizhou Zhangsglang-botb8zhongmmangkad
 		0c19540550 [Fix] Fix gpt oss triton kernels and upgrade flashinfer back to 0.6.11.post1 (#25335) 
Co-authored-by: sglang-bot <sglang-bot@users.noreply.github.com>
Co-authored-by: b8zhong <b8zhong@users.noreply.github.com>
Co-authored-by: mmangkad <mmangkad@users.noreply.github.com>
 2026-05-15 01:04:56 -07:00
shuwennandClaude Opus 4.7 66ef97c00f [Lint] Fix Optional[X] = (None,) typo defaults in two dataclasses (#25252) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-15 00:32:40 -07:00
Xiaoyu ZhangandBBuf Codex aaaad9e7c2 [Codex] Diffusion tune Hunyuan3D shape export chunks (#24358) 
Co-authored-by: BBuf Codex <bbuf-codex@users.noreply.github.com>
 2026-05-15 15:31:01 +08:00
Chi McIsaac 4b6b434dfe [diffusion] fix: fix Z-Image Cache-DiT sequence-parallel override (#25305) 2026-05-15 13:25:00 +08:00
WenhaoZhangandniehen6174 6cfa4c9c2f [diffusion] fix: respect dit_precision config instead of hardcoded bfloat16 in DenoisingStagge (#24988) 
Co-authored-by: niehen6174 <niehen.6174@gmail.com>
 2026-05-15 13:22:24 +08:00
Cheng WanandClaude Opus 4.7 ad4994dc1d DeepseekV2MoE: defer shared experts when routed kernel is non-mutating (#25279) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-14 22:20:55 -07:00
origithub-actions[bot]zhiguo.qinR0CKSTAR
 		897587b03a [MUSA]: Add flashinfer sampling backend (#24978) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: zhiguo.qin <zhiguo.qin@mthreads.com>
Co-authored-by: R0CKSTAR <yeahdongcn@gmail.com>
 2026-05-14 20:23:15 -07:00
Zhanghengispobock晟海
 		d9fa84b25b [UnifiedTree]: Support HiCache For DeepSeek_V4 (#24691) 
Co-authored-by: ispobock <ispobaoke@gmail.com>
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
 2026-05-14 20:20:52 -07:00
Liangsheng Yin d89b678d69 move dead sglang.test files to test/manual (#25316) 2026-05-14 20:02:44 -07:00
YAMY 8d5b347edd Support Qwen3.5 NVFP4 MTP DeepEP (#24906) 2026-05-15 10:49:14 +08:00
Qiaolin Yu 7af4320d67 [perf] fix kimi tokenizer to improve ttft (#25265) 2026-05-14 19:11:25 -07:00
Cheng Wan dca9ba6321 perf(mla): TMA bulk-store set_mla_kv_buffer (up to 12× over baseline) (#25311) 2026-05-14 18:23:41 -07:00
Alison Shaohnyls2002Liangsheng Yin
 		ba214ef3d3 ci: tag-gated nightly migration — foundation + 40 whole-file moves (#24725) 
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
 2026-05-14 16:11:15 -07:00
Liangsheng Yin 67096f48bf Revert "[MoE] Decouple Mega MoE from DeepEP backend" (#25317) 2026-05-14 16:00:41 -07:00
Charles Chen 78408d2300 Fix frozen kv MTP crash when bonus_tokens is None (#25204) 2026-05-14 15:33:55 -07:00
Liangsheng Yin 22dfcdaa04 revert flashinfer 0.6.11 bumps (#25310) 2026-05-14 15:28:58 -07:00
Yuhao Yang 88d3ed7df1 Enable SGLANG_OPT_FP8_WO_A_GEMM by default (#25181) 2026-05-15 02:09:13 +08:00
Yuhao Yang 37f030a0de [MoE] Decouple Mega MoE from DeepEP backend (#24884) 2026-05-15 02:01:44 +08:00
Le ZhangandClaude Opus 4.7 3fc60e5748 [MLX] bench_one_batch: thread --quantization through to MlxModelRunner (#25221) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-14 09:56:42 -07:00
Jae B. 90afd680f3 [Apple Silicon] [MLX] Auto-detect MLX-format quantization_config dict (#25191) 2026-05-14 09:55:37 -07:00
Liangsheng Yin 4593bbdf31 ci: dynamic partition + LPT from live sglang-ci-stats model (#25263) 2026-05-14 02:35:59 -07:00
Xingyu Liu 2279b79f35 Add --model-config-parser registry for pluggable config formats (#25050) 
Signed-off-by: Xingyu Liu <charlotteliu12x@gmail.com>
 2026-05-14 01:54:06 -07:00
YAMY 4be25f2428 fix(disagg): broadcast bootstrap port across multi-node prefill ranks (#24378) 2026-05-14 16:39:01 +08:00
Hsiu-Chun, Hunggithub-actions[bot]YC Yen-Ching Tseng
 		bc265c5f82 [AMD] Add amd jit resolve token ids bench ci (#25210) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: YC Yen-Ching Tseng <yctseng@amd.com>
 2026-05-14 00:13:12 -07:00
Hsiu-Chun, Hunggithub-actions[bot]YC Yen-Ching Tseng
 		7b128e143a [AMD] Add amd jit clamp position bench ci (#25209) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: YC Yen-Ching Tseng <yctseng@amd.com>
 2026-05-14 00:03:56 -07:00
Yihao Wang 41eb2d861b [fix] load_audio: fall back to soundfile when torchcodec fails on WAV with trailing metadata (#24185) 2026-05-14 15:02:49 +08:00
YC Yen-Ching Tseng 22bfae0d1d [AMD] Auto-fallback NSA indexer to page_size=1 when aiter preshuffle gluon kernel is unavailable (Deepseek v3.2) (#25205) 2026-05-14 00:01:56 -07:00
Qiaolin Yu 421179c453 [perf] avoid hidden states d2h when return_hidden_states=false (#25155) 2026-05-13 23:47:17 -07:00
AndyLi429 3640116397 [NPU]Bugfix:Set default values for npu_wrapper_preprocess parameters (#25130) 2026-05-14 14:21:11 +08:00
Teng Macopilot-swe-agent[bot]stmatengssZhangheng
 		be156d6804 [HiCache] enable ssd offload support for mooncake store (#24277) 
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: stmatengss <11641725+stmatengss@users.noreply.github.com>
Co-authored-by: Zhangheng <hzh0425@apache.org>
 2026-05-13 23:07:02 -07:00
Ke Bao e4378ff37f [UnifiedRadixCache] Fix HiCache load back start node (#25088) 2026-05-14 13:18:12 +08:00
Tejas Dharamsi c016246b0f [Rerank] Early-exit logprob scan and hoist math import (#25046) 2026-05-13 22:01:07 -07:00
Piotr MazurekandPiotr Mazurek a6a6c3119b LFM2: pass has_initial_state to causal_conv1d_fn for prefill (#24717) 
Co-authored-by: Piotr Mazurek <piotr.mazurek@liquid.ai>
 2026-05-13 21:57:58 -07:00
Liangsheng Yin 5c11c2492f ci: emit machine-readable TIMINGS block at end of run_unittest_files (#25232) 2026-05-13 21:56:54 -07:00
Pai Liu fd889097dc [Bug Fix] Add priority property to DecodeRequest to fix AttributeError with --enable-priority-scheduling (#25064) 2026-05-14 11:54:25 +08:00
xutizhouSisyphusrootCheng Wan
 		c701a08765 feat: [2/2][DeepEP] Add waterfill load balancing for shared expert dispatch (#19290) 
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Co-authored-by: root <aichenf@nvidia.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
 2026-05-13 19:23:41 -07:00
Lianmin Zhengandcctry 5fb6bde6c0 Add sglang:get_loads_duration_seconds metric (#25163) 
Co-authored-by: cctry <cctry@meta.com>
 2026-05-13 18:38:49 -07:00
Baizhou Zhangandpranjalssh b7f856df70 DeepSeek V4 w4a4 MegaMoE (#25052) 
Co-authored-by: pranjalssh <adkz.photos@gmail.com>
 2026-05-13 18:35:32 -07:00
Colin ZBowen BaoHAI
 		34c0029f0a [diffusion] [AMD] feat: support online MXFP4 and fp8 quantization (#21431) 
Co-authored-by: Bowen Bao <bowenbao@amd.com>
Co-authored-by: HAI <hixiao@gmail.com>
 2026-05-14 08:52:01 +08:00
Qiaolin Yu 7618ad7075 [attn backend] Integrate tokenspeed_mla prefill/decode kernels (fp8 kv cache, blackwell) (#24925) 2026-05-13 17:36:17 -07:00
Liangsheng Yin 22d3f3996c ci: decouple stage and runner for cuda registry (#25197) 2026-05-13 17:28:21 -07:00
Cheng WanandClaude Opus 4.7 6c0633b0b1 fix(nvfp4): make process_weights_after_loading hot-reload-safe via alias-when-same-shape (#25190) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-13 16:57:39 -07:00
shiyu7 37f18438c5 [rebase]Deepseek_v4 support w4(mxfp4)a16 on hopper (#24986) 2026-05-13 16:33:46 -07:00