minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 15 Packages Projects Releases Wiki Activity
16,201 Commits 2 Branches 0 Tags
fe6a05a8e81f29f63e0bb8a721bdac5ac5762b1e
Commit Graph
11103 Commits
Author SHA1 Message Date
Nikhil KulkarniandClaude Opus 5 fe6a05a8e8 fix: preserve priority for batched embedding requests (#32977) 
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-06 19:26:18 -07:00
Yuwei An db8f3cdd11 fix(gdn): skip the -1 padding sentinel in the chunked extend kernel (#33810) 2026-08-06 18:48:47 -07:00
MickandClaude Fable 5 1e08b865f9 [diffusion] feat: support K/V-gather style sequence parallel (CP-like) attention (#32667) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-07 09:39:28 +08:00
Mick 9ee658d4f6 [diffusion] CI: fix output-rank test fixture (#33878) 2026-08-07 09:33:50 +08:00
Dustin LuongandJimmy Shong 2c3ecf32f1 [Fix] Reformat /vertex_generate successful predictions (#33446) 
Co-authored-by: Jimmy Shong <69131491+Jiminator@users.noreply.github.com>
 2026-08-07 00:00:48 +00:00
Hao Zhangandzhisbug af7c62e337 Fix paged SWA retraction resume accounting (#33794) 
Co-authored-by: zhisbug <1654062+zhisbug@users.noreply.github.com>
 2026-08-06 16:32:54 -07:00
Po-Han Huang (NVIDIA) bae29f716a Fix inference mode mismatch in FlashInfer warmup (#33788) 2026-08-06 16:11:41 -07:00
Oguz UlgenandLianmin Zheng b38caebf09 [AMD] Enable gfx1250 sgl-kernel builds (#32466) 
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
 2026-08-06 15:47:17 -07:00
valechen 18e6c61c21 [AMD] perf: compact Triton extend-attention for ragged prefill (AMD/HIP-only) (#29677) 2026-08-06 14:46:10 -07:00
Brayden Zhong dd7e4c91e2 Fix Mistral-Large-3 EAGLE draft skipping DeepseekV2Model.__init__ (#33785) 2026-08-06 13:59:28 -07:00
Yuhao Yang 971932d661 [Kimi-K3] Allow DSPARK verify on cutedsl_mla (fold_sq) (#33650) 2026-08-06 13:54:25 -07:00
YAMY 2fc557254b fix(PP): size the mamba pool per pipeline stage, not per whole model (#33666) 2026-08-06 13:10:43 -07:00
Mohammad Miadh AngkadandBrayden Zhong 434e646282 [Deps] Upgrade CUDA PyTorch stack to 2.13 (#28836) 
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
 2026-08-06 12:08:44 -07:00
Ziang LiandBrayden Zhong 4ad990ba7d [ModelOpt FP4] Support online MoE weight quantization (#33115) 
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
 2026-08-06 11:01:55 -07:00
YAMYandShangming Cai 05c7ebf64c [Disagg][StagingBuffer][2/2] Support radix cache (#30545) 
Co-authored-by: Shangming Cai <csmthu@gmail.com>
 2026-08-06 23:59:35 +08:00
Mohammad Miadh Angkad 8a1637a479 Fix serving benchmark post-warmup cache flush race (#33663) 2026-08-06 15:27:10 +00:00
Mick 7195b8e4c7 [diffusion] refactor: validate and document spectrum controls (#33851) 2026-08-06 23:23:11 +08:00
Mick 44bde3911a [diffusion] fix: resolve IPC A2A peers from process groups (#33848) 2026-08-06 23:22:41 +08:00
Mick c212a6938c [diffusion] chore: retire released warmup and decoder flags (#33850) 2026-08-06 23:02:32 +08:00
Xiaoyu ZhangandClaude Fable 5 591cfb0881 [diffusion] FLUX.2 bit-exact residual-gate fast path (H200 klein-4B 50-step denoise -1.2%) (#33823) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-06 22:54:40 +08:00
Xiaoyu ZhangandClaude Fable 5 dd98c9572a [diffusion] Generalize the FLUX.2 VAE decoder fast path to AutoencoderKL (Z-Image / FLUX.1) behind quality=high (#33818) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-06 22:53:15 +08:00
Peng Wu 21225aba3d [Scheduler] Fix to restrict the SWA chunk-cap escape hatch to true head-of-line livelock (#32700) 2026-08-06 22:09:04 +08:00
Mick 183bd80add [diffusion] chore: centralize entrypoint API hygiene (#33845) 2026-08-06 21:53:40 +08:00
Mick 2132cdef16 [diffusion] chore: consolidate pipeline core hygiene (#33843) 2026-08-06 21:51:34 +08:00
Mick 45dfd80674 [diffusion] refactor: simplify disaggregation transport hygiene (#33844) 2026-08-06 21:50:32 +08:00
Mohammad Miadh Angkad 1a15cf1536 Gate multimodal feature transport by model capability (#33653) 2026-08-06 06:48:32 -07:00
AuFlowandAuFlow e8d0fe92e9 [diffusion] Fix GLM-Image resolution alignment (#32999) 
Co-authored-by: AuFlow <AuFlow@users.noreply.github.com>
 2026-08-06 15:41:45 +03:00
Xiaoyu ZhangandClaude Fable 5 3654740347 [diffusion] ERNIE-Image bit-exact fused RMSNorm+scale/shift (H200 1024^2 e2e 15.63 -> 15.00 s, denoise -3.3%) (#33854) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-06 19:58:44 +08:00
Xiaoyu ZhangandClaude Fable 5 295784723a [diffusion] Ideogram 4: fuse RMSNorm modulate/gate chains via the Z-Image Triton suite behind quality=high (H200 e2e -2.9%/-3.4%) (#33822) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-06 19:57:34 +08:00
Xiaoyu ZhangandClaude Fable 5 eff6a11350 [diffusion] FLUX.1 bit-exact residual-gate fast path + tanh-GELU epilogue behind quality=high (H200 e2e -1.1% lossless / -4.3% high) (#33819) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-06 19:56:38 +08:00
mohbasitBasitHAI
 		f8f2870a84 Profiling Enhancements [1/3]: cuda graph profile traces (#24370) 
Co-authored-by: Basit <mohbasit@ctr2-alola-ctrl-01.amd.com>
Co-authored-by: HAI <hixiao@gmail.com>
 2026-08-06 03:19:35 -07:00
Zhiqiang Xie f6de147b8d Remove revoke queue after hit-then-alloc refactoring (#33613) 2026-08-06 03:10:10 -07:00
YAMYandLee Nau 8b29c90218 [NVIDIA] Enable CuTe DSL BF16 GEMM on SM107 (#33617) 
Co-authored-by: Lee Nau <lnau@nvidia.com>
 2026-08-06 02:06:08 -07:00
kangwangamd 4cdab7b4f4 [AMD] Add msgpack to ROCm diffusion deps (fix multimodal-gen unit test ModuleNotFoundError) (#31899) 2026-08-06 01:52:23 -07:00
weireweireandweireweire fe55d78b7d Fix MXFP4 scale placeholder initialization (#33500) 
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
 2026-08-06 01:46:54 -07:00
Liangsheng Yin efc99a86ff [CI] Restore the full prefill CUDA graph capture range in test launches (#33847) 2026-08-06 01:40:10 -07:00
Baizhou Zhang 0e584529f5 [CI] Remove profiling from nightly tests (#33832) 2026-08-06 01:16:08 -07:00
sglang-botandsglang-bot c11ce7c514 chore: bump sgl-kernel version to 0.4.6.post1 (#33842) 
Co-authored-by: sglang-bot <sglang-bot@users.noreply.github.com>
 2026-08-06 01:09:48 -07:00
Mohammad Miadh Angkad 04374ba5e0 Update AOT kernels for Torch 2.13 (#33841) 2026-08-06 01:03:26 -07:00
YC Yen-Ching Tseng b8140f36ea Fix lint error (#33830) 2026-08-06 00:53:40 -07:00
MickandClaude Fable 5 bfce378e5f [diffusion] feat: capture-safe pynccl all-to-all (#33775) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-06 15:41:05 +08:00
Xinyuan Tong 31c1e5943f Facade DSA index-cache: MTP topk-reuse state + index-K storage (#28609) 2026-08-06 00:34:31 -07:00
YC Yen-Ching TsengandHaiShaw cf7923615c [AMD] Gate DFLASH non-greedy verify on the target-only kernel being registered (#33694) 
Co-authored-by: HaiShaw <hixiao@gmail.com>
 2026-08-06 00:10:22 -07:00
Baizhou Zhang 5d1a0c7129 Revert "Warn on risky serving-time Triton work" (#33826) 2026-08-05 23:36:10 -07:00
Kan WuClaude Fable 5Cursor
 		32e5d788bd [mm] rust-server: native multimodal processing for Qwen VL (integrate sglang-mm, e2e) (#32365) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-08-05 23:35:50 -07:00
hjzhanghjzhangZhanghengshuwenn
 		8e11feb68e [HiCache] Support packed and sidecar draft caches for MTP/EAGLE/DSpark (#30393) 
Co-authored-by: hjzhang <hjzhang89.gmail.com>
Co-authored-by: Zhangheng <hzh0425@apache.org>
Co-authored-by: shuwenn <47200617+alphabetc1@users.noreply.github.com>
 2026-08-06 14:31:11 +08:00
Xiaoyu ZhangandClaude Fable 5 b6876fc652 [diffusion] ERNIE-Image bit-exact residual-gate fast path (H200 1024^2 e2e 16.17 -> 15.75 s) (#33734) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-06 13:43:09 +08:00
MickandClaude Fable 5 604d3561b0 [diffusion] feat: data-parallel serving (--dp-size) (#33725) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-06 11:50:13 +08:00
Liangsheng Yin 9bd1461757 [CI] Bound the CUDA graph capture range in test launches and lift the spec fixture's admission cap (#33776) 2026-08-05 20:00:30 -07:00
Cheng Wan 4ea227fa91 config: the draft runner carries its own attention backend 
`build_draft_tp_worker` built a `ServerArgs` variant whose only job was to make
four config reads answer with the draft's backend instead of the target's, and
published it for the duration of the build so the bags agreed. The backend is a
per-runner fact — target and draft coexist in one process — so it moves onto the
runner, and the variant and the construction-time publish both go away.

`ModelRunner` takes `draft_attention_backend` and resolves the runner's effective
value once (`resolve_draft_attention_backend`: the algorithm's resolved backend,
else `--speculative-draft-attention-backend`, else None for a target runner);
`TpModelWorker` threads it to both runner constructions.
`resolve_attention_backend_strs` reads it off the runner, and `ModelRunner`
stamps the resolved pair *before* building backends so a backend can read it
while it constructs — which is what the FlashInfer KV-access check needs now that
it no longer asks the config. `configure_kv_cache_dtype` and the draft backend
factory read the runner too.

One latent bug falls out: the non-hybrid branch of the backend build ignored the
resolved pair and re-read `server_args.attention_backend`, which is why the
variant had to set that field as well as the split pair. It now uses the value
that was resolved for the runner.

`draft_server_args_overrides` and the `preserve_config()` publish switch are
deleted; with them goes the last production `ServerArgs.derive` outside
pre-publish config building, and the last construction-time publish. The
chunked-prefix gate the target resolved simply stays in the bags, since nothing
re-projects them.
 2026-08-05 19:32:24 -07:00