minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 13 Packages Projects Releases Wiki Activity
18,675 Commits 4 Branches 0 Tags
main
Commit Graph
18675 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Jonah Bernardandjonahbernard bcd45d3ac7 [AMD][DFlash] Add DFlash test to AMD CI (#28112) 
Co-authored-by: jonahbernard <Jonah.Bernard@amd.com>
 2026-06-12 22:06:41 -07:00
Mick 8becb37519 [diffusion] warmup: improve diffusion server warmup (#28119) 2026-06-13 13:04:10 +08:00
Yuan Luoandluoyuan.luo eb18416f9f [jit-kernel] Support per token group quant 8bit v2 jit kernel (#27449) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-06-13 12:15:09 +08:00
Michael 053e153183 [AMD] ci: register 8 attention-backend unit tests to run on AMD CI (#27817) 2026-06-12 20:52:45 -07:00
Xinyu Zhang 1800d7caa6 Bump ray minimum version to 2.55.1 (#27724) 2026-06-12 20:49:11 -07:00
mqhc2020 f288283c07 [AMD] move shared expert check function to quark (#27057) 2026-06-12 20:45:59 -07:00
EvanderFBoopandgemini-code-assist[bot] d8f8e89ffb [Test] Add unit tests for srt/mem_cache/utils.py (#27913) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-06-12 20:38:57 -07:00
Liangsheng Yin 32ef040618 [Spec] Move eagle verify prepare_for_verify/sample to eagle_utils free helpers (#28117) 2026-06-12 20:37:33 -07:00
Qiaolin Yu e02f7ca482 [perf] remove several h2d sync (#28076) 2026-06-12 20:30:17 -07:00
Lianmin Zheng b001d3e815 Add prajjwal1 to CI_PERMISSIONS.json (#28120) 2026-06-12 20:26:38 -07:00
Khoa Pham a0c6e0b3a4 chore: bump tokenspeed_mla 0.1.1 -> 0.1.6 (#28116) 2026-06-12 19:57:36 -07:00
Ting SUN ade30fd494 fix(server): serialize nested dict config values as JSON (#28094) 
Signed-off-by: Ting Sun <suntcrick@gmail.com>
 2026-06-12 19:27:31 -07:00
Ting SUN e9c3b262e4 [Bugfix][Spec] Fix multi-layer EAGLE DRAFT_EXTEND_V2 attn-TP logprob metadata capture (#28026) 
Signed-off-by: Ting Sun <suntcrick@gmail.com>
 2026-06-12 19:26:35 -07:00
Khoa PhamandQiaolin Yu d1a39b0c74 [DeepSeek V3] Defer moe finalize and fused it with main stream add (#27720) 
Co-authored-by: Qiaolin Yu <liin1211@outlook.com>
 2026-06-12 19:00:51 -07:00
Baizhou Zhang a14d1a5656 Add DeepSeek V4 MTP acceptance length checks (#28098) 2026-06-12 18:53:20 -07:00
IBRAHIM IBRAHIMandClaude Opus 4.8 45f8d48994 docs: add llm-d page under Advanced Features (#28078) 
Signed-off-by: IBRAHIM IBRAHIM <66755652+Ibrahim2595@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-12 18:40:31 -07:00
Brayden ZhongandBrayden Zhong 1a19f66acb Fix DP attention + EP mode of Nemotron (#28102) 
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
 2026-06-12 18:39:04 -07:00
Yuwei AnandClaude Fable 5 6c3e429ba1 [Tiny] Cuda Graph Refactor Code Style Follow up (#28107) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-06-12 18:38:13 -07:00
zijiexiaandClaude Fable 5 29ac249be1 docs: add cookbook-migrate-model skill from the Qwen3.5 pilot (#27845) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-06-12 18:31:52 -07:00
Liangsheng Yin 5633ca8599 [Spec] Move prepare_for_draft to EagleDraftWorkerBase (#28105) 2026-06-12 17:42:43 -07:00
Liangsheng Yin 5f315c74da [CI] Enforce modern stage=/runner_config= form for dispatchable test suites (#28108) 2026-06-12 17:32:54 -07:00
Ting SUN 335a9c7837 fix(frontend): return HTTP 400 for out-of-vocabulary token_ids_logprob (#28088) 
Signed-off-by: Ting Sun <suntcrick@gmail.com>
 2026-06-12 17:18:26 -07:00
Khoa PhamandMohammad Miadh Angkad 82eedd5bd0 [DeepEP] Enable fabric handles automatically when supported (#27107) 
Co-authored-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>
 2026-06-12 17:17:44 -07:00
amote-i c26669e37b [NPU] [DOC] Update server arguments to NPU support features page (#28083) 2026-06-12 17:16:52 -07:00
Jonah Bernardandjonahbernard fda7955890 [AMD][DFlash] Enable Fused KV Materialization (#27854) 
Co-authored-by: jonahbernard <Jonah.Bernard@amd.com>
 2026-06-12 16:57:46 -07:00
Cheng Wan 3a1417a0c1 [refactor] Fold FrozenKVMTPCudaGraphRunner onto the shared DecodeCudaGraphRunner base (#28081) 2026-06-12 16:55:53 -07:00
Liangsheng Yin 87554c7855 [Spec] Move draft-extend prep to EagleDraftWorkerBase; unify prepare_for_* names (#28093) 2026-06-12 16:44:04 -07:00
YAMY f23f48df98 fix(moe): make FlashInfer A2A robust to collapsed global_num_tokens (moe_dense_tp_size NaN) (#27945) 2026-06-12 16:11:44 -07:00
Cheng WanandClaude Opus 4.8 d601edab73 [Spec] Fix EagleDraftWorker draft-extend attn backend assignment (#28096) 
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-12 15:47:05 -07:00
Liangsheng Yin fd977adbd6 Fix PR-close cancellation skipping workflows beyond the first 30 (#27785) 2026-06-12 15:39:33 -07:00
Kangyan-ZhouandClaude Fable 5 b0b8436f1c [Fix] Unquote ResponseTool annotation breaking lint on all PRs (#28095) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-06-12 15:20:36 -07:00
Michael 54989b1fd0 [AMD] ci: add label-gated extra-a tier (kv_canary + mock_model unit tests) (#27822) 2026-06-12 15:12:56 -07:00
Brayden ZhongandBrayden Zhong 95867f0932 [Doc] Fix some inconsistencies in the Nemotron Cookbook (#28087) 
Co-authored-by: Brayden Zhong <brayden.zhong@radixark.ai>
 2026-06-12 14:51:58 -07:00
Xinyuan TongKai-Hsun ChenKristin CowalcijkaerostaglaziermagBlake LeddenPanJasonLeoyzenkennyu
 		85712fa5b0 Fix Responses API request handling (#25881) 
Co-authored-by: Kai-Hsun Chen <kaihsun@apache.org>
Co-authored-by: Kristin Cowalcijk <kristincowalcijk@gmail.com>
Co-authored-by: aerosta <63026763+aerosta@users.noreply.github.com>
Co-authored-by: glaziermag <glaziermag@users.noreply.github.com>
Co-authored-by: Blake Ledden <blake.ledden@gmail.com>
Co-authored-by: PanJason <pyyjason@gmail.com>
Co-authored-by: Leoyzen <leoyzen@gmail.com>
Co-authored-by: kennyu <966806+kennyu@users.noreply.github.com>
 2026-06-12 14:47:55 -07:00
+3
Xinyuan TongJairo David Campaña RoseroKaran BansaleasonYufeng HeqingchanghanClaude Opus 4.7Ajay AnuboluRavitez DondetiRatish PXiaoshuai ZhangRicardo-M-LXinyuan Tong
 		b3270264e4 Fix Anthropic Messages API compatibility (#25876) 
Co-authored-by: Jairo David Campaña Rosero <jairocampana10001@gmail.com>
Co-authored-by: Karan Bansal <3264937+karanb192@users.noreply.github.com>
Co-authored-by: eason <85663565+mango766@users.noreply.github.com>
Co-authored-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
Co-authored-by: qingchanghan <17794466+qingchanghan@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Ajay Anubolu <124525760+AjAnubolu@users.noreply.github.com>
Co-authored-by: Ravitez Dondeti <13931987+dondetir@users.noreply.github.com>
Co-authored-by: Ratish P <114130421+Ratish1@users.noreply.github.com>
Co-authored-by: Xiaoshuai Zhang <15795935+jetd1@users.noreply.github.com>
Co-authored-by: Ricardo-M-L <69202550+Ricardo-M-L@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <xinyuan.tong@radixark.ai>
 2026-06-12 14:46:57 -07:00
Liangsheng Yin caf59759ea [Spec] Centralize dummy verify-input capture; add carries_draft_hidden_states (#28032) 2026-06-12 14:23:29 -07:00
Michael 65d76bd3f6 [AMD] Fix CI base-a fwd_occupancy: disable SGLANG_SANITIZE_NAN_LOGITS in AMD CI (#28075) 2026-06-12 14:14:34 -07:00
ybyang 1e71c1a859 fix(pd): disable overlap for spec+grammar in disagg decode loop (#28039) 2026-06-12 14:08:36 -07:00
Mohammad Miadh Angkad cb9140ee61 Enable PDL for GPT-OSS tinygemm router (#27941) 2026-06-12 13:51:50 -07:00
shuwenn 9d37e710b7 [Bench] Add consistent p90/p95/p99 percentiles for all latency metrics (#27662) 2026-06-12 13:44:38 -07:00
Zach Zhu 627ed3476b Fix invalid KVFP4QuantizeUtil references (#28013) 
Signed-off-by: Zach Zhu <zzqshu@126.com>
 2026-06-12 13:44:11 -07:00
Vedant V JhaveriandVedant Jhaveri 3be5a7ec89 Respect explicit --max-running-requests instead of clamping to heuristic (#27399) 
Co-authored-by: Vedant Jhaveri <vjhaveri@linkedin.com>
 2026-06-12 12:30:47 -07:00
Zhichen ZengJiajun LiZhichenzzz
 		6e0fa5afe1 Support Nemotron DP attention and MTP (#24955) 
Co-authored-by: Jiajun Li <48857426+guapisolo@users.noreply.github.com>
Co-authored-by: Zhichenzzz <northwesterniemsteaching@gmail.com>
 2026-06-12 12:21:12 -07:00
David Wang bb33594c1a flashinfer swa kv pool fix (dflash gemma 4) (#27737) 2026-06-12 11:35:05 -07:00
Mohammad Miadh Angkadandzijiexia fa4273d2db [Docs] Add Kimi K2.7 Code cookbook (#28064) 
Co-authored-by: zijiexia <37504505+zijiexia@users.noreply.github.com>
 2026-06-12 11:19:44 -07:00
cctry 75998d0421 Fix --mem-fraction-static not accounting for EAGLE draft model KV cache (#23862) 2026-06-12 10:35:48 -07:00
Alex NailsandClaude Opus 4.8 533b59d00c Add @alexnails as codeowner for srt/platforms (#28066) 
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-12 10:28:58 -07:00
Yuan Luoandluoyuan.luo c80d8fe78a [Perf] Skip per-call mat_a/scales_a padding in cutlass FP8 blockwise GEMM (#27896) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-06-12 12:01:53 -04:00
Xinyuan Tong 9f6b2339f9 docs(minimax-m3): warm-steady-state benchmark numbers (#28062) 2026-06-12 08:42:23 -07:00
Yuhao YangandXinyuan Tong 50815d54a7 docs (#28061) 
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2026-06-12 21:19:40 +08:00