minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
13,962 Commits 2 Branches 0 Tags
d1a39b0c74ed7f20dd67a68624a3bab1e588ec71
Commit Graph
13962 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Khoa PhamandQiaolin Yu d1a39b0c74 [DeepSeek V3] Defer moe finalize and fused it with main stream add (#27720) 
Co-authored-by: Qiaolin Yu <liin1211@outlook.com>
 2026-06-12 19:00:51 -07:00
Baizhou Zhang a14d1a5656 Add DeepSeek V4 MTP acceptance length checks (#28098) 2026-06-12 18:53:20 -07:00
IBRAHIM IBRAHIMandClaude Opus 4.8 45f8d48994 docs: add llm-d page under Advanced Features (#28078) 
Signed-off-by: IBRAHIM IBRAHIM <66755652+Ibrahim2595@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-12 18:40:31 -07:00
Brayden ZhongandBrayden Zhong 1a19f66acb Fix DP attention + EP mode of Nemotron (#28102) 
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
 2026-06-12 18:39:04 -07:00
Yuwei AnandClaude Fable 5 6c3e429ba1 [Tiny] Cuda Graph Refactor Code Style Follow up (#28107) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-06-12 18:38:13 -07:00
zijiexiaandClaude Fable 5 29ac249be1 docs: add cookbook-migrate-model skill from the Qwen3.5 pilot (#27845) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-06-12 18:31:52 -07:00
Liangsheng Yin 5633ca8599 [Spec] Move prepare_for_draft to EagleDraftWorkerBase (#28105) 2026-06-12 17:42:43 -07:00
Liangsheng Yin 5f315c74da [CI] Enforce modern stage=/runner_config= form for dispatchable test suites (#28108) 2026-06-12 17:32:54 -07:00
Ting SUN 335a9c7837 fix(frontend): return HTTP 400 for out-of-vocabulary token_ids_logprob (#28088) 
Signed-off-by: Ting Sun <suntcrick@gmail.com>
 2026-06-12 17:18:26 -07:00
Khoa PhamandMohammad Miadh Angkad 82eedd5bd0 [DeepEP] Enable fabric handles automatically when supported (#27107) 
Co-authored-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>
 2026-06-12 17:17:44 -07:00
amote-i c26669e37b [NPU] [DOC] Update server arguments to NPU support features page (#28083) 2026-06-12 17:16:52 -07:00
Jonah Bernardandjonahbernard fda7955890 [AMD][DFlash] Enable Fused KV Materialization (#27854) 
Co-authored-by: jonahbernard <Jonah.Bernard@amd.com>
 2026-06-12 16:57:46 -07:00
Cheng Wan 3a1417a0c1 [refactor] Fold FrozenKVMTPCudaGraphRunner onto the shared DecodeCudaGraphRunner base (#28081) 2026-06-12 16:55:53 -07:00
Liangsheng Yin 87554c7855 [Spec] Move draft-extend prep to EagleDraftWorkerBase; unify prepare_for_* names (#28093) 2026-06-12 16:44:04 -07:00
YAMY f23f48df98 fix(moe): make FlashInfer A2A robust to collapsed global_num_tokens (moe_dense_tp_size NaN) (#27945) 2026-06-12 16:11:44 -07:00
Cheng WanandClaude Opus 4.8 d601edab73 [Spec] Fix EagleDraftWorker draft-extend attn backend assignment (#28096) 
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-12 15:47:05 -07:00
Liangsheng Yin fd977adbd6 Fix PR-close cancellation skipping workflows beyond the first 30 (#27785) 2026-06-12 15:39:33 -07:00
Kangyan-ZhouandClaude Fable 5 b0b8436f1c [Fix] Unquote ResponseTool annotation breaking lint on all PRs (#28095) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-06-12 15:20:36 -07:00
Michael 54989b1fd0 [AMD] ci: add label-gated extra-a tier (kv_canary + mock_model unit tests) (#27822) 2026-06-12 15:12:56 -07:00
Brayden ZhongandBrayden Zhong 95867f0932 [Doc] Fix some inconsistencies in the Nemotron Cookbook (#28087) 
Co-authored-by: Brayden Zhong <brayden.zhong@radixark.ai>
 2026-06-12 14:51:58 -07:00
Xinyuan TongKai-Hsun ChenKristin CowalcijkaerostaglaziermagBlake LeddenPanJasonLeoyzenkennyu
 		85712fa5b0 Fix Responses API request handling (#25881) 
Co-authored-by: Kai-Hsun Chen <kaihsun@apache.org>
Co-authored-by: Kristin Cowalcijk <kristincowalcijk@gmail.com>
Co-authored-by: aerosta <63026763+aerosta@users.noreply.github.com>
Co-authored-by: glaziermag <glaziermag@users.noreply.github.com>
Co-authored-by: Blake Ledden <blake.ledden@gmail.com>
Co-authored-by: PanJason <pyyjason@gmail.com>
Co-authored-by: Leoyzen <leoyzen@gmail.com>
Co-authored-by: kennyu <966806+kennyu@users.noreply.github.com>
 2026-06-12 14:47:55 -07:00
+3
Xinyuan TongJairo David Campaña RoseroKaran BansaleasonYufeng HeqingchanghanClaude Opus 4.7Ajay AnuboluRavitez DondetiRatish PXiaoshuai ZhangRicardo-M-LXinyuan Tong
 		b3270264e4 Fix Anthropic Messages API compatibility (#25876) 
Co-authored-by: Jairo David Campaña Rosero <jairocampana10001@gmail.com>
Co-authored-by: Karan Bansal <3264937+karanb192@users.noreply.github.com>
Co-authored-by: eason <85663565+mango766@users.noreply.github.com>
Co-authored-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
Co-authored-by: qingchanghan <17794466+qingchanghan@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Ajay Anubolu <124525760+AjAnubolu@users.noreply.github.com>
Co-authored-by: Ravitez Dondeti <13931987+dondetir@users.noreply.github.com>
Co-authored-by: Ratish P <114130421+Ratish1@users.noreply.github.com>
Co-authored-by: Xiaoshuai Zhang <15795935+jetd1@users.noreply.github.com>
Co-authored-by: Ricardo-M-L <69202550+Ricardo-M-L@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <xinyuan.tong@radixark.ai>
 2026-06-12 14:46:57 -07:00
Liangsheng Yin caf59759ea [Spec] Centralize dummy verify-input capture; add carries_draft_hidden_states (#28032) 2026-06-12 14:23:29 -07:00
Michael 65d76bd3f6 [AMD] Fix CI base-a fwd_occupancy: disable SGLANG_SANITIZE_NAN_LOGITS in AMD CI (#28075) 2026-06-12 14:14:34 -07:00
ybyang 1e71c1a859 fix(pd): disable overlap for spec+grammar in disagg decode loop (#28039) 2026-06-12 14:08:36 -07:00
Mohammad Miadh Angkad cb9140ee61 Enable PDL for GPT-OSS tinygemm router (#27941) 2026-06-12 13:51:50 -07:00
shuwenn 9d37e710b7 [Bench] Add consistent p90/p95/p99 percentiles for all latency metrics (#27662) 2026-06-12 13:44:38 -07:00
Zach Zhu 627ed3476b Fix invalid KVFP4QuantizeUtil references (#28013) 
Signed-off-by: Zach Zhu <zzqshu@126.com>
 2026-06-12 13:44:11 -07:00
Vedant V JhaveriandVedant Jhaveri 3be5a7ec89 Respect explicit --max-running-requests instead of clamping to heuristic (#27399) 
Co-authored-by: Vedant Jhaveri <vjhaveri@linkedin.com>
 2026-06-12 12:30:47 -07:00
Zhichen ZengJiajun LiZhichenzzz
 		6e0fa5afe1 Support Nemotron DP attention and MTP (#24955) 
Co-authored-by: Jiajun Li <48857426+guapisolo@users.noreply.github.com>
Co-authored-by: Zhichenzzz <northwesterniemsteaching@gmail.com>
 2026-06-12 12:21:12 -07:00
David Wang bb33594c1a flashinfer swa kv pool fix (dflash gemma 4) (#27737) 2026-06-12 11:35:05 -07:00
Mohammad Miadh Angkadandzijiexia fa4273d2db [Docs] Add Kimi K2.7 Code cookbook (#28064) 
Co-authored-by: zijiexia <37504505+zijiexia@users.noreply.github.com>
 2026-06-12 11:19:44 -07:00
cctry 75998d0421 Fix --mem-fraction-static not accounting for EAGLE draft model KV cache (#23862) 2026-06-12 10:35:48 -07:00
Alex NailsandClaude Opus 4.8 533b59d00c Add @alexnails as codeowner for srt/platforms (#28066) 
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-12 10:28:58 -07:00
Yuan Luoandluoyuan.luo c80d8fe78a [Perf] Skip per-call mat_a/scales_a padding in cutlass FP8 blockwise GEMM (#27896) 
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
 2026-06-12 12:01:53 -04:00
Xinyuan Tong 9f6b2339f9 docs(minimax-m3): warm-steady-state benchmark numbers (#28062) 2026-06-12 08:42:23 -07:00
Yuhao YangandXinyuan Tong 50815d54a7 docs (#28061) 
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2026-06-12 21:19:40 +08:00
Xinyuan Tong dba617f2ec doc: update docs for new model (#28060) 2026-06-12 21:14:35 +08:00
qiaozpandishandhanani fcae6767d5 Add bucketed multi-dir layout for NIXL file storage (#27672) 
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
 2026-06-12 05:34:55 -07:00
luoroger37 18989f3d48 [PD] Fix resource leak on prealloc/transfer abort and idle check (#28022) 2026-06-12 18:20:24 +08:00
ming_wang f308abc052 Revise the mimo-v2-flash best practice (#28016) 2026-06-12 17:12:33 +08:00
liuxproandunknown 83c0007f8d examples: add Chain-of-Verification (CoVe) hallucination reduction demo (#27866) 
Co-authored-by: unknown <liuxiao.209@360buyad.local>
 2026-06-12 01:57:33 -07:00
Xinyi Song 371b96e210 [AMD] Fix DeepSeek-V4-Flash-FP8 on MI300 (#27972) 2026-06-12 01:51:32 -07:00
Joectwm 694cea8656 Add EPD disaggregated encode tracing (#25994) 2026-06-12 16:16:14 +08:00
iridiumine 60e4f14953 [NPU][Bugfix] Fix accuracy issue in no-graph with MTP (#27752) 2026-06-12 16:05:06 +08:00
Liangsheng Yin a52ccd2179 [Spec] Install EagleDraftExtendInput as the V2 draft-extend spec_info (#24860) 2026-06-12 00:46:16 -07:00
Bingxu Chen 1cd5cb1220 [AMD] [CI] Add dsv4 accuracy PR gate to pr-test-amd-rocm720 (#27149) 2026-06-11 23:42:16 -07:00
Xinyuan Tong 66ab5c9c7a fix(gateway): make sgl-model-gateway a cargo workspace so maturin 1.14 accepts the parent README (#27997) 2026-06-11 23:37:29 -07:00
ashwini rathi fe887e6935 test(xpu): add multi-feature and embedding stage-b tests (#27861) 2026-06-12 14:36:28 +08:00
Bi Xueandispobock 3c1f9eafa5 [sgl] proactively release out-of-window SWA slots after chunked prefill (#27402) 
Co-authored-by: ispobock <ispobaoke@gmail.com>
 2026-06-12 14:17:18 +08:00