minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
17,848 Commits 3 Branches 0 Tags
30705c004ca4bbfc92216dfaf845da14d84c4c4d
Commit Graph
73 Commits
Author SHA1 Message Date
Alex NailsandAlison Shao 28262c20df [CI][RFC] Replace black-jupyter with ruff-format (#37210) 
Co-authored-by: Alison Shao <a.shao@wustl.edu>
 2026-09-02 19:46:08 -07:00
Oguz Ulgen f15748d965 [bench] Support real-traffic replay with early-stop-aware steady-state metrics in bench_one_batch_server (#37469) 2026-09-02 17:20:25 -07:00
Wes 2d9c64394f Fix reasoning metrics and add TPOT to bench_multiturn (#35443) 2026-09-02 11:28:47 +08:00
Liangsheng Yin 2530204502 [mem_cache] Make release, row-reuse asserts, and presence checks read the KV record (#37167) 2026-08-31 12:46:15 -07:00
Shuwen Wang 9e9d26a4af Fix Mooncake serving benchmark trace rows (#37201) 2026-08-31 11:43:59 -07:00
Cheng WanandClaude Opus 5 fd40a331bf config: a parallel size has one spelling; a patched scope declares its own (#36621) 
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-27 12:56:42 -07:00
Cheng WanandClaude Opus 5 ca1d7ed8e6 config: a parallel leaf with no live counterpart is read bare (#36620) 
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-27 12:56:11 -07:00
Cheng Wan 413df1f8db config: ServerArgs holds the raw input (#36255) 2026-08-26 05:14:05 -07:00
Cheng Wan 937af8538b config: the runtime readers take the published bags (#36254) 2026-08-26 05:08:25 -07:00
Cheng Wan ae5feb4b9c config: stop handing the record to code that does not read it (#36252) 2026-08-26 05:02:17 -07:00
Cheng Wan d7b144f64e config: publishing is the process entry's job (#36251) 2026-08-26 04:58:41 -07:00
Cheng Wan 8005df61d3 config: spell the parallel config tier at the call site (#36250) 2026-08-26 03:00:28 -07:00
BingjiaWang 3c481b9421 [Benchmark] Add optional steady-state window for serving metrics (#30918) 2026-08-24 14:49:20 -07:00
Cheng WanandClaude Opus 5 362c2ee849 config: borrowed-record reads follow the config bags (#35908) 
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-23 01:19:20 -07:00
Cheng WanandClaude Opus 5 64aa859da2 config: constructing a config no longer resolves it (#35907) 
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-23 01:18:53 -07:00
R0CKSTARandAlex Nails d90318b3e2 [MLX] Upgrade to Torch 2.13/MLX 0.32+ and redesign the Torch-MLX tensor bridge (#32984) 
Co-authored-by: Alex Nails <alex.nails@radixark.ai>
 2026-08-21 18:51:42 -07:00
Han YuWenhui ZhuAlex Nails
 		6b94d39f13 [Model Loading] Overlap checkpoint staging with CUDA graph capture during startup (#32017) 
Co-authored-by: Wenhui Zhu <wzhu59@asu.edu>
Co-authored-by: Alex Nails <alex.nails@radixark.ai>
 2026-08-13 12:26:25 -07:00
sglang-botsglang-botBrayden ZhongJimmy Shonghnyls2002Alex NailsLiangsheng Yin
 		fde9ad2531 [Feature] Add Muse Glimmer model support (#34262) 
Co-authored-by: sglang-bot <232288953+sglang-bot@users.noreply.github.com>
Co-authored-by: Brayden Zhong <brayden.zhong@radixark.ai>
Co-authored-by: Jimmy Shong <69131491+Jiminator@users.noreply.github.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: Alex Nails <alex.nails@radixark.ai>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
 2026-08-11 15:41:52 -07:00
Khoa PhamandClaude Opus 5 ce1b9f88b6 config: derive the runner's DCP topology from its ParallelState (#34133) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-09 01:18:24 -07:00
Mohammad Miadh Angkad 8a1637a479 Fix serving benchmark post-warmup cache flush race (#33663) 2026-08-06 15:27:10 +00:00
silencejade 593777c046 [FIX] [benchmark] Fix flush_cache failure after warmup by waiting for server idle (#33527) 2026-08-05 21:27:43 +08:00
Liangsheng Yin 558c9bdcc2 [misc] Improve benchmark determinism and dataset API coverage (#33255) 2026-08-02 01:39:50 -07:00
Yuwei AnandClaude Fable 5 3e0f7c3f30 [BCG][3/N] Enable bcg on dsa & deepep a2a backend (#31987) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-31 16:45:18 -07:00
Rain Jiang 4af8ddb576 support rust sglang server (#29799) 2026-07-31 11:56:31 -07:00
Mick 22faf9fef8 embedding: centralize capabilities and complete OpenAI compatibility (#32481) 2026-07-30 10:28:52 +08:00
Xiaoyu Zhang 075bd97952 [Benchmark] Remove obsolete auto-benchmark remnants (#31941) 2026-07-21 20:44:52 +08:00
fzyzcjy e77d95c3d5 Pass per-forward overrides to ForwardBatch.init_new as explicit arguments (#30670) 2026-07-15 14:25:59 +08:00
Mick 43124cdd90 fix: fix image benchmark backend parity (#30867) 2026-07-15 10:11:22 +08:00
fzyzcjy 6999007a13 Drop ModelRunner's duplicated parallel-degree fields and read them via self.ps (#31165) 2026-07-14 16:02:40 +08:00
fzyzcjy 725920915f Introduce ModelRunner.ps ParallelState (#31161) 2026-07-14 16:01:14 +08:00
sglang-botsglang-botClaude CodeCodexLiangsheng YinLiangsheng Yin
 		6cc9352dfe [Spec] Add DSpark: confidence-scheduled speculative decoding (#30261) 
Co-authored-by: sglang-bot <232288953+sglang-bot@users.noreply.github.com>
Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
 2026-07-12 17:25:26 -05:00
Mick af66370d81 bench: support random image resolutions (#30879) 2026-07-12 08:28:56 +08:00
Liangsheng Yin 348e6fd29b [Fix] Guard kernel OOB accesses and harden runtime edge cases (#30847) 2026-07-11 14:22:13 -05:00
Liangsheng Yin 078f06fbf4 [Refactor] Share chat encoding dispatch between serving and offline tools (#30623) 2026-07-09 02:45:42 -07:00
Cheng Wan e703f9e566 [refactor] Adopt get_parallel() everywhere and close out the parallel wrapper surface (#30492) 2026-07-09 02:09:39 -07:00
Liangsheng Yin bc5d376c2c [Bench] Add fixed-prompt mode and per-request spec accept length metrics (#30615) 2026-07-09 02:06:04 -07:00
Khoa PhamCursorClaude Fable 5
 		3a679459e5 [bench] Add agentic-trace multi-turn dataset to bench_serving (#29215) 
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-06 19:45:44 -07:00
Lijuan Tang 3cdc2415b1 [MLX] Fix step-bounded profiling for bench tools on Apple Silicon (#29217) 2026-06-30 22:55:29 -07:00
Polisetty V R K Jyothendra Varma d5abafcc1c [Intel GPU] add pytorch profiling support for XPU in bench offline throughput and enhance num steps (#28308) 
Signed-off-by: P V R K Jyothendra Varma <polisettyvarma@gmail.com>
 2026-06-29 09:10:15 +08:00
fzyzcjy 7002a37ea1 Inline extend_range accessors and remove the extend_input_len/fill_len properties (#27611) 2026-06-25 08:56:44 +08:00
Liangsheng Yin e3f1fa9d8e [misc] Unify benchmark deprecation shims and one_batch_server CLI entrypoint (#29108) 2026-06-23 21:10:25 -07:00
Alex Tumanov 52b89c4948 [misc] Add bench_serving compatibility shim (#28997) 2026-06-23 19:49:53 -07:00
Alex Tumanov 33373cbb12 [misc] Move bench_serving into sglang.benchmark (#28996) 2026-06-23 19:34:11 -07:00
Liangsheng Yin f76c6c95be [misc] Add sglang.bench_offline_throughput deprecation shim (#29085) 2026-06-23 18:39:58 -07:00
Liangsheng Yin b448b08401 [misc] Move bench_offline_throughput into sglang/benchmark/ with a back-compat shim (#28747) 2026-06-23 18:37:52 -07:00
Liangsheng Yin 11e7c9e0e6 [misc] Add sglang.bench_one_batch deprecation shim (#29082) 2026-06-23 14:57:35 -07:00
Liangsheng Yin c864c8d9c2 [misc] Move bench_one_batch into sglang/benchmark/ with a back-compat shim (#28687) 2026-06-23 14:48:35 -07:00
vikram singh shekhawatandClaude Sonnet 4.6 e63b57da0b [Fix] model init / XPU / transformers-v5 / bench-image fixes (#28292) 
Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
 2026-06-23 12:58:13 +08:00
Lianmin ZhengandYinghai Lu 45d203fb08 Fix tokenizer state cleanup on dispatch failure (#28694) 
Co-authored-by: Yinghai Lu <yinghai@meta.com>
 2026-06-19 21:55:39 -07:00
Liangsheng Yin d271de64fe [misc] Move bench_one_batch_server into sglang/benchmark/ with a back-compat shim (#28625) 2026-06-19 14:19:29 -07:00