minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
6,884 Commits 2 Branches 0 Tags
4a56fa5cf2e2efb7eb4e6fd730bf581b39be21fa
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Lianmin ZhengandStefan He 22f641ab4f [minor] remove debug code in python/sglang/srt/compilation/weak_ref_tensor_jit.py (#13235) 
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
 2025-11-14 12:22:24 -08:00
Lianmin Zhenggithub-actions[bot]Stefan He
 		7a2254b260 [Auto Sync] Update backend.py, forward_batch_info.py, piece... (20251113) (#13221) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
 2025-11-13 18:54:11 -08:00
Lianmin Zhenggithub-actions[bot]gemini-code-assist[bot]
 		19f6a33ce9 [Auto Sync] Update pynccl_wrapper.py, environ.py, registry.... (20251111) (#13097) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-11-13 15:58:47 -08:00
Lianmin ZhengandStefan He d646cf6347 [Auto Sync] Update test_deterministic.py (20251112) (#13128) 
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
 2025-11-12 11:12:23 -08:00
Lianmin Zheng 40b26b456b Simplify the BatchMultimodalOutput in io_struct.py (#12993) 2025-11-10 13:54:56 -08:00
Lianmin Zhenggithub-actions[bot]Stefan He
 		9ea2c686c7 [Auto Sync] Update batch_invariant_ops.py (20251109) (#12916) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
 2025-11-10 01:51:39 -08:00
Lianmin Zhenggithub-actions[bot]Stefan He
 		0296f1cdad [Auto Sync] Update activation.py, logits_processor.py, rota... (20251107) (#12853) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
 2025-11-07 22:07:51 -08:00
Lianmin Zheng ae62279071 Fix data parallel controller launch for num nodes > 2 (#12822) 2025-11-07 12:13:21 -08:00
Lianmin Zheng c7d57d5bb3 Fix CI and style (#12658) 2025-11-05 15:08:15 -08:00
Lianmin Zheng fb2e816e83 Fix server args for gpt oss so users can override the moe runner backend (#12696) 2025-11-05 11:36:59 -08:00
Lianmin Zhenggithub-actions[bot]SangBin Cho
 		55dfb539cf [Auto Sync] Update scheduler_metrics_mixin.py, collector.py (20251104) (#12647) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
 2025-11-04 13:56:14 -08:00
Lianmin ZhengKan Wucctry
 		f600866a44 Improve the metrics for PD (#12580) 
Co-authored-by: Kan Wu <wukanustc@gmail.com>
Co-authored-by: cctry <shiyang@x.ai>
 2025-11-03 22:10:57 -08:00
Lianmin Zheng 243c064df2 Remove the dependency of nccl.h in symmetric memory (#12571) 2025-11-03 16:11:00 -08:00
Lianmin ZhengandNicolas Castet 7a21d8b276 Reduce the overhead of nccl symmetric memory (#12524) 
Co-authored-by: Nicolas Castet <ncastet@nvidia.com>
 2025-11-03 11:56:27 -08:00
Lianmin Zheng 20315697f4 move all get_stream in sgl_kernel to c++ to reduce the launch overhead (#12521) 2025-11-02 13:15:05 -08:00
Lianmin Zheng 2d5605e89b Fix ci install to allow prerelease (#12449) 2025-10-31 02:22:15 -07:00
Lianmin ZhengandByron Hsu c0652d907b Clean up sgl kernel (#12413) 
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
 2025-10-31 01:13:34 -07:00
Lianmin Zheng 6a63a9852e minor code sync (#12403) 2025-10-30 12:49:31 -07:00
Lianmin Zheng 7cd716f776 Update news section in README.md (#12409) 2025-10-30 12:41:21 -07:00
Lianmin Zheng 7b36c47b3b Clean up attention backend selection code & Other minor rename (#12136) 2025-10-25 23:50:12 -07:00
Lianmin Zheng 8e70064c37 Clean up server launch code and multi tokenizer (#12132) 2025-10-25 16:40:27 -07:00
Lianmin Zhengandsglang-bot 3862661cd4 Fix token for CI monitor (#12127) 
Co-authored-by: sglang-bot <sglangbot@gmail.com>
 2025-10-25 12:38:26 -07:00
Lianmin Zhengandsglang-bot 428710c2ec clean up github tokens (#12126) 
Co-authored-by: sglang-bot <sglangbot@gmail.com>
 2025-10-25 12:23:54 -07:00
Lianmin Zheng 4caca1ba04 Clean up server args & Add CI scripts (#12124) 2025-10-25 11:53:57 -07:00
Lianmin Zhenggithub-actions[bot]Stefan He
 		ea13cb1452 [Auto Sync] Update test_deterministic.py, test_deterministi... (20251024) (#12083) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
 2025-10-24 22:28:01 -07:00
Lianmin Zheng ffc722a690 Revert "lang: support direct video inference" (#12038) 2025-10-23 19:21:31 -07:00
Lianmin ZhengandStefan He ab07cd3e5a [Auto Sync] Update test_deterministic_utils.py (20251023) (#12022) 
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
 2025-10-23 11:20:45 -07:00
Lianmin Zheng 6d535b719f Revert "Recapture cuda graph after model weight update to resolve IMA error " (#11980) 2025-10-22 11:50:26 -07:00
Lianmin Zheng 01f14a7ad2 [code move] move pp into a separate mixin (#11838) 2025-10-20 18:46:56 -07:00
Lianmin Zhenggithub-actions[bot]Kan Wu
 		43ad05907c [Auto Sync] Update scheduler.py, server_args.py (20251020) (#11875) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Kan Wu <wukanustc@gmail.com>
 2025-10-20 17:41:19 -07:00
Lianmin Zheng 6058fb520c Update CODEOWNERS for layer quantization path (#11818) 2025-10-18 21:17:17 -07:00
Lianmin Zheng 67e34c56d7 Fix install instructions and pyproject.tomls (#11781) 2025-10-18 01:08:01 -07:00
Lianmin ZhengSangBin ChoCheng WanCheng Wan
 		9eefe2c0b7 Set CUDA_VISIBLE_DEVICES to achieve one GPU per process (#9170) 
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: Cheng Wan <cwan@x.ai>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
 2025-10-17 17:30:06 -07:00
Lianmin Zhenggithub-actions[bot]Cheng Wan
 		fdd7c69d65 [Auto Sync] Update common.py (20251017) (#11782) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
 2025-10-17 15:03:42 -07:00
Lianmin ZhengStefan HeByron Hsu
 		b9a54e0968 [minor] sync code on python/sglang/test/test_deterministic.py and improve ci tests (#11777) 
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
 2025-10-17 14:25:22 -07:00
Lianmin Zheng cd7e1bd591 Sync code and test CI; rename some env vars (#11686) 2025-10-15 18:37:03 -07:00
Lianmin Zheng d314bf6010 Update install.md (#11631) 2025-10-14 14:34:46 -07:00
Lianmin Zhenggithub-actions[bot]Byron Hsu
 		b98cf39866 [Auto Sync] Update collector.py (20251014) (#11625) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
 2025-10-14 13:34:33 -07:00
Lianmin Zhenggithub-actions[bot]Stefan He
 		27d710457c [Auto Sync] Update scheduler.py, server_args.py (20251014) (#11623) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
 2025-10-14 13:20:03 -07:00
Lianmin Zheng f7f9f8eceb Update news section in README.md (#11598) 2025-10-14 00:49:39 -07:00
Lianmin Zhenggithub-actions[bot]Hanming Lu
 		ba2ce28fe9 [Auto Sync] Update model_config.py (20251014) (#11580) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
 2025-10-13 22:16:34 -07:00
Lianmin Zheng 5e3f7e7fa9 Minor: improve sampler & remove unused fields from model_config.py (#11531) 2025-10-13 11:04:44 -07:00
Lianmin Zheng 2ac46e94ef Sync changes on io_struct.py and deterministic ops (#11498) 2025-10-12 16:03:10 -07:00
Lianmin Zheng 5a6ec8f999 Fix unit tests (#11503) 2025-10-12 07:45:57 -07:00
Lianmin Zheng 6a653bb11b temporarily remove b200 tests (#11502) 2025-10-12 06:48:49 -07:00
Lianmin Zheng 548a57b1f3 Fix port conflicts in CI (#11497) 2025-10-12 06:46:36 -07:00
Lianmin Zheng 88e73ed048 Temporarily remove b200 tests (#11501) 2025-10-12 06:41:37 -07:00
Lianmin Zheng 61055cb309 Reorder PD disagg CI tests (#11438) 2025-10-10 17:56:49 -07:00
Lianmin Zheng b4408e6098 Revert "fix: fix video input for qwen3-vl" (#11437) 2025-10-10 12:44:40 -07:00
Lianmin Zheng 9b8ebb2798 move more files under srt/utils (#11285) 2025-10-09 16:46:15 -07:00
Lianmin Zheng 708f4ff490 Rename max_micro_batch_size -> pp_max_micro_batch_size (#11279) 2025-10-06 15:50:56 -07:00
Lianmin Zhenggithub-actions[bot]Sehoon Kim
 		e2daeb351c [Auto Sync] Update test_utils.py (20251006) (#11280) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Sehoon Kim <sehoon@x.ai>
 2025-10-06 15:49:57 -07:00
Lianmin Zheng 0e7b353009 Fix code sync scripts (#11276) 2025-10-06 15:35:01 -07:00
Lianmin Zheng eb30b888db Remove env var warnings for release (#11262) 2025-10-06 10:09:17 -07:00
Lianmin Zheng b6b4b56395 Update condition for sgl-kernel-benchmark-test (#11254) 2025-10-05 20:55:02 -07:00
Lianmin Zheng f8924ad74b update sgl kernel version to 0.3.14.post1 (#11242) 2025-10-05 20:30:40 -07:00
Lianmin Zheng 366a603e95 Use cu128 for torch audio to fix some CI tests (#11251) 2025-10-05 19:52:32 -07:00
Lianmin Zhenggithub-actions[bot]jzhou-xai
 		6b30e097ab [Auto Sync] Update io_struct.py (20251004) (#11206) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: jzhou-xai <jzhou@x.ai>
 2025-10-05 18:06:07 -07:00
Lianmin Zheng d645ae90a3 Rename runner labels (#11228) 2025-10-05 18:05:41 -07:00
Lianmin Zheng 148d8d485d Update DeepGEMM repository tag to specific commit (#11229) 2025-10-05 13:47:36 -07:00
Lianmin Zheng 6a261aaca5 Minor fixes for server_args, parallel_state, and test_deterministic.py (#11159) 2025-10-02 12:12:49 -07:00
Lianmin Zheng 097725bb66 Clean up parallel_state.py (#11148) 2025-10-02 01:09:13 -07:00
Lianmin Zheng 609f65ba23 Remove debug print statement from scheduler output (#11145) 2025-10-01 13:37:05 -07:00
Lianmin Zheng 2d62af6be5 Fix metrics and request tracing (TimeStats) (#11123) 2025-10-01 13:03:07 -07:00
Lianmin Zhenggithub-actions[bot]Sehoon Kim
 		5dddb331c4 [Auto Sync] Update base_grammar_backend.py, xgrammar_backen... (20250930) (#11115) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Sehoon Kim <sehoon@x.ai>
 2025-09-30 21:50:43 -07:00
Lianmin Zhenggithub-actions[bot]Cheng Wan
 		1237aa19ce [Auto Sync] Update fused_moe_triton_config.py (20250930) (#11099) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
 2025-09-30 00:57:16 -07:00
Lianmin Zheng e5281f84d5 Update CODEOWNERS for attention/ascend_backend.py (#11092) 2025-09-29 21:28:49 -07:00
Lianmin Zheng dda34c2f93 Fix mem fraction static for nightly tests (#11076) 2025-09-29 12:57:41 -07:00
Lianmin Zhengandsglang-bot a17e70f5cc Use more general heuristics to set the default value of --mem-fraction-static (#10975) 
Co-authored-by: sglang-bot <sglangbot@gmail.com>
 2025-09-29 10:11:03 -07:00
Lianmin Zheng 07440f5f34 Fix FusedSetKVBufferArg in RotaryEmbedding (#11003) 2025-09-28 11:17:27 -07:00
Lianmin Zheng f6bc3f529b Fix profiler (#10997) 2025-09-27 14:56:18 -07:00
Lianmin Zheng e56c64bfaf Update label field comment to indicate deprecation (#10970) 2025-09-26 12:59:59 -07:00
Lianmin Zheng 3e95aa1a09 Remove pull_request trigger from CI monitor workflow (#10932) 2025-09-25 19:40:38 -07:00
Lianmin ZhengYingchun Laigemini-code-assist[bot]
 		f68dd998b9 Rename customer label -> custom label (#10899) 
Co-authored-by: Yingchun Lai <laiyingchun@apache.org>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-25 16:19:53 -07:00
Lianmin Zheng 35ec2a45a8 [minor] Remove deprecated function get_ip (#10883) 2025-09-25 16:18:04 -07:00
Lianmin Zhenggithub-actions[bot]Hanming Lu
 		3e43eb137b [Auto Sync] Update model_config.py (20250925) (#10885) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
 2025-09-24 22:59:16 -07:00
Lianmin Zheng b1f0fc1c0b Add CI timeout guidelines (#10829) 2025-09-23 22:08:02 -07:00
Lianmin Zheng 32d893730f Revert "[fix][pd-disag]no need set next batch sampling info done in prefill" (#10828) 2025-09-23 17:02:01 -07:00
Lianmin Zhengandgithub-actions[bot] f47a2c67e6 [Auto Sync] Update load_config.py, model_config.py, configu... (20250923) (#10825) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-09-23 16:48:12 -07:00
Lianmin Zhenggithub-actions[bot]cctry
 		312bfc4c95 [Auto Sync] Update simple_eval_common.py (20250923) (#10824) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: cctry <shiyang@x.ai>
 2025-09-23 13:50:47 -07:00
Lianmin Zhenggithub-actions[bot]Cheng Wan
 		e290303ea1 [Auto Sync] Update elementwise.py (20250923) (#10823) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
 2025-09-23 13:50:22 -07:00
Lianmin Zheng ea338676b5 Clean up server args (#10770) 2025-09-23 00:22:32 -07:00
Lianmin Zhengandgithub-actions[bot] 113f8f65a2 [Auto Sync] Update configurer.py (20250923) (#10765) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-09-22 17:34:34 -07:00
Lianmin Zhengandcctry 38c00ed7a1 Fix multimodal registry and code sync scripts (#10759) 
Co-authored-by: cctry <shiyang@x.ai>
 2025-09-22 15:36:01 -07:00
Lianmin Zhengandgithub-actions[bot] f949ad5794 [Auto Sync] Update activation.py, chunk_cache.py, utils.py (20250917) (#10538) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-09-16 17:06:43 -07:00
Lianmin Zhenggithub-actions[bot]cctry
 		c49484a658 [Auto Sync] Update scheduler_profiler_mixin.py, rpd_utils.p... (20250916) (#10494) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: cctry <shiyang@x.ai>
 2025-09-16 17:02:20 -07:00
Lianmin Zhengandcctry 31fb19a0a2 [Auto Sync] Update registry.py (20250915) (#10484) 
Co-authored-by: cctry <shiyang@x.ai>
 2025-09-15 17:34:28 -07:00
Lianmin Zheng 50dc0c1e9c Run tests based on labels (#10456) 2025-09-15 00:29:20 -07:00
Lianmin Zheng f73aae0bfc Update GITHUB_TOKEN secret for documentation push (#10458) 2025-09-14 21:59:13 -07:00
Lianmin Zheng b354e3c90d [CI] Fix token key in label-pr.yml workflow (#10452) 2025-09-14 20:45:53 -07:00
Lianmin Zheng 65e6f48ce4 Update permissions in label-pr.yml (#10450) 2025-09-14 20:41:43 -07:00
Lianmin Zheng 0ec580a86c Fix label PR (#10445) 2025-09-14 20:33:09 -07:00
Lianmin Zheng 8f6a175803 Fix label pr for ci (#10441) 2025-09-14 19:48:06 -07:00
Lianmin Zheng b7d385e812 automatically label pr for ci (#10435) 2025-09-14 19:13:11 -07:00
Lianmin Zheng c9ec4cae5b Fix the style of sgl kernel (#10398) 2025-09-12 22:20:21 -07:00
Lianmin Zhengandgithub-actions[bot] cdddab056c [Auto Sync] Update xgrammar_backend.py (20250913) (#10395) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-09-12 17:46:56 -07:00
Lianmin Zhengandgithub-actions[bot] 2269cf1e2f [Auto Sync] Update base_grammar_backend.py, llguidance_back... (20250911) (#10333) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-09-12 12:55:55 -07:00
Lianmin Zhenggithub-actions[bot]Kan Wu
 		144ee5f37c [Auto Sync] Update server_args.py (20250912) (#10347) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Kan Wu <wukanustc@gmail.com>
 2025-09-11 19:18:07 -07:00
Lianmin Zheng 64f296f8e6 [Minor] Improve the style of server args (#10328) 2025-09-11 07:06:29 -07:00
Lianmin Zhenggithub-actions[bot]Byron Hsu
 		956d805dde [Auto Sync] Update parallel_state.py (20250911) (#10326) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
 2025-09-11 06:36:29 -07:00