minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
12,181 Commits 3 Branches 0 Tags
ece8a1a78813f6f0bd8394b03589d144125d08fb
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Lianmin Zheng ece8a1a788 Refactor device timer, clean up metrics collector, and add fwd occupancy metric (#24197) 2026-05-01 10:25:25 -07:00
Lianmin ZhengXiaozhu MengYinghai Lu
 		d9e8a4a7f8 [SWA] Ensure we use pre-computed SWA cache location during prefill (#24138) 
Co-authored-by: Xiaozhu Meng <mxz297@gmail.com>
Co-authored-by: Yinghai Lu <yinghai@meta.com>
 2026-05-01 00:01:49 -07:00
Lianmin Zheng b1ef99f65f [CI] Remove orphaned test/srt/ascend and test/srt/configs (#24145) 2026-04-30 04:43:11 -07:00
Lianmin Zheng e60c60eff0 [SWA] Fix missing mamba_indices parameter in cpu copy interface (#24026) 2026-04-29 17:33:38 -07:00
Lianmin Zheng 2a771a40ac Add engine_type label to tokenizer manager metrics (#23978) 2026-04-28 19:52:58 -07:00
Lianmin Zheng d66eb3a91b docs: update contribution guide with coding style guidelines (#23977) 2026-04-28 19:51:58 -07:00
Lianmin Zheng 14b4e6fa69 Support --model as alias for --model-path in CLI (#23894) 2026-04-28 17:23:27 -07:00
Lianmin Zheng 8536d4b402 Clean up noisy startup warnings from third-party deps (#23669) 2026-04-27 03:10:46 -07:00
Lianmin Zheng a4facdf3f6 [CI] Refactor ci_install_dependency.sh into standalone functions (#23592) 2026-04-24 17:39:39 -07:00
Lianmin Zheng 6344b546c8 Deprecate --collect-tokens-histogram, auto-collect with --enable-metrics (#23595) 2026-04-24 12:00:16 -07:00
Lianmin Zheng 95d021b523 Pre-set SWA cache location in CudaGraphRunner (#23552) 2026-04-23 16:51:29 -07:00
Lianmin Zheng bb962b0046 Fix MoE no_combine: skip router weight in down projection (#23545) 2026-04-23 16:47:58 -07:00
Lianmin Zheng 6a3c070ee3 Add 'allready' to ignore words list in .codespellrc (#23465) 2026-04-22 02:39:04 -07:00
Lianmin Zheng 9c47bbad13 Clean up bench_one_batch warning and simplify norm dispatch (#23110) 2026-04-17 17:42:20 -07:00
Lianmin Zheng 44e67c6835 Remove deprecated double sparsity feature (#23009) 2026-04-17 13:33:12 -07:00
Lianmin Zheng 35da90cb76 [misc] Configure logging before ServerArgs.__post_init__ (#22926) 2026-04-15 23:53:15 -07:00
Lianmin Zheng ccff59254c Update .codespellrc (#22912) 2026-04-15 16:25:55 -07:00
Lianmin Zheng 43925d179d [Speculative] Fix Eagle3/DFLASH aux hidden state capture during CUDA graph init (#22836) 2026-04-15 14:04:54 -07:00
Lianmin Zheng adb310b976 Cleanup server_args.py and minor code tidying (#22820) 2026-04-14 18:52:41 -07:00
Lianmin Zheng 222eda1598 [Misc] Use cache_once for is_arch_support_pdl in sgl-kernel (#22725) 2026-04-14 15:22:10 -07:00
Lianmin Zheng 9fb00ede15 Clean up TokenizerManager and req_time_stats: reduce overhead and simplify (#21646) 2026-04-13 16:47:32 -07:00
Lianmin Zheng 8f9553bccb [Misc] Migrate SGLANG_SET_CPU_AFFINITY to envs and refactor model config building (#22730) 2026-04-13 16:10:31 -07:00
Lianmin Zheng f81b6e8f51 [Misc] Add @cache_once to is_arch_support_pdl in jit_kernel (#22724) 2026-04-13 14:42:49 -07:00
Lianmin Zheng ba7bcca6b3 Use reshape instead of contiguous().view() in TRTLLMHAAttnBackend (#22517) 2026-04-13 14:29:12 -07:00
Lianmin Zheng ddc8ef1038 Lazy import flash_attention_v4 to avoid loading flash_attn.cute at startup (#22306) 2026-04-08 20:40:25 -07:00
Lianmin ZhengandClaude Opus 4.6 494bb86169 Cache sub-objects in __getitem__ to ensure identity stability (#22184) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-06 18:53:38 -07:00
Lianmin Zheng a80961333b Clean up req_time_stats: reduce overhead and simplify (#22186) 2026-04-06 14:20:51 -07:00
Lianmin Zheng e835601fb7 Cache gfx95 quant format detection in DeepseekV2DecoderLayer (#22143) 2026-04-05 20:20:54 -07:00
Lianmin Zheng eb407b80f3 [Kernel] Make FA3/FA4 imports lazy in FlashAttentionBackend (#22028) 2026-04-03 13:49:00 -07:00
Lianmin Zheng fe38410c3e Remove logging for subprocess watchdog start (#21968) 2026-04-02 11:30:33 -07:00
Lianmin Zheng afb32d7622 README: coding agent sponsorship for long-term contributors (#21642) 2026-03-29 16:02:51 -07:00
Lianmin ZhengandClaude Opus 4.6 9f7792415a Clean up TokenizerManager: remove dead code and improve rid validation (#21639) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-29 15:12:49 -07:00
Lianmin ZhengandClaude Opus 4.6 f3970b17ef [Cleanup] Remove unused BatchMultimodalOutput and BatchMultimodalDecodeReq (#21640) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-29 14:54:25 -07:00
Lianmin ZhengandClaude Opus 4.6 1d9c8e8c9e Simplify routed experts test and move base64 encoding to tokenizer manager (#21634) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-29 12:44:01 -07:00
Lianmin ZhengandClaude Opus 4.6 ba6b501f3a Clean up detokenizer and remove dead multimodal_gen code (#21588) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-28 21:44:40 -07:00
Lianmin Zheng 83997080a6 docs: flesh out MAINTAINER.md oncall lists and link GitHub profiles (#21575) 2026-03-27 17:39:16 -07:00
Lianmin Zheng 9a91323c9f test: point DSV3 int8 MLA CI models to lmsys Hugging Face org (#21561) 2026-03-27 13:04:01 -07:00
Lianmin Zheng 3dfaa47d5e Change cron schedule to run every 6 hours (#21285) 2026-03-24 00:04:23 -07:00
Lianmin Zheng d6ce1b8dce docs: update add-jit-kernel skill for run_suite CI registration (#21264) 2026-03-23 21:31:22 -07:00
Lianmin Zheng 260abe1fb1 Refactor JIT kernel CI to use run_suite.py registration system (#21239) 2026-03-23 21:17:27 -07:00
Lianmin Zheng 4779755eb9 Split pr-test.yml: extract sgl-kernel, jit-kernel, and multimodal-gen tests into separate workflow files (#21219) 2026-03-23 13:17:05 -07:00
Lianmin ZhengandClaude Opus 4.6 27ac831a84 docs: improve CI and testing documentation (#21202) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-23 10:48:50 -07:00
Lianmin Zheng 814202704b ci: unify PR test suite naming (#21187) 2026-03-23 00:18:45 -07:00
Lianmin Zheng 7757a9ddd0 ci: remove IS_BLACKWELL env var; auto-detect Blackwell (#21118) 2026-03-22 23:44:48 -07:00
Lianmin Zheng 7050011dee Enable JIT clamp_position and resolve_future_token_ids on ROCm (#21116) 2026-03-22 22:33:54 -07:00
Lianmin Zheng 3343a79466 Fix jit-kernel unit test CI dependency install in pr-test workflow (#21089) 2026-03-21 22:13:06 -07:00
Lianmin Zheng 1ff0ee95e3 Update pr-test.yml to remove the temp hack - cron: '15 8 * * *' (#21115) 2026-03-21 21:53:31 -07:00
Lianmin ZhengandClaude Opus 4.6 76e4a8662c Replace clamp_position with JIT kernel + platform dispatch (#20999) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-21 21:26:26 -07:00
Lianmin Zheng 9b98d55ba4 Change PR test schedule time to 7:50 AM UTC (#21087) 2026-03-21 00:36:05 -07:00
Lianmin Zheng f2c9d5696f Change daily cron job time in pr-test.yml (#21084) 2026-03-21 00:24:53 -07:00
Lianmin Zheng dba6fb3d30 Fix streaming logprobs corruption caused by shared mutable list reference (#21030) 2026-03-21 00:18:48 -07:00
Lianmin Zheng d089db0563 ci: add daily PR Test schedule at 12:18am PDT (#21083) 2026-03-21 00:15:48 -07:00
Lianmin Zheng 182dd20d57 ci: bypass maintenance gate for PR Test on main (#21082) 2026-03-21 00:08:47 -07:00
Lianmin Zheng 507db41504 Add CI maintenance mode gate to nightly-test-nvidia.yml (#21079) 2026-03-20 23:44:37 -07:00
Lianmin Zheng 427442304a Add CI maintenance mode gate to nightly-test-nvidia.yml (#21073) 2026-03-20 22:07:34 -07:00
Lianmin Zheng 50404e0d1f ci: refactor CUDA dependency install script (#21017) 2026-03-20 21:36:58 -07:00
Lianmin Zheng d5473f87c4 Add CI maintenance mode gate to pr-test.yml (#21069) 2026-03-20 21:27:38 -07:00
Lianmin Zheng a75e74b534 Add revocation of permissions section (#21066) 2026-03-20 20:22:24 -07:00
Lianmin Zheng 2d7a262ca3 ci: rename 1/2-gpu-runner labels to 1/2-gpu-h100 (#21008) 2026-03-20 06:04:15 -07:00
Lianmin Zheng 104b10f70a refactor: consolidate is_in_ci (jit_kernel, sgl-kernel benchmarks, tests) (#21009) 2026-03-20 05:55:36 -07:00
Lianmin Zheng c1da420799 ci: run Stage A CUDA tests as stage-a-test-small-1-gpu on 5090 (#20988) 2026-03-20 02:55:16 -07:00
Lianmin Zheng a0a4dae67f Revert "Fix DeepSeek V32 FP4 test" (#21003) 2026-03-20 02:19:28 -07:00
Lianmin ZhengandClaude Opus 4.6 112b628227 Replace _resolve_future_token_ids with JIT kernel + platform dispatch (#20976) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-20 01:47:03 -07:00
Lianmin Zheng 193bbf9b66 chore(ci): remove deprecated CI Monitor workflow (#20993) 2026-03-20 00:05:22 -07:00
Lianmin Zheng cea79530f8 ci: merge CPU default and stage-a-cpu-only suites (#20992) 2026-03-20 00:04:55 -07:00
Lianmin Zheng 3ebe6e9894 ci: remove redundant RUNNER_LABELS from pr-test.yml (#20991) 2026-03-19 23:59:47 -07:00
Lianmin Zheng 712a48c5d2 ci: move metrics scripts under scripts/ci/utils (#20986) 2026-03-19 23:47:57 -07:00
Lianmin Zheng 0be88ac44c ci: rename pr-test workflow_call input ref to git_ref 
Rename workflow_call input to avoid confusion with checkout/dispatch ref. Only caller: release-branch-cut.yml.
 2026-03-19 23:45:06 -07:00
Lianmin ZhengandClaude Opus 4.6 0949b138af Simplify server startup output (#20885) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-19 13:11:37 -07:00
Lianmin Zheng 3b89c6d3ea simplify and improve pr-test.yml (#20911) 2026-03-19 11:01:06 -07:00
Lianmin ZhengandClaude Opus 4.6 46a392658e Refine RL & Post-Training description in README (#20877) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-18 12:43:42 -07:00
Lianmin ZhengandClaude Opus 4.6 39e83d1401 [CI] Eliminate per-arch Docker tags by using push-by-digest (#20793) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-18 12:35:06 -07:00
Lianmin Zheng a515cc38a7 Delete .editorconfig (#20795) 2026-03-17 13:02:04 -07:00
Lianmin Zheng dc1ab5d2b2 Delete CODE_OF_CONDUCT.md (#20794) 2026-03-17 13:01:51 -07:00
Lianmin Zhenggithub-actions[bot]Jiayi Yuan
 		463baafe10 [Auto Sync] Update batch_invariant_ops.py (20260221) (#19098) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Jiayi Yuan <34369239+jy-yuan@users.noreply.github.com>
 2026-02-20 18:27:31 -08:00
Lianmin Zhenggithub-actions[bot]Hanming Lu
 		2928dfb8fa [Auto Sync] Update bench_one_batch_server_internal.py (20260221) (#19097) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
 2026-02-20 18:19:14 -08:00
Lianmin ZhengandSehoon Kim e02a9bec8d Refactor sampler: Use a better hash function for deterministic sampling and clear dispatch for probs/logprobs/logits sampling paths (#18915) 
Co-authored-by: Sehoon Kim <sehoon@x.ai>
 2026-02-17 15:41:23 -08:00
Lianmin Zhengandgithub-actions[bot] b33769786f [Auto Sync] Update grpc_request_manager.py, tokenizer_manag... (20260214) (#18838) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2026-02-14 18:12:32 -08:00
Lianmin Zhenggithub-actions[bot]Jiayi Yuan
 		8b2020584c [Auto Sync] Update test_deterministic.py (20260214) (#18839) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Jiayi Yuan <34369239+jy-yuan@users.noreply.github.com>
 2026-02-14 17:19:30 -08:00
Lianmin Zhenggithub-actions[bot]Xiuyu LiCheng Wan
 		008ea46af1 [Auto Sync] Update loader.py, weight_utils.py (20260213) (#18779) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Xiuyu Li <xiuyu@x.ai>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
 2026-02-13 12:22:50 -08:00
Lianmin Zhenggithub-actions[bot]Cheng Wan
 		c56a5efbaa [Auto Sync] Update grok.py (20260213) (#18765) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
 2026-02-12 18:41:41 -08:00
Lianmin Zheng d5f66fec15 Revert changes to weight_utils.py (#18759) 2026-02-12 17:15:16 -08:00
Lianmin Zhenggithub-actions[bot]Dan Zheng
 		9815ee934c [Auto Sync] Update weight_utils.py (20260212) (#18692) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Dan Zheng <dzheng@x.ai>
 2026-02-12 16:26:05 -08:00
Lianmin Zheng 5875ef0a34 Clean up noisy startup log messages and refactor loader.py (#18531) 2026-02-11 16:12:57 -08:00
Lianmin Zhenggithub-actions[bot]Hanming Lugemini-code-assist[bot]
 		b027c5aca6 [Auto Sync] Update cache_init_params.py (20260209) (#18502) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-02-09 14:49:41 -08:00
Lianmin Zhengandgithub-actions[bot] ce95f203b0 [Auto Sync] Update logits_processor.py (20260209) (#18503) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2026-02-09 14:33:02 -08:00
Lianmin Zheng 75997ebe8d Update author information in pyproject.toml (#18453) 2026-02-08 12:22:55 -08:00
Lianmin Zhenggithub-actions[bot]Jiayi Yuangemini-code-assist[bot]
 		1acae30806 [Auto Sync] Update test_deterministic.py (20260131) (#18034) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Jiayi Yuan <34369239+jy-yuan@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-02-01 07:40:07 -08:00
Lianmin Zhengandgithub-actions[bot] fb609669ca [Auto Sync] Update elementwise.py (20260131) (#18033) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2026-02-01 07:39:37 -08:00
Lianmin Zhenggithub-actions[bot]Byron Hsu
 		7412ceb4eb [Auto Sync] Update linear.py to assert shapes (20260130) (#17966) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
 2026-01-31 01:01:55 -08:00
Lianmin Zhenggithub-actions[bot]Archit Patke
 		0e184609d3 Add launch_command assignment in crash dump (#17967) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Archit Patke <apatke@x.ai>
 2026-01-31 01:00:40 -08:00
Lianmin Zheng d4180815a4 Make the functions in logits_processor.py and sampler.py more modular (#17885) 2026-01-28 16:24:23 -08:00
Lianmin Zhenggithub-actions[bot]Jiayi Yuan
 		0834f9afeb [Auto Sync] Update test_deterministic.py (20260124) (#17665) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Jiayi Yuan <34369239+jy-yuan@users.noreply.github.com>
 2026-01-24 02:52:47 -08:00
Lianmin Zhenggithub-actions[bot]yehu-ux
 		bc6f0b5ce7 [Auto Sync] Update logits_processor.py, test_logprobs.py (20260124) (#17664) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: yehu-ux <yehu@x.ai>
 2026-01-23 17:57:41 -08:00
Lianmin Zheng 56e6652d1d Lazy import torchao (#17626) 2026-01-22 22:04:51 -08:00
Lianmin Zhenggithub-actions[bot]Wangfan Fu
 		b74a57a8d9 [Auto Sync] Update detokenizer_manager.py, io_struct.py, mu... (20260120) (#17442) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Wangfan Fu <wangfan@x.ai>
 2026-01-21 14:48:32 -08:00
Lianmin Zhenggithub-actions[bot]Binyao Jiang
 		1fdf5cac39 [Auto Sync] Update environ.py, fp8.py (20260121) (#17486) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Binyao Jiang <byjiang1996@gmail.com>
 2026-01-21 12:04:09 -08:00
Lianmin Zhenggithub-actions[bot]Binyao Jiang
 		20ed3822bb [Auto Sync] Update piecewise_cuda_graph_runner.py (20260119) (#17313) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Binyao Jiang <byjiang1996@gmail.com>
 2026-01-20 14:05:05 -08:00
Lianmin Zhengandgithub-actions[bot] e619f53113 [Auto Sync] Update tokenizer_manager.py (20260119) (#17317) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2026-01-18 20:32:19 -08:00
Lianmin Zheng fc4b932f4e Update code sync scripts (#17319) 2026-01-18 19:57:28 -08:00