minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
12,410 Commits 3 Branches 0 Tags
62c2e091f6ba7181d69b660e57280ec98c0e55ad
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Lianmin Zheng e40e339c72 Filter non-int token ids in benchmark and observe decode-side bootstrap/alloc metrics (#24684) 2026-05-08 11:45:37 -07:00
Lianmin Zheng 3c3f0bd55e Cache empty MatchResult in RadixCache (#24470) 2026-05-07 17:13:20 -07:00
Lianmin Zheng b859f7ffba Improve metrics, observability, and PD deploy tooling (#24521) 2026-05-06 11:27:35 -07:00
Lianmin Zheng d4d4b04d66 [PD] Fix missing update_status call in abort() across all KV backends (#24522) 2026-05-06 05:30:11 -07:00
Lianmin Zheng b91b05ae27 Add --random-input-len to send_one.py (#24464) 2026-05-05 17:49:33 -07:00
Lianmin Zheng 64f80eabbe Register aten::rms_norm and aten::mm.dtype in batch invariant mode (#24459) 2026-05-05 17:21:34 -07:00
Lianmin Zheng 46bde1f426 Add fwd_occupancy metric to SchedulerStats and Prometheus collector (#24458) 2026-05-05 17:04:34 -07:00
Lianmin Zheng 710fed10fb Revert "[fix] /pause_generation and /continue_generation wrong for --tokenizer-worker-num > 1" (#24461) 2026-05-05 16:44:34 -07:00
Lianmin Zheng 29dd3a36c0 Refactor device timer installation and rename prefill prealloc to bootstrap (#24341) 2026-05-04 13:57:13 -07:00
Lianmin Zheng c7566a860b Add CI permissions for user luccafong, 842974287 (#24238) 2026-05-01 12:50:36 -07:00
Lianmin Zheng ece8a1a788 Refactor device timer, clean up metrics collector, and add fwd occupancy metric (#24197) 2026-05-01 10:25:25 -07:00
Lianmin ZhengXiaozhu MengYinghai Lu
 		d9e8a4a7f8 [SWA] Ensure we use pre-computed SWA cache location during prefill (#24138) 
Co-authored-by: Xiaozhu Meng <mxz297@gmail.com>
Co-authored-by: Yinghai Lu <yinghai@meta.com>
 2026-05-01 00:01:49 -07:00
Lianmin Zheng b1ef99f65f [CI] Remove orphaned test/srt/ascend and test/srt/configs (#24145) 2026-04-30 04:43:11 -07:00
Lianmin Zheng e60c60eff0 [SWA] Fix missing mamba_indices parameter in cpu copy interface (#24026) 2026-04-29 17:33:38 -07:00
Lianmin Zheng 2a771a40ac Add engine_type label to tokenizer manager metrics (#23978) 2026-04-28 19:52:58 -07:00
Lianmin Zheng d66eb3a91b docs: update contribution guide with coding style guidelines (#23977) 2026-04-28 19:51:58 -07:00
Lianmin Zheng 14b4e6fa69 Support --model as alias for --model-path in CLI (#23894) 2026-04-28 17:23:27 -07:00
Lianmin Zheng 8536d4b402 Clean up noisy startup warnings from third-party deps (#23669) 2026-04-27 03:10:46 -07:00
Lianmin Zheng a4facdf3f6 [CI] Refactor ci_install_dependency.sh into standalone functions (#23592) 2026-04-24 17:39:39 -07:00
Lianmin Zheng 6344b546c8 Deprecate --collect-tokens-histogram, auto-collect with --enable-metrics (#23595) 2026-04-24 12:00:16 -07:00
Lianmin Zheng 95d021b523 Pre-set SWA cache location in CudaGraphRunner (#23552) 2026-04-23 16:51:29 -07:00
Lianmin Zheng bb962b0046 Fix MoE no_combine: skip router weight in down projection (#23545) 2026-04-23 16:47:58 -07:00
Lianmin Zheng 6a3c070ee3 Add 'allready' to ignore words list in .codespellrc (#23465) 2026-04-22 02:39:04 -07:00
Lianmin Zheng 9c47bbad13 Clean up bench_one_batch warning and simplify norm dispatch (#23110) 2026-04-17 17:42:20 -07:00
Lianmin Zheng 44e67c6835 Remove deprecated double sparsity feature (#23009) 2026-04-17 13:33:12 -07:00
Lianmin Zheng 35da90cb76 [misc] Configure logging before ServerArgs.__post_init__ (#22926) 2026-04-15 23:53:15 -07:00
Lianmin Zheng ccff59254c Update .codespellrc (#22912) 2026-04-15 16:25:55 -07:00
Lianmin Zheng 43925d179d [Speculative] Fix Eagle3/DFLASH aux hidden state capture during CUDA graph init (#22836) 2026-04-15 14:04:54 -07:00
Lianmin Zheng adb310b976 Cleanup server_args.py and minor code tidying (#22820) 2026-04-14 18:52:41 -07:00
Lianmin Zheng 222eda1598 [Misc] Use cache_once for is_arch_support_pdl in sgl-kernel (#22725) 2026-04-14 15:22:10 -07:00
Lianmin Zheng 9fb00ede15 Clean up TokenizerManager and req_time_stats: reduce overhead and simplify (#21646) 2026-04-13 16:47:32 -07:00
Lianmin Zheng 8f9553bccb [Misc] Migrate SGLANG_SET_CPU_AFFINITY to envs and refactor model config building (#22730) 2026-04-13 16:10:31 -07:00
Lianmin Zheng f81b6e8f51 [Misc] Add @cache_once to is_arch_support_pdl in jit_kernel (#22724) 2026-04-13 14:42:49 -07:00
Lianmin Zheng ba7bcca6b3 Use reshape instead of contiguous().view() in TRTLLMHAAttnBackend (#22517) 2026-04-13 14:29:12 -07:00
Lianmin Zheng ddc8ef1038 Lazy import flash_attention_v4 to avoid loading flash_attn.cute at startup (#22306) 2026-04-08 20:40:25 -07:00
Lianmin ZhengandClaude Opus 4.6 494bb86169 Cache sub-objects in __getitem__ to ensure identity stability (#22184) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-06 18:53:38 -07:00
Lianmin Zheng a80961333b Clean up req_time_stats: reduce overhead and simplify (#22186) 2026-04-06 14:20:51 -07:00
Lianmin Zheng e835601fb7 Cache gfx95 quant format detection in DeepseekV2DecoderLayer (#22143) 2026-04-05 20:20:54 -07:00
Lianmin Zheng eb407b80f3 [Kernel] Make FA3/FA4 imports lazy in FlashAttentionBackend (#22028) 2026-04-03 13:49:00 -07:00
Lianmin Zheng fe38410c3e Remove logging for subprocess watchdog start (#21968) 2026-04-02 11:30:33 -07:00
Lianmin Zheng afb32d7622 README: coding agent sponsorship for long-term contributors (#21642) 2026-03-29 16:02:51 -07:00
Lianmin ZhengandClaude Opus 4.6 9f7792415a Clean up TokenizerManager: remove dead code and improve rid validation (#21639) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-29 15:12:49 -07:00
Lianmin ZhengandClaude Opus 4.6 f3970b17ef [Cleanup] Remove unused BatchMultimodalOutput and BatchMultimodalDecodeReq (#21640) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-29 14:54:25 -07:00
Lianmin ZhengandClaude Opus 4.6 1d9c8e8c9e Simplify routed experts test and move base64 encoding to tokenizer manager (#21634) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-29 12:44:01 -07:00
Lianmin ZhengandClaude Opus 4.6 ba6b501f3a Clean up detokenizer and remove dead multimodal_gen code (#21588) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-28 21:44:40 -07:00
Lianmin Zheng 83997080a6 docs: flesh out MAINTAINER.md oncall lists and link GitHub profiles (#21575) 2026-03-27 17:39:16 -07:00
Lianmin Zheng 9a91323c9f test: point DSV3 int8 MLA CI models to lmsys Hugging Face org (#21561) 2026-03-27 13:04:01 -07:00
Lianmin Zheng 3dfaa47d5e Change cron schedule to run every 6 hours (#21285) 2026-03-24 00:04:23 -07:00
Lianmin Zheng d6ce1b8dce docs: update add-jit-kernel skill for run_suite CI registration (#21264) 2026-03-23 21:31:22 -07:00
Lianmin Zheng 260abe1fb1 Refactor JIT kernel CI to use run_suite.py registration system (#21239) 2026-03-23 21:17:27 -07:00
Lianmin Zheng 4779755eb9 Split pr-test.yml: extract sgl-kernel, jit-kernel, and multimodal-gen tests into separate workflow files (#21219) 2026-03-23 13:17:05 -07:00
Lianmin ZhengandClaude Opus 4.6 27ac831a84 docs: improve CI and testing documentation (#21202) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-23 10:48:50 -07:00
Lianmin Zheng 814202704b ci: unify PR test suite naming (#21187) 2026-03-23 00:18:45 -07:00
Lianmin Zheng 7757a9ddd0 ci: remove IS_BLACKWELL env var; auto-detect Blackwell (#21118) 2026-03-22 23:44:48 -07:00
Lianmin Zheng 7050011dee Enable JIT clamp_position and resolve_future_token_ids on ROCm (#21116) 2026-03-22 22:33:54 -07:00
Lianmin Zheng 3343a79466 Fix jit-kernel unit test CI dependency install in pr-test workflow (#21089) 2026-03-21 22:13:06 -07:00
Lianmin Zheng 1ff0ee95e3 Update pr-test.yml to remove the temp hack - cron: '15 8 * * *' (#21115) 2026-03-21 21:53:31 -07:00
Lianmin ZhengandClaude Opus 4.6 76e4a8662c Replace clamp_position with JIT kernel + platform dispatch (#20999) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-21 21:26:26 -07:00
Lianmin Zheng 9b98d55ba4 Change PR test schedule time to 7:50 AM UTC (#21087) 2026-03-21 00:36:05 -07:00
Lianmin Zheng f2c9d5696f Change daily cron job time in pr-test.yml (#21084) 2026-03-21 00:24:53 -07:00
Lianmin Zheng dba6fb3d30 Fix streaming logprobs corruption caused by shared mutable list reference (#21030) 2026-03-21 00:18:48 -07:00
Lianmin Zheng d089db0563 ci: add daily PR Test schedule at 12:18am PDT (#21083) 2026-03-21 00:15:48 -07:00
Lianmin Zheng 182dd20d57 ci: bypass maintenance gate for PR Test on main (#21082) 2026-03-21 00:08:47 -07:00
Lianmin Zheng 507db41504 Add CI maintenance mode gate to nightly-test-nvidia.yml (#21079) 2026-03-20 23:44:37 -07:00
Lianmin Zheng 427442304a Add CI maintenance mode gate to nightly-test-nvidia.yml (#21073) 2026-03-20 22:07:34 -07:00
Lianmin Zheng 50404e0d1f ci: refactor CUDA dependency install script (#21017) 2026-03-20 21:36:58 -07:00
Lianmin Zheng d5473f87c4 Add CI maintenance mode gate to pr-test.yml (#21069) 2026-03-20 21:27:38 -07:00
Lianmin Zheng a75e74b534 Add revocation of permissions section (#21066) 2026-03-20 20:22:24 -07:00
Lianmin Zheng 2d7a262ca3 ci: rename 1/2-gpu-runner labels to 1/2-gpu-h100 (#21008) 2026-03-20 06:04:15 -07:00
Lianmin Zheng 104b10f70a refactor: consolidate is_in_ci (jit_kernel, sgl-kernel benchmarks, tests) (#21009) 2026-03-20 05:55:36 -07:00
Lianmin Zheng c1da420799 ci: run Stage A CUDA tests as stage-a-test-small-1-gpu on 5090 (#20988) 2026-03-20 02:55:16 -07:00
Lianmin Zheng a0a4dae67f Revert "Fix DeepSeek V32 FP4 test" (#21003) 2026-03-20 02:19:28 -07:00
Lianmin ZhengandClaude Opus 4.6 112b628227 Replace _resolve_future_token_ids with JIT kernel + platform dispatch (#20976) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-20 01:47:03 -07:00
Lianmin Zheng 193bbf9b66 chore(ci): remove deprecated CI Monitor workflow (#20993) 2026-03-20 00:05:22 -07:00
Lianmin Zheng cea79530f8 ci: merge CPU default and stage-a-cpu-only suites (#20992) 2026-03-20 00:04:55 -07:00
Lianmin Zheng 3ebe6e9894 ci: remove redundant RUNNER_LABELS from pr-test.yml (#20991) 2026-03-19 23:59:47 -07:00
Lianmin Zheng 712a48c5d2 ci: move metrics scripts under scripts/ci/utils (#20986) 2026-03-19 23:47:57 -07:00
Lianmin Zheng 0be88ac44c ci: rename pr-test workflow_call input ref to git_ref 
Rename workflow_call input to avoid confusion with checkout/dispatch ref. Only caller: release-branch-cut.yml.
 2026-03-19 23:45:06 -07:00
Lianmin ZhengandClaude Opus 4.6 0949b138af Simplify server startup output (#20885) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-19 13:11:37 -07:00
Lianmin Zheng 3b89c6d3ea simplify and improve pr-test.yml (#20911) 2026-03-19 11:01:06 -07:00
Lianmin ZhengandClaude Opus 4.6 46a392658e Refine RL & Post-Training description in README (#20877) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-18 12:43:42 -07:00
Lianmin ZhengandClaude Opus 4.6 39e83d1401 [CI] Eliminate per-arch Docker tags by using push-by-digest (#20793) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-18 12:35:06 -07:00
Lianmin Zheng a515cc38a7 Delete .editorconfig (#20795) 2026-03-17 13:02:04 -07:00
Lianmin Zheng dc1ab5d2b2 Delete CODE_OF_CONDUCT.md (#20794) 2026-03-17 13:01:51 -07:00
Lianmin Zhenggithub-actions[bot]Jiayi Yuan
 		463baafe10 [Auto Sync] Update batch_invariant_ops.py (20260221) (#19098) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Jiayi Yuan <34369239+jy-yuan@users.noreply.github.com>
 2026-02-20 18:27:31 -08:00
Lianmin Zhenggithub-actions[bot]Hanming Lu
 		2928dfb8fa [Auto Sync] Update bench_one_batch_server_internal.py (20260221) (#19097) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
 2026-02-20 18:19:14 -08:00
Lianmin ZhengandSehoon Kim e02a9bec8d Refactor sampler: Use a better hash function for deterministic sampling and clear dispatch for probs/logprobs/logits sampling paths (#18915) 
Co-authored-by: Sehoon Kim <sehoon@x.ai>
 2026-02-17 15:41:23 -08:00
Lianmin Zhengandgithub-actions[bot] b33769786f [Auto Sync] Update grpc_request_manager.py, tokenizer_manag... (20260214) (#18838) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2026-02-14 18:12:32 -08:00
Lianmin Zhenggithub-actions[bot]Jiayi Yuan
 		8b2020584c [Auto Sync] Update test_deterministic.py (20260214) (#18839) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Jiayi Yuan <34369239+jy-yuan@users.noreply.github.com>
 2026-02-14 17:19:30 -08:00
Lianmin Zhenggithub-actions[bot]Xiuyu LiCheng Wan
 		008ea46af1 [Auto Sync] Update loader.py, weight_utils.py (20260213) (#18779) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Xiuyu Li <xiuyu@x.ai>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
 2026-02-13 12:22:50 -08:00
Lianmin Zhenggithub-actions[bot]Cheng Wan
 		c56a5efbaa [Auto Sync] Update grok.py (20260213) (#18765) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
 2026-02-12 18:41:41 -08:00
Lianmin Zheng d5f66fec15 Revert changes to weight_utils.py (#18759) 2026-02-12 17:15:16 -08:00
Lianmin Zhenggithub-actions[bot]Dan Zheng
 		9815ee934c [Auto Sync] Update weight_utils.py (20260212) (#18692) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Dan Zheng <dzheng@x.ai>
 2026-02-12 16:26:05 -08:00
Lianmin Zheng 5875ef0a34 Clean up noisy startup log messages and refactor loader.py (#18531) 2026-02-11 16:12:57 -08:00
Lianmin Zhenggithub-actions[bot]Hanming Lugemini-code-assist[bot]
 		b027c5aca6 [Auto Sync] Update cache_init_params.py (20260209) (#18502) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-02-09 14:49:41 -08:00
Lianmin Zhengandgithub-actions[bot] ce95f203b0 [Auto Sync] Update logits_processor.py (20260209) (#18503) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2026-02-09 14:33:02 -08:00
Lianmin Zheng 75997ebe8d Update author information in pyproject.toml (#18453) 2026-02-08 12:22:55 -08:00
Lianmin Zhenggithub-actions[bot]Jiayi Yuangemini-code-assist[bot]
 		1acae30806 [Auto Sync] Update test_deterministic.py (20260131) (#18034) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Jiayi Yuan <34369239+jy-yuan@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-02-01 07:40:07 -08:00
Lianmin Zhengandgithub-actions[bot] fb609669ca [Auto Sync] Update elementwise.py (20260131) (#18033) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2026-02-01 07:39:37 -08:00
Lianmin Zhenggithub-actions[bot]Byron Hsu
 		7412ceb4eb [Auto Sync] Update linear.py to assert shapes (20260130) (#17966) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
 2026-01-31 01:01:55 -08:00