minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
11,684 Commits 2 Branches 0 Tags
34fef07a15ea01add2d4b324530acaa401b0655e
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Kangyan-ZhouandClaude Opus 4.6 89553ff82b [Observability] Add Prometheus metrics endpoint for gRPC mode (#20801) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-09 20:04:54 -07:00
Kangyan-ZhouandClaude Opus 4.6 9d905efa2c [Docker] Fix Trivy CVEs, cubin download 403s, and kernels command order (#22322) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-09 12:26:22 -07:00
Kangyan-ZhouandClaude Opus 4.6 46c2b77627 [CI] Add GLM-5.1 nightly tests and update Qwen3.5 model (#22399) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-08 17:04:57 -07:00
Kangyan-ZhouandClaude Opus 4.6 cc8ea08b8b [CI] Replace upload/download-artifact with job outputs in release-docker-runtime (#22388) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-08 16:13:33 -07:00
Kangyan-Zhou dd73e9a62e Revert "[CI] Update nightly test models for H200/B200 (#22288)" (#22297) 2026-04-07 17:04:06 -07:00
Kangyan-ZhouandClaude Opus 4.6 e6652309c4 [CI] Update nightly test models for H200/B200 (#22288) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-07 15:44:52 -07:00
Kangyan-ZhouClaude Opus 4.6Baizhou Zhang
 		93109cc89b [Fix] Fix setuptools-scm version resolution for rc tags (#22165) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
 2026-04-05 16:55:32 -07:00
Kangyan-ZhouandClaude Opus 4.6 596c34ee04 Update ci_auto_bisect.py to have streak 1 so that all failures will b… (#22161) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-05 10:39:19 -07:00
Kangyan-ZhouClaude Opus 4.6Baizhou Zhang
 		5dd2c243eb fix: TRT-LLM MHA CUDA illegal address with EAGLE v2 + DP attention (#21649) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
 2026-04-05 09:41:14 -07:00
Kangyan-Zhou edee9ae929 Update ci_auto_bisect.py to use correct model (#22142) 2026-04-04 23:57:52 -07:00
Kangyan-ZhouandClaude Opus 4.6 8cbeacd783 feat: CI auto-bisect workflow for automated regression analysis (#22119) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-04 18:58:18 -07:00
Kangyan-ZhouClaude Opus 4.6Baizhou Zhang
 		56ac9c9932 [Fix] Add _MOE_TP to graph_capture for MoE models with ep>1 (#21907) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
 2026-04-03 02:33:16 -07:00
Kangyan-ZhouandClaude Opus 4.6 1f97714f9b [CI] Add timeouts to Slack upload urlopen and WebClient (#21903) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-02 19:30:55 -07:00
Kangyan-ZhouandClaude Opus 4.6 afa14ffac9 Skip Go stdlib and NVIDIA tool CVEs in Trivy scan (#21905) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-01 21:41:16 -07:00
Kangyan-Zhou 648632b6c4 [CI] Remove crashing Kimi K2.5 EAGLE3/MTP variants, keep TP8 and TP8+DP8 (#21898) 2026-04-01 20:27:24 -07:00
Kangyan-Zhou 9a7f198343 [CI] Increase multimodal server test timeout from 60 to 90 minutes (#21897) 2026-04-01 20:19:45 -07:00
Kangyan-ZhouandClaude Opus 4.6 ea6b22fb85 Fix CVEs in Docker image: pillow, linux-libc-dev, and broken sgl-model-gateway build (#21789) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-31 20:07:15 -07:00
Kangyan-ZhouandClaude Opus 4.6 8c66f4a90f Add Trivy vulnerability scanning to nightly dev Docker builds (#21772) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-31 16:09:11 -07:00
Kangyan-ZhouandClaude Opus 4.6 d8ab41dce5 [Fix] Handle pre-release tags in nightly wheel version parsing (#21656) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-29 22:29:40 -07:00
Kangyan-ZhouandClaude Opus 4.6 9d64a82173 feat(ci): add GB300 nightly benchmark test suites (#21487) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-28 21:54:03 -07:00
Kangyan-ZhouandClaude Opus 4.6 33cca495ae [CI] Replace upload/download-artifact with job outputs in release-docker workflow (#21579) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-27 22:12:55 -07:00
Kangyan-Zhou 62361e8fd5 Temporarily adjust the scheulde for pr-test.yml to 12 hours (#20951) 2026-03-19 14:48:10 -07:00
Kangyan-ZhouandClaude Opus 4.6 b6055e59cd [HiCache] Reduce per-request backup log noise (#20813) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-17 22:47:14 -07:00
Kangyan-Zhou 3d8fc9a0ca Revert "[Nvidia] Add trtllm mnnvl allreduce with unified flashinfer allreduce fusion api" (#20792) 2026-03-17 11:59:02 -07:00
Kangyan-ZhouandClaude Opus 4.6 f5a4a5429f Revert early HTTP port reservation (#17754, #19805) (#20468) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-12 16:17:33 -07:00
Kangyan-ZhouandClaude Opus 4.6 7a12255b6e fix: set first_token_time before computing decode_throughput for single-batch completions (#19984) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-06 16:11:41 -08:00
Kangyan-ZhouandClaude Opus 4.6 e89069ee64 Fallback to torch.cuda.mem_get_info() when nvidia-smi is unavailable (#18957) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-06 15:00:08 -08:00
Kangyan-ZhouandClaude Opus 4.6 25e678d933 [diffusion] endpoint: add /server_info and /model_info endpoints for gateway discovery (#20020) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-06 21:36:13 +08:00
Kangyan-Zhouand 198381d9ce Add SSL/TLS support for HTTP and gRPC servers (#18973) 
Co-authored-by: guys@spotify.com
 2026-03-04 19:27:16 -08:00
Kangyan-ZhouandClaude Opus 4.6 0eb64c1e72 [smg] Extract tokenizer_path from /model_info into discovered labels (#19905) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-04 17:12:20 -08:00
Kangyan-ZhouandClaude Opus 4.6 c03deb8175 Fix disagg PD bootstrap and KV transfer metrics (#19009) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-04 09:08:10 -08:00
Kangyan-ZhouandClaude Opus 4.6 dc92f88a21 Enhance bench_multiturn.py with OpenAI API support and richer metrics (#19724) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-03 13:48:04 -08:00
Kangyan-ZhouandClaude Opus 4.6 ec9775491f Add bisect ci claude code skill (#19649) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-01 20:53:40 -08:00
Kangyan-ZhouandClaude Opus 4.6 98224de29b [Bugfix] Add missing auto_create_handle_loop to communicator methods (#19610) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-01 15:00:05 -08:00
Kangyan-ZhouandClaude Opus 4.6 dc02e5bea7 [HiCache] Re-land spec v2 + decode KV cache offloading compatibility (#19615) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-01 13:58:31 -08:00
Kangyan-Zhou dcf462cfba Revert "[HiCache] Enable spec v2 + decode KV cache offloading compatibility" (#19613) 2026-02-28 21:54:32 -08:00
Kangyan-ZhouandClaude Opus 4.6 8167346609 [HiCache] Enable spec v2 + decode KV cache offloading compatibility (#19518) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-28 21:53:52 -08:00
Kangyan-Zhou 306c552639 Revert "Fix HybridAttnBackend forward for linear attention" (#19356) 2026-02-25 11:49:50 -08:00
Kangyan-ZhouandClaude Opus 4.6 8aeb16f3fc fix: add missing blank line after docstring in serving_transcription.py (#19206) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-23 18:29:38 -08:00
Kangyan-ZhouandClaude Opus 4.6 b2d8cc8cf0 Fix dev Docker build OOM on ARM64 cu13 by adding docker system prune (#18947) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-22 10:29:31 +08:00
Kangyan-ZhouandClaude Opus 4.6 ae95869292 Enable SGLANG_ENABLE_SPEC_V2 for nightly speculative decoding tests (#18719) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-14 23:00:33 +08:00
Kangyan-Zhou 3a1c388b43 Update performance dashboard for nightly tests (#18824) 2026-02-14 09:28:28 +08:00
Kangyan-ZhouandClaude Opus 4.6 eccf875d49 [CI] Revive 8-GPU trace upload in nightly test workflow (#18820) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-14 08:37:08 +08:00
Kangyan-Zhou 710d873ba6 Update notified user in post_ci_failures_to_slack.py (#18817) 2026-02-14 06:48:56 +08:00
Kangyan-Zhou 1b8f68af57 Fix B200 installation issue (#18725) 2026-02-12 22:06:23 +08:00
Kangyan-Zhou f116b3a51b Make PR based docker and pypi workflow work for forked PR (#18720) 2026-02-12 21:05:17 +08:00
Kangyan-Zhou 0db6fd4dbe Revert broken sgl_kernel exclusion patterns in paths-filter (#18193) 2026-02-03 10:56:44 -08:00
Kangyan-Zhou cd31540fd7 Improve Per Commit Test job filtering for sglang-kernel (#18054) 2026-02-01 21:15:23 -08:00
Kangyan-Zhou 9c168fcac7 Fix Diffusion Request Validation to allow missing input artifacts if the input only contains text (#16610) 2026-01-31 23:38:40 -08:00
Kangyan-Zhou e5ac6229e1 Fix installation script for H200 runners (#18050) 2026-01-31 23:30:51 -08:00
Kangyan-Zhou e884b17632 Fix rerun stage command with merged commit history (#17960) 2026-01-31 20:37:55 -08:00
Kangyan-Zhou d443d2d2ae Improve error output in tnightly tets (#18053) 2026-01-31 19:26:19 -08:00
Kangyan-Zhou a16aba8340 Increase install dependency for gb200 (#17977) 2026-01-29 23:55:25 -08:00
Kangyan-ZhouandClaude Opus 4.5 606ff09ef8 [Fix] Remove unused Type import in gpt_j.py (#17975) 
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
 2026-01-29 21:11:11 -08:00
Kangyan-ZhouandClaude Opus 4.5 2cd2c3118d Add concurrency tracking to runner utilization report (#17963) 
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
 2026-01-29 17:31:55 -08:00
Kangyan-Zhou c3bf53c7c1 Fix ci weight validation logic to check the safetensor completeness (#17917) 2026-01-29 13:00:42 -08:00
Kangyan-Zhou c0b4dd68a2 Add a performance dashboard server and frontend for nightly CUDA tests (#17725) 2026-01-27 22:22:33 -08:00
Kangyan-Zhou 48f4340b14 Exclude some diffusion package for ARM in docker release (#17745) 2026-01-25 23:32:39 -08:00
Kangyan-Zhou 52e0f65fce Update nightly-test-nvidia.yml to remove push trigger (#17625) 2026-01-25 21:42:36 -08:00
Kangyan-Zhou 5aaedac3c6 Add EP=2 to qwen235b nightly tests (#17738) 2026-01-25 21:36:04 -08:00
Kangyan-ZhouandXinyuan Tong 592603d77b Fix flaky streaming logprobs test by handling detokenizer text buffering (#17687) 
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2026-01-25 15:09:06 -08:00
Kangyan-Zhou 344eeaee90 Upload nightly test metrics to GH artifacts (#17696) 2026-01-25 14:35:14 -08:00
Kangyan-Zhou 7ca8c12e0e Extend b200 kernel tests timeout for CPU differences (#17718) 2026-01-25 12:59:39 -08:00
Kangyan-Zhou 8d3e1ac0c8 Add an all type in pyproject.tml to include diffusion support (#17697) 2026-01-25 12:52:13 -08:00
Kangyan-Zhou 9123491430 A few updates to the night tests (#17694) 2026-01-25 11:20:17 -08:00
Kangyan-Zhou b829b797ef Fix slash command handler trigger condition by trimming the comments (#17691) 2026-01-24 18:54:46 -08:00
Kangyan-Zhou 69a7a70e47 Temporarily disable lora overlap loading test due to flakiness (#17683) 2026-01-24 12:24:44 -08:00
Kangyan-Zhou 137eb5b95c Fix NSA indexer test and move it to pre commit test (#17682) 2026-01-24 12:06:18 -08:00
Kangyan-Zhou 8656a146a6 Fix test timeout issue in pr-test (#17681) 2026-01-24 11:46:08 -08:00
Kangyan-Zhou f2ae066a6b Update release-branch-cut.yml for actions: write (#17539) 2026-01-21 18:03:27 -08:00
Kangyan-Zhou be5121b452 Fix NSA indexer in the nightly test (#17452) 2026-01-20 19:58:13 -08:00
Kangyan-Zhou 18e2ef09d7 Add v1/models endpoint to diffusion model APIs so that they can be discovered by model gateway (#16425) 2026-01-06 14:28:24 -08:00
Kangyan-Zhou d93f37a625 Add harvenstar to CI_PERMISSIONS.json 2026-01-04 22:35:26 -08:00
Kangyan-Zhou ca80c19b55 Revert "[diffusion] feat: support warmup with resolutions" (#16433) 2026-01-04 18:44:05 -08:00
Kangyan-Zhou 130c6911f8 Add a GH action for cherrypick (#16243) 2025-12-31 20:05:38 -08:00
Kangyan-Zhou 12b89e51d8 Add P90/99 e2e latency in bench_serving script (#16245) 2025-12-31 15:40:33 -08:00
Kangyan-Zhou 2da49eec50 Fix XEON docker release workflow (#16107) 2025-12-31 09:47:01 -08:00
Kangyan-Zhou d65ae0ec7a Use a different concurrency group for release branch testing (#16202) 2025-12-31 09:46:43 -08:00
Kangyan-Zhou fc643ffbc9 Download missing shards in model weights files when not in CI (#16211) 2025-12-31 20:42:34 +08:00
Kangyan-Zhou 0d003e34b0 Reduce CI failure monitor to run once every 12 hours (#16123) 2025-12-29 19:44:25 -08:00
Kangyan-Zhou 9c4eb46099 Add a new branch cut GH workflow, and adopt setuptools-scm for version control (#15985) 2025-12-29 13:51:21 -08:00
Kangyan-Zhou a91e072f33 Use allow auto truncate in the OpenAI API endpoint (#15369) 2025-12-25 18:45:57 -08:00
Kangyan-Zhou 011d8d8970 Reserve more memory for DeepSeekOCR model and adjust server start timeout for DeepGEMM to reduce flakiness (#15277) 2025-12-17 13:13:05 -08:00
Kangyan-Zhou 272c5fe43e Increase timeout for TestDeepseekV3MTP for potential DeepGEMM cold start (#15239) 2025-12-16 00:26:10 -08:00
Kangyan-Zhou e7157c9b77 Add cache for flashinfer installation (#15153) 2025-12-15 10:37:34 -08:00
Kangyan-Zhou 037c3982af Fix H200 CI by commenting out Warmup Weights and JIT Compilation (#15139) 2025-12-14 18:04:28 -08:00
Kangyan-Zhou 291396544e Add a special label for b200 CI runner that can run kernel tests (#15033) 2025-12-12 19:41:36 -08:00
Kangyan-Zhou b243154614 Fix CI by reverting incorrect metric check logic (#15004) 2025-12-12 10:07:38 -08:00
Kangyan-Zhou aa3716b29d Update CI_PERMISSIONS.json 2025-12-11 15:55:50 -08:00
Kangyan-Zhou c5f1e86117 Update CI_PERMISSIONS.json (#14917) 2025-12-11 10:14:42 -08:00
Kangyan-Zhou 2856624156 Only count limitations for previous runs that reaches the test stages (#14856) 2025-12-10 23:17:34 -08:00
Kangyan-Zhou 32829b1638 Remove myself to test CI gate issue (#14871) 2025-12-10 22:37:01 -08:00
Kangyan-Zhou d9dca28247 Update pr-test.yml to fix unknown job name deepep-8-gpu 2025-12-01 10:02:29 -08:00
Kangyan-Zhou 41b7aab848 Disable Deepep 8 GPU tests (#14152) 2025-12-01 09:37:38 -08:00
Kangyan-Zhou 5ddd2f6b6b Always run model evaluation even if the trace upload step fails (#14157) 2025-11-29 20:13:59 -08:00
Kangyan-Zhou 1d3d8b3418 Fix Minimax M2 loading issue (#13956) 2025-11-29 17:07:19 -05:00
Kangyan-Zhou d7cb08c5be Always run all stages in cron based PR tests (#14151) 2025-11-29 12:32:06 -08:00
Kangyan-Zhou d6c88d519d Trigger PR test on main every 3 hours instead of push event (#14130) 2025-11-29 10:12:50 -08:00
Kangyan-Zhou a102a0507a Disable Deepep 2 GPU tests (#14111) 2025-11-28 13:56:34 -08:00
Kangyan-Zhou 4c9f7c97d3 Temporarily disabled test (#14069) 2025-11-27 14:58:46 -08:00