minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
14,605 Commits 2 Branches 0 Tags
bd3b252e0af0e7bfbcc1f329a672a7ee47a36721
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Kangyan-ZhouandClaude Opus 4.8 364bf976be [router] Tokenize prompt once at ingress; forward input_ids to the engine (all policies) (#28744) 
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-19 16:07:17 -07:00
Kangyan-ZhouandClaude Opus 4.8 abf7011cdd [router] Raise chat body cap to 5 MiB for long contexts (#28742) 
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-19 15:50:34 -07:00
Kangyan-ZhouandClaude Opus 4.8 cab62855f5 [router] Align sgl_router_ttft_seconds buckets with engine TTFT grid (#28717) 
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-19 08:53:46 -07:00
Kangyan-ZhouandClaude Opus 4.8 827fc56e00 [router] Multi-arch experimental sgl-router image + fix distroless libpcre2 startup crash (#28474) 
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-16 20:09:39 -07:00
Kangyan-ZhouandClaude Fable 5 cad43d3212 [CI] Reclaim leaked /dev/shm segments on server startup (#28089) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-06-15 16:14:08 -07:00
Kangyan-ZhouandClaude Opus 4.8 c0dfe4c8ec [router] Reconcile workers that registered without resolving model_ids (#27980) 
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-14 18:47:50 -07:00
Kangyan-ZhouandClaude Fable 5 b0b8436f1c [Fix] Unquote ResponseTool annotation breaking lint on all PRs (#28095) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-06-12 15:20:36 -07:00
Kangyan-ZhouandClaude Opus 4.8 21647f1f5d [router] Apply chat template before cache-aware hashing (fix overlap=0 on chat traffic) (#27386) 
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
 2026-06-10 10:27:54 -07:00
Kangyan-ZhouandClaude Opus 4.8 0c7faf01fb test(sgl-router): cover sticky scale-up no-redistribution e2e (#27742) 
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-10 08:54:31 -07:00
Kangyan-ZhouandClaude Opus 4.8 badab6b136 [router] Add request/TTFT/worker metrics + Grafana dashboard to experimental sgl-router (#27591) 
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
 2026-06-09 07:25:56 -07:00
Kangyan-ZhouandClaude Opus 4.8 95090b837e [router] Add /flush_cache endpoint to experimental sgl-router (#27612) 
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
 2026-06-08 18:53:29 -07:00
Kangyan-ZhouandClaude Opus 4.8 6bc3953b48 feat(agentic router): add sticky-session routing policy (#27394) 
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
 2026-06-07 20:05:27 -07:00
Kangyan-ZhouandClaude Opus 4.8 bcf89928b4 [router] Configure experimental sgl-router via CLI flags instead of a config file (#27073) 
Signed-off-by: Kangyan Zhou <zky314343421@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
 2026-06-05 10:02:10 +08:00
Kangyan-ZhouandClaude Opus 4.7 7c04b9e942 fix(docker): generate Cargo.lock in chef stage for sgl-router build (#26279) 
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-25 16:54:47 +08:00
Kangyan-ZhouandClaude Opus 4.7 81704ad602 ci: add nightly Docker workflow for experimental sgl-router (#26273) 
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-25 16:19:07 +08:00
Kangyan-ZhouandClaude Opus 4.7 6e8fe176be sgl-router: experimental Rust HTTP router for SGLang worker pools (#25851) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-25 15:34:05 +08:00
Kangyan-ZhouandClaude Opus 4.7 085777210c feat(kv-events): expose structured KV-event publisher block on /server_info (#25844) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-23 01:59:45 +08:00
Kangyan-ZhouandClaude Opus 4.7 d4082eab4d [CI] pr-test-extra: add run_all_tests to workflow_dispatch inputs (#26110) 
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-22 18:03:39 +08:00
Kangyan-ZhouandClaude Opus 4.7 4486a339a3 [CI] bot-cherry-pick: remove concurrency group to enable batch dispatch (#26074) 
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-22 16:27:46 +08:00
Kangyan-ZhouandClaude Opus 4.7 16d049f898 Add sglang-cherrypick skill for batching bot-cherry-pick dispatches (#26068) 
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-22 16:20:06 +08:00
Kangyan-Zhou 63ecf2f62b [CI] Drop unused 'environment: prod' from bot-cherry-pick job (#26067) 2026-05-22 16:16:07 +08:00
Kangyan-ZhouandClaude Opus 4.7 049bb83134 [CI] bot-cherry-pick: surface created PR number/URL in job summary (#26001) 
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-21 22:58:21 +08:00
Kangyan-ZhouandLiangsheng Yin caa9f08294 [CI] Force-reinstall nvidia-cutlass-dsl-libs-cu13 last to avoid wheel-mix TypeError (#25958) 
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
 2026-05-21 22:01:42 +08:00
Kangyan-Zhou 32352f7edf [CI] Fix bot-cherry-pick: use state == "MERGED" instead of invalid merged field (#25987) 2026-05-21 20:10:12 +08:00
Kangyan-Zhou 64f21b1589 [CI] Improve bot-cherry-pick: accept PR number, require merged, explicit title (#25981) 2026-05-21 18:32:57 +08:00
Kangyan-Zhou 4ea8282cb7 [Revert] nvidia-cutlass-dsl[cu13] 4.5.1 -> 4.5.0 (#25938) 2026-05-21 14:36:56 +08:00
Kangyan-ZhouandClaude Opus 4.7 4868b92d47 [CI] Fix bot-cherry-pick auth: GITHUB_TOKEN for push, dedicated PAT for PR (#25926) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-21 10:28:42 +08:00
Kangyan-ZhouandClaude Opus 4.7 a1843524a5 ci(sgl-router): add PR test workflow (pre-positioned for feature PR) (#25854) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-20 15:20:03 +08:00
Kangyan-Zhou 974948a701 fix: strip "[asctime]" prefix when parsing JSON log lines in nightly tests (#25340) 2026-05-14 20:17:13 -07:00
Kangyan-ZhouandClaude Opus 4.7 8b9ff4a68c [SMG] Expand K8s integration tests: cross-namespace, lifecycle, multi-model (#24375) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-13 20:08:09 -07:00
Kangyan-ZhouandClaude Opus 4.7 e8d57e7243 [SMG] Fix cache-aware policy pool isolation in PD mode (#25184) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-13 20:07:14 -07:00
Kangyan-ZhouandClaude 992fc0d6fe ci(sgl-kernel-build): actually reclaim disk in the wheel-build cleanup step (#25206) 
Co-authored-by: Claude <noreply@anthropic.com>
 2026-05-13 18:45:48 -07:00
Kangyan-ZhouandClaude Opus 4.7 af7511e0e8 [sgl-model-gateway] Close PyO3 binding gaps and add regression tests (#24719) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-13 17:43:01 -07:00
Kangyan-ZhouandClaude Opus 4.7 b1db9f71ee [SMG] Fix matrix-sibling concurrency collision in PR Test (SMG) (#25188) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-13 13:25:40 -07:00
Kangyan-ZhouandClaude Opus 4.7 a2586f1c53 [CI] pin NeMo-Skills install to known-good SHA in accuracy_test_runner (#24581) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-06 22:24:48 -07:00
Kangyan-ZhouandClaude Opus 4.7 0f283a515a [Docker] fix: install nixl stub alongside nixl-cuXX binary (#24369) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-04 12:46:38 -07:00
Kangyan-ZhouandClaude Opus 4.7 8ffd39e140 [CI] Exclude flaky h20 stage from check-stage-health root cause set (#24374) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-04 11:41:25 -07:00
Kangyan-ZhouandClaude Opus 4.7 c545a5b1c0 [SMG][CI] Add K8s integration tests + wire into pr-test-rust (#24278) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-04 10:27:37 -07:00
Kangyan-ZhouClaude Opus 4.7Baizhou Zhang
 		52b4609789 [Docker] Prep for torch 2.11: cu129 fix, image validator, dep cleanup (#23593) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
 2026-05-04 00:37:55 -07:00
Kangyan-ZhouandClaude Opus 4.7 266d994368 [CI] Fix nightly NV jobs cancelling each other via shared concurrency groups (#24282) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-02 21:53:03 -07:00
Kangyan-ZhouClaude Opus 4.7Liangsheng Yin
 		24a6b3084d [CI] drop --prerelease allow from uv pip install suffix (#24265) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
 2026-05-02 12:25:01 -07:00
Kangyan-ZhouandClaude Opus 4.7 d74d9bda49 [gateway] Align /v1/loads and /model_info with sglang server; drop dead /rerank (#24167) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-02 11:43:30 -07:00
Kangyan-ZhouandClaude Opus 4.7 2e72a36420 [CI] Restore SMG e2e on 2-gpu-h100 / 4-gpu-h100 runners (#24222) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-01 23:55:20 -07:00
Kangyan-ZhouandClaude Opus 4.7 cd27baaffd [ci][cu13] Bump torch_memory_saver to 0.0.9.post1; restore manual tests (#23182) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-01 22:50:38 -07:00
Kangyan-ZhouandClaude Opus 4.7 674a80d5d8 [CI] Add 1-gpu-h100-h200 to rerun-test runner_label choices (#24264) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-01 22:45:48 -07:00
Kangyan-ZhouandClaude Opus 4.7 cf346bb15d [CI] Skip worker-dependent SMG e2e tests pending runner-image debug (#24166) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-30 16:45:10 -07:00
Kangyan-Zhou c5f1339773 Revert "ci: add rebase-required mode to check-maintenance action" (#24179) 2026-04-30 16:11:36 -07:00
Kangyan-ZhouandClaude Opus 4.7 e45b8ec1ec [CI] Publish nightly sglang wheel under both cu129 and cu130 indexes (#24176) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-30 16:08:59 -07:00
Kangyan-ZhouClaude Opus 4.7Baizhou Zhang
 		340efca244 [sgl-kernel] Prep for torch 2.11 upgrade and switch PyPI default to cu130 (#24162) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
 2026-04-30 14:54:39 -07:00
Kangyan-ZhouandClaude Opus 4.7 6575aea128 [CI] Fix black formatting on main (unblocks PR #21247 lint) (#24093) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-29 14:59:17 -07:00
Kangyan-ZhouandClaude Opus 4.7 d4040e7010 [CI] Broaden stage-b-test-1-gpu-large runner pool to H100 + H200 (#24080) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-29 12:18:10 -07:00
Kangyan-ZhouandClaude Opus 4.7 feec1ac7f9 ci: clean up stale-CUDA mooncake variant in install_extra_deps (#23960) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-28 19:32:38 -07:00
Kangyan-ZhouandClaude Opus 4.7 35591c7d51 fix(lora): don't assert on non-LoRA lm_head adapter weights (#23433) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-26 12:10:07 -07:00
Kangyan-ZhouandClaude Opus 4.7 8efa177f1e [CI] release-pypi-nightly: install protoc before building wheel (#23750) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-25 22:37:29 -07:00
Kangyan-ZhouandClaude Opus 4.7 4be853e1c6 [CI] release-whl-kernel: strip +cu129 local version before PyPI upload (#23749) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-25 22:14:45 -07:00
Kangyan-ZhouandClaude Opus 4.7 282b47fcfd [CI] release-whl-kernel: clean root-owned build artifacts before checkout (#23747) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-25 21:39:41 -07:00
Kangyan-ZhouandClaude Opus 4.7 921e14dcac [CI] release-docker-deepseek-v4: select which flavors to push (#23730) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-25 14:09:34 -07:00
Kangyan-Zhou 0c826374a8 ci: add docker release workflow for deepseek_v4 branch (#23728) 2026-04-25 11:45:23 -07:00
Kangyan-ZhouandClaude Opus 4.7 acaa35664d [CI] sgl-kernel: prune dangling images before each wheel build (#23723) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-25 10:44:47 -07:00
Kangyan-ZhouandClaude Opus 4.6 2882a136bf [CI] Consolidate Docker release workflows into reusable workflow (#22541) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-23 14:06:01 -07:00
Kangyan-ZhouandClaude Opus 4.6 f1a70b4666 [Observability] Add HTTP sidecar endpoints and FlushCache gRPC RPC for gRPC mode (#22500) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-22 23:06:10 -07:00
Kangyan-ZhouandClaude Opus 4.6 18359aadc8 [CI] Lower GSM8K baselines for B200 nightly after eval unification (#22136) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-22 22:30:54 -07:00
Kangyan-ZhouandClaude Opus 4.7 c689f774a4 [CI] /rerun-stage: fix workflow-run URL lookup for sgl-kernel PRs (#23510) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-22 17:48:38 -07:00
Kangyan-ZhouandClaude Opus 4.7 0bee211335 [CI] Broaden stage-b-test-4-gpu-b200 runner pool to low-disk label (#23505) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-22 17:14:09 -07:00
Kangyan-ZhouandClaude Opus 4.7 14ac14287c [CI] /rerun-stage: auto-include wheel build when PR modifies sgl-kernel/ (#23492) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-22 11:28:06 -07:00
Kangyan-ZhouandClaude Opus 4.7 88c9bab830 [diffusion] ci: allow using prebuilt sgl-kernel wheel for GT regeneration (#23443) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-21 23:44:02 -07:00
Kangyan-ZhouandClaude Opus 4.7 77fd86f89e [ci] split stage-c-test-4-gpu-b200 to enable a low-disk runner pool (#23417) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-21 18:33:33 -07:00
Kangyan-ZhouandClaude Opus 4.7 4698f4cd10 [CI] Fix wait-for-jobs hanging when matrix job skipped at job level (#23277) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-20 11:16:10 -07:00
Kangyan-ZhouandClaude Opus 4.7 97baf17557 Fix test_modelopt_export using stale ModelConfig kwargs (#23214) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-19 23:18:09 -07:00
Kangyan-ZhouandClaude Opus 4.7 1ebe1c57ed [CI] Partition stage-a-test-cpu into 4 matrix shards (#23208) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-19 22:07:37 -07:00
Kangyan-ZhouandClaude Opus 4.7 1d252803f5 fix(ci): repair path filters regressed by #21482 (#23201) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-19 20:34:57 -07:00
Kangyan-ZhouandClaude Opus 4.6 89553ff82b [Observability] Add Prometheus metrics endpoint for gRPC mode (#20801) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-09 20:04:54 -07:00
Kangyan-ZhouandClaude Opus 4.6 9d905efa2c [Docker] Fix Trivy CVEs, cubin download 403s, and kernels command order (#22322) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-09 12:26:22 -07:00
Kangyan-ZhouandClaude Opus 4.6 46c2b77627 [CI] Add GLM-5.1 nightly tests and update Qwen3.5 model (#22399) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-08 17:04:57 -07:00
Kangyan-ZhouandClaude Opus 4.6 cc8ea08b8b [CI] Replace upload/download-artifact with job outputs in release-docker-runtime (#22388) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-08 16:13:33 -07:00
Kangyan-Zhou dd73e9a62e Revert "[CI] Update nightly test models for H200/B200 (#22288)" (#22297) 2026-04-07 17:04:06 -07:00
Kangyan-ZhouandClaude Opus 4.6 e6652309c4 [CI] Update nightly test models for H200/B200 (#22288) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-07 15:44:52 -07:00
Kangyan-ZhouClaude Opus 4.6Baizhou Zhang
 		93109cc89b [Fix] Fix setuptools-scm version resolution for rc tags (#22165) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
 2026-04-05 16:55:32 -07:00
Kangyan-ZhouandClaude Opus 4.6 596c34ee04 Update ci_auto_bisect.py to have streak 1 so that all failures will b… (#22161) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-05 10:39:19 -07:00
Kangyan-ZhouClaude Opus 4.6Baizhou Zhang
 		5dd2c243eb fix: TRT-LLM MHA CUDA illegal address with EAGLE v2 + DP attention (#21649) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
 2026-04-05 09:41:14 -07:00
Kangyan-Zhou edee9ae929 Update ci_auto_bisect.py to use correct model (#22142) 2026-04-04 23:57:52 -07:00
Kangyan-ZhouandClaude Opus 4.6 8cbeacd783 feat: CI auto-bisect workflow for automated regression analysis (#22119) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-04 18:58:18 -07:00
Kangyan-ZhouClaude Opus 4.6Baizhou Zhang
 		56ac9c9932 [Fix] Add _MOE_TP to graph_capture for MoE models with ep>1 (#21907) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
 2026-04-03 02:33:16 -07:00
Kangyan-ZhouandClaude Opus 4.6 1f97714f9b [CI] Add timeouts to Slack upload urlopen and WebClient (#21903) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-02 19:30:55 -07:00
Kangyan-ZhouandClaude Opus 4.6 afa14ffac9 Skip Go stdlib and NVIDIA tool CVEs in Trivy scan (#21905) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-01 21:41:16 -07:00
Kangyan-Zhou 648632b6c4 [CI] Remove crashing Kimi K2.5 EAGLE3/MTP variants, keep TP8 and TP8+DP8 (#21898) 2026-04-01 20:27:24 -07:00
Kangyan-Zhou 9a7f198343 [CI] Increase multimodal server test timeout from 60 to 90 minutes (#21897) 2026-04-01 20:19:45 -07:00
Kangyan-ZhouandClaude Opus 4.6 ea6b22fb85 Fix CVEs in Docker image: pillow, linux-libc-dev, and broken sgl-model-gateway build (#21789) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-31 20:07:15 -07:00
Kangyan-ZhouandClaude Opus 4.6 8c66f4a90f Add Trivy vulnerability scanning to nightly dev Docker builds (#21772) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-31 16:09:11 -07:00
Kangyan-ZhouandClaude Opus 4.6 d8ab41dce5 [Fix] Handle pre-release tags in nightly wheel version parsing (#21656) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-29 22:29:40 -07:00
Kangyan-ZhouandClaude Opus 4.6 9d64a82173 feat(ci): add GB300 nightly benchmark test suites (#21487) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-28 21:54:03 -07:00
Kangyan-ZhouandClaude Opus 4.6 33cca495ae [CI] Replace upload/download-artifact with job outputs in release-docker workflow (#21579) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-27 22:12:55 -07:00
Kangyan-Zhou 62361e8fd5 Temporarily adjust the scheulde for pr-test.yml to 12 hours (#20951) 2026-03-19 14:48:10 -07:00
Kangyan-ZhouandClaude Opus 4.6 b6055e59cd [HiCache] Reduce per-request backup log noise (#20813) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-17 22:47:14 -07:00
Kangyan-Zhou 3d8fc9a0ca Revert "[Nvidia] Add trtllm mnnvl allreduce with unified flashinfer allreduce fusion api" (#20792) 2026-03-17 11:59:02 -07:00
Kangyan-ZhouandClaude Opus 4.6 f5a4a5429f Revert early HTTP port reservation (#17754, #19805) (#20468) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-12 16:17:33 -07:00
Kangyan-ZhouandClaude Opus 4.6 7a12255b6e fix: set first_token_time before computing decode_throughput for single-batch completions (#19984) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-06 16:11:41 -08:00
Kangyan-ZhouandClaude Opus 4.6 e89069ee64 Fallback to torch.cuda.mem_get_info() when nvidia-smi is unavailable (#18957) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-06 15:00:08 -08:00
Kangyan-ZhouandClaude Opus 4.6 25e678d933 [diffusion] endpoint: add /server_info and /model_info endpoints for gateway discovery (#20020) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-06 21:36:13 +08:00
Kangyan-Zhouand 198381d9ce Add SSL/TLS support for HTTP and gRPC servers (#18973) 
Co-authored-by: guys@spotify.com
 2026-03-04 19:27:16 -08:00