minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
16,633 Commits 2 Branches 0 Tags
ab810e40524c8489a9eb9c3cbcdb7ade1fa6b947
Commit Graph
350 Commits
Author SHA1 Message Date
William ChangandPatrick Toulme 735995e7bd Implement random tie breadking for cache_aware sglang router policy (#33138) 
Co-authored-by: Patrick Toulme <ptoulme@google.com>
 2026-08-06 15:31:40 +08:00
chenkaiyue 2b3223a953 [model-gateway] PD router: cancel paired decode when prefill fails (#29017) 2026-07-02 17:03:14 -07:00
Xiaoyu Zhang b276a9acee chore: cleanup garbage code (#29770) 2026-07-02 16:14:01 +08:00
weireweireandweireweire 5f30fa258c Support DP-aware PD router dispatch (#26245) 
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
 2026-06-24 15:02:51 -07:00
Prajjwal ChittoriandClaude Opus 4.8 adc203dfee fix(router): use full conversation for PD chat cache-aware routing (#26263) (#27430) 
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-22 11:55:48 -07:00
Xinyuan Tong 66ab5c9c7a fix(gateway): make sgl-model-gateway a cargo workspace so maturin 1.14 accepts the parent README (#27997) 2026-06-11 23:37:29 -07:00
Mohammad Miadh Angkad 76b1efaffd Fix SMG service discovery Clippy lint (#26034) 2026-05-22 10:51:13 +08:00
ybyang b5b9c809e1 fix(model-gateway): rustfmt nightly in conversations/handlers.rs (#25947) 2026-05-20 23:29:51 -07:00
Cheng WanandClaude Sonnet 4.6 a528eb7564 fix: rustfmt service_discovery.rs warn! line length (#25927) 
Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
 2026-05-20 19:25:11 -07:00
grunerandAmit Gruner b7d0df4b6f [SMG] Support regular worker discovery alongside PD workers in IGW mode (#25294) 
Co-authored-by: Amit Gruner <agruner@crusoe.ai>
 2026-05-21 07:23:55 +08:00
grunerandAmit Gruner 0ab427d0e1 [SMG] Add /v1/models fallback for model name discovery (#25293) 
Co-authored-by: Amit Gruner <agruner@crusoe.ai>
 2026-05-18 22:02:35 +08:00
Douglas YangKangyan ZhouClaude Opus 4.7
 		e5589843a3 feature: upstream cancel (#19524) 
Co-authored-by: Kangyan Zhou <zky314343421@gmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-18 16:20:09 +08:00
Kangyan-ZhouandClaude Opus 4.7 8b9ff4a68c [SMG] Expand K8s integration tests: cross-namespace, lifecycle, multi-model (#24375) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-13 20:08:09 -07:00
Kangyan-ZhouandClaude Opus 4.7 e8d57e7243 [SMG] Fix cache-aware policy pool isolation in PD mode (#25184) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-13 20:07:14 -07:00
Kangyan-ZhouandClaude Opus 4.7 af7511e0e8 [sgl-model-gateway] Close PyO3 binding gaps and add regression tests (#24719) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-13 17:43:01 -07:00
Mick ff70aeac30 [diffusion] feat: add performance mode server args (#24491) 2026-05-14 00:57:46 +08:00
Revanth Reddy Airre be088f8076 fix(router): configure HTTP client connection settings (#24330) 
Signed-off-by: Revanth Reddy Airre <revanthreddy@hippocraticai.com>
 2026-05-07 11:42:45 -07:00
Revanth Reddy Airre d363315de9 fix(router): make HTTP pool idle timeout configurable (#24329) 
Signed-off-by: Revanth Reddy Airre <revanthreddy@hippocraticai.com>
 2026-05-06 22:11:11 -07:00
Kangyan-ZhouandClaude Opus 4.7 c545a5b1c0 [SMG][CI] Add K8s integration tests + wire into pr-test-rust (#24278) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-04 10:27:37 -07:00
Kangyan-ZhouandClaude Opus 4.7 d74d9bda49 [gateway] Align /v1/loads and /model_info with sglang server; drop dead /rerank (#24167) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-02 11:43:30 -07:00
Kangyan-ZhouandClaude Opus 4.7 2e72a36420 [CI] Restore SMG e2e on 2-gpu-h100 / 4-gpu-h100 runners (#24222) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-01 23:55:20 -07:00
Kangyan-ZhouandClaude Opus 4.7 cf346bb15d [CI] Skip worker-dependent SMG e2e tests pending runner-image debug (#24166) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-30 16:45:10 -07:00
Alex NailsandClaude Opus 4.7 c3ab5bec7d ci: consolidate rust + protoc install across workflows (#23700) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-29 13:39:02 -07:00
Liangsheng Yin 573e12a7fc Merge /get_load into /v1/loads (#23010) 2026-04-17 13:36:51 -07:00
ybyang 9e84f53785 [PD] Add a fallback to bypass rust dep for mini_lb (#21982) 2026-04-15 22:34:36 +08:00
Zhai Feiyue c456cba7fd [gateway] Support SGLANG_LOG_MS for millisecond precision in router logs (#22506) 2026-04-13 17:28:00 -07:00
David Cheung ed427e1299 Migrate all callers from /get_server_info to /server_info (#21463) 2026-04-01 21:17:50 -07:00
Liangsheng YinandClaude Opus 4.6 0138708576 [Misc] Add network timeout to eval dataset downloads (#21873) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-01 13:16:14 -07:00
Liangsheng Yin f0458e0b49 [Utils] Move network/socket utilities from common.py to network.py (#20646) 2026-03-15 20:35:24 -07:00
Simo Lin a16f03c1e3 Revert "feat: add banner to sgl-model-gateway (#20471)" (#20536) 2026-03-13 09:20:32 -07:00
Simo Lin b227e53ebf feat: add banner to sgl-model-gateway (#20471) 
Signed-off-by: Simo Lin <linsimo.mark@gmail.com>
 2026-03-12 14:25:27 -07:00
Feng Su 8a411a9a2a [Tracing] Remove the deprecated tracing code from mini_lb (#19409) 2026-03-07 11:19:23 +08:00
Jonathan Leegithub_usernameClaude Opus 4.6
 		e58391dd7d Add --json-log flag to enable structured JSON logging (#19968) 
Co-authored-by: github_username <github_email>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-05 11:24:12 -08:00
Kangyan-ZhouandClaude Opus 4.6 0eb64c1e72 [smg] Extract tokenizer_path from /model_info into discovered labels (#19905) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-04 17:12:20 -08:00
Eric Zhang e6411ba315 Increase max_concurrent_jobs in job queue (#19797) 2026-03-03 15:54:51 -08:00
Duyi-Wang 98e433e305 [PD-Disagg][Fix] Remove 'test_external_dp_routing' from Rust Router constructor parameters. (#19492) 2026-02-27 02:40:01 -08:00
Liangsheng YinandRatish P 539f772f54 [PD-Disagg] Fully support external DP dispatch w/ PD-disaggregation mode. (#19268) 
Co-authored-by: Ratish P <114130421+ratish1@users.noreply.github.com>
 2026-02-24 19:58:01 -08:00
Simo Lin b2166a2f33 fix(gateway): pin internal crate dependencies to exact versions (#19224) 
Signed-off-by: Simo Lin <linsimo.mark@gmail.com>
 2026-02-23 22:35:47 -08:00
Blake Ledden f759960ac2 feat: expose consistent_hashing policy in Python router CLI args (#17972) 2026-02-16 00:33:24 +08:00
Bingxu Chen 38473f8ee0 [AMD] Fix sgl-model-gateway Build Errors in ROCm Docker Release (#18836) 2026-02-14 00:07:26 -08:00
Simo Lin 0236522214 refactor: remove crate re-export aliases from lib.rs (#18737) 2026-02-12 09:17:24 -08:00
Simo Lin 2f283d8152 refactor: consolidate gRPC client into shared crate dependency (#18730) 2026-02-12 06:43:24 -08:00
Simo Lin ac16d4450e [smg][mesh] extract mesh to mesh crate to reduce compile time (#17907) 2026-01-28 17:58:42 -08:00
Praneth Paruchuri 6f009961bb [model-gateway] Optimize HashRing construction to reduce heap allocations (#17575) 2026-01-27 22:46:40 -08:00
Praneth Paruchuri 897c35b457 [model-gateway] Optimize consistent hashing hot path to eliminate allocations (#17467) 2026-01-27 22:46:11 -08:00
shuwenn a723d1c5ef [model-gateway] ignore error for embeddings/classify in PD router (#15931) 2026-01-26 22:49:14 -05:00
Simo Lin 6756cf18c6 remove multimodal as this is completely dead code (#17750) 2026-01-26 00:07:40 -08:00
Simo Lin d4adff31aa update wasm endpoint (#17748) 2026-01-25 23:43:29 -08:00
Simo Lin ed75136e85 remove self managed wasm as it has been replaced with official smg wa… (#17746) 2026-01-25 22:56:33 -08:00
Praneth Paruchuri 8c2d8b51e9 [model-gateway] fix wasm example2 (#17244) 2026-01-25 22:45:36 -08:00