minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
18,662 Commits 2 Branches 0 Tags
018b73c7a06261bd70a77ddfc402c736bcb2333d
Commit Graph
360 Commits
Author SHA1 Message Date
inkcherryhuanglongShangming CaiHuang Long
 		1f60ddef5d [PD] Introduce runtime role switching between prefill and decode (#28403) 
Signed-off-by: huanglong <huanglong@linux.alibaba.com>
Signed-off-by: inkcherry <mingzhi.liu@amd.com>
Co-authored-by: huanglong <huanglong@linux.alibaba.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Huang Long <121648372+LLLL114@users.noreply.github.com>
 2026-09-18 01:45:12 +08:00
lzShangming CaiXinyuan Tong
 		6220f45d8e [PD] Add /v1/responses support to the HTTP PD router (#36141) 
Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2026-09-13 22:48:13 +08:00
Shangming CaiandClaude Fable 5 981b947568 [Fix] Seed raw tokenizer_path for smg-grpc-servicer in gRPC mode (#39105) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-09-12 14:41:15 +08:00
Xiaoyu ZhangandMick Qian 4d23a4fa6d [Test] Consolidate test cleanup and CI taxonomy (net -11.4K lines) (#37436) 
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
 2026-09-07 15:13:59 +08:00
Alex NailsandAlison Shao 28262c20df [CI][RFC] Replace black-jupyter with ruff-format (#37210) 
Co-authored-by: Alison Shao <a.shao@wustl.edu>
 2026-09-02 19:46:08 -07:00
Juhyun-Kim-Memphis 549166b819 fix(gateway): bump wfaas to 1.0.2 so ContinueNextStep unblocks dependents (#37249) 2026-09-01 02:30:34 +08:00
Cheng Wan 413df1f8db config: ServerArgs holds the raw input (#36255) 2026-08-26 05:14:05 -07:00
Shangming Cai f464e77d17 fix(mini-lb): forward the flush_cache timeout param to workers (#36150) 2026-08-24 15:32:57 +08:00
Cheng WanandClaude Opus 5 64aa859da2 config: constructing a config no longer resolves it (#35907) 
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-23 01:18:53 -07:00
Cheng Wan c70c7d72a8 config: the readback and the resolving view say what they are (#35027) 2026-08-17 16:18:19 -07:00
William ChangandPatrick Toulme 735995e7bd Implement random tie breadking for cache_aware sglang router policy (#33138) 
Co-authored-by: Patrick Toulme <ptoulme@google.com>
 2026-08-06 15:31:40 +08:00
chenkaiyue 2b3223a953 [model-gateway] PD router: cancel paired decode when prefill fails (#29017) 2026-07-02 17:03:14 -07:00
Xiaoyu Zhang b276a9acee chore: cleanup garbage code (#29770) 2026-07-02 16:14:01 +08:00
weireweireandweireweire 5f30fa258c Support DP-aware PD router dispatch (#26245) 
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
 2026-06-24 15:02:51 -07:00
Prajjwal ChittoriandClaude Opus 4.8 adc203dfee fix(router): use full conversation for PD chat cache-aware routing (#26263) (#27430) 
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-22 11:55:48 -07:00
Xinyuan Tong 66ab5c9c7a fix(gateway): make sgl-model-gateway a cargo workspace so maturin 1.14 accepts the parent README (#27997) 2026-06-11 23:37:29 -07:00
Mohammad Miadh Angkad 76b1efaffd Fix SMG service discovery Clippy lint (#26034) 2026-05-22 10:51:13 +08:00
ybyang b5b9c809e1 fix(model-gateway): rustfmt nightly in conversations/handlers.rs (#25947) 2026-05-20 23:29:51 -07:00
Cheng WanandClaude Sonnet 4.6 a528eb7564 fix: rustfmt service_discovery.rs warn! line length (#25927) 
Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
 2026-05-20 19:25:11 -07:00
grunerandAmit Gruner b7d0df4b6f [SMG] Support regular worker discovery alongside PD workers in IGW mode (#25294) 
Co-authored-by: Amit Gruner <agruner@crusoe.ai>
 2026-05-21 07:23:55 +08:00
grunerandAmit Gruner 0ab427d0e1 [SMG] Add /v1/models fallback for model name discovery (#25293) 
Co-authored-by: Amit Gruner <agruner@crusoe.ai>
 2026-05-18 22:02:35 +08:00
Douglas YangKangyan ZhouClaude Opus 4.7
 		e5589843a3 feature: upstream cancel (#19524) 
Co-authored-by: Kangyan Zhou <zky314343421@gmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-18 16:20:09 +08:00
Kangyan-ZhouandClaude Opus 4.7 8b9ff4a68c [SMG] Expand K8s integration tests: cross-namespace, lifecycle, multi-model (#24375) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-13 20:08:09 -07:00
Kangyan-ZhouandClaude Opus 4.7 e8d57e7243 [SMG] Fix cache-aware policy pool isolation in PD mode (#25184) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-13 20:07:14 -07:00
Kangyan-ZhouandClaude Opus 4.7 af7511e0e8 [sgl-model-gateway] Close PyO3 binding gaps and add regression tests (#24719) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-13 17:43:01 -07:00
Mick ff70aeac30 [diffusion] feat: add performance mode server args (#24491) 2026-05-14 00:57:46 +08:00
Revanth Reddy Airre be088f8076 fix(router): configure HTTP client connection settings (#24330) 
Signed-off-by: Revanth Reddy Airre <revanthreddy@hippocraticai.com>
 2026-05-07 11:42:45 -07:00
Revanth Reddy Airre d363315de9 fix(router): make HTTP pool idle timeout configurable (#24329) 
Signed-off-by: Revanth Reddy Airre <revanthreddy@hippocraticai.com>
 2026-05-06 22:11:11 -07:00
Kangyan-ZhouandClaude Opus 4.7 c545a5b1c0 [SMG][CI] Add K8s integration tests + wire into pr-test-rust (#24278) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-04 10:27:37 -07:00
Kangyan-ZhouandClaude Opus 4.7 d74d9bda49 [gateway] Align /v1/loads and /model_info with sglang server; drop dead /rerank (#24167) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-02 11:43:30 -07:00
Kangyan-ZhouandClaude Opus 4.7 2e72a36420 [CI] Restore SMG e2e on 2-gpu-h100 / 4-gpu-h100 runners (#24222) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-01 23:55:20 -07:00
Kangyan-ZhouandClaude Opus 4.7 cf346bb15d [CI] Skip worker-dependent SMG e2e tests pending runner-image debug (#24166) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-30 16:45:10 -07:00
Alex NailsandClaude Opus 4.7 c3ab5bec7d ci: consolidate rust + protoc install across workflows (#23700) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-29 13:39:02 -07:00
Liangsheng Yin 573e12a7fc Merge /get_load into /v1/loads (#23010) 2026-04-17 13:36:51 -07:00
ybyang 9e84f53785 [PD] Add a fallback to bypass rust dep for mini_lb (#21982) 2026-04-15 22:34:36 +08:00
Zhai Feiyue c456cba7fd [gateway] Support SGLANG_LOG_MS for millisecond precision in router logs (#22506) 2026-04-13 17:28:00 -07:00
David Cheung ed427e1299 Migrate all callers from /get_server_info to /server_info (#21463) 2026-04-01 21:17:50 -07:00
Liangsheng YinandClaude Opus 4.6 0138708576 [Misc] Add network timeout to eval dataset downloads (#21873) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-01 13:16:14 -07:00
Liangsheng Yin f0458e0b49 [Utils] Move network/socket utilities from common.py to network.py (#20646) 2026-03-15 20:35:24 -07:00
Simo Lin a16f03c1e3 Revert "feat: add banner to sgl-model-gateway (#20471)" (#20536) 2026-03-13 09:20:32 -07:00
Simo Lin b227e53ebf feat: add banner to sgl-model-gateway (#20471) 
Signed-off-by: Simo Lin <linsimo.mark@gmail.com>
 2026-03-12 14:25:27 -07:00
Feng Su 8a411a9a2a [Tracing] Remove the deprecated tracing code from mini_lb (#19409) 2026-03-07 11:19:23 +08:00
Jonathan Leegithub_usernameClaude Opus 4.6
 		e58391dd7d Add --json-log flag to enable structured JSON logging (#19968) 
Co-authored-by: github_username <github_email>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-05 11:24:12 -08:00
Kangyan-ZhouandClaude Opus 4.6 0eb64c1e72 [smg] Extract tokenizer_path from /model_info into discovered labels (#19905) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-04 17:12:20 -08:00
Eric Zhang e6411ba315 Increase max_concurrent_jobs in job queue (#19797) 2026-03-03 15:54:51 -08:00
Duyi-Wang 98e433e305 [PD-Disagg][Fix] Remove 'test_external_dp_routing' from Rust Router constructor parameters. (#19492) 2026-02-27 02:40:01 -08:00
Liangsheng YinandRatish P 539f772f54 [PD-Disagg] Fully support external DP dispatch w/ PD-disaggregation mode. (#19268) 
Co-authored-by: Ratish P <114130421+ratish1@users.noreply.github.com>
 2026-02-24 19:58:01 -08:00
Simo Lin b2166a2f33 fix(gateway): pin internal crate dependencies to exact versions (#19224) 
Signed-off-by: Simo Lin <linsimo.mark@gmail.com>
 2026-02-23 22:35:47 -08:00
Blake Ledden f759960ac2 feat: expose consistent_hashing policy in Python router CLI args (#17972) 2026-02-16 00:33:24 +08:00
Bingxu Chen 38473f8ee0 [AMD] Fix sgl-model-gateway Build Errors in ROCm Docker Release (#18836) 2026-02-14 00:07:26 -08:00