|
1f60ddef5d
|
[PD] Introduce runtime role switching between prefill and decode (#28403)
Signed-off-by: huanglong <huanglong@linux.alibaba.com>
Signed-off-by: inkcherry <mingzhi.liu@amd.com>
Co-authored-by: huanglong <huanglong@linux.alibaba.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Huang Long <121648372+LLLL114@users.noreply.github.com>
|
2026-09-18 01:45:12 +08:00 |
|
|
Xiaoyu ZhangandMick Qian
|
4d23a4fa6d
|
[Test] Consolidate test cleanup and CI taxonomy (net -11.4K lines) (#37436)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
|
2026-09-07 15:13:59 +08:00 |
|
|
Alex NailsandAlison Shao
|
28262c20df
|
[CI][RFC] Replace black-jupyter with ruff-format (#37210)
Co-authored-by: Alison Shao <a.shao@wustl.edu>
|
2026-09-02 19:46:08 -07:00 |
|
|
Cheng Wan
|
413df1f8db
|
config: ServerArgs holds the raw input (#36255)
|
2026-08-26 05:14:05 -07:00 |
|
|
Shangming Cai
|
f464e77d17
|
fix(mini-lb): forward the flush_cache timeout param to workers (#36150)
|
2026-08-24 15:32:57 +08:00 |
|
|
Cheng WanandClaude Opus 5
|
64aa859da2
|
config: constructing a config no longer resolves it (#35907)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
|
2026-08-23 01:18:53 -07:00 |
|
|
Xiaoyu Zhang
|
b276a9acee
|
chore: cleanup garbage code (#29770)
|
2026-07-02 16:14:01 +08:00 |
|
|
Xinyuan Tong
|
66ab5c9c7a
|
fix(gateway): make sgl-model-gateway a cargo workspace so maturin 1.14 accepts the parent README (#27997)
|
2026-06-11 23:37:29 -07:00 |
|
|
grunerandAmit Gruner
|
b7d0df4b6f
|
[SMG] Support regular worker discovery alongside PD workers in IGW mode (#25294)
Co-authored-by: Amit Gruner <agruner@crusoe.ai>
|
2026-05-21 07:23:55 +08:00 |
|
|
Kangyan-ZhouandClaude Opus 4.7
|
af7511e0e8
|
[sgl-model-gateway] Close PyO3 binding gaps and add regression tests (#24719)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
2026-05-13 17:43:01 -07:00 |
|
|
Kangyan-ZhouandClaude Opus 4.7
|
cf346bb15d
|
[CI] Skip worker-dependent SMG e2e tests pending runner-image debug (#24166)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
2026-04-30 16:45:10 -07:00 |
|
|
ybyang
|
9e84f53785
|
[PD] Add a fallback to bypass rust dep for mini_lb (#21982)
|
2026-04-15 22:34:36 +08:00 |
|
|
David Cheung
|
ed427e1299
|
Migrate all callers from /get_server_info to /server_info (#21463)
|
2026-04-01 21:17:50 -07:00 |
|
|
Liangsheng Yin
|
f0458e0b49
|
[Utils] Move network/socket utilities from common.py to network.py (#20646)
|
2026-03-15 20:35:24 -07:00 |
|
|
Simo Lin
|
a16f03c1e3
|
Revert "feat: add banner to sgl-model-gateway (#20471)" (#20536)
|
2026-03-13 09:20:32 -07:00 |
|
|
Simo Lin
|
b227e53ebf
|
feat: add banner to sgl-model-gateway (#20471)
Signed-off-by: Simo Lin <linsimo.mark@gmail.com>
|
2026-03-12 14:25:27 -07:00 |
|
|
Feng Su
|
8a411a9a2a
|
[Tracing] Remove the deprecated tracing code from mini_lb (#19409)
|
2026-03-07 11:19:23 +08:00 |
|
|
|
e58391dd7d
|
Add --json-log flag to enable structured JSON logging (#19968)
Co-authored-by: github_username <github_email>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
2026-03-05 11:24:12 -08:00 |
|
|
Duyi-Wang
|
98e433e305
|
[PD-Disagg][Fix] Remove 'test_external_dp_routing' from Rust Router constructor parameters. (#19492)
|
2026-02-27 02:40:01 -08:00 |
|
|
Liangsheng YinandRatish P
|
539f772f54
|
[PD-Disagg] Fully support external DP dispatch w/ PD-disaggregation mode. (#19268)
Co-authored-by: Ratish P <114130421+ratish1@users.noreply.github.com>
|
2026-02-24 19:58:01 -08:00 |
|
|
Blake Ledden
|
f759960ac2
|
feat: expose consistent_hashing policy in Python router CLI args (#17972)
|
2026-02-16 00:33:24 +08:00 |
|
|
Simo Lin
|
3f44268fe5
|
[smg] release 0.3.2 (#17168)
|
2026-01-15 11:22:18 -08:00 |
|
|
Ziwen Zhao
|
cf25852a1d
|
[model-gateway] add --disable-health-check option to skip worker health probes (#17002)
|
2026-01-13 17:28:32 -08:00 |
|
|
Tony LuandKun
|
5938c3b06a
|
[model-gateway] HA - Lightweight State Layer + gRPC Mesh (#14108)
Signed-off-by: Kun(llfl) <i@imux.top>
Signed-off-by: Tony Lu <tonylu@linux.alibaba.com>
Co-authored-by: Kun(llfl) <i@imux.top>
|
2026-01-13 17:03:39 -08:00 |
|
|
fzyzcjy
|
ff3ddb9d9b
|
Support min num routing keys in key-based load balancing policy (#16564)
|
2026-01-12 21:38:03 -08:00 |
|
|
fzyzcjy
|
9d3018f484
|
Support min load besides random routing key assignment policy in ManualPolicy (#16767)
|
2026-01-12 18:31:18 -08:00 |
|
|
Wenyi Xu
|
3c16c58619
|
[model-gateway] Add Redis support as a history backend (#16300)
|
2026-01-11 01:03:00 -08:00 |
|
|
Simo Lin
|
7460240737
|
[model-gateway] release 0.3.1 (#16254)
|
2026-01-08 17:49:37 -08:00 |
|
|
Simo Lin
|
0ff3747ca1
|
[model-gateway]: move unit tests to bindings/python/tests/ (#16430)
|
2026-01-04 18:10:35 -08:00 |
|
|
Chang Su
|
d8ac5eecf7
|
[model-gateway] bug fix on module name (#16332)
|
2026-01-02 21:56:18 -08:00 |
|
|
fzyzcjy
|
00562ee14a
|
Support cache eviction for Manual Policy (#16263)
|
2026-01-02 08:52:01 +08:00 |
|
|
Simo Linandgemini-code-assist[bot]
|
41a1d16b02
|
[model-gateway] Organize CLI arguments into logical groups for better --help output (#16035)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2025-12-28 20:25:53 -08:00 |
|
|
Simo Lin
|
3645ed0f73
|
[model-gateway] Add PrefixHash load balancing policy for KV cache-aware routing (#15935)
|
2025-12-27 05:58:05 -05:00 |
|
|
Simo Lin
|
171912a9e3
|
[model-gateway] Add consistent hashing for ManualPolicy routing (#15907)
|
2025-12-26 22:53:03 -08:00 |
|
|
Simo Lin
|
4edee6954a
|
[model-gateway] add JWT/OIDC authentication for control plane APIs (#15850)
|
2025-12-26 18:30:00 -08:00 |
|
|
fzyzcjy
|
45adad37d0
|
Add manual routing policy for router (#15586)
|
2025-12-25 11:27:36 +08:00 |
|
|
Simo Lin
|
5454d2a7ae
|
[model-gateway] release smg 0.3.0 (#15781)
|
2025-12-24 13:23:57 -08:00 |
|
|
Simo Lin
|
50cad014dc
|
[model-gateway] fix graceful shutdown for TLS/Non-TLS server (#15491)
|
2025-12-19 12:35:50 -08:00 |
|
|
ratish
|
0e4108ba29
|
feat(gateway): Add server-side TLS support (#15052)
|
2025-12-14 15:37:55 -08:00 |
|
|
fzyzcjy
|
617e9b3bf8
|
[model-gateway] support customizing Prometheus duration buckets (#14716)
|
2025-12-10 08:30:03 -08:00 |
|
|
Wenyi Xu
|
d7f6320bb8
|
[model-gateway] Dynamically Populate Tool Call Parser Choices (#14807)
|
2025-12-10 06:04:22 -08:00 |
|
|
Simo Lin
|
390406c46c
|
[model-gateway] release gateway 0.2.4 (#14763)
|
2025-12-09 16:34:29 -08:00 |
|
|
fzyzcjy
|
a2ca9bd4f1
|
Super tiny fix unused code in router (#14618)
|
2025-12-07 23:12:09 -08:00 |
|
|
Simo Lin
|
aff1238ef2
|
[model-gateway] reorganize metrics, logging, and otel to its own module (#14590)
|
2025-12-07 16:50:39 -08:00 |
|
|
Alison Shao
|
cee93a6f26
|
[Bug fix] Add /model_info endpoint to mini_lb (#14535)
|
2025-12-06 11:34:03 -08:00 |
|
|
Feng Su
|
e12c6b320f
|
[model-gateway][tracing]: implement request tracing using OpenTelemetry with trace context propagation (HTTP) (#13897)
|
2025-12-06 05:59:04 -08:00 |
|
|
Simo Lin
|
49dfa1d891
|
[model-gateway] change sgl-router to sgl-model-gateway (#14312)
|
2025-12-05 12:04:48 -08:00 |
|