 Simo Lin
|
e6fe092dcc
|
[model-gateway] rename py_test to e2e_test (#16454)
|
2026-01-04 22:24:34 -08:00 |
|
|
Simo Lin
|
e53160bb31
|
[model-gateway] reorganize integration tests into logical subdirectories (#16451)
|
2026-01-04 21:49:20 -08:00 |
|
|
Simo Lin
|
2181bc9e51
|
[model-gateway] Delete Python integration_mock tests (#16448)
|
2026-01-04 21:34:06 -08:00 |
|
|
Simo Lin
|
f84487af59
|
[model-gateway] : Rust integration tests for integration_mock replacement (#16441)
|
2026-01-04 21:01:47 -08:00 |
|
|
Simo Lin
|
0ff3747ca1
|
[model-gateway]: move unit tests to bindings/python/tests/ (#16430)
|
2026-01-04 18:10:35 -08:00 |
|
|
Simo Lin
|
4436dc0f6c
|
[model-gateway] improve lock contention and allocation in middleware (#16405)
|
2026-01-04 10:40:07 -08:00 |
|
|
Simo Lin
|
cf6800f6cc
|
[model-gateway] optimize Vec and HashMap allocations in responses api (#16406)
|
2026-01-04 10:07:55 -08:00 |
|
|
Simo Lin
|
65b0b5b248
|
fix(logging): use Display format for model_id instead of Debug (#16337)
|
2026-01-03 00:21:39 -08:00 |
|
|
Simo Lin
|
31ed68e7c1
|
[model-gateway] address embedding similarity threshold in ut (#16321)
|
2026-01-02 16:46:55 -08:00 |
|
|
Simo LinandChang Su
|
d401d23876
|
[model-gateway] Add embedding correctness test comparing against HuggingFace (#16092)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2026-01-02 10:58:20 -08:00 |
|
|
Simo Lin
|
851845579c
|
perf(tree): optimize input_char_count calculation (#16238)
|
2025-12-31 15:29:37 -08:00 |
|
|
Simo Lin
|
ace6f3005f
|
ci(benchmark): disable sccache summary annotations and fix tree output (#16233)
|
2025-12-31 11:05:21 -08:00 |
|
|
Simo Lin
|
60d7279c46
|
test(tree): add comprehensive unit tests and fix input_char_count bug (#16228)
|
2025-12-31 07:31:39 -08:00 |
|
|
Simo Lin
|
b4ce7a6d71
|
[model-gateway] cache_aware eliminate String allocations in hot path (#16209)
|
2025-12-31 00:26:10 -08:00 |
|
|
Simo Lin
|
9940c6f592
|
[model-gateway]: ASCII byte comparison and probabilistic timestamp updates (#16181)
|
2025-12-30 12:28:01 -08:00 |
|
|
Simo LinandChang Su
|
4bc2f2e0bc
|
[model-gateway][docs] Add Classification API documentation (#16182)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-12-30 10:53:31 -08:00 |
|
|
Simo Lin
|
8c6f865afb
|
[model-gateway]: optimize prefix_match with zero-copy tenant and deferred char count (#16099)
|
2025-12-29 11:32:57 -08:00 |
|
|
Simo LinandChang Su
|
f39382c6ca
|
[model-gateway] Fix duplicate classify prefix in response ID (#16101)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-12-29 11:07:21 -08:00 |
|
|
Simo LinandChang Su
|
2ff289e2dc
|
[model-gateway] Generate UUID-based request IDs for embedding/classify (#16100)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-12-29 10:49:58 -08:00 |
|
|
Simo LinandChang Su
|
b2a3f0551a
|
[model-gateway] Wire classify pipeline to gRPC router (#16098)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-12-29 10:44:45 -08:00 |
|
|
Simo Lin
|
684e148e38
|
[model-gateway] Optimize INSERT with leaf-only timestamp updates (#16097)
|
2025-12-29 10:43:20 -08:00 |
|
|
Simo LinandChang Su
|
f44c4b3715
|
[model-gateway] Add classify pipeline stages and protocol types (#16094)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-12-29 10:23:36 -08:00 |
|
|
Simo Lin
|
ac78f96ec5
|
[model-gateway] Optimize radix tree timestamp updates for multi-tenant scaling (#16093)
|
2025-12-29 10:08:19 -08:00 |
|
|
Simo Lin
|
7f5879985c
|
[model-gateway] Improve tree benchmark with realistic multi-tenant scenarios (#14838)
|
2025-12-29 08:37:24 -08:00 |
|
|
Simo LinandChang Su
|
162d1cf9be
|
[model-gateway] Add classification model support infrastructure (#16061)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-12-29 08:34:05 -08:00 |
|
|
Simo Lin
|
c31f62722c
|
[model-gateway] fix tokenizer to match transformers special token handling (#16087)
|
2025-12-29 08:13:03 -08:00 |
|
|
Simo Lin
|
de2799f3f5
|
[docs] Fix non-clickable ToC links in model gateway documentation (#16054)
|
2025-12-28 23:35:37 -08:00 |
|
|
Simo Lin
|
097330909f
|
[model-gateway] update WorkerRegistryStats with connection mode and circuit breaker info (#16046)
|
2025-12-28 23:14:57 -08:00 |
|
|
Simo Lin
|
a44d0079f8
|
[ci] update genai bench to 0.0.3 for pd testing (#16051)
|
2025-12-28 23:14:44 -08:00 |
|
|
Simo Lin
|
ec8c831d5a
|
[model-gateway]: optimize metrics for minimal CPU and memory overhead (#16041)
|
2025-12-28 22:28:08 -08:00 |
|
|
Simo Lin
|
f13949e5a9
|
[model-gateway] perf: optimize observability logging for minimal CPU/memory overhead (#16039)
|
2025-12-28 21:20:46 -08:00 |
|
|
Simo Lin
|
c236a3fdf0
|
[model-gateway][CI] Display benchmark results in GitHub Actions summary (#16037)
|
2025-12-28 20:57:27 -08:00 |
|
|
Simo Linandgemini-code-assist[bot]
|
41a1d16b02
|
[model-gateway] Organize CLI arguments into logical groups for better --help output (#16035)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2025-12-28 20:25:53 -08:00 |
|
|
Simo Lin
|
9884c9fd4c
|
[model-gateway] Organize Rust CLI arguments into logical groups for better --help output (#16036)
|
2025-12-28 20:25:38 -08:00 |
|
|
Simo Lin
|
3645ed0f73
|
[model-gateway] Add PrefixHash load balancing policy for KV cache-aware routing (#15935)
|
2025-12-27 05:58:05 -05:00 |
|
|
Simo Lin
|
0e25aa439b
|
[model-gateway] optimize radix tree memory and reduce allocations (#15933)
|
2025-12-27 02:47:30 -08:00 |
|
|
Simo Lin
|
171912a9e3
|
[model-gateway] Add consistent hashing for ManualPolicy routing (#15907)
|
2025-12-26 22:53:03 -08:00 |
|
|
Simo Lin
|
4edee6954a
|
[model-gateway] add JWT/OIDC authentication for control plane APIs (#15850)
|
2025-12-26 18:30:00 -08:00 |
|
|
Simo Lin
|
acddb8e0ec
|
Revert embedding integration tests from 5f3a47d8a (#15914)
|
2025-12-26 20:29:03 -05:00 |
|
|
Simo Lin
|
caa95c7eb4
|
[model-gateway] update ManualPolicy with header-based routing (#15847)
|
2025-12-25 13:04:43 -08:00 |
|
|
Simo Lin
|
5454d2a7ae
|
[model-gateway] release smg 0.3.0 (#15781)
|
2025-12-24 13:23:57 -08:00 |
|
|
Simo Lin
|
643aeefe6d
|
[model-gateway] Fix logging module name, parse endpoint context, and tokenizer factory (#15782)
|
2025-12-24 10:54:48 -08:00 |
|
|
Simo Lin
|
b27b5a834e
|
[bug] fix code formatting which blocks ci (#15780)
|
2025-12-24 09:14:13 -08:00 |
|
|
Simo Lin
|
2f7c629235
|
[model-gateway] Fix IGW routing and optimize RouterManager (#15741)
|
2025-12-24 11:28:53 -05:00 |
|
|
Simo Lin
|
9665574937
|
[docs] major SGL Model Gateway documentation update (#15715)
|
2025-12-23 20:26:09 -08:00 |
|
|
Simo Lin
|
846953d9f1
|
[model-gateway] Add tokenize/detokenize HTTP endpoints and tokenizer management (#15702)
|
2025-12-23 17:32:07 -08:00 |
|
|
Simo Lin
|
aa6ac96635
|
[model-gateway] Fix tokenizer caching and improve error handling (#15695)
|
2025-12-23 10:33:38 -08:00 |
|
|
Simo Lin
|
2142881b7e
|
[model-gateway] return 503 when all workers are circuit-broken (#15611)
|
2025-12-22 04:06:09 -10:00 |
|
|
Simo Lin
|
1167867e8d
|
[model-gateway] add retry support to OpenAI router chat endpoint (#15589)
|
2025-12-21 20:28:51 -08:00 |
|
|
Simo Lin
|
796969caf0
|
Optimize Rust CI builds with proper sccache configuration (#15581)
|
2025-12-21 17:32:13 -08:00 |
|
|
Simo Lin
|
122c250336
|
[model-gateway] add retry and circuit breaker support to gRPC routers (#15585)
|
2025-12-21 17:12:36 -08:00 |
|
|
Simo Lin
|
b9d786059f
|
[model-gateway] refactor WorkerManager with fan_out helper and thin handlers (#15583)
|
2025-12-21 15:29:37 -08:00 |
|
|
Simo Lin
|
1354063a48
|
[model-gateway] add WorkerService abstraction for worker business logic (#15580)
|
2025-12-21 14:35:35 -08:00 |
|
|
Simo Lin
|
e1dcd0df3e
|
[model-gateway] minor code clean up (#15578)
|
2025-12-21 11:00:32 -10:00 |
|
|
Simo Lin
|
69412ccb1f
|
[model-gateway] add model gateway multi-arch docker build, test and document docker image (#15544)
|
2025-12-20 11:56:59 -08:00 |
|
|
Simo Lin
|
d3885d4ba6
|
Fix docker gateway image name and add latest tag (#15542)
|
2025-12-20 11:05:03 -08:00 |
|
|
Simo Lin
|
5529ab5895
|
[model-gateway] Implement RAII load guard with response body attachment (#15507)
|
2025-12-19 19:14:52 -08:00 |
|
|
Simo Lin
|
74a3349bea
|
[ci] remove rust benchmark in unit test ci (#15510)
|
2025-12-19 18:59:41 -08:00 |
|
|
Simo Lin
|
71a602883a
|
[model-gateway] simplify workflow engine backoff and reduce duplicate reads (#15505)
|
2025-12-19 16:23:02 -08:00 |
|
|
Simo Lin
|
61405b3d00
|
[model-gateway] Run workflow event subscribers concurrently (#15504)
|
2025-12-19 15:49:51 -08:00 |
|
|
Simo Lin
|
0adfc42b99
|
[model-gateway] Optimize workflow engine with pre-computed dependency graph (#15503)
|
2025-12-19 15:44:57 -08:00 |
|
|
Simo Lin
|
ba72e759ca
|
[model-gateway] Improve logging across core modules (#15497)
|
2025-12-19 13:35:28 -08:00 |
|
|
Simo Lin
|
bd16244d93
|
[model-gateway] Improve logging in policies module (#15496)
|
2025-12-19 13:02:57 -08:00 |
|
|
Simo Lin
|
b5eb0214ca
|
[model-gateway] Improve logging in data_connector module (#15495)
|
2025-12-19 12:57:18 -08:00 |
|
|
Simo Lin
|
d72e908ba5
|
[model-gateway] refactor: extract common graceful shutdown code before TLS branch (#15494)
|
2025-12-19 12:56:33 -08:00 |
|
|
Simo Lin
|
50cad014dc
|
[model-gateway] fix graceful shutdown for TLS/Non-TLS server (#15491)
|
2025-12-19 12:35:50 -08:00 |
|
|
Simo Lin
|
70607e55e8
|
[model-gateway] Replace PolicyRegistry RwLock with DashMap for lock-free policy lookups (#15361)
|
2025-12-18 00:11:20 -08:00 |
|
|
Simo Lin
|
53e151945a
|
[model-gateway] optimize worker registry and reduce lock contention in grpc client fetch (#15336)
|
2025-12-17 08:49:10 -10:00 |
|
|
Simo Lin
|
d747147a26
|
[model-gateway] reduce cpu overhead (#15316)
|
2025-12-17 09:42:01 -08:00 |
|
|
Simo Lin
|
3518b33178
|
[model-gateway] Remove legacy RouterMetrics and Rename SmgMetrics to Metrics and smg_labels to metrics_labels (#15160)
|
2025-12-15 04:57:42 -08:00 |
|
|
Simo Lin
|
7b9156c773
|
[model-gateway] add mcp and discovery metrics (#15156)
|
2025-12-14 22:54:53 -08:00 |
|
|
Simo Lin
|
bd9c3a47d6
|
[model-gateway] Add streaming metrics for harmony gRPC router (#15147)
|
2025-12-14 20:54:43 -08:00 |
|
|
Simo Lin
|
1e641ee4a7
|
[model-gateway] upgrade axum and axum server (#15146)
|
2025-12-14 20:54:01 -08:00 |
|
|
Simo Lin
|
fb96669ff9
|
[model-gateway] Add Layer 3 worker metrics (smg_worker_*) (#15130)
|
2025-12-14 20:25:46 -08:00 |
|
|
Simo Lin
|
0612175ce1
|
[model-gateway] add streaming metrics (TTFT, TPOT, tokens, duration) for gRPC router (#15125)
|
2025-12-14 14:39:10 -08:00 |
|
|
Simo Lin
|
b11af135eb
|
[model-gateway] feat(metrics): implement Layer 2 router metrics (smg_router_*) (#15124)
|
2025-12-14 10:34:12 -08:00 |
|
|
Simo Lin
|
f9bceea064
|
[model-gateway] Implement Layer 1 HTTP metrics instrumentation (#15121)
|
2025-12-14 09:39:15 -08:00 |
|
|
Simo Lin
|
47633c191f
|
[model-gateway] Add new SMG metrics architecture with 6 layers (#15106)
|
2025-12-14 08:37:19 -08:00 |
|
|
Simo Lin
|
74ea45cc4b
|
[model-gateway] optimize metric labels to avoid unnecessary allocations (#15095)
|
2025-12-13 21:10:45 -08:00 |
|
|
Simo Lin
|
0fa044ad63
|
[model-gateway] Add circuit breaker and discovery watcher metrics (#15094)
|
2025-12-13 20:47:04 -08:00 |
|
|
Simo Lin
|
7d8e42c979
|
[model-gateway] Fix metric emission gaps and name mismatch (#15093)
|
2025-12-13 20:16:30 -08:00 |
|
|
Simo Lin
|
20ce9938b5
|
[model-gateway] Remove unused TokenizerMetrics to reduce CPU overhead (#15087)
|
2025-12-13 18:54:18 -08:00 |
|
|
Simo Lin
|
fd37cc5d38
|
[model-gateway] Refactor worker steps and add update workflow (#15085)
|
2025-12-13 15:00:16 -08:00 |
|
|
Simo Lin
|
5d9c6bac07
|
[bug] fix grpc secheduler launcher breaking change (#15080)
|
2025-12-13 10:36:49 -08:00 |
|
|
Simo Lin
|
e048ee90fc
|
[model-gateway] Simplify error response creation (#15079)
|
2025-12-13 10:28:24 -08:00 |
|
|
Simo Lin
|
9a56273ad2
|
[model-gateway] refactor: unify worker management into modular workflow structure (#15010)
|
2025-12-12 15:25:35 -08:00 |
|
|
Simo Lin
|
526fd0082f
|
[model-gateway] refactor: workflow engine cleanup and minor optimization (#15001)
|
2025-12-12 09:03:28 -08:00 |
|
|
Simo Lin
|
56d0ad47f4
|
[model-gateway] fix: handle workflow deadlock and optimize cycle detection (#15000)
|
2025-12-12 08:53:45 -08:00 |
|
|
Simo Lin
|
306e5b8d0b
|
[model-gateway] feat: add DAG parallel execution support and workflow optimization (#14999)
|
2025-12-12 08:31:12 -08:00 |
|
|
Simo Lin
|
10c68f6236
|
[model-gateway] refactor: extract workflow engine to src/workflow module (#14996)
|
2025-12-12 06:29:12 -08:00 |
|
|
Simo Lin
|
6f99dc97f7
|
[model-gateway] fix imports and delete unused code (#14911)
|
2025-12-11 07:42:47 -08:00 |
|
|
Simo Lin
|
cd1c1fa5bc
|
[model-gateway] fix annotation error and code formating (#14910)
|
2025-12-11 07:06:32 -08:00 |
|
|
Simo Lin
|
5b0872d2d7
|
[model-gateway] code clean up on oai router in responses (#14852)
|
2025-12-11 06:55:09 -08:00 |
|
|
Simo Lin
|
bcc5483ed7
|
[model-gateway] fix import order in oai conversation (#14851)
|
2025-12-10 15:36:42 -08:00 |
|
|
Simo Lin
|
ccf2602773
|
[model-gateway] code clean up on oai router (#14850)
|
2025-12-10 15:34:27 -08:00 |
|
|
Simo Lin
|
c032b559a8
|
[model-gateway] adds default implementations to RouterTrait in mod.rs (#14841)
|
2025-12-10 11:46:35 -08:00 |
|
|
Simo Lin
|
2543666c8e
|
[model-gateway] add anthropic message api spec (#14834)
|
2025-12-10 08:26:51 -08:00 |
|
|
Simo Lin
|
390406c46c
|
[model-gateway] release gateway 0.2.4 (#14763)
|
2025-12-09 16:34:29 -08:00 |
|
|
Simo Lin
|
9ad02b799d
|
[Perf] Optimize radix tree for cache-aware load balancin (#14758)
|
2025-12-09 15:35:29 -08:00 |
|
|
Simo Lin
|
73df7a4e8d
|
[SMG] perf: optimize tokenizer for reduced CPU and memory overhead (#14752)
|
2025-12-09 11:41:13 -08:00 |
|