 Simo Lin
|
53e151945a
|
[model-gateway] optimize worker registry and reduce lock contention in grpc client fetch (#15336)
|
2025-12-17 08:49:10 -10:00 |
|
|
Simo Lin
|
d747147a26
|
[model-gateway] reduce cpu overhead (#15316)
|
2025-12-17 09:42:01 -08:00 |
|
|
Simo Lin
|
3518b33178
|
[model-gateway] Remove legacy RouterMetrics and Rename SmgMetrics to Metrics and smg_labels to metrics_labels (#15160)
|
2025-12-15 04:57:42 -08:00 |
|
|
Simo Lin
|
7b9156c773
|
[model-gateway] add mcp and discovery metrics (#15156)
|
2025-12-14 22:54:53 -08:00 |
|
|
Simo Lin
|
bd9c3a47d6
|
[model-gateway] Add streaming metrics for harmony gRPC router (#15147)
|
2025-12-14 20:54:43 -08:00 |
|
|
Simo Lin
|
1e641ee4a7
|
[model-gateway] upgrade axum and axum server (#15146)
|
2025-12-14 20:54:01 -08:00 |
|
|
Simo Lin
|
fb96669ff9
|
[model-gateway] Add Layer 3 worker metrics (smg_worker_*) (#15130)
|
2025-12-14 20:25:46 -08:00 |
|
|
Simo Lin
|
0612175ce1
|
[model-gateway] add streaming metrics (TTFT, TPOT, tokens, duration) for gRPC router (#15125)
|
2025-12-14 14:39:10 -08:00 |
|
|
Simo Lin
|
b11af135eb
|
[model-gateway] feat(metrics): implement Layer 2 router metrics (smg_router_*) (#15124)
|
2025-12-14 10:34:12 -08:00 |
|
|
Simo Lin
|
f9bceea064
|
[model-gateway] Implement Layer 1 HTTP metrics instrumentation (#15121)
|
2025-12-14 09:39:15 -08:00 |
|
|
Simo Lin
|
47633c191f
|
[model-gateway] Add new SMG metrics architecture with 6 layers (#15106)
|
2025-12-14 08:37:19 -08:00 |
|
|
Simo Lin
|
74ea45cc4b
|
[model-gateway] optimize metric labels to avoid unnecessary allocations (#15095)
|
2025-12-13 21:10:45 -08:00 |
|
|
Simo Lin
|
0fa044ad63
|
[model-gateway] Add circuit breaker and discovery watcher metrics (#15094)
|
2025-12-13 20:47:04 -08:00 |
|
|
Simo Lin
|
7d8e42c979
|
[model-gateway] Fix metric emission gaps and name mismatch (#15093)
|
2025-12-13 20:16:30 -08:00 |
|
|
Simo Lin
|
20ce9938b5
|
[model-gateway] Remove unused TokenizerMetrics to reduce CPU overhead (#15087)
|
2025-12-13 18:54:18 -08:00 |
|
|
Simo Lin
|
fd37cc5d38
|
[model-gateway] Refactor worker steps and add update workflow (#15085)
|
2025-12-13 15:00:16 -08:00 |
|
|
Simo Lin
|
5d9c6bac07
|
[bug] fix grpc secheduler launcher breaking change (#15080)
|
2025-12-13 10:36:49 -08:00 |
|
|
Simo Lin
|
e048ee90fc
|
[model-gateway] Simplify error response creation (#15079)
|
2025-12-13 10:28:24 -08:00 |
|
|
Simo Lin
|
9a56273ad2
|
[model-gateway] refactor: unify worker management into modular workflow structure (#15010)
|
2025-12-12 15:25:35 -08:00 |
|
|
Simo Lin
|
526fd0082f
|
[model-gateway] refactor: workflow engine cleanup and minor optimization (#15001)
|
2025-12-12 09:03:28 -08:00 |
|
|
Simo Lin
|
56d0ad47f4
|
[model-gateway] fix: handle workflow deadlock and optimize cycle detection (#15000)
|
2025-12-12 08:53:45 -08:00 |
|
|
Simo Lin
|
306e5b8d0b
|
[model-gateway] feat: add DAG parallel execution support and workflow optimization (#14999)
|
2025-12-12 08:31:12 -08:00 |
|
|
Simo Lin
|
10c68f6236
|
[model-gateway] refactor: extract workflow engine to src/workflow module (#14996)
|
2025-12-12 06:29:12 -08:00 |
|
|
Simo Lin
|
6f99dc97f7
|
[model-gateway] fix imports and delete unused code (#14911)
|
2025-12-11 07:42:47 -08:00 |
|
|
Simo Lin
|
cd1c1fa5bc
|
[model-gateway] fix annotation error and code formating (#14910)
|
2025-12-11 07:06:32 -08:00 |
|
|
Simo Lin
|
5b0872d2d7
|
[model-gateway] code clean up on oai router in responses (#14852)
|
2025-12-11 06:55:09 -08:00 |
|
|
Simo Lin
|
bcc5483ed7
|
[model-gateway] fix import order in oai conversation (#14851)
|
2025-12-10 15:36:42 -08:00 |
|
|
Simo Lin
|
ccf2602773
|
[model-gateway] code clean up on oai router (#14850)
|
2025-12-10 15:34:27 -08:00 |
|
|
Simo Lin
|
c032b559a8
|
[model-gateway] adds default implementations to RouterTrait in mod.rs (#14841)
|
2025-12-10 11:46:35 -08:00 |
|
|
Simo Lin
|
2543666c8e
|
[model-gateway] add anthropic message api spec (#14834)
|
2025-12-10 08:26:51 -08:00 |
|
|
Simo Lin
|
390406c46c
|
[model-gateway] release gateway 0.2.4 (#14763)
|
2025-12-09 16:34:29 -08:00 |
|
|
Simo Lin
|
9ad02b799d
|
[Perf] Optimize radix tree for cache-aware load balancin (#14758)
|
2025-12-09 15:35:29 -08:00 |
|
|
Simo Lin
|
73df7a4e8d
|
[SMG] perf: optimize tokenizer for reduced CPU and memory overhead (#14752)
|
2025-12-09 11:41:13 -08:00 |
|
|
Simo Lin
|
8b98bb768c
|
[model-gateway] optimize core modules (#14751)
|
2025-12-09 10:54:07 -08:00 |
|
|
Simo Lin
|
af60cad05d
|
[ci][smg] fix docker release ci and add it to pr test (#14683)
|
2025-12-08 20:56:38 -08:00 |
|
|
Simo Lin
|
edde5e5d40
|
[model-gateway] add OTEL integration to grpc router (#14671)
|
2025-12-08 14:45:05 -08:00 |
|
|
Simo Lin
|
8550822d6b
|
[model-gateway] Optimize memory usage in HTTP router (#14667)
|
2025-12-08 13:10:31 -08:00 |
|
|
Simo Lin
|
7bf16c6339
|
[model-gateway] fix WASM arbitrary file read security vol (#14664)
|
2025-12-08 12:11:16 -08:00 |
|
|
Simo Lin
|
39f9a9c2a5
|
[model-gateway] reduce cpu overhead in grpc router (#14663)
|
2025-12-08 11:54:56 -08:00 |
|
|
Simo Lin
|
d69ecc19b8
|
[model-gateway] reducing cpu overhead in various of places (#14658)
|
2025-12-08 09:44:40 -08:00 |
|
|
Simo Lin
|
2970f22917
|
[model-gateway] fix WASM unbounded request/response body read vuln (#14612)
|
2025-12-07 20:47:25 -08:00 |
|
|
Simo Lin
|
8fbf7dd56f
|
[model-gateway] refactor otel to be more efficient (#14604)
|
2025-12-07 20:09:25 -08:00 |
|
|
Simo Lin
|
a4ffd665c0
|
[model-gateway] fix WASM memory limit per module (#14600)
|
2025-12-07 19:15:19 -08:00 |
|
|
Simo Lin
|
aff1238ef2
|
[model-gateway] reorganize metrics, logging, and otel to its own module (#14590)
|
2025-12-07 16:50:39 -08:00 |
|
|
Simo Lin
|
5e2cda6158
|
[model-gateway] Fixed WASM Security Vulnerability - Execution Timeout (#14588)
|
2025-12-07 16:05:53 -08:00 |
|
|
Simo Lin
|
b0bbc7f53d
|
[model-gateway] extra accumulator and tool handler in oai router (#14587)
|
2025-12-07 15:52:12 -08:00 |
|
|
Simo Lin
|
d30d6b368f
|
[bug] fix notebook to include new keys from model_info (#14528)
|
2025-12-05 20:44:03 -08:00 |
|
|
Simo Lin
|
959a17466d
|
[model-gateway] fix code owner for wasm (#14516)
|
2025-12-05 13:48:25 -08:00 |
|
|
Simo Lin
|
b72f0268c7
|
[model-gateway] fix left over sgl-router names in wasm (#14514)
|
2025-12-05 13:16:58 -08:00 |
|
|
Simo Lin
|
09376fd738
|
[model-gateway] fix logs in smg workflow (#14513)
|
2025-12-05 12:50:50 -08:00 |
|
|
Simo Lin
|
aed835e32d
|
[model-gateway] fix left over sgl-router names to sgl-model-gateway (#14512)
|
2025-12-05 12:41:48 -08:00 |
|
|
Simo Lin
|
49dfa1d891
|
[model-gateway] change sgl-router to sgl-model-gateway (#14312)
|
2025-12-05 12:04:48 -08:00 |
|
|
Simo Lin
|
cf9a774cf8
|
[model-gateway] fix server info comment (#14508)
|
2025-12-05 10:09:58 -08:00 |
|
|
Simo Linandkey4ng
|
1569fc7f49
|
[model-gateway] reorganized conversation handler (#14507)
Co-authored-by: key4ng <rukeyang@gmail.com>
|
2025-12-05 09:31:27 -08:00 |
|
|
Simo Linandkey4ng
|
05284378d6
|
[model-gateway] move conversation to first class routing (#14506)
Co-authored-by: key4ng <rukeyang@gmail.com>
|
2025-12-05 08:44:13 -08:00 |
|
|
Simo Lin
|
7235a7fbe9
|
[misc] add model arch and type to server info and use it for harmony (#14456)
|
2025-12-05 06:51:00 -08:00 |
|
|
Simo LinandChang Su
|
c1006fd8a1
|
[model-gateway] grpc to leverage event type (#14450)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-12-04 14:30:19 -08:00 |
|
|
Simo LinandChang Su
|
29c6c2ea9d
|
[model-gateway] add mistral 3 image processor (#14445)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-12-04 13:16:04 -08:00 |
|
|
Simo Linandkey4ng
|
eb85fa6daf
|
[model-gateway] move all responses api event from oai to proto (#14446)
Co-authored-by: key4ng <rukeyang@gmail.com>
|
2025-12-04 13:11:03 -08:00 |
|
|
Simo Linandkey4ng
|
d8faf2f361
|
[model-gateway] move oai header util to router header util (#14441)
Co-authored-by: key4ng <rukeyang@gmail.com>
|
2025-12-04 11:02:50 -08:00 |
|
|
Simo Linandkey4ng
|
7f3308bc68
|
[model-gateway] extract conversation out of oai router (#14440)
Co-authored-by: key4ng <rukeyang@gmail.com>
|
2025-12-04 10:16:14 -08:00 |
|
|
Simo Lin
|
fdc2ef58db
|
[model-gateway] add llama4 vision image processor (#14438)
|
2025-12-04 09:52:36 -08:00 |
|
|
Simo Linandkey4ng
|
b01fc161eb
|
[model-gateway] introduce request ctx for oai router (#14434)
Co-authored-by: key4ng <rukeyang@gmail.com>
|
2025-12-04 08:31:44 -08:00 |
|
|
Simo Lin
|
11d33c0e8f
|
[model-gateway] add phi4 vision image processor (#14430)
|
2025-12-04 07:40:33 -08:00 |
|
|
Simo Lin
|
fa78c44a3d
|
[model-gateway] introduce provider in openai router (#14394)
|
2025-12-03 22:47:20 -08:00 |
|
|
Simo Lin
|
654a78f9c8
|
[model-gateway] add phi3 vision image processor (#14381)
|
2025-12-03 22:32:40 -08:00 |
|
|
Simo Lin
|
d42c167bfd
|
[model-gateway] add qwen3_vl model image processor (#14377)
|
2025-12-03 17:16:37 -08:00 |
|
|
Simo Lin
|
388151053d
|
[model-gateway] use worker crate in openai router (#14330)
|
2025-12-03 13:36:32 -08:00 |
|
|
Simo Lin
|
8ab5d8b40d
|
[model-gateway] add qwen2.5_vl model image processor (#14375)
|
2025-12-03 12:51:30 -08:00 |
|
|
Simo Lin
|
03575ce38d
|
[model-gateway] add qwen2_vl model image processor and tests (#14374)
|
2025-12-03 12:34:06 -08:00 |
|
|
Simo Lin
|
abf6272bdd
|
[model-gateway] add llava model image processor and tests (#14371)
|
2025-12-03 11:18:00 -08:00 |
|
|
Simo Lin
|
58ac3f317b
|
[model-gateway] add image processor and transformer structure (#14344)
|
2025-12-03 00:04:06 -08:00 |
|
|
Simo Lin
|
d5ea8c7176
|
[model-gateway] multimodality initialization (#13350)
|
2025-12-02 19:41:40 -08:00 |
|
|
Simo Lin
|
5dad1ff1b6
|
[model-gateway] add workflow for external model providers (#14323)
|
2025-12-02 16:47:06 -08:00 |
|
|
Simo Lin
|
51a86ce69e
|
[model-gateway] change rust package name to sgl-model-gateway instead (#14283)
|
2025-12-02 00:06:54 -08:00 |
|
|
Simo Lin
|
383689e3ad
|
[model-gateway] fix version output (#14276)
|
2025-12-01 22:51:57 -08:00 |
|
|
Simo Lin
|
73df525382
|
[model-gateway] include smg version command in py binding (#14274)
|
2025-12-01 21:21:02 -08:00 |
|
|
Simo Lin
|
cd4151abc7
|
[model-gateway] add audio and moderation in model card (#14263)
|
2025-12-01 18:36:38 -08:00 |
|
|
Simo Lin
|
57f933fd7d
|
[model-gateway] Migrate Worker trait to model-aware methods (#14250)
|
2025-12-01 14:32:09 -08:00 |
|
|
Simo Lin
|
e7d6027e4a
|
[model-gateway] add ModelCard support to WorkerMetadata (#14243)
|
2025-12-01 13:04:09 -08:00 |
|
|
Simo Lin
|
9325f94534
|
[model-gateway] add ModelCard and ProviderType for model configuration (#14237)
|
2025-12-01 09:47:51 -08:00 |
|
|
Simo Lin
|
92ca629582
|
[model-gateway] add ModelType bitflags and Endpoint enum for worker (#14230)
|
2025-12-01 08:14:44 -08:00 |
|
|
Simo Lin
|
79b389da6b
|
[model-gateway] refactor oai router 1/n (#14228)
|
2025-12-01 07:16:35 -08:00 |
|
|
Simo Lin
|
e9e90460ed
|
[model-gateway] allow refill rate to be zero (#14030)
|
2025-11-26 17:15:27 -08:00 |
|
|
Simo Lin
|
69a03bc3f7
|
[ci] allow manual label to trigger ci in rust, change ci order (#14016)
|
2025-11-26 11:38:50 -08:00 |
|
|
Simo Lin
|
7e964b5198
|
[ci] mark skip as success instead of failure (#14014)
|
2025-11-26 10:39:38 -08:00 |
|
|
Simo Lin
|
a0d9f6cdd8
|
[model-gateway] fix xpu ci (#14012)
|
2025-11-26 10:33:59 -08:00 |
|
|
Simo Lin
|
4852aa054c
|
[misc] add llama3.1 chat template (#13935)
|
2025-11-25 09:31:54 -08:00 |
|
|
Simo Lin
|
9b4b344115
|
[model-gateway] add grpc server code owner (#13865)
|
2025-11-24 12:29:57 -08:00 |
|
|
Simo Lin
|
9054e844ea
|
remove package json which is not used (#13810)
|
2025-11-23 15:00:30 -08:00 |
|
|
Simo Lin
|
5354d7b7fd
|
[model-gateway] clean up router manager function order (#13776)
|
2025-11-22 16:16:20 -08:00 |
|
|
Simo Lin
|
e01486778a
|
[model-gateway] update smg code owner (#13777)
|
2025-11-22 15:50:53 -08:00 |
|
|
Simo Lin
|
3ae664d786
|
[model-gateway] add both python and rust cli alias (#13678)
|
2025-11-20 17:00:28 -08:00 |
|
|
Simo Lin
|
6b9459e86c
|
[docker] fix dockerfile naming for diffusion (#13534)
|
2025-11-18 12:18:32 -08:00 |
|
|
Simo Lin
|
9188feccca
|
[model-gateway] use worker startup time out for worker registration (#13473)
|
2025-11-17 19:28:21 -08:00 |
|
|
Simo Lin
|
25acbbc612
|
Remove verbs from GET endpoint paths to follow REST standards (#13273)
|
2025-11-17 12:34:36 -08:00 |
|
|
Simo Lin
|
b436113fc2
|
[model-gateway] Add Gateway Release Tooling (#13420)
|
2025-11-17 03:26:48 -08:00 |
|
|
Simo Lin
|
b24235b8bb
|
[model-gateway] update workflow names for gateway and exclude npu (#13415)
|
2025-11-17 02:04:07 -08:00 |
|
|
Simo Lin
|
efc5d8f5ed
|
[model-gateway] fix SDist step readme path (#13373)
|
2025-11-16 03:16:52 -08:00 |
|
|
Simo Lin
|
ef32a252d0
|
[model-gateway] fix model gateway pypi release workflow path (#13372)
|
2025-11-16 02:33:49 -08:00 |
|