 Kangyan-ZhouandClaude Opus 4.6
|
89553ff82b
|
[Observability] Add Prometheus metrics endpoint for gRPC mode (#20801)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
2026-04-09 20:04:54 -07:00 |
|
|
Chang Su
|
1389962f06
|
[gRPC] Preserve original ImportError in grpc_server.py (#21801)
Signed-off-by: Chang Su <chang.s.su@oracle.com>
|
2026-03-31 18:22:29 -07:00 |
|
|
Simo Lin
|
654fc02cf1
|
[gRPC] Extract gRPC servicer into standalone package (#20478)
Signed-off-by: Simo Lin <linsimo.mark@gmail.com>
|
2026-03-13 09:13:29 -07:00 |
|
|
Simo LinandChang Su
|
3f3eb206fa
|
feat(grpc): add SubscribeKvEvents RPC for KV cache event streaming (#20112)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2026-03-08 16:00:29 -07:00 |
|
|
Simo Lin
|
9502369488
|
fix(grpc): add server-side keepalive options to prevent GOAWAY (#19986)
Signed-off-by: Simo Lin <linsimo.mark@gmail.com>
|
2026-03-05 18:56:35 -08:00 |
|
|
Chang Su
|
dd2bbe6d62
|
fix(grpc): use context.abort() with proper status codes instead of in-band errors (#19972)
Signed-off-by: Chang Su <chang.s.su@oracle.com>
|
2026-03-05 14:53:18 -08:00 |
|
|
Kangyan-Zhouand
|
198381d9ce
|
Add SSL/TLS support for HTTP and gRPC servers (#18973)
Co-authored-by: guys@spotify.com
|
2026-03-04 19:27:16 -08:00 |
|
|
Chang Su
|
0ee9d3c8e9
|
fix(grpc): send last chunk before completion during streaming (#19895)
|
2026-03-04 13:21:21 -08:00 |
|
|
|
d939e26585
|
[model gateway][0/N] router EPD support: add encoder grpc server backend support (#16552)
Co-authored-by: Zongyao Chen <ZongYao.Chen@linux.alibaba.com>
Co-authored-by: Zongyao Chen <solar1s@163.com>
|
2026-03-03 19:38:15 +08:00 |
|
|
Chang Su
|
71620122c9
|
feat(grpc): add multimodal TensorData parsing for vision inference (#19535)
Signed-off-by: Chang Su <chang.s.su@oracle.com>
|
2026-02-27 19:29:43 -08:00 |
|
|
Simo Lin
|
92c5749f41
|
refactor: replace local proto compilation with smg-grpc-proto package (#18682)
|
2026-02-12 05:29:24 -08:00 |
|
|
JooYeonandjoo_yeon.lee
|
c9297661b9
|
fix: /metrics endpoint always reports engine_type="unified" in PD disaggregation mode (#18552)
Co-authored-by: joo_yeon.lee <joo_yeon.lee@samsung.com>
|
2026-02-12 14:20:43 +08:00 |
|
|
Simo Lin
|
f091858304
|
[gRPC] Add GetLoads RPC for comprehensive load metrics (#17087)
|
2026-01-14 12:05:10 -08:00 |
|
|
Simo LinandChang Su
|
162d1cf9be
|
[model-gateway] Add classification model support infrastructure (#16061)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-12-29 08:34:05 -08:00 |
|
|
Ratish P
|
ca740a41f3
|
[model-gateway]: fix grpc embedding test (#15934)
|
2025-12-27 05:56:54 -05:00 |
|
|
Ratish P
|
886e038329
|
[model-gateway]: fix crash in embedding worker health check (#15910)
|
2025-12-26 12:57:13 -08:00 |
|
|
ratish
|
5f3a47d8a7
|
[model-gateway]: add gRPC router embeddings endpoint implementation (#15273)
|
2025-12-23 09:12:22 -08:00 |
|
|
Lianmin Zheng
|
5e1a495c65
|
Improve engine customization interface (#15635)
|
2025-12-22 14:24:16 -08:00 |
|
|
Simo Lin
|
7235a7fbe9
|
[misc] add model arch and type to server info and use it for harmony (#14456)
|
2025-12-05 06:51:00 -08:00 |
|
|
Chang Su
|
94aad0de99
|
[misc][grpc] Remove duplicate log (#12168)
|
2025-10-26 14:06:59 -07:00 |
|
|
Chang Su
|
6ade6a02d4
|
[grpc] Support gRPC standard health check (#11955)
|
2025-10-22 16:59:09 -07:00 |
|
|
Chang Su
|
9c0b1eb5ad
|
[router][grpc] Fix wram-up random token ids for small models (#11887)
|
2025-10-20 19:22:17 -07:00 |
|
|
Simo Lin
|
4f24ab1718
|
[router][grpc] add dissag info to warm up in grpc server (#11727)
|
2025-10-16 14:19:55 -07:00 |
|
|
Simo LinandChang Su
|
325951460f
|
[router][grpc] add warm up to grpc server (#11627)
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2025-10-14 16:11:16 -07:00 |
|
|
Chang Su
|
887c2b4575
|
[router][grpc] Add serve_grpc to launch_server and log id for HealthCheck (#11564)
|
2025-10-13 16:07:19 -07:00 |
|
|
Simo Lin
|
c495833186
|
[router] leverage RAII to actively cancel request during client disconnect (#11399)
|
2025-10-10 20:43:38 -04:00 |
|
|
Lianmin Zheng
|
9b8ebb2798
|
move more files under srt/utils (#11285)
|
2025-10-09 16:46:15 -07:00 |
|
|
Chang Su
|
b520958ec8
|
[router][grpc] Replace fake health check with correct ones (#11387)
|
2025-10-09 09:13:57 -07:00 |
|
|
Simo Lin
|
368fd20622
|
[router][grpc] disable health check generation and increase timeout (#11353)
|
2025-10-08 19:23:08 -07:00 |
|
|
Chang Su
|
a65ca73911
|
[router][grpc] Cleanup debug logs in grpc_server and grpc_router (#11340)
|
2025-10-08 13:26:19 -07:00 |
|
|
Chang Su
|
6f1e03a456
|
[router][grpc] Fix sampling_params.stop_strs is None (#11306)
|
2025-10-07 10:57:38 -07:00 |
|
|
Simo Lin
|
2fcd56eaf6
|
[router] add get server info and get model info in grpc server (#11303)
|
2025-10-07 08:36:52 -07:00 |
|
|
Chang Su
|
a578d300ba
|
[router][grpc] Fix proto3 default value mismatches and cleanup unused fields (#11283)
|
2025-10-06 18:54:51 -07:00 |
|
|
Simo Lin
|
d736e0b65e
|
[router] add grpc router pd mode for chat and generate (#11140)
|
2025-10-04 06:58:28 -07:00 |
|
|
Chang Su
|
963175d5c0
|
[router][grpc] Support streaming for v1/chat/completions (#11179)
|
2025-10-02 14:35:16 -07:00 |
|
|
Simo Lin
|
96fe2d0f15
|
[router] add pd service in grpc router for pd (#11120)
|
2025-10-01 08:09:21 -07:00 |
|
|
Chang Su
|
8ce830a8b0
|
[router][bugfix] Fix input_logprobs handling with None value and logprob_start_len = -1 (#11113)
|
2025-09-30 16:09:40 -07:00 |
|
|
Simo Lin
|
5a290a5644
|
[router][grpc-server] Fix gRPC server shutdown (#11094)
|
2025-09-30 07:12:12 -04:00 |
|
|
Chang Su
|
5937a56d47
|
[router][grpc] Add logprobs support to router (#11082)
|
2025-09-29 15:55:06 -07:00 |
|
|
Simo Lin
|
5519766a4d
|
[router] fix chat template loading and tokenizer path (#10999)
|
2025-09-27 23:54:12 -04:00 |
|
|
Chang Su
|
37f3325b06
|
[router][grpc] Support E2E non-stream chat completions (#10980)
|
2025-09-26 22:02:06 -07:00 |
|
|
Chang Su
|
37158f2018
|
router: Support parallel sampling num > 1 in grpc_server and non-stream handling (#10929)
|
2025-09-25 20:03:35 -07:00 |
|
|
Chang Su
|
7dcd689b47
|
[router][refactor] Clean up protobuf fields (#10923)
|
2025-09-25 17:48:47 -07:00 |
|
|
Chang Su
|
916784746b
|
router: Fix constraint proto and build_constraint in grpc router (#10881)
|
2025-09-25 11:12:06 -04:00 |
|
|
Chang Su
|
53ca15529a
|
Implement Standalone gRPC Server for SGLang Python Scheduler (#10283)
|
2025-09-11 20:57:17 -07:00 |
|