 Chang Su
|
2ca25a8aab
|
Revert "fix: display served_model_name in /v1/models" (#13093)
|
2025-11-11 15:58:46 -08:00 |
|
|
Chang Su
|
838bcb0d93
|
[misc][ci] Add run-ci after auto-labeler (#13013)
|
2025-11-10 10:41:54 -08:00 |
|
|
Chang Su
|
e434187289
|
[router][grpc] Move all error logs to their call sites (#12859)
|
2025-11-07 15:55:34 -08:00 |
|
|
Chang Su
|
fe19a580fb
|
[router][grpc] Refactor: Add builders for chat and responses (#12852)
|
2025-11-07 15:43:32 -08:00 |
|
|
Chang Su
|
2cb42dc184
|
[router][grpc] Add more mcp test cases to responses api (#12749)
|
2025-11-06 10:20:00 -08:00 |
|
|
Chang Su
|
837b08eb2e
|
[router][grpc] Support mixin tool calls in Responses API (#12736)
|
2025-11-05 21:29:45 -08:00 |
|
|
Chang Su
|
ffba61a100
|
[router][grpc] Make harmony parser checks recipient first before channel (#12713)
|
2025-11-05 15:33:52 -08:00 |
|
|
Chang Su
|
3c219eb0e4
|
[misc] Change sync-labels to false (#12714)
|
2025-11-05 15:33:33 -08:00 |
|
|
Chang Su
|
83b104ee8c
|
[misc] Add labeler for automatic labeling (#12710)
|
2025-11-05 14:44:43 -08:00 |
|
|
Chang Su
|
4fe53e5888
|
[router][grpc] Support streaming parsing with Tool Choice in chat completions API (#12677)
|
2025-11-05 12:12:21 -08:00 |
|
|
Chang Su
|
9f5e701879
|
[router][grpc] Implement tool_choice support for Responses API (#12668)
|
2025-11-04 21:43:42 -08:00 |
|
|
Chang Su
|
1357ab025a
|
[router][grpc] Emit OutputItemDone event and store output item array (#12656)
|
2025-11-04 17:01:03 -08:00 |
|
|
Chang Su
|
887742a1e7
|
[router][grpc] Fix index issues in reasoning content and missing streaming events (#12650)
|
2025-11-04 15:38:43 -08:00 |
|
|
Chang Su
|
0e82fd3df4
|
[router][grpc] Fix model validation, tool call check, streaming logic and misc in responses (#12616)
|
2025-11-04 02:21:58 -08:00 |
|
|
Chang Su
|
83804bc626
|
[router][grpc] Restructure modules and code clean up (#12598)
|
2025-11-03 23:59:27 -08:00 |
|
|
Chang Su
|
0b41a293fa
|
[router][grpc] Consolidate error messages build in error.rs (#12301)
|
2025-11-03 14:41:45 -08:00 |
|
|
Chang Suandybyang
|
ecb9fa14e6
|
[docker] clean up main dockerfile for router and dev configurations (#12364)
Co-authored-by: ybyang <ybyang7@iflytek.com>
|
2025-10-30 07:46:09 -07:00 |
|
|
Chang Su
|
db15148c8a
|
[router] update router docker to use maturin and build from local (#12350)
|
2025-10-29 13:00:57 -07:00 |
|
|
Chang Su
|
d717e73ea8
|
[router] refactor mcp to use LRU and fix pooling bug (#12346)
|
2025-10-29 12:18:54 -07:00 |
|
|
Chang Su
|
400bddf24c
|
[router] fix router release workflow and add build test in PR (#12315)
|
2025-10-29 08:11:45 -07:00 |
|
|
Chang Su
|
b48354c537
|
[router][grpc] Fix inconsistent behavior of conversation_id not found (#12299)
|
2025-10-28 16:24:43 -07:00 |
|
|
Chang Su
|
a124b517f6
|
[router] Remove SharedXxxStorage type aliases to make Arc explicit (#12171)
|
2025-10-26 14:48:07 -07:00 |
|
|
Chang Su
|
d05a968ba8
|
[router][grpc] Add ResponsesContext and fix error propagation in responses api (#12164)
|
2025-10-26 14:07:19 -07:00 |
|
|
Chang Su
|
94aad0de99
|
[misc][grpc] Remove duplicate log (#12168)
|
2025-10-26 14:06:59 -07:00 |
|
|
Chang Su
|
6dbb569bc3
|
[router][grpc] Fix tool call id in parse_json_schema_response (#12152)
|
2025-10-26 01:40:53 -07:00 |
|
|
Chang Su
|
773d89da8c
|
[router] cleaned up all the redundant comments in the config module (#12147)
|
2025-10-25 23:10:59 -07:00 |
|
|
Chang Su
|
4463e90dd2
|
[router][grpc] Remove gpt_oss parsers and remove _parser suffix in tool parser files (#12091)
|
2025-10-24 15:14:41 -07:00 |
|
|
Chang Su
|
28b8a4064d
|
[router][CI] Clean up imports and prints statements in sgl-router/py_test (#12024)
|
2025-10-23 11:56:57 -07:00 |
|
|
Chang Su
|
6ade6a02d4
|
[grpc] Support gRPC standard health check (#11955)
|
2025-10-22 16:59:09 -07:00 |
|
|
Chang Su
|
590bc4b7a7
|
[router][grpc] Fix background tasks stored with wrong id (#11945)
|
2025-10-21 18:38:51 -07:00 |
|
|
Chang Su
|
70f6309cd4
|
[router][grpc] Support v1/responses API (#11926)
|
2025-10-21 17:41:48 -07:00 |
|
|
Chang Su
|
9c0b1eb5ad
|
[router][grpc] Fix wram-up random token ids for small models (#11887)
|
2025-10-20 19:22:17 -07:00 |
|
|
Chang Su
|
e69094df64
|
[router][grpc] Remove continue_final_message in ChatTemplateParams and add minijinja-contrib (#11882)
|
2025-10-20 18:03:09 -07:00 |
|
|
Chang Su
|
ca240eefb4
|
[router][grpc] Support parallel queue puts in grpc_request_manager and remove mutex for grpc_client (#11798)
|
2025-10-17 20:49:43 -07:00 |
|
|
Chang Su
|
627974405d
|
[Lint] Add python/sglang to ruff F401 checks and remove unused imports in files (#11685)
|
2025-10-17 16:49:46 -07:00 |
|
|
Chang Su
|
d1984e218c
|
[router][grpc] Remove timeout for connections and remove max_tokens deprecation warning log (#11775)
|
2025-10-17 12:36:36 -07:00 |
|
|
Chang Su
|
9e87b60f37
|
[router][CI] Clean up deprecated fields in pr-test-pd-router.yml (#11739)
|
2025-10-16 19:01:00 -07:00 |
|
|
Chang Su
|
dc01313da1
|
[router] Add rustfmt and set group imports by default (#11732)
|
2025-10-16 17:33:29 -07:00 |
|
|
Chang Su
|
c7962868c1
|
[router] Fix tool_choice normalization in ChatCompletionRequest and fix ut (#11731)
|
2025-10-16 14:20:13 -07:00 |
|
|
Chang Su
|
2479b89405
|
[router][grpc] Simplify model_id determination (#11684)
|
2025-10-15 15:56:58 -07:00 |
|
|
Chang Su
|
f226d3da2a
|
Fix missing json imports in serving_responses.py (#11681)
|
2025-10-15 13:01:55 -07:00 |
|
|
Chang SuandXinyuan Tong
|
30ea4c462b
|
[tool call] Fix prev_tool_call_arr management in base_format_detector.py (#11367)
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
|
2025-10-15 09:51:51 -07:00 |
|
|
Chang Su
|
27ef1459e6
|
[router][protocols] Add Axum validate extractor and use it for /v1/chat/completions endpoint (#11588)
|
2025-10-13 22:51:15 -07:00 |
|
|
Chang Su
|
887c2b4575
|
[router][grpc] Add serve_grpc to launch_server and log id for HealthCheck (#11564)
|
2025-10-13 16:07:19 -07:00 |
|
|
Chang Su
|
4b694e7d5a
|
[router][grpc] Add error handling to generate_tool_constraints (#11562)
|
2025-10-13 12:26:09 -07:00 |
|
|
Chang Su
|
7b59b0b8b0
|
[router][grpc] Further delegate non-stream processing to processing.rs (#11553)
|
2025-10-13 10:36:27 -07:00 |
|
|
Chang Su
|
92777135a0
|
[router][grpc] Consolidate parser checks for chat completions (#11439)
|
2025-10-10 20:44:29 -04:00 |
|
|
Chang Su
|
b520958ec8
|
[router][grpc] Replace fake health check with correct ones (#11387)
|
2025-10-09 09:13:57 -07:00 |
|
|
Chang Su
|
ab926dd697
|
[router][grpc] Fix streaming bugs: empty tool names, state pollution, and panics (#11373)
|
2025-10-09 06:53:23 -04:00 |
|
|
Chang Su
|
a0557642ea
|
[router][lint] Add unused_qualifications to cargo lint warnings (#11366)
|
2025-10-08 22:17:11 -07:00 |
|
|
Chang Su
|
fccac7d126
|
[router][grpc] Add dependencies in Cargo.toml to support chat template rendering (#11342)
|
2025-10-08 15:38:37 -07:00 |
|
|
Chang Su
|
a1080b72a0
|
[router] Fix all unused_qualifications (#11341)
|
2025-10-08 13:55:27 -07:00 |
|
|
Chang Su
|
a65ca73911
|
[router][grpc] Cleanup debug logs in grpc_server and grpc_router (#11340)
|
2025-10-08 13:26:19 -07:00 |
|
|
Chang SuandSimo Lin
|
edd86b8853
|
[router][grpc] Refactor chat handler in grpc/ to use centralized orchestrator (#11314)
Co-authored-by: Simo Lin <linsimo.mark@gmail.com>
|
2025-10-07 20:50:20 -07:00 |
|
|
Chang Su
|
7ba3de0e92
|
[oai serving chat] Add argument --sampling-defaults and fix ChatCompletionRequest defaults (#11304)
|
2025-10-08 00:36:05 +00:00 |
|
|
Chang Su
|
f094e0a490
|
[router][grpc] Fix request_id extraction when n > 1 (#11311)
|
2025-10-07 19:27:56 -04:00 |
|
|
Chang Su
|
420c99acfe
|
[router][grpc] Fix error message format in grpc chat handler (#11307)
|
2025-10-07 13:54:02 -07:00 |
|
|
Chang Su
|
6f1e03a456
|
[router][grpc] Fix sampling_params.stop_strs is None (#11306)
|
2025-10-07 10:57:38 -07:00 |
|
|
Chang Su
|
64582caa84
|
[router][grpc] Refactor chat template content format detection (#11288)
|
2025-10-07 08:38:51 -07:00 |
|
|
Chang Su
|
a578d300ba
|
[router][grpc] Fix proto3 default value mismatches and cleanup unused fields (#11283)
|
2025-10-06 18:54:51 -07:00 |
|
|
Chang Su
|
b07c9c76c5
|
[router][grpc] Refine streaming processes (#11277)
|
2025-10-06 15:15:01 -07:00 |
|
|
Chang Su
|
466992b2d0
|
[router][tool call] Clean up redundant detect_format and has_tool_markers (#11270)
|
2025-10-06 14:04:02 -07:00 |
|
|
Chang Su
|
963175d5c0
|
[router][grpc] Support streaming for v1/chat/completions (#11179)
|
2025-10-02 14:35:16 -07:00 |
|
|
Chang Su
|
3511b37099
|
[proto] Add script to compile python protos (#11171)
|
2025-10-02 08:45:51 -07:00 |
|
|
Chang Su
|
b658be6f6a
|
[router][grpc] Support tool call parser in streaming (#11160)
|
2025-10-02 03:18:50 -07:00 |
|
|
Chang Su
|
8ce830a8b0
|
[router][bugfix] Fix input_logprobs handling with None value and logprob_start_len = -1 (#11113)
|
2025-09-30 16:09:40 -07:00 |
|
|
Chang SuandSimo Lin
|
d1676cd483
|
[router][tool call] Full support for ToolChoice (#11085)
Co-authored-by: Simo Lin <linsimo.mark@gmail.com>
|
2025-09-29 22:36:03 -07:00 |
|
|
Chang Su
|
5937a56d47
|
[router][grpc] Add logprobs support to router (#11082)
|
2025-09-29 15:55:06 -07:00 |
|
|
Chang Su
|
f065e5bea5
|
[router] Use get_pooled in process_single_choice (#11079)
|
2025-09-29 15:48:00 -07:00 |
|
|
Chang Su
|
4eeaff74a0
|
[router][tool call] Separate JsonParser and LlamaParser (#11073)
|
2025-09-29 10:26:37 -07:00 |
|
|
Chang Su
|
af4ab65606
|
[router][tool call] Improve normal content extraction and error handling (non-stream) (#11050)
|
2025-09-29 00:19:30 -07:00 |
|
|
Chang Su
|
dba751a896
|
[router][tool call] Support normal content extraction before tool call (streaming) (#11038)
|
2025-09-28 19:46:06 -07:00 |
|
|
Chang Su
|
c1c8dd1dd0
|
[router][tool parser] Modify tool parser to return both normal text and tool calls (non-stream) (#10995)
|
2025-09-27 18:10:17 -04:00 |
|
|
Chang Su
|
37f3325b06
|
[router][grpc] Support E2E non-stream chat completions (#10980)
|
2025-09-26 22:02:06 -07:00 |
|
|
Chang Su
|
0c3db88978
|
[router][grpc] Add helpfer functions for decoder in router.rs and fix specs (#10971)
|
2025-09-26 20:10:45 -04:00 |
|
|
Chang Su
|
37158f2018
|
router: Support parallel sampling num > 1 in grpc_server and non-stream handling (#10929)
|
2025-09-25 20:03:35 -07:00 |
|
|
Chang Su
|
7dcd689b47
|
[router][refactor] Clean up protobuf fields (#10923)
|
2025-09-25 17:48:47 -07:00 |
|
|
Chang Su
|
5e21d6aec0
|
refactor: Move grpc/client.rs to grpc_client/sglang_scheduler.rs (#10924)
|
2025-09-25 17:21:22 -04:00 |
|
|
Chang Su
|
916784746b
|
router: Fix constraint proto and build_constraint in grpc router (#10881)
|
2025-09-25 11:12:06 -04:00 |
|
|
Chang Su
|
4a87ba217f
|
router-grpc: Add tools processing and other paramters for apply_chat_template (#10877)
|
2025-09-24 15:23:06 -07:00 |
|
|
Chang Su
|
9209b209be
|
router-grpc: Support jinja chat template content format detection (#10832)
|
2025-09-24 11:45:01 -07:00 |
|
|
Chang Su
|
ee704e6265
|
[router] add auth middleware for api key auth (#10826)
|
2025-09-23 16:07:34 -07:00 |
|
|
Chang Su
|
08b8c0c3cd
|
[router] fix axum default body limit (#10818)
|
2025-09-23 12:44:17 -07:00 |
|
|
Chang Su
|
7ff93e613f
|
router(grpc): Implement route for chat_cmpl endpoint (#10761)
|
2025-09-23 11:26:33 -07:00 |
|
|
Chang SuandSimo Lin
|
16adf3dcab
|
[router] fix logger type mismatch (#10774)
Co-authored-by: Simo Lin <linsimo.mark@gmail.com>
|
2025-09-22 21:02:28 -07:00 |
|
|
Chang Su
|
60dbbd086a
|
bugfix: Fix get_worker_urls_for_model in http/router.rs (#10754)
|
2025-09-22 14:10:31 -04:00 |
|
|
Chang Su
|
03ce92e594
|
router-spec: Reorder ChatCompletionRequest and fix validation logic (#10675)
|
2025-09-19 16:41:21 -07:00 |
|
|
Chang Su
|
c1815a99b7
|
model support: Sarashina2VisionForCausalLM (#10632)
|
2025-09-18 17:30:38 -07:00 |
|
|
Chang Su
|
5fe39e85a2
|
[router] fix router manager and router init in server (#10499)
|
2025-09-15 22:23:26 -07:00 |
|
|
Chang Su
|
35ef3f2902
|
[router] fix worker registration in multi model mode (#10486)
|
2025-09-15 21:05:00 -04:00 |
|
|
Chang Su
|
2689f0bf02
|
[router] multi model registration fix (#10481)
|
2025-09-15 15:22:21 -07:00 |
|
|
Chang Su
|
b93acd7020
|
[router] minor code clean up in server startup (#10470)
|
2025-09-15 07:28:25 -07:00 |
|
|
Chang Su
|
69b35793a0
|
[router] fix logger ordering git ctx (#10457)
|
2025-09-14 21:37:21 -07:00 |
|
|
Chang Su
|
53ca15529a
|
Implement Standalone gRPC Server for SGLang Python Scheduler (#10283)
|
2025-09-11 20:57:17 -07:00 |
|
|
Chang Su
|
8b3b995ac9
|
[router] fix release workflow to include protobuf (#10055)
|
2025-09-04 22:09:30 -07:00 |
|
|
Chang Su
|
11dcabc545
|
Grpc client (#9939)
|
2025-09-02 11:47:35 -07:00 |
|
|
Chang Su
|
9a0cac1be0
|
[router] add grpc pd and regular router init (#9893)
|
2025-09-01 20:06:15 -07:00 |
|
|
Chang Su
|
598c0bc19d
|
[router] add tokenizer download support from hf hub (#9882)
|
2025-09-01 10:40:37 -07:00 |
|
|
Chang Su
|
c112bcc461
|
[router] global tool parser registry (#9840)
|
2025-08-30 23:35:39 -07:00 |
|
|
Chang Su
|
fd5ce576a4
|
Tool parser.benchmark (#9835)
|
2025-08-30 21:08:11 -07:00 |
|