 Chang Su
|
1389962f06
|
[gRPC] Preserve original ImportError in grpc_server.py (#21801)
Signed-off-by: Chang Su <chang.s.su@oracle.com>
|
2026-03-31 18:22:29 -07:00 |
|
|
Chang Su
|
0d23a461a0
|
feat(mm)(grpc): compute M-RoPE positions for preprocessed VL inputs (#19973)
Signed-off-by: Chang Su <chang.s.su@oracle.com>
Co-authored-by: Chang Su <chang.s.su@oracle.com>
|
2026-03-18 15:34:50 -07:00 |
|
|
Chang Su
|
dd2bbe6d62
|
fix(grpc): use context.abort() with proper status codes instead of in-band errors (#19972)
Signed-off-by: Chang Su <chang.s.su@oracle.com>
|
2026-03-05 14:53:18 -08:00 |
|
|
Chang Su
|
0ee9d3c8e9
|
fix(grpc): send last chunk before completion during streaming (#19895)
|
2026-03-04 13:21:21 -08:00 |
|
|
Chang Su
|
71620122c9
|
feat(grpc): add multimodal TensorData parsing for vision inference (#19535)
Signed-off-by: Chang Su <chang.s.su@oracle.com>
|
2026-02-27 19:29:43 -08:00 |
|
|
Chang Su
|
4229de3b13
|
[model-gateway] Refine TokenizerRegisty.load() to handle duplication (#17230)
|
2026-01-16 13:53:21 -08:00 |
|
|
Chang Su
|
a7f5677abe
|
refactor: unify registration through tokenizer_registration workflow (#17187)
|
2026-01-16 10:53:36 -08:00 |
|
|
Chang Su
|
669d309a8b
|
[model-gateway] Consolidate "unknown" model id usage (#17186)
|
2026-01-15 21:02:40 -08:00 |
|
|
Chang Su
|
16880235d1
|
[grpc] Auto-generate protobuf files during wheel build (#16409)
|
2026-01-08 09:09:54 -08:00 |
|
|
Chang Su
|
fb5b71d015
|
[router][openai] Rename prepare_mcp_payload_for_streaming and patch_streaming_response_json (#16596)
|
2026-01-06 16:33:29 -08:00 |
|
|
Chang Su
|
05b54b6d7b
|
[router][grpc] Replace Vec<(String, String, String)> with ExtractedToolCall (#16598)
|
2026-01-06 16:32:59 -08:00 |
|
|
Chang Su
|
4f443f445a
|
[model-gateway][cleanup] Fix wrong comment in manager.rs (#16601)
|
2026-01-06 16:32:37 -08:00 |
|
|
Chang Su
|
21da2dc1ce
|
[grpc] Unify ResponsesContext and HarmonyResponsesContext (#16549)
|
2026-01-06 07:28:06 -08:00 |
|
|
Chang Su
|
9bf76c11a9
|
[responses API] Add list_tools_for_servers and threading server_keys in routers (#16540)
|
2026-01-05 22:42:22 -08:00 |
|
|
Chang Su
|
4cf2bbd084
|
[router] Remove deadcode and add note for unused API completeness methods (#16528)
|
2026-01-05 19:53:28 -08:00 |
|
|
Chang Su
|
b98bd9a5fb
|
[model-gateway] Tighten visibility in modules and remove unused re-exports (#16524)
|
2026-01-05 17:38:50 -08:00 |
|
|
Chang Su
|
1751c75b5d
|
[model-gateway] Tighten visibility across data_connector and grpc module (#16516)
|
2026-01-05 12:31:32 -08:00 |
|
|
Chang Su
|
51541404f8
|
[grpc] Refactor openai module (#16511)
|
2026-01-05 11:39:33 -08:00 |
|
|
Chang Su
|
5a2b1ed407
|
[grpc] Refactor grpc/regular/responses (#16509)
|
2026-01-05 10:21:34 -08:00 |
|
|
Chang Su
|
454dc9e242
|
[model-gateway][grpc] Refactor harmony/responses.rs (#16508)
|
2026-01-05 10:21:05 -08:00 |
|
|
Chang Su
|
8d58b3dcdd
|
[model-gateway] Clear architectual debt in responses API (#16359)
|
2026-01-03 21:35:28 -08:00 |
|
|
Chang Su
|
7bc13c906c
|
[grpc] update api to scheduler in grpc request manager (#16350)
|
2026-01-03 10:54:31 -08:00 |
|
|
Chang Su
|
d8ac5eecf7
|
[model-gateway] bug fix on module name (#16332)
|
2026-01-02 21:56:18 -08:00 |
|
|
Chang Su
|
b23e7ed13c
|
refactor(gateway): shorten logging targets from sgl_model_gateway to smg (#16328)
|
2026-01-02 21:12:33 -08:00 |
|
|
Chang Su
|
24c91001cf
|
[model-gateway] code clean up in tokenizer register step workflow (#16316)
|
2026-01-02 16:05:10 -08:00 |
|
|
Chang Su
|
c4edcac6d7
|
refactor(core): remove get_by_model_fast alias in worker_registry (#16313)
|
2026-01-02 16:04:23 -08:00 |
|
|
Chang Su
|
e93433892b
|
refactor(steps): consolidate duplicate strip_protocol function (#16318)
|
2026-01-02 16:02:42 -08:00 |
|
|
Chang Su
|
a2d4f58a96
|
fix(http): use 504 Gateway Timeout for upstream timeouts (#16320)
|
2026-01-02 16:01:44 -08:00 |
|
|
Chang Su
|
f66b091699
|
refactor(http): improve pd_types.rs documentation and style (#16319)
|
2026-01-02 16:01:11 -08:00 |
|
|
Chang Su
|
2f623368ab
|
refactor(core): use idiomatic .min() in calculate_delay (#16314)
|
2026-01-02 15:46:13 -08:00 |
|
|
Chang Su
|
bd9a2ced47
|
refactor(core): use thiserror for WorkerError (#16315)
|
2026-01-02 15:33:07 -08:00 |
|
|
Chang Su
|
0ca417d9ae
|
chore(core): remove unused serde_json import in worker.rs (#16312)
|
2026-01-02 15:31:41 -08:00 |
|
|
Chang SuandSimo Lin
|
ec92d7f14e
|
[model-gateway] fix v1/models response format to be oai compatible (#13693)
Co-authored-by: Simo Lin <linsimo.mark@gmail.com>
|
2025-12-01 07:20:38 -08:00 |
|
|
Chang SuandSimo Lin
|
66f242b98f
|
[model gateway][grpc] Add tojson filter to override minijinja's tojson (#14013)
Co-authored-by: Simo Lin <linsimo.mark@gmail.com>
|
2025-11-26 11:05:01 -08:00 |
|
|
Chang Su
|
c4db77f8a9
|
[model-gateway] fix gateway cli arg parser to not use = (#13685)
|
2025-11-20 17:57:17 -08:00 |
|
|
Chang Su
|
0d2d687812
|
[router][grpc] Support num_reasoning_tokens in haromy models (#13047)
|
2025-11-18 16:40:32 -08:00 |
|
|
Chang Su
|
3a6ec47b0b
|
[model-gateway] limit opened files in docker build to fix edge case (#13536)
|
2025-11-18 12:17:38 -08:00 |
|
|
Chang Su
|
b8e32e7965
|
[model-gateway] fix gateway docker build due to recent py code change (#13532)
|
2025-11-18 11:24:25 -08:00 |
|
|
Chang Su
|
c9e5799bd1
|
[router][grpc] Refine docs in minimax_m2 to match other parsers (#13218)
|
2025-11-14 06:47:24 +09:00 |
|
|
Chang Su
|
e5e65e3d2a
|
[router][grpc] Support vllm backend for grpc router (#13120)
|
2025-11-12 02:29:20 -08:00 |
|
|
Chang Su
|
018123b5b0
|
[misc] Remove performance and router-benchmark label matching (#13135)
|
2025-11-12 00:04:20 -08:00 |
|
|
Chang Su
|
2ca25a8aab
|
Revert "fix: display served_model_name in /v1/models" (#13093)
|
2025-11-11 15:58:46 -08:00 |
|
|
Chang Su
|
838bcb0d93
|
[misc][ci] Add run-ci after auto-labeler (#13013)
|
2025-11-10 10:41:54 -08:00 |
|
|
Chang Su
|
e434187289
|
[router][grpc] Move all error logs to their call sites (#12859)
|
2025-11-07 15:55:34 -08:00 |
|
|
Chang Su
|
fe19a580fb
|
[router][grpc] Refactor: Add builders for chat and responses (#12852)
|
2025-11-07 15:43:32 -08:00 |
|
|
Chang Su
|
2cb42dc184
|
[router][grpc] Add more mcp test cases to responses api (#12749)
|
2025-11-06 10:20:00 -08:00 |
|
|
Chang Su
|
837b08eb2e
|
[router][grpc] Support mixin tool calls in Responses API (#12736)
|
2025-11-05 21:29:45 -08:00 |
|
|
Chang Su
|
ffba61a100
|
[router][grpc] Make harmony parser checks recipient first before channel (#12713)
|
2025-11-05 15:33:52 -08:00 |
|
|
Chang Su
|
3c219eb0e4
|
[misc] Change sync-labels to false (#12714)
|
2025-11-05 15:33:33 -08:00 |
|
|
Chang Su
|
83b104ee8c
|
[misc] Add labeler for automatic labeling (#12710)
|
2025-11-05 14:44:43 -08:00 |
|
|
Chang Su
|
4fe53e5888
|
[router][grpc] Support streaming parsing with Tool Choice in chat completions API (#12677)
|
2025-11-05 12:12:21 -08:00 |
|
|
Chang Su
|
9f5e701879
|
[router][grpc] Implement tool_choice support for Responses API (#12668)
|
2025-11-04 21:43:42 -08:00 |
|
|
Chang Su
|
1357ab025a
|
[router][grpc] Emit OutputItemDone event and store output item array (#12656)
|
2025-11-04 17:01:03 -08:00 |
|
|
Chang Su
|
887742a1e7
|
[router][grpc] Fix index issues in reasoning content and missing streaming events (#12650)
|
2025-11-04 15:38:43 -08:00 |
|
|
Chang Su
|
0e82fd3df4
|
[router][grpc] Fix model validation, tool call check, streaming logic and misc in responses (#12616)
|
2025-11-04 02:21:58 -08:00 |
|
|
Chang Su
|
83804bc626
|
[router][grpc] Restructure modules and code clean up (#12598)
|
2025-11-03 23:59:27 -08:00 |
|
|
Chang Su
|
0b41a293fa
|
[router][grpc] Consolidate error messages build in error.rs (#12301)
|
2025-11-03 14:41:45 -08:00 |
|
|
Chang Suandybyang
|
ecb9fa14e6
|
[docker] clean up main dockerfile for router and dev configurations (#12364)
Co-authored-by: ybyang <ybyang7@iflytek.com>
|
2025-10-30 07:46:09 -07:00 |
|
|
Chang Su
|
db15148c8a
|
[router] update router docker to use maturin and build from local (#12350)
|
2025-10-29 13:00:57 -07:00 |
|
|
Chang Su
|
d717e73ea8
|
[router] refactor mcp to use LRU and fix pooling bug (#12346)
|
2025-10-29 12:18:54 -07:00 |
|
|
Chang Su
|
400bddf24c
|
[router] fix router release workflow and add build test in PR (#12315)
|
2025-10-29 08:11:45 -07:00 |
|
|
Chang Su
|
b48354c537
|
[router][grpc] Fix inconsistent behavior of conversation_id not found (#12299)
|
2025-10-28 16:24:43 -07:00 |
|
|
Chang Su
|
a124b517f6
|
[router] Remove SharedXxxStorage type aliases to make Arc explicit (#12171)
|
2025-10-26 14:48:07 -07:00 |
|
|
Chang Su
|
d05a968ba8
|
[router][grpc] Add ResponsesContext and fix error propagation in responses api (#12164)
|
2025-10-26 14:07:19 -07:00 |
|
|
Chang Su
|
94aad0de99
|
[misc][grpc] Remove duplicate log (#12168)
|
2025-10-26 14:06:59 -07:00 |
|
|
Chang Su
|
6dbb569bc3
|
[router][grpc] Fix tool call id in parse_json_schema_response (#12152)
|
2025-10-26 01:40:53 -07:00 |
|
|
Chang Su
|
773d89da8c
|
[router] cleaned up all the redundant comments in the config module (#12147)
|
2025-10-25 23:10:59 -07:00 |
|
|
Chang Su
|
4463e90dd2
|
[router][grpc] Remove gpt_oss parsers and remove _parser suffix in tool parser files (#12091)
|
2025-10-24 15:14:41 -07:00 |
|
|
Chang Su
|
28b8a4064d
|
[router][CI] Clean up imports and prints statements in sgl-router/py_test (#12024)
|
2025-10-23 11:56:57 -07:00 |
|
|
Chang Su
|
6ade6a02d4
|
[grpc] Support gRPC standard health check (#11955)
|
2025-10-22 16:59:09 -07:00 |
|
|
Chang Su
|
590bc4b7a7
|
[router][grpc] Fix background tasks stored with wrong id (#11945)
|
2025-10-21 18:38:51 -07:00 |
|
|
Chang Su
|
70f6309cd4
|
[router][grpc] Support v1/responses API (#11926)
|
2025-10-21 17:41:48 -07:00 |
|
|
Chang Su
|
9c0b1eb5ad
|
[router][grpc] Fix wram-up random token ids for small models (#11887)
|
2025-10-20 19:22:17 -07:00 |
|
|
Chang Su
|
e69094df64
|
[router][grpc] Remove continue_final_message in ChatTemplateParams and add minijinja-contrib (#11882)
|
2025-10-20 18:03:09 -07:00 |
|
|
Chang Su
|
ca240eefb4
|
[router][grpc] Support parallel queue puts in grpc_request_manager and remove mutex for grpc_client (#11798)
|
2025-10-17 20:49:43 -07:00 |
|
|
Chang Su
|
627974405d
|
[Lint] Add python/sglang to ruff F401 checks and remove unused imports in files (#11685)
|
2025-10-17 16:49:46 -07:00 |
|
|
Chang Su
|
d1984e218c
|
[router][grpc] Remove timeout for connections and remove max_tokens deprecation warning log (#11775)
|
2025-10-17 12:36:36 -07:00 |
|
|
Chang Su
|
9e87b60f37
|
[router][CI] Clean up deprecated fields in pr-test-pd-router.yml (#11739)
|
2025-10-16 19:01:00 -07:00 |
|
|
Chang Su
|
dc01313da1
|
[router] Add rustfmt and set group imports by default (#11732)
|
2025-10-16 17:33:29 -07:00 |
|
|
Chang Su
|
c7962868c1
|
[router] Fix tool_choice normalization in ChatCompletionRequest and fix ut (#11731)
|
2025-10-16 14:20:13 -07:00 |
|
|
Chang Su
|
2479b89405
|
[router][grpc] Simplify model_id determination (#11684)
|
2025-10-15 15:56:58 -07:00 |
|
|
Chang Su
|
f226d3da2a
|
Fix missing json imports in serving_responses.py (#11681)
|
2025-10-15 13:01:55 -07:00 |
|
|
Chang SuandXinyuan Tong
|
30ea4c462b
|
[tool call] Fix prev_tool_call_arr management in base_format_detector.py (#11367)
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
|
2025-10-15 09:51:51 -07:00 |
|
|
Chang Su
|
27ef1459e6
|
[router][protocols] Add Axum validate extractor and use it for /v1/chat/completions endpoint (#11588)
|
2025-10-13 22:51:15 -07:00 |
|
|
Chang Su
|
887c2b4575
|
[router][grpc] Add serve_grpc to launch_server and log id for HealthCheck (#11564)
|
2025-10-13 16:07:19 -07:00 |
|
|
Chang Su
|
4b694e7d5a
|
[router][grpc] Add error handling to generate_tool_constraints (#11562)
|
2025-10-13 12:26:09 -07:00 |
|
|
Chang Su
|
7b59b0b8b0
|
[router][grpc] Further delegate non-stream processing to processing.rs (#11553)
|
2025-10-13 10:36:27 -07:00 |
|
|
Chang Su
|
92777135a0
|
[router][grpc] Consolidate parser checks for chat completions (#11439)
|
2025-10-10 20:44:29 -04:00 |
|
|
Chang Su
|
b520958ec8
|
[router][grpc] Replace fake health check with correct ones (#11387)
|
2025-10-09 09:13:57 -07:00 |
|
|
Chang Su
|
ab926dd697
|
[router][grpc] Fix streaming bugs: empty tool names, state pollution, and panics (#11373)
|
2025-10-09 06:53:23 -04:00 |
|
|
Chang Su
|
a0557642ea
|
[router][lint] Add unused_qualifications to cargo lint warnings (#11366)
|
2025-10-08 22:17:11 -07:00 |
|
|
Chang Su
|
fccac7d126
|
[router][grpc] Add dependencies in Cargo.toml to support chat template rendering (#11342)
|
2025-10-08 15:38:37 -07:00 |
|
|
Chang Su
|
a1080b72a0
|
[router] Fix all unused_qualifications (#11341)
|
2025-10-08 13:55:27 -07:00 |
|
|
Chang Su
|
a65ca73911
|
[router][grpc] Cleanup debug logs in grpc_server and grpc_router (#11340)
|
2025-10-08 13:26:19 -07:00 |
|
|
Chang SuandSimo Lin
|
edd86b8853
|
[router][grpc] Refactor chat handler in grpc/ to use centralized orchestrator (#11314)
Co-authored-by: Simo Lin <linsimo.mark@gmail.com>
|
2025-10-07 20:50:20 -07:00 |
|
|
Chang Su
|
7ba3de0e92
|
[oai serving chat] Add argument --sampling-defaults and fix ChatCompletionRequest defaults (#11304)
|
2025-10-08 00:36:05 +00:00 |
|
|
Chang Su
|
f094e0a490
|
[router][grpc] Fix request_id extraction when n > 1 (#11311)
|
2025-10-07 19:27:56 -04:00 |
|
|
Chang Su
|
420c99acfe
|
[router][grpc] Fix error message format in grpc chat handler (#11307)
|
2025-10-07 13:54:02 -07:00 |
|
|
Chang Su
|
6f1e03a456
|
[router][grpc] Fix sampling_params.stop_strs is None (#11306)
|
2025-10-07 10:57:38 -07:00 |
|
|
Chang Su
|
64582caa84
|
[router][grpc] Refactor chat template content format detection (#11288)
|
2025-10-07 08:38:51 -07:00 |
|