 Chang Su
|
963175d5c0
|
[router][grpc] Support streaming for v1/chat/completions (#11179)
|
2025-10-02 14:35:16 -07:00 |
|
|
Chang Su
|
3511b37099
|
[proto] Add script to compile python protos (#11171)
|
2025-10-02 08:45:51 -07:00 |
|
|
Chang Su
|
b658be6f6a
|
[router][grpc] Support tool call parser in streaming (#11160)
|
2025-10-02 03:18:50 -07:00 |
|
|
Chang Su
|
8ce830a8b0
|
[router][bugfix] Fix input_logprobs handling with None value and logprob_start_len = -1 (#11113)
|
2025-09-30 16:09:40 -07:00 |
|
|
Chang SuandSimo Lin
|
d1676cd483
|
[router][tool call] Full support for ToolChoice (#11085)
Co-authored-by: Simo Lin <linsimo.mark@gmail.com>
|
2025-09-29 22:36:03 -07:00 |
|
|
Chang Su
|
5937a56d47
|
[router][grpc] Add logprobs support to router (#11082)
|
2025-09-29 15:55:06 -07:00 |
|
|
Chang Su
|
f065e5bea5
|
[router] Use get_pooled in process_single_choice (#11079)
|
2025-09-29 15:48:00 -07:00 |
|
|
Chang Su
|
4eeaff74a0
|
[router][tool call] Separate JsonParser and LlamaParser (#11073)
|
2025-09-29 10:26:37 -07:00 |
|
|
Chang Su
|
af4ab65606
|
[router][tool call] Improve normal content extraction and error handling (non-stream) (#11050)
|
2025-09-29 00:19:30 -07:00 |
|
|
Chang Su
|
dba751a896
|
[router][tool call] Support normal content extraction before tool call (streaming) (#11038)
|
2025-09-28 19:46:06 -07:00 |
|
|
Chang Su
|
c1c8dd1dd0
|
[router][tool parser] Modify tool parser to return both normal text and tool calls (non-stream) (#10995)
|
2025-09-27 18:10:17 -04:00 |
|
|
Chang Su
|
37f3325b06
|
[router][grpc] Support E2E non-stream chat completions (#10980)
|
2025-09-26 22:02:06 -07:00 |
|
|
Chang Su
|
0c3db88978
|
[router][grpc] Add helpfer functions for decoder in router.rs and fix specs (#10971)
|
2025-09-26 20:10:45 -04:00 |
|
|
Chang Su
|
37158f2018
|
router: Support parallel sampling num > 1 in grpc_server and non-stream handling (#10929)
|
2025-09-25 20:03:35 -07:00 |
|
|
Chang Su
|
7dcd689b47
|
[router][refactor] Clean up protobuf fields (#10923)
|
2025-09-25 17:48:47 -07:00 |
|
|
Chang Su
|
5e21d6aec0
|
refactor: Move grpc/client.rs to grpc_client/sglang_scheduler.rs (#10924)
|
2025-09-25 17:21:22 -04:00 |
|
|
Chang Su
|
916784746b
|
router: Fix constraint proto and build_constraint in grpc router (#10881)
|
2025-09-25 11:12:06 -04:00 |
|
|
Chang Su
|
4a87ba217f
|
router-grpc: Add tools processing and other paramters for apply_chat_template (#10877)
|
2025-09-24 15:23:06 -07:00 |
|
|
Chang Su
|
9209b209be
|
router-grpc: Support jinja chat template content format detection (#10832)
|
2025-09-24 11:45:01 -07:00 |
|
|
Chang Su
|
ee704e6265
|
[router] add auth middleware for api key auth (#10826)
|
2025-09-23 16:07:34 -07:00 |
|
|
Chang Su
|
08b8c0c3cd
|
[router] fix axum default body limit (#10818)
|
2025-09-23 12:44:17 -07:00 |
|
|
Chang Su
|
7ff93e613f
|
router(grpc): Implement route for chat_cmpl endpoint (#10761)
|
2025-09-23 11:26:33 -07:00 |
|
|
Chang SuandSimo Lin
|
16adf3dcab
|
[router] fix logger type mismatch (#10774)
Co-authored-by: Simo Lin <linsimo.mark@gmail.com>
|
2025-09-22 21:02:28 -07:00 |
|
|
Chang Su
|
60dbbd086a
|
bugfix: Fix get_worker_urls_for_model in http/router.rs (#10754)
|
2025-09-22 14:10:31 -04:00 |
|
|
Chang Su
|
03ce92e594
|
router-spec: Reorder ChatCompletionRequest and fix validation logic (#10675)
|
2025-09-19 16:41:21 -07:00 |
|
|
Chang Su
|
c1815a99b7
|
model support: Sarashina2VisionForCausalLM (#10632)
|
2025-09-18 17:30:38 -07:00 |
|
|
Chang Su
|
5fe39e85a2
|
[router] fix router manager and router init in server (#10499)
|
2025-09-15 22:23:26 -07:00 |
|
|
Chang Su
|
35ef3f2902
|
[router] fix worker registration in multi model mode (#10486)
|
2025-09-15 21:05:00 -04:00 |
|
|
Chang Su
|
2689f0bf02
|
[router] multi model registration fix (#10481)
|
2025-09-15 15:22:21 -07:00 |
|
|
Chang Su
|
b93acd7020
|
[router] minor code clean up in server startup (#10470)
|
2025-09-15 07:28:25 -07:00 |
|
|
Chang Su
|
69b35793a0
|
[router] fix logger ordering git ctx (#10457)
|
2025-09-14 21:37:21 -07:00 |
|
|
Chang Su
|
53ca15529a
|
Implement Standalone gRPC Server for SGLang Python Scheduler (#10283)
|
2025-09-11 20:57:17 -07:00 |
|
|
Chang Su
|
8b3b995ac9
|
[router] fix release workflow to include protobuf (#10055)
|
2025-09-04 22:09:30 -07:00 |
|
|
Chang Su
|
11dcabc545
|
Grpc client (#9939)
|
2025-09-02 11:47:35 -07:00 |
|
|
Chang Su
|
9a0cac1be0
|
[router] add grpc pd and regular router init (#9893)
|
2025-09-01 20:06:15 -07:00 |
|
|
Chang Su
|
598c0bc19d
|
[router] add tokenizer download support from hf hub (#9882)
|
2025-09-01 10:40:37 -07:00 |
|
|
Chang Su
|
c112bcc461
|
[router] global tool parser registry (#9840)
|
2025-08-30 23:35:39 -07:00 |
|
|
Chang Su
|
fd5ce576a4
|
Tool parser.benchmark (#9835)
|
2025-08-30 21:08:11 -07:00 |
|
|
Chang Su
|
28684f909d
|
[router] upgrade kernel version in pd ci (#9720)
|
2025-08-27 16:02:41 -07:00 |
|
|
Chang Su
|
90313fb09a
|
[router] add token bucket rate limiter (#9656)
|
2025-08-26 10:36:26 -07:00 |
|
|
Chang Su
|
c9dd70fbde
|
tool-call(dsv3): Improve deepseek-v3 chat template and tool_choice = required (#9525)
|
2025-08-23 01:46:56 -07:00 |
|
|
Chang Su
|
816c4c8572
|
[router] add tool parser base structure and partial json parser (#9482)
|
2025-08-21 22:08:56 -07:00 |
|
|
Chang Su
|
53e2cd464f
|
[router] remove all tokenizer metrics for performance (#9474)
|
2025-08-21 18:35:24 -07:00 |
|
|
Chang Su
|
5cfbb4c136
|
[router] add glm and step3 reasoning parser (#9415)
|
2025-08-20 18:33:10 -07:00 |
|
|
Chang Su
|
e65231022f
|
[router] add tokenizer integration test with real mini tokenizer (#9413)
|
2025-08-20 17:56:23 -07:00 |
|
|
Chang Su
|
7638f5e44e
|
[router] Implement gRPC SGLangSchedulerClient (#9364)
|
2025-08-19 16:44:11 -07:00 |
|
|
Chang Su
|
46fe8b8cb2
|
[CI] Fix lint issues (#9361)
|
2025-08-19 13:05:36 -07:00 |
|
|
Chang Su
|
439df4548a
|
[router] Add spec for sglang scheduler (#9322)
|
2025-08-18 17:20:20 -07:00 |
|
|
Chang Su
|
ad359d1c71
|
router: Fix user guide link README.md (#9122)
|
2025-08-12 12:29:10 -07:00 |
|
|
Chang Su
|
f2a5de284b
|
[Bugfix] Fix accuracy-test-1-gpu failure caused by builtin_tools (#9114)
|
2025-08-12 09:56:13 -07:00 |
|
|
Chang Su
|
9c83d74da3
|
bugfix: Fix the commentary msg extraction in GptOssDetector (#9097)
|
2025-08-11 23:53:10 -07:00 |
|
|
Chang Su
|
a218490136
|
(gpt-oss, oai, chat): Remove Harmony Integration and Implement Native GPT-OSS Tool Call Support (#9043)
|
2025-08-11 18:59:18 -07:00 |
|
|
Chang Su
|
a6452b7188
|
bugfix: Fix output_ids extraction in detokenizer_manager (#9047)
|
2025-08-11 03:17:32 -07:00 |
|
|
|
92cc32d9fc
|
Support v1/responses and use harmony in serving_chat (#8837)
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
|
2025-08-06 16:20:34 -07:00 |
|
|
Chang Su
|
743638bc03
|
misc: Remove debug print to logger.info (#8633)
|
2025-07-31 12:56:52 -07:00 |
|
|
|
51c38163c1
|
model: support Step3V (#8583)
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: nnnobody-code <nnnobody@foxmail.com>
Co-authored-by: ispobock <ispobaoke@gmail.com>
Co-authored-by: Qiaolin-Yu <qy254@cornell.edu>
Co-authored-by: Qiaolin-Yu <liin1211@outlook.com>
Co-authored-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
|
2025-07-31 02:41:00 -07:00 |
|
|
Chang Su
|
a79a5d7012
|
Revert "Fix the input tools format and history tool_calls in OpenAI API (#6556)" (#8584)
|
2025-07-30 13:12:05 -07:00 |
|
|
Chang Su
|
dd487e5553
|
bugfix: Fix XGrammar backend to use model's EOS tokens for constrained generation (#8422)
|
2025-07-28 10:01:02 +08:00 |
|
|
Chang Su
|
58dd95fbc8
|
Fix test_openai_server (#8419)
|
2025-07-27 13:36:01 -07:00 |
|
|
Chang Su
|
b47eda3316
|
bugfix: Fix multiple finish_reason chunks and tool_calls finish reason check (#8417)
|
2025-07-27 13:31:06 -07:00 |
|
|
Chang Su
|
d8ee15643b
|
[Feat] Add reasoning parser for Qwen/Qwen3-235B-A22B-Thinking-2507 (#8363)
|
2025-07-25 14:59:42 -07:00 |
|
|
Chang Su
|
f8260f2539
|
[Bugfix][Feat] Add XML-ish grammar in EBNFComposer and fix misc bugs in Qwen3 detector (#8357)
|
2025-07-25 12:03:16 -07:00 |
|
|
Chang Su
|
01079e174f
|
feat(function call): complete utility method for KimiK2Detector and enhance documentation (#8043)
|
2025-07-23 17:37:31 -07:00 |
|
|
Chang Su
|
e6d5988442
|
Update CODEOWNERS (#8044)
|
2025-07-15 14:38:15 +08:00 |
|
|
Chang Su
|
112b496a6c
|
misc: Improvement to serving_chat.py and add more ut (#7489)
|
2025-06-24 17:19:51 -07:00 |
|
|
Chang Su
|
3562256bb2
|
fix: Add --model as an alias for --model-path in server_args (#7505)
|
2025-06-24 12:08:08 -07:00 |
|
|
Chang Su
|
d04163b3fa
|
Fix RequestValidationError response format (#7487)
|
2025-06-23 20:35:11 -07:00 |
|
|
Chang Su
|
fa42e41962
|
ci: Revert openai_server related tests in AMD suites (#7449)
|
2025-06-23 15:28:22 -07:00 |
|
|
Chang Su
|
34b6b8426f
|
feat(func_call): Add more check in BaseFormatDetector.parse_streaming_increment (#7479)
|
2025-06-23 11:15:47 -07:00 |
|
|
Chang Su
|
b7a2df0a44
|
refactor(test): reorganize OpenAI test file structure (#7408)
|
2025-06-21 19:37:48 -07:00 |
|
|
Chang SuandJin Pan
|
72676cd6c0
|
feat(oai refactor): Replace openai_api with entrypoints/openai (#7351)
Co-authored-by: Jin Pan <jpan236@wisc.edu>
|
2025-06-21 13:21:06 -07:00 |
|
|
Chang Su
|
fdfd5224bf
|
fix: Fix CI test_function_call_parser.py (#7425)
|
2025-06-22 00:25:08 +08:00 |
|
|
Chang Su
|
d20a073bc3
|
Put _normalize_rid before other normalization in io_struct (#7363)
|
2025-06-19 11:21:25 -07:00 |
|
|
Chang Su
|
fc554105f6
|
ci: Fix test_ebnf_generate_all_optional_function_params (#7288)
|
2025-06-17 16:39:42 -07:00 |
|
|
Chang Su
|
e726131523
|
bugfix(tool call ebnf): Fix EBNF generation for optional function parameters (#7283)
|
2025-06-17 13:36:07 -07:00 |
|
|
Chang Su
|
8b2474898b
|
bugfix(OAI): Fix image_data processing for jinja chat templates (#6877)
|
2025-06-05 13:37:01 -07:00 |
|
|
Chang Su
|
e39bca0756
|
ci: relax test_function_call_required (#6786)
|
2025-05-30 19:18:42 -07:00 |
|
|
Chang Su
|
f18b068f15
|
feat(tool call): Enhance Llama32Detector for improved JSON parsing in non-stream (#6784)
|
2025-05-30 17:05:17 -07:00 |
|
|
Chang Su
|
c673727e0e
|
refactor(tool call): Fix BaseFormatDetector tool_index issue and refactor parse_streaming_increment (#6715)
|
2025-05-29 00:08:45 -07:00 |
|
|
Chang Su
|
461a730280
|
fix(deepseekv3): Fix DeepSeekV3Detector tool_index assignment and multi-tool call streaming support (#6655)
|
2025-05-28 00:22:53 -07:00 |
|
|
Chang Su
|
41ba767f0c
|
feat: Add warnings for invalid tool_choice and UTs (#6582)
|
2025-05-27 16:53:19 -07:00 |
|
|
Chang Su
|
bdb962d755
|
fix(tool call): Fix tool_index in PythonicDetector and issues with mixed output in non-streaming (#6678)
|
2025-05-27 16:18:42 -07:00 |
|
|
Chang Su
|
ae33584235
|
[Bugfix]: Fix call for function_call_parser.multi_format_detector in adapter.py (#6650)
|
2025-05-26 21:57:10 -07:00 |
|
|
Chang Su
|
16f69b1f65
|
feat: Improve Mistral and Qwen25 function call parsing (#6597)
|
2025-05-25 23:07:23 -07:00 |
|
|
Chang Su
|
681e7af32b
|
[OAI] Support non-normalized logprobs in OpenAI server (#5961)
|
2025-05-24 21:35:55 -07:00 |
|
|
Chang Su
|
ed0c3035cd
|
feat(Tool Calling): Support required and specific function mode (#6550)
|
2025-05-23 21:00:37 -07:00 |
|
|
Chang Su
|
7b02c32679
|
[Bugfix](gemma3_mm): handle flatten_batch constraint for multiple images (#6562)
|
2025-05-23 18:11:54 -07:00 |
|
|
Chang Suandyizhang2077
|
4685fbb888
|
[VLM] Support chunk prefill for VLM (#6355)
Co-authored-by: yizhang2077 <1109276519@qq.com>
|
2025-05-22 20:32:41 -07:00 |
|
|
Chang Su
|
1be6956d1b
|
[Bugfix] Fix field error in v1_embedding_request (#6400)
|
2025-05-18 15:58:29 -07:00 |
|
|
Chang Su
|
ebe58d545d
|
[Misc] Implement RankZeroFilter for rank-specific logging in model_runner.py (#6333)
|
2025-05-18 15:27:13 -07:00 |
|
|
Chang Su
|
066cf44546
|
[OAI] Add rid tracing for v1/embeddings and fix rid type in Chat (#6397)
|
2025-05-18 13:05:38 -07:00 |
|
|
Chang Su
|
205d5cb407
|
perf: Optimize local attention memory allocation in FlashAttentionBackend (#6356)
|
2025-05-17 01:45:46 -07:00 |
|
|
Chang Su
|
912788c095
|
perf: optimize local_block_table memory allocation (#6273)
|
2025-05-13 17:18:38 -07:00 |
|
|
Chang Su
|
1940cdec61
|
[Bugfix] Fix Llama4 gibberish output with long context and CUDA graph (#6162)
|
2025-05-09 15:33:02 -07:00 |
|
|
Chang Su
|
170d1f218a
|
feat: Refactor DeepSeekV3 function call (#5908)
|
2025-05-01 21:28:57 -07:00 |
|
|
Chang Su
|
28b26dbf48
|
[Bugfix]: fix missing queue_time_start for requests from grammar_queue (#5696)
|
2025-04-29 17:31:44 -07:00 |
|
|
Chang Su
|
2b06484bd1
|
feat: support pythonic tool call and index in tool call streaming (#5725)
|
2025-04-29 17:30:44 -07:00 |
|
|
Chang Su
|
9419e75d60
|
[CI] Add test_function_calling.py to run_suite.py (#5896)
|
2025-04-29 15:54:53 -07:00 |
|
|
Chang Su
|
05ee219286
|
Support max_completion_tokens for OpenAIChatCompletions (#5857)
|
2025-04-28 13:50:13 -07:00 |
|
|
Chang Su
|
c776234b45
|
Enable local attention during decode (#5479)
|
2025-04-17 02:07:43 -07:00 |
|