 Chang Su
|
e65231022f
|
[router] add tokenizer integration test with real mini tokenizer (#9413)
|
2025-08-20 17:56:23 -07:00 |
|
|
Chang Su
|
7638f5e44e
|
[router] Implement gRPC SGLangSchedulerClient (#9364)
|
2025-08-19 16:44:11 -07:00 |
|
|
Chang Su
|
46fe8b8cb2
|
[CI] Fix lint issues (#9361)
|
2025-08-19 13:05:36 -07:00 |
|
|
Chang Su
|
439df4548a
|
[router] Add spec for sglang scheduler (#9322)
|
2025-08-18 17:20:20 -07:00 |
|
|
Chang Su
|
ad359d1c71
|
router: Fix user guide link README.md (#9122)
|
2025-08-12 12:29:10 -07:00 |
|
|
Chang Su
|
f2a5de284b
|
[Bugfix] Fix accuracy-test-1-gpu failure caused by builtin_tools (#9114)
|
2025-08-12 09:56:13 -07:00 |
|
|
Chang Su
|
9c83d74da3
|
bugfix: Fix the commentary msg extraction in GptOssDetector (#9097)
|
2025-08-11 23:53:10 -07:00 |
|
|
Chang Su
|
a218490136
|
(gpt-oss, oai, chat): Remove Harmony Integration and Implement Native GPT-OSS Tool Call Support (#9043)
|
2025-08-11 18:59:18 -07:00 |
|
|
Chang Su
|
a6452b7188
|
bugfix: Fix output_ids extraction in detokenizer_manager (#9047)
|
2025-08-11 03:17:32 -07:00 |
|
|
|
92cc32d9fc
|
Support v1/responses and use harmony in serving_chat (#8837)
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
|
2025-08-06 16:20:34 -07:00 |
|
|
Chang Su
|
743638bc03
|
misc: Remove debug print to logger.info (#8633)
|
2025-07-31 12:56:52 -07:00 |
|
|
|
51c38163c1
|
model: support Step3V (#8583)
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: nnnobody-code <nnnobody@foxmail.com>
Co-authored-by: ispobock <ispobaoke@gmail.com>
Co-authored-by: Qiaolin-Yu <qy254@cornell.edu>
Co-authored-by: Qiaolin-Yu <liin1211@outlook.com>
Co-authored-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
|
2025-07-31 02:41:00 -07:00 |
|
|
Chang Su
|
a79a5d7012
|
Revert "Fix the input tools format and history tool_calls in OpenAI API (#6556)" (#8584)
|
2025-07-30 13:12:05 -07:00 |
|
|
Chang Su
|
dd487e5553
|
bugfix: Fix XGrammar backend to use model's EOS tokens for constrained generation (#8422)
|
2025-07-28 10:01:02 +08:00 |
|
|
Chang Su
|
58dd95fbc8
|
Fix test_openai_server (#8419)
|
2025-07-27 13:36:01 -07:00 |
|
|
Chang Su
|
b47eda3316
|
bugfix: Fix multiple finish_reason chunks and tool_calls finish reason check (#8417)
|
2025-07-27 13:31:06 -07:00 |
|
|
Chang Su
|
d8ee15643b
|
[Feat] Add reasoning parser for Qwen/Qwen3-235B-A22B-Thinking-2507 (#8363)
|
2025-07-25 14:59:42 -07:00 |
|
|
Chang Su
|
f8260f2539
|
[Bugfix][Feat] Add XML-ish grammar in EBNFComposer and fix misc bugs in Qwen3 detector (#8357)
|
2025-07-25 12:03:16 -07:00 |
|
|
Chang Su
|
01079e174f
|
feat(function call): complete utility method for KimiK2Detector and enhance documentation (#8043)
|
2025-07-23 17:37:31 -07:00 |
|
|
Chang Su
|
e6d5988442
|
Update CODEOWNERS (#8044)
|
2025-07-15 14:38:15 +08:00 |
|
|
Chang Su
|
112b496a6c
|
misc: Improvement to serving_chat.py and add more ut (#7489)
|
2025-06-24 17:19:51 -07:00 |
|
|
Chang Su
|
3562256bb2
|
fix: Add --model as an alias for --model-path in server_args (#7505)
|
2025-06-24 12:08:08 -07:00 |
|
|
Chang Su
|
d04163b3fa
|
Fix RequestValidationError response format (#7487)
|
2025-06-23 20:35:11 -07:00 |
|
|
Chang Su
|
fa42e41962
|
ci: Revert openai_server related tests in AMD suites (#7449)
|
2025-06-23 15:28:22 -07:00 |
|
|
Chang Su
|
34b6b8426f
|
feat(func_call): Add more check in BaseFormatDetector.parse_streaming_increment (#7479)
|
2025-06-23 11:15:47 -07:00 |
|
|
Chang Su
|
b7a2df0a44
|
refactor(test): reorganize OpenAI test file structure (#7408)
|
2025-06-21 19:37:48 -07:00 |
|
|
Chang SuandJin Pan
|
72676cd6c0
|
feat(oai refactor): Replace openai_api with entrypoints/openai (#7351)
Co-authored-by: Jin Pan <jpan236@wisc.edu>
|
2025-06-21 13:21:06 -07:00 |
|
|
Chang Su
|
fdfd5224bf
|
fix: Fix CI test_function_call_parser.py (#7425)
|
2025-06-22 00:25:08 +08:00 |
|
|
Chang Su
|
d20a073bc3
|
Put _normalize_rid before other normalization in io_struct (#7363)
|
2025-06-19 11:21:25 -07:00 |
|
|
Chang Su
|
fc554105f6
|
ci: Fix test_ebnf_generate_all_optional_function_params (#7288)
|
2025-06-17 16:39:42 -07:00 |
|
|
Chang Su
|
e726131523
|
bugfix(tool call ebnf): Fix EBNF generation for optional function parameters (#7283)
|
2025-06-17 13:36:07 -07:00 |
|
|
Chang Su
|
8b2474898b
|
bugfix(OAI): Fix image_data processing for jinja chat templates (#6877)
|
2025-06-05 13:37:01 -07:00 |
|
|
Chang Su
|
e39bca0756
|
ci: relax test_function_call_required (#6786)
|
2025-05-30 19:18:42 -07:00 |
|
|
Chang Su
|
f18b068f15
|
feat(tool call): Enhance Llama32Detector for improved JSON parsing in non-stream (#6784)
|
2025-05-30 17:05:17 -07:00 |
|
|
Chang Su
|
c673727e0e
|
refactor(tool call): Fix BaseFormatDetector tool_index issue and refactor parse_streaming_increment (#6715)
|
2025-05-29 00:08:45 -07:00 |
|
|
Chang Su
|
461a730280
|
fix(deepseekv3): Fix DeepSeekV3Detector tool_index assignment and multi-tool call streaming support (#6655)
|
2025-05-28 00:22:53 -07:00 |
|
|
Chang Su
|
41ba767f0c
|
feat: Add warnings for invalid tool_choice and UTs (#6582)
|
2025-05-27 16:53:19 -07:00 |
|
|
Chang Su
|
bdb962d755
|
fix(tool call): Fix tool_index in PythonicDetector and issues with mixed output in non-streaming (#6678)
|
2025-05-27 16:18:42 -07:00 |
|
|
Chang Su
|
ae33584235
|
[Bugfix]: Fix call for function_call_parser.multi_format_detector in adapter.py (#6650)
|
2025-05-26 21:57:10 -07:00 |
|
|
Chang Su
|
16f69b1f65
|
feat: Improve Mistral and Qwen25 function call parsing (#6597)
|
2025-05-25 23:07:23 -07:00 |
|
|
Chang Su
|
681e7af32b
|
[OAI] Support non-normalized logprobs in OpenAI server (#5961)
|
2025-05-24 21:35:55 -07:00 |
|
|
Chang Su
|
ed0c3035cd
|
feat(Tool Calling): Support required and specific function mode (#6550)
|
2025-05-23 21:00:37 -07:00 |
|
|
Chang Su
|
7b02c32679
|
[Bugfix](gemma3_mm): handle flatten_batch constraint for multiple images (#6562)
|
2025-05-23 18:11:54 -07:00 |
|
|
Chang Suandyizhang2077
|
4685fbb888
|
[VLM] Support chunk prefill for VLM (#6355)
Co-authored-by: yizhang2077 <1109276519@qq.com>
|
2025-05-22 20:32:41 -07:00 |
|
|
Chang Su
|
1be6956d1b
|
[Bugfix] Fix field error in v1_embedding_request (#6400)
|
2025-05-18 15:58:29 -07:00 |
|
|
Chang Su
|
ebe58d545d
|
[Misc] Implement RankZeroFilter for rank-specific logging in model_runner.py (#6333)
|
2025-05-18 15:27:13 -07:00 |
|
|
Chang Su
|
066cf44546
|
[OAI] Add rid tracing for v1/embeddings and fix rid type in Chat (#6397)
|
2025-05-18 13:05:38 -07:00 |
|
|
Chang Su
|
205d5cb407
|
perf: Optimize local attention memory allocation in FlashAttentionBackend (#6356)
|
2025-05-17 01:45:46 -07:00 |
|
|
Chang Su
|
912788c095
|
perf: optimize local_block_table memory allocation (#6273)
|
2025-05-13 17:18:38 -07:00 |
|
|
Chang Su
|
1940cdec61
|
[Bugfix] Fix Llama4 gibberish output with long context and CUDA graph (#6162)
|
2025-05-09 15:33:02 -07:00 |
|
|
Chang Su
|
170d1f218a
|
feat: Refactor DeepSeekV3 function call (#5908)
|
2025-05-01 21:28:57 -07:00 |
|
|
Chang Su
|
28b26dbf48
|
[Bugfix]: fix missing queue_time_start for requests from grammar_queue (#5696)
|
2025-04-29 17:31:44 -07:00 |
|
|
Chang Su
|
2b06484bd1
|
feat: support pythonic tool call and index in tool call streaming (#5725)
|
2025-04-29 17:30:44 -07:00 |
|
|
Chang Su
|
9419e75d60
|
[CI] Add test_function_calling.py to run_suite.py (#5896)
|
2025-04-29 15:54:53 -07:00 |
|
|
Chang Su
|
05ee219286
|
Support max_completion_tokens for OpenAIChatCompletions (#5857)
|
2025-04-28 13:50:13 -07:00 |
|
|
Chang Su
|
c776234b45
|
Enable local attention during decode (#5479)
|
2025-04-17 02:07:43 -07:00 |
|
|
Chang Su
|
27a009bb00
|
Fix ignore_eos parameter when loading a chat template (#5264)
|
2025-04-15 17:09:45 -07:00 |
|
|
Chang Suandch-wan
|
aee62d744b
|
Optimize GPU memory usage in FlashAttentionBackend's strided indexing (#5262)
Co-authored-by: ch-wan <cwan39@gatech.edu>
|
2025-04-11 00:34:17 -07:00 |
|
|
Chang Su
|
aac531c53b
|
[Bugfix] Fix index out of bounds in local attention with large sequences (#5173)
|
2025-04-08 18:43:13 -07:00 |
|
|
|
f04c80dc42
|
Add Llama4 support (#5092)
Co-authored-by: Cheng Wan <cwan39@gatech.edu>
Co-authored-by: fzyzcjy <ch271828n@outlook.com>
Co-authored-by: ispobock <ispobaoke@163.com>
|
2025-04-07 00:29:36 -07:00 |
|
|
Chang Su
|
5fe79605a8
|
Fix Llama3.3 tool call support (#4320)
|
2025-03-13 14:01:41 -07:00 |
|
|
Chang Suandvoidxb
|
eec3f6d1eb
|
[Bugfix] Fix tokenizer_manager not getting 400 when req is too long (#3678)
Co-authored-by: voidxb <unkown>
|
2025-02-27 22:59:43 -08:00 |
|
|
Chang Su
|
4d4cdb3fe7
|
Frontend: better error message handling for FINISH_ABORT in scheduler.py (#2956)
|
2025-01-18 19:37:30 -08:00 |
|
|
Chang Su
|
a8ccacc8b8
|
[Frontend] Fix request length check and add option to disallow auto truncation in scheduler (#2876)
|
2025-01-16 14:51:19 -08:00 |
|