[Reasoning + Structured Output] make reasoning compatible with structured output (#12551)

Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
This commit is contained in:
Muqi Li
2025-12-08 01:28:39 -08:00
committed by GitHub
co-authored by Xinyuan Tong
parent f72a77038f
commit 06836ad02a
8 changed files with 72 additions and 33 deletions
@@ -360,6 +360,13 @@
"## Native API and SGLang Runtime (SRT)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"> Note: For native API, as a work-around, you need to set `reasoning` argument to `True` to ensure the model will think before generating the structured output. It's not required for chat-completion API."
]
},
{
"cell_type": "markdown",
"metadata": {},
@@ -407,6 +414,7 @@
" f\"http://localhost:{port}/generate\",\n",
" json={\n",
" \"text\": text,\n",
" \"reasoning\": True,\n",
" \"sampling_params\": {\n",
" \"temperature\": 0,\n",
" \"max_new_tokens\": 2048,\n",
@@ -454,6 +462,7 @@
" f\"http://localhost:{port}/generate\",\n",
" json={\n",
" \"text\": text,\n",
" \"reasoning\": True,\n",
" \"sampling_params\": {\n",
" \"temperature\": 0,\n",
" \"max_new_tokens\": 2048,\n",
@@ -482,6 +491,7 @@
" f\"http://localhost:{port}/generate\",\n",
" json={\n",
" \"text\": \"Give me the information of the capital of France.\",\n",
" \"reasoning\": True,\n",
" \"sampling_params\": {\n",
" \"max_new_tokens\": 2048,\n",
" \"temperature\": 0,\n",
@@ -519,6 +529,7 @@
" f\"http://localhost:{port}/generate\",\n",
" json={\n",
" \"text\": \"Paris is the capital of\",\n",
" \"reasoning\": True,\n",
" \"sampling_params\": {\n",
" \"temperature\": 0,\n",
" \"max_new_tokens\": 2048,\n",
@@ -547,6 +558,7 @@
")\n",
"payload = {\n",
" \"text\": text,\n",
" \"reasoning\": True,\n",
" \"sampling_params\": {\n",
" \"max_new_tokens\": 2048,\n",
" \"structural_tag\": json.dumps(\n",