minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

Add native Exa-backed web_search support (#29342)

Browse Source 
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
This commit is contained in:
jonah-berman
2026-06-27 14:41:09 +01:00
committed by GitHub
co-authored by Xinyuan Tong
parent 3306233961
commit 2f34dbe372
10 changed files with 737 additions and 51 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs_new/cookbook/autoregressive/OpenAI/GPT-OSS.mdx
+24 -23
View File
@@ -41,12 +41,12 @@ import { GPTOSSDeployment } from "/src/snippets/autoregressive/gpt-oss-deploymen
### 3.2 Configuration Tips
- **Demo tool server:** Launch with `--tool-server demo` to enable the built-in web-search (Exa) and Python interpreter tools.
- **Web search tool:** Requires an Exa API key — set `EXA_API_KEY` in your environment.
- **Python tool:** Runs in a Docker sandbox by default. Set `PYTHON_EXECUTION_BACKEND=UV` to run on the host (model-generated code executes locally — use with care).
- **Native web search:** Set `EXA_API_KEY` in the SGLang server environment to enable built-in web search (Exa). No `--tool-server` is required, and requests are tagged with `x-exa-integration: sglang`.
- **Web search defaults:** `numResults=10`, search `type="auto"`, and `contents.highlights=true`. Override with `SGLANG_EXA_NUM_RESULTS`, `SGLANG_EXA_SEARCH_TYPE`, and `SGLANG_EXA_INCLUDE_HIGHLIGHTS`.
- **Python tool:** Add `--tool-server demo` to enable the Python interpreter. Runs in a Docker sandbox by default; set `PYTHON_EXECUTION_BACKEND=UV` to run on the host (model-generated code executes locally — use with care).
- **MCP tool servers:** For production, point SGLang at external MCP SSE servers with `--tool-server ip-1:port-1,ip-2:port-2`.
- **Responses API:** GPT-OSS supports OpenAI's Responses API (`client.responses.create`) in addition to the standard Chat Completions API (see section 4.2.3).
- **Use Python 3.12** when running the demo tools.
- **Responses API:** GPT-OSS supports OpenAI's Responses API (`client.responses.create`) in addition to the standard Chat Completions API (see section 4.2.4).
- **Use Python 3.12** when running the demo Python tool.
## 4.Model Invocation
@@ -433,16 +433,19 @@ The spec-v2 overlap scheduler is enabled by default. It improves performance by
#### 4.2.4 Responses API and Built-in Tools
GPT-OSS supports the OpenAI Responses API with built-in tool use (web search and Python interpreter). Set up your environment and launch with `--tool-server demo`:
GPT-OSS supports the OpenAI Responses API with built-in tool use (web search and Python interpreter). Set `EXA_API_KEY` to enable native web search; add `--tool-server demo` only when you also want the Python tool:
```shell Command
export EXA_API_KEY=YOUR_EXA_KEY
# Optional: server-side Exa tuning (defaults shown)
export SGLANG_EXA_NUM_RESULTS=10
export SGLANG_EXA_SEARCH_TYPE=auto
export SGLANG_EXA_INCLUDE_HIGHLIGHTS=true
# Optional: run Python tool on host instead of Docker (model code executes locally)
export PYTHON_EXECUTION_BACKEND=UV
python3 -m sglang.launch_server \
--model-path openai/gpt-oss-120b \
--tool-server demo \
--tp 2
```
@@ -464,10 +467,8 @@ from openai import OpenAI
client = OpenAI(base_url="http://localhost:30000/v1", api_key="sk-123456")
tools = [
{"type": "code_interpreter"},
{"type": "web_search_preview"},
]
search_tools = [{"type": "web_search"}]
python_tools = [{"type": "code_interpreter"}]
# Configurable reasoning effort: "high", "medium", or "low"
response = client.responses.create(
@@ -478,24 +479,24 @@ response = client.responses.create(
)
print(response.output_text)
# Python tool usage
# Web search (requires EXA_API_KEY on the SGLang server)
response = client.responses.create(
model="openai/gpt-oss-120b",
instructions="You are a helpful assistant.",
instructions="You are a helpful assistant, you can search the web when needed.",
input="Search the web for the latest news about Nvidia stock price",
tools=search_tools,
)
print(response.output_text)
# Python tool (requires launching SGLang with --tool-server demo)
response = client.responses.create(
model="openai/gpt-oss-120b",
instructions="You are a helpful assistant, you could use python tool to execute code.",
input="Use python tool to calculate the sum of 29138749187 and 29138749187",
tools=tools,
tools=python_tools,
)
print(response.output_text)
# Output: The sum is 58,277,498,374.
# Web search usage
response = client.responses.create(
model="openai/gpt-oss-120b",
instructions="You are a helpful assistant.",
input="Search the web for the latest news about Nvidia stock price",
tools=tools,
)
print(response.output_text)
```
## 5.Benchmark
docs_new/docs/advanced_features/server_arguments.mdx
+1 -1
View File
@@ -1096,7 +1096,7 @@ Please consult the documentation below and [server_args.py](https://github.com/s
</tr>
<tr>
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>`--tool-server`</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Either 'demo' or a comma-separated list of tool server urls to use for the model. If not specified, no tool server will be used.</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Either 'demo' or a comma-separated list of tool server urls to use for the model. If not specified, no external tool server will be used. Native GPT-OSS `web_search` can still be enabled with `EXA_API_KEY`.</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>`None`</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Type: str</td>
</tr>