minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

feat(openai): Accept the input_audio content part in chat completions (#33606)

Browse Source
This commit is contained in:
Jason Wiemels
2026-08-19 13:37:50 -07:00
committed by GitHub
parent 746418a1ec
commit defb2a3100
6 changed files with 150 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/cookbook/autoregressive/Google/Gemma4.mdx
+1 -1
View File
@@ -531,7 +531,7 @@ Tool Call: get_weather
### 4.5 Audio Input
The audio-capable Gemma 4 variants (`gemma-4-E2B-it`, `gemma-4-E4B-it`, `gemma-4-12B-it`) accept raw audio alongside text. Pass the waveform as a base64 `audio_url` data URI (16 kHz mono WAV works well):
The audio-capable Gemma 4 variants (`gemma-4-E2B-it`, `gemma-4-E4B-it`, `gemma-4-12B-it`) accept raw audio alongside text. Pass the waveform as a base64 `audio_url` data URI (16 kHz mono WAV works well), or as OpenAI's `input_audio` part with the base64 bytes in `data` and a `format` of `wav`:
```python Example
import base64
docs/cookbook/autoregressive/ThinkingMachines/Inkling-Small.mdx
+1 -1
View File
@@ -216,7 +216,7 @@ Tool calls: [ChatCompletionMessageFunctionToolCall(id='call_98f772f3a0044f45b80c
### 3.3 Multimodal Input (Image + Audio)
Inkling-Small is multimodal: a single user message can mix **text**, **images**, and **audio**. Pass each media item as its own content part — `image_url` for images, `audio_url` for audio — with the `url` set to either an HTTP(S) link or a base64 `data:` URI. The server must be started with `--enable-multimodal` (already included in every recipe above).
Inkling-Small is multimodal: a single user message can mix **text**, **images**, and **audio**. Pass each media item as its own content part — `image_url` for images, `audio_url` for audio — with the `url` set to either an HTTP(S) link or a base64 `data:` URI. Audio can also be sent as OpenAI's `input_audio` part, carrying the base64 bytes in `data` alongside a `format` of `wav` or `mp3`. The server must be started with `--enable-multimodal` (already included in every recipe above).
<Accordion title="Image + Audio Example (Python)">
docs/cookbook/autoregressive/ThinkingMachines/Inkling.mdx
+1 -1
View File
@@ -213,7 +213,7 @@ Tool calls: [ChatCompletionMessageFunctionToolCall(id='call_98f772f3a0044f45b80c
### 3.3 Multimodal Input (Image + Audio)
Inkling is multimodal: a single user message can mix **text**, **images**, and **audio**. Pass each media item as its own content part — `image_url` for images, `audio_url` for audio — with the `url` set to either an HTTP(S) link or a base64 `data:` URI. The server must be started with `--enable-multimodal` (already included in every recipe above).
Inkling is multimodal: a single user message can mix **text**, **images**, and **audio**. Pass each media item as its own content part — `image_url` for images, `audio_url` for audio — with the `url` set to either an HTTP(S) link or a base64 `data:` URI. Audio can also be sent as OpenAI's `input_audio` part, carrying the base64 bytes in `data` alongside a `format` of `wav` or `mp3`. The server must be started with `--enable-multimodal` (already included in every recipe above).
<Accordion title="Image + Audio Example (Python)">