It seems like any request for the gpt-4o-audio-preview with functions/tools unvariably fail. If there are no functions/tools present, it works fine.
We make tons of requests with functions/tools of the same format to all other models with no issues.
Here is an example body, it works 100% of the time when the tools field is removed, and 0% of the time when it is present:
{
"model": "gpt-4o-audio-preview",
"messages": [
{
"role": "user",
"content": [
{
"type": "text",
"text": "Can you make out any of the words in the following message?"
},
{
"type": "input_audio",
"input_audio": {
"data": "some valid well tested base64 audio data",
"format": "wav"
}
}
]
}
],
"modalities": [
"text",
"audio"
],
"audio": {
"voice": "alloy",
"format": "wav"
},
"tools": [
{
"type": "function",
"function": {
"name": "getCurrentTemperature",
"description": "Get the current temperature for a specific location",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g., San Francisco, CA"
},
"unit": {
"type": "string",
"enum": [
"Celsius",
"Fahrenheit"
],
"description": "The temperature unit to use. Infer this from the user's location."
}
},
"required": [
"location",
"unit"
]
}
}
}
]
}
The error we get is:
{
"error": {
"message": "The server had an error processing your request. Sorry about that! You can retry your request, or contact us through our help center at help.openai.com if you keep seeing this error. (Please include the request ID req_eb56f4338ab342cfc264f438d774d008 in your email.)",
"type": "server_error",
"param": null,
"code": null
}
}