Realtime API tool calling is NOT working, any idea?

API Bugs
1

The Realtime API function calling used to work pretty good, at rare times, it failed in function calling but once you refreshed the web page to get a new session, the problem was solved, however, this morning , things are different, the tool calling is not working anymore

part of my source code for the backend is here:

        session = openai.beta.realtime.sessions.create(
            input_audio_transcription={"model": "gpt-4o-mini-transcribe"},
            model='gpt-4o-mini-realtime-preview',
            modalities=["text", "audio"],
            output_audio_format="pcm16",
            input_audio_format="pcm16",
            temperature=0.6,
            voice="coral",
            instructions="""Your name XXX, you can help with file upload,
            Do NOT include any additional commentary such as the data you are processing.""",
            tool_choice='auto',
            tools=[
                    {
                        "type": "function",
                        "name": "toggle_loader",
                        "description" : "when users mention to show or hide file uploaders, this tool could be used",
                        "parameters": {
                            "type" : "object",
                            "properties" : {
                                "on_off" : {
                                    "type": "boolean",
                                    "description": "when users ask to show the file loader, on_off should be true, otherwise false"
                                },
                            },
                            "required" : ["on_off"]
                        }
                    },
 ...

after registering this model , the response is clear that the tool was registered correctly

Tool(description='when users mention to show or hide file uploaders, this tool could be used', name='toggle_loader', parameters={'type': 'object', 'properties': {'on_off': {'type': 'boolean', 'description': 'when users ask to show the file loader, on_off should be true, otherwise false'}}, 'required': ['on_off']}, type='function'),

But when I started talking with the agent , asking for file uploader, it never gave correct response, I checked the resonse info, the item.type has no ‘function_call’ at all, which mean the model internally does not realize the necessity to call tools.

Anybody has any idea about what is going on with the function calling in realtime model?

2

I experience the same issue currently. I trippled checked everything and had a working state. There must be something on the api side that changed.

The api does as if there are no tool definitions and even if i explicitely tell the model to execute a tool, it won’t do it.

3

Thanks for your Kalumet! then it’s the model problem instead of our own code, the Realtime model at times go crazy, probably it has taken a lot of garbage … =)

4

it is working all right now for the function calling in realtime model without changing a single line of code …, which means there’s model regression, 6-hour out of work , nightmare for business