Realtime API - tool calling kinda weird?

API
1 
tools=[
    {
        "name": "get_weather",
        "type": "function",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {"type": "string"},
                "unit": {"type": "string", "enum": ["c", "f"]},
            },
            "required": ["location", "unit"],
            "additionalProperties": False,
        },
    }
  ]

This is the only format that works. The various other formats in the Functions Calling docs such as the one below, do not work.

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "strict": True,
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {"type": "string"},
                    "unit": {"type": "string", "enum": ["c", "f"]},
                },
                "required": ["location", "unit"],
                "additionalProperties": False,
            },
        },
    }
]

using the strict tag gives an error, which is the part I don’t love. Not being sure that you’ll get the expected structure is a bit worrisome. Any workarounds? or plans to fix?

2

I don’t know the exact reasoning for them not supporting strict but I can speculate… It’s likely that the version of GPT-4o that realtime is based on, is the older version of GPT-4o that doesn’t have structured outputs support. Currently, Structured Outputs is only supported on a small set of models. The o1 models don’t even support JSON mode, let alone Structured Outputs.

I’m sure Structured Outputs will eventually make its way to the Realtime API but it does require special changes to the model so who knows when that will happen. It’s not like its just a switch they turned on. They had to physically modify the models code to make Structured Outputs work.

2 Likes