Structured JSON resetting strict to false

API
10

Hi! Thanks for this. I’m not sure if it is being effective.

We just did some calls using functions and it seems that it isn’t “strictly” returning. I have a schema with 4 top level properties (of type string), and it is now only returning 2.

I started noticing this behaviour at the start of the week, until now it has been extremely good at returning the exact schema (without the ability to set strict). I had found earlier in the week that if I tweak the schema, sometimes it starts returning all data again. I tried this trick again today and it didn’t seem to have an effect.

I thought it might be something to do with the new parsing you have, and turning strict on would fix it, but that seems not to be the case! I could do several calls that ask for 1 thing only I guess, but that would be less than ideal.

Note: I wrote this post a few hours ago and deleted it because I thought that we were putting “strict” in the wrong location, but it turns out that seems to be an issue with the guide vs the API docs - see this forum post here I made asking you to correct the guide

Is any one else noticing this behaviour, or am I using function calls incorrectly? We have been trying a whole range of things just to get our old function calling working again - but will next attempt using the new structured output and see if that has an effect. I would still expect function calls to work though :slight_smile:

Edit:
Just to prove I am not crazy, here is the data of an anonymsed call we are doing to openai:

{
  "request": {
    "call": "client.chat.completions.create",
    "data": {
      "model": "gpt-4o",
      "tools": [
        {
          "type": "function",
          "function": {
            "name": "summary",
            "strict": true,
            "parameters": {
              "type": "object",
              "required": [
                "x1",
                "x2",
                "x3",
                "x4"
              ],
              "properties": {
                "x1": {
                  "type": "string",
                  "description": "aaa"
                },
                "x2": {
                  "type": "string",
                  "description": "bbb"
                },
                "x3": {
                  "type": "string",
                  "description": "ccc"
                },
                "x4": {
                  "type": "string",
                  "description": "dddd"
                }
              },
              "additionalProperties": false
            },
            "description": "foo"
          }
        }
      ],
      "messages": [
        {
          "role": "user",
          "content": "bar"
        }
      ],
      "max_tokens": 4096,
      "temperature": 0.7,
      "tool_choice": "required"
    },
    "vendor": "openai"
  },
  "task_id": 454
}

I get back this response:

{
  "x1": "yyy",
  "x2": "zzz"
}

It does say strict = true so it should in theory send a full json that matches the schema I would have thought!

Update: If I change the model to 4o-mini it seems to work, but 4o and 4-turbo both seem to fail