Function JSON Schema is still ignored by GPT-4, 4o, and 4o-mini when calling tools

When calling tools, all models of GPT appear to only understand top-level property names and primitive typing. Take this modestly complex function schema:

{
   "type": "function",
   "function": {
      "name": "tool_save_recipe",
      "description": "Saves a recipe for later recall. Ensure the user provides all required properties, and prompt when any are missing. Pay special attention to the 'ingredients' property, which the user will provide as a series of terms such as '200g apple, 10g ginger' or '6oz of banana and 2oz of blueberry'.",
      "parameters": {
         "type": "object",
         "properties": {
            "name": {
               "type": "string",
               "description": "The name of the recipe. e.g. Mean Green Machine"
            },
            "author": {
               "type": "string",
               "description": "The creator of the recipe. e.g. Joseph Gold"
            },
            "sku": {
               "type": "string",
               "description": "The sku of the recipe. e.g. GJN001"
            },
            "ingredients": {
               "type": "object",
               "additionalProperties": {
                  "type": "string"
               },
               "minProperties": 1,
               "description": "The ingredients that are used in the recipe. Each key:val pair in this dictionary uses the ingredient name as the key and the amount as the value. e.g. {apple: 200g, ginger: 10g}"
            },
            "yield_amount": {
               "type": "string",
               "description": "The amount that the user wants to make. e.g. 8oz"
            },
            "type": {
               "type": "string",
               "enum": ["food", "beverage", "juice"],
               "description": "The type of food that a recipe makes. e.g. juice"
            },
            "instructions": {
               "type": "string",
               "description": "Optional instructions on how to make the recipe. Can be many sentences. e.g. Put all ingredients in a bowl and mix well."
            },
            "description": {
               "type": "string",
               "description": "A user-facing description of the recipe. e.g. A delicious citrus juice high in vitamin C and antioxidants."
            }
         },
         "required": [
            "name",
            "ingredients",
            "yield_amount",
            "type"
         ]
      }
   }
}

A prompt such as:
"Save a new recipe. It's called Booster Joe. 325g apple, 360g carrot, 40g lemon, 5g ginger. It's a juice that makes 12oz."

Will often result in a message instead of a tool call:

I can save that recipe for you! Could you please provide me with the name of the author and any specific instructions on how to make the recipe? Additionally, do you have a description for "Booster Joe"? 

It is ignoring the “required” part of the schema entirely, and is instead prompting for optional parameters. Rarely, it will generate a tool call from a prompt like this, but the ingredients will be formatted incorrectly (as a string, array of strings, etc), or not included at all.

What it DOES seem to pay attention to is the function description. Changing the description to the following seems to get it to work correctly:

"Saves a recipe for later recall. Ensure the user provides all required properties (name, ingredients, yield_amount, type), and prompt when any of these are missing. Optional properties (author, instructions, description) can be provided if available, but the function should proceed with the given information even if they are not provided. Format ingredients as a dictionary in which the ingredient name is the key and the amount is the value. For example: {apple:200g, carrot 150g}"

Resulting in the expected function arguments:

{'name': 'Booster Joe', 'yield_amount': '12oz', 'type': 'juice', 'ingredients': {'apple': '325g', 'carrot': '360g', 'lemon': '40g', 'ginger': '5g'}} 

This is a serious bug. RAG is one of the primary uses of GPT for real-world applications, and as it stands there is no way to enforce function schema at the level of the assistant. It also fails to understand non-primitive data types, and will only follow them with a detailed description of expected output.

Try to change the schema of ingredients to something like this:

"ingredients": {
        "type": "array",
        "items": {
          "type": "object",
          "properties": {
            "name": {
              "type": "string",
              "description": "Name of ingredient"
            },
            "amount": {
              "type": "string",
              "description": "Amount of the ingredient"
            }
          }
        },
        "description": "List of ingredients."
      },

https://openai.com/index/introducing-structured-outputs-in-the-api/

Consider trying the new gpt-4o-2024-08-06 model, it should guarantee to exactly match JSON Schemas provided by developers

1 Like

Heh, wow. Talk about good timing.

A quick test shows that this also improves comprehension. There was another function I was having difficulty getting the previous model to even recognize with a dozen function descriptions. This new model recognized it on the first try with my most recent revision.

If I find anything noteworthy I’ll report back. At first glance this seems to be the fix I’ve been waiting for.

2 Likes