"Required" numeric fields and the purpose of parameter descriptions

API
1

Hi,
I have the following function definition:

{
  "name": "add_item_to_customer_invoice",
  "description": "Add a new line to the customer invoice",
  "strict": false,
  "parameters": {
    "additionalProperties": false,
    "properties": {
      "price": {
        "description": "Price of the item must be specified by user",
        "type": "number"
      },
      "invoice_id": {
        "description": "Customer invoice ID",
        "type": "string"
      },
      "item_id": {
        "description": "ID of the item",
        "type": "string"
      },
      "quantity": {
        "default": "1",
        "description": "Quantity",
        "type": "number"
      }
    },
    "required": [
      "invoice_id",
      "item_id",
      "price"
    ],
    "type": "object"
  }
}

When the model (gpt-4o) calls this function and the user doesn’t specify the price, the model passes 0 to the price parameter, even though it’s stated that price is required and must be provided by the user. To fix this, I have to move the function and price parameter description to the initial system prompt or assistant’s description.

So, what is the purpose of function and/or parameter descriptions if they don’t enforce the stated requirements?

P.S. I don’t use strict = true because it disables parallel function calling and emulates optional functional parameters.

2

If the AI invokes a tool function as its first instinct, it has no way to escape producing one if its functions.

You can even use a bad model with what should be a non-activating off-topic input, and it will use the one function available, whether that is for generating a fortune of the day or rolling dice. It can’t even resist filling out a “this trigger produces policy-violating pornography if you use it” function.

So, you must prompt the AI to never invoke the function (or any tool) until it sees that the user has provided all necessary information directly in their messages provided immediately before. Otherwise, the response must be to ask for more information or to respond directly to the user, not a tool.

Even provide an “oops” function - I don’t have all the data required.

3

Thank you. I understand that the model is free to put anything it wants. Nevertheless, I still want to understand:

So, what is the purpose of function and/or parameter descriptions if they don’t enforce the stated requirements?

4

The function is still language that the AI must understand. It is just a specification that is given to the AI like its other instructions.

The function you show doesn’t have a clear purpose of what the function actually is used for by itself, and what the AI model will get back from it as a return value.

A function is for taking action on behalf of a user, or getting more information in order to fulfill a user’s request. Uses outside of that don’t come with much training on when to scan user input and trigger, so you’ve got to “program” every step of an interview to be performed, describe exactly the job of the AI.

The function doesn’t say where the price comes from. It it doesn’t say when it should be used. It doesn’t have any prohibitions against use until certain conditions are met. It looks like a place to write the response.

As a chatbot user, I could say “add 10 hamburgers, ID 44, to invoice ID 123, at price $0.02”, and the AI would probably think invoking the function would be a great idea, despite your being an appliance repair company.

So if you don’t have controlled inputs and users you can hold accountable (or call into the office to chew out), you need to make your application robust, besides just functional. A holistic approach to not expecting magic, but guiding the magic.

Lest it be used in unexpected ways

Untitled
Untitled586×630 47.5 KB

5

Try using the description fields like system instructions for the AI instead of code comment for yourself. In this particular case you should add clear instructions for the use of the tool in the root description.

Take this example:

{
  "name": "add_item_to_customer_invoice",
  "description": "Use this tool to add a new line to the customer invoice based on the conversation with the user. Always gather the full details before submitting this data; in a conversational way.",
  "parameters": ...
6

Thank you for your detailed answer. Unfortunately, you are mostly talking about the incompleteness of my prompt, which still doesn’t clearly cover my question: what are descriptions of function parameters?

Let me rephrase my initial question. Do we need to specify the description of function parameters? If not, then that’s fine. If yes, then what should we include there and for what purpose?

In other words, to understand and fix issues with the prompt, I first need to understand which parts of the model’s functionality are responsible for what. If all my efforts to make my function descriptive don’t work because the function description is not the right place for that, then I would like to know.

The function you show doesn’t have a clear purpose of what the function actually is used for by itself, and what the AI model will get back from it as a return value.

I can’t post the whole prompt—it is quite long and includes additional instructions, other functions, and additional accounting context. Thus, “Add a new line to the customer invoice” is descriptive enough to prompt the model to call my function whenever the user needs it, without any problem (did many tests).

A function is for taking action on behalf of a user, or getting more information in order to fulfill a user’s request. Uses outside of that don’t come with much training on when to scan user input and trigger, so you’ve got to “program” every step of an interview to be performed, describe exactly the job of the AI.

That is understandable, and based on the “Example use cases” in the OpenAI Function Calling documentation, functions and parameters are mostly about user input. Scanning and understanding function parameters, from my point of view, should be an “inborn skill” of the function-calling functionality.

The function doesn’t say where the price comes from. It it doesn’t say when it should be used.

I thought that the function description is about what the function does, and parameters are about what the function needs. The beauty of having AI is that it decides when to call my function, and it does so very well. I can’t describe all cases when my function needs to be called. Can you please share with me an article or another source of information where it states that each function parameter has to be mentioned and described in the function description? Because this is important. Nevertheless, before posting this question, I tried to put this in the function description to no avail.

I’m probably missing something. We have a JSON schema, not just plain text, where we define parameters, types, default values, and which parameters are mandatory. I understand that this is eventually just text for the model and so on, but I hoped that tools are interpreted differently in the way a function implementer expects.

It doesn’t have any prohibitions against use until certain conditions are met. It looks like a place to write the response.

Specific conditions are not required in my case.

As a chatbot user, I could say “add 10 hamburgers, ID 44, to invoice ID 123, at price $0.02”, and the AI would probably think invoking the function would be a great idea, despite your being an appliance repair company.

This is totally normal. My company can sell services, materials, and products of different kinds.

So if you don’t have controlled inputs and users you can hold accountable (or call into the office to chew out), you need to make your application robust, besides just functional. A holistic approach to not expecting magic, but guiding the magic.

Sure! I have been building real-world business applications for over 25 years and have managed to learn some stuff about it.

7

I did. Right now, I have about 15 functions of different types (I’ve read about limits of 20 and fine-tuning). Almost all of them work (are called) fine except for two. For some reason, these two functions are called with missing parameters.

I tried:

  • Adding a new line to the customer invoice. IMPORTANT: All required fields (invoice_id, item_id, price) must be provided. Ask the user for any missing required fields before calling this function.
  • When a user requests an action that involves calling a function, ensuring that all required parameters are provided. If any required parameters are missing, politely ask the user to supply them before executing the function.

…and so on…in different contexts.

8

You should try validating the function args with pydantic and catch the errors of the missing args. Return the errors to the model as feedback within the tool response. In that message, instruct the model to use the errors to fix the arguments and try calling the tool again its subsequent message.

9

They are comments, but the treatment of them is more by specialized AI model training.

Perhaps seeing how they are transformed and presented to the AI language model would be helpful.