Function Calling - fill parameters conditionally

API
, ,
1

I have a function called add_user_to_class(). I’m trying to use function calling capability to call this function. This function has the following parameters:
class_name, user_name, user_address, user_disply_id, b_create_user. If the user asks something on the line “add a user to a class”, I wanted llm to confirm with the user
if he/she wants to create a new user or add an existing one. If the user wants to create a new user, llm should get the parameters user_name and user_address(only these two).
And if the user wants to add an existing user, llm should only get the user display id. Also, llm should set the field “b_create_user” to True if the user wants to create new user. Else, its value should be False.
I have tried the following system prompt:

" You are a friendly AI assistant whos capable of adding a user to a class. Before you call the function add_user_to_class(), make sure you ask the user if he wants to create a new user or if he wants to add an existing user to the class. Only if the user wants to create a new user ask the user name, user address and the name of the class. In this case, the value of ‘b_create_user’ is True. However, if the user wants to add an existing user to the class, only ask him the display id of the user and the name of the class. Do not ask him user name and user address in this case. In this case, the value of ‘b_create_user’ is False. Strictly DO NOT make assumptions about what values to plug into functions. Ask for clarification if a user request is ambiguous."

tools:

    {
        "type": "function",
        "function": {
            "name": "add_user_to_class",
            "description": "This function will create a new user and a new class add the user to the class",
            "parameters": {
                "type": "object",
                "properties": {
                    "class_name": {
                        "type": "string",
                        "description": "Class name. This can be any string. Stricly ask this from the user. DO NOT hallucinate the value of this parameter.",
                    },
                    "user_name": {
                        "type": "string",
                        "description": "The name of the user. This can be any string. Only ask this from the user if the user wants to create a new user.",
                    },
                    "user_display_id": {
                        "type": "string",
                        "description": "Display ID of a user. Only ask this if the user wants to add an existing user to the tab.",
                    },
                    "b_create_user": {
                        "type": "string",
                        "description": "This parameter is true if the user wants to create a new user else False."
                    }
                },
                "required": ["class_name"],
            },
        },
    },

This doesn’t work for me all the time. Most of the times, the llm hallucinate the parameter values. How can I make the model work according to my requirement?

2

What if you made two different functions, maybe create_and_add_user_to_class and add_existing_user_to_class? Maybe it will be less likely to be confused.

3

I tested your function and edited some parts. I merged the user name and display id under one parameter.

{
  "name": "add_user_to_class",
  "description": "Adds user to the class",
  "parameters": {
    "type": "object",
    "properties": {
      "class_name": {
        "type": "string",
        "description": "Class name"
      },
      "user_identifier": {
        "type": "object",
        "description": "User identifier",
        "properties": {
          "user_name": {
            "type": "string",
            "description": "Name of the new user"
          },
          "user_display_id": {
            "type": "string",
            "description": "User display id for existing user"
          }
        }
      },
      "isNewUser": {
        "type": "boolean",
        "description": "Set true for new user"
      }
    },
    "required": [
      "class_name",
      "user_identifier"
    ]
  }
}

I set the temp to 0.1 to minimize hallucination. Here’s a sample conversation:

Screenshot 2024-06-07 at 9.07.32
Screenshot 2024-06-07 at 9.07.322122×1594 347 KB

4

Hi @supershaneski thanks for the response. I have few questions on your query… is there any reason why you have combined user name and display id under one parameter?
Also, you for create new user, I can see the model is still hallucinating the displayId parameter.
Also what kind of system message to use here?

5

i combined for simplicity, to only check one parameter for user since that seem to be your specification.

as for the hallucination, what is your temperature? set it lower. however, you should expect hallucination. if you have specific id format, you can validate the response and check if the value is hallucinated, disregard it then set the appropriate status message in tool outputs.

i only have simple system message as you can see in the image.