Assistant goes crazy when combining file retrival and functions

Hello, I have been using the API for a WhatsApp bot. So far I managed to get consistent responses based on a file that is table markdown formatted.

I just tried to implement function calling, with the idea of providing a picture of the menu for the restaurant this bot should work for.

This created some problems :

  • The bot seems to be calling the function whenever he wants and sometimes multiple times with simple sentences such as “hi”
  • The bot no longer has accuracy when asked about products on the provided file.

I have specified in all the messages to use that file, and also in the instructions. Whenever I remove the file-id from the messages he stops doing the function calls so frequently, but I lose the product information.

How can I solve this? I only need the bot to call the function whenever someone wants a picture of the menu, not every time someone ask anything, because there is a file with all the information about the products.

Adding the instructions : (Yes they are in Spanish)

Obten la informacion acerca de los productos del negocio del archivo file-7DbNFIEgrPKTf4KWTK60i1NA.

De no encontrar informacion acerca de un producto notifica al usuario que ese producto no existe. No intentes predecir o asumir que un producto pueda existir por similitudes con otro.

Actua de forma amable al usuario. Utiliza emoticones en la conversacion y frases cortas. 

Confirma la orden del cliente, menciona el precio de cada producto para que el cliente lo conozca.

Cuando el cliente confirme su orden completa , debes mostrar un mensaje con estilo de factura , por ejemplo :  

Producto 1. 600 colones
Producto 2.100 colones
Producto 3.3000 colones

Total : 4700 colones.

Despues de esto , puedes preguntar al cliente en que forma va a pagar su orden. Los pagos actuales son : Tarjeta, Efectivo y SINPE Movil. Debes de modificar la factura y mantener la informacion anterior , para que agregues el metodo de pago, por ejemplo :

Producto 1. 600 colones
Producto 2.100 colones
Producto 3.3000 colones

Metodo de pago : SINPE Movil

Total : 4700 colones.

Yes, I also see this behavior. How do you handle when it triggers the function calling when it is not supposed to be? You can probably send something like:

{ “status”: “ignore”, “message”: “not relevant” }

Well, I am under the assumption that I cant really tell when was the function called correctly.

For example. I expect the bot to call the function in cases where the user wants to see the menu, but I have seen parameters with values that did not come from user messages.

As I said before, the function would trigger at the very first message with a simple Hello, passing a value of “menu” for the parameter I had defined, I don’t know where this came from.

This is the function. I was thinking about maybe adding more required parameters, but what if the bot makes them up anyways?

{
  "name": "get_picture_of_menu",
  "description": "Gets a picture of the menu",
  "parameters": {
    "type": "object",
    "properties": {
      "userWantsPicture": {
        "type": "string",
        "description": "The value of the user's choice if the wants to see a picture of the menu. Yes or No"
      }
    },
    "required": [
      "userWantsPicture"
    ]
  }
}

That’s the problem. The bot do make up the parameters not sent by the user.

Do you know any way to prevent this? It seems to me that low parameter counts and files with related information to the function description might not work properly.

I was also thinking of making a function to help book appointments for a barber shop, since that takes more parameters, it could work.

Also, is there a way for a function to “Fail”. For example, what if there is no space available in the barbershop? Am I supposed to create a simple “checkForSpace” function that has all the complex logic behind the scenes that returns either a yes or no?

This function can be improved.

The main issue is that it technically can always be called. If the user wants to see the menu or not.

The function should instead have no parameters (because why else would the function be called if the user wants to see the menu?)

In regards to hallucinating: ideally you are validating the code first and returning an error message for GPT to pass on failures.