I am in love with the function-calling feature introduced in
gpt-3.5-turbo-0613, and it’s been a game-changer for my GPT-3.5-Turbo bot. I have a lot of functions, though, and the problem is the token usage. Even if you only include one function, it still costs a lot of tokens – say, 400 to 550 tokens per function in my testing. High token usage, especially with multiple functions, can cost you a lot more money, and it’ll also reduce the number of tokens available for responses in total.
Because of this, I’ve had to implement conditional checks to see which functions are added to the functions that my bot is able to call. For instance, if certain words or phrases exist in a user’s message, a relevant function will be available for my bot.
The issue with this is that my bot won’t always be able to call the correct function.
A feature for function-calling that would be fantastic is a way to simply pass the function names to the bot – perhaps with short descriptions. Say, GPT-3.5 only sees
image_generation as a function and is able to call it, and if it decides to call it, it is then able to fetch its description, parameter information, etc. to call it with arguments.
As far as I’m aware, this is not possible currently. If there is a way to do this, please let me know! I suppose it would be possible by sending a regular response first to get the correct function before generating a function-call response, but that wouldn’t be ideal since it’d require an additional response before the function-call and wouldn’t be perfect.