Hey guys,
I am interested in what you think about an idea I have concerning function_calling.
I am currently using completion calls to do function_calling. For the end-user it would be nice to understand why a certain function has been triggered, that’s why I am thinking of adding a new property called “reasoning” (or something similar) and of course giving further instructions to the system prompt.
What do you think about this method? Has anybody experience with this or has a better solution?
Example:
{
"type": "function",
"function": {
"name": "track_order",
"description": "Track a package based on it's order number",
"parameters": {
"type": "object",
"properties": {
"order_number": {
"type": "string",
"description": "6-digit order number"
},
"reasoning": {
"type": "string",
"description": "The reason why you called this function."
}
},
"required": [
"order_number",
"reasoning"
]
}
}
},
....
]```
While all hindsight is hallucination, you can ask the model to lay out logical steps of things that it will do before doing it. The chance that it will then do something else is quite low (unless it actually can’t do something).
I suspect that with functions it’s quite clunky: we’ve established that you can’t ask for reasoning after (or while) invoking the function, so you’d need to create a prompt that generates reasoning first, then calls the function, and then copies that reasoning into your function… big waste of tokens but it’s one approach, I guess.
What about shifting the overhead around?
For example using a 1D classifier to determine if a function call will be needed and then return a pre-determined string to the user.
This would look something like this:
User: what’s the name of that fish restaurant at Main street?
Classifier Model returns ‘1’
Mapping the ‘1’ to an enum value
Send message to user: Will look up information in Internet.
This could still be token and time efficient while being less precise with regards to the explanation.
But I guess it’s really not perfect either.