It would be nice to have a warning or error message when incorrect function specification is passed while using the Function Calling.
Example:
Right Usage:
tools = [
{
"type": "function",
"function": {
"name": "get_current_weather",
"description": "Get the current weather in a given location",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA",
},
"unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
},
"required": ["location"],
},
},
}
]
Wrong Usage:
tools = [
{
"type": "function",
"function": {
"name": "get_current_weather",
"description": "Get the current weather in a given location",
"input_schema": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA",
},
"unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
},
"required": ["location"],
},
},
}
]
While in this case the model generalizes and passes the correct arguments, for a different function, the model passes empty arguments.
Example:
tools = [
{
"type": "function",
"function": {
"name": "sematic_search",
"description": "Use this function to retrieve a list of the most relevant and semantically similar answers to the user's question.",
"input_schema": {
"type": "object",
"properties": {
"user_question": {
"type": "string",
"description": "The user_question for which sematically similar answers need to be retrieved. This parameter is mandatory and should be a meaningful question or phrase.",
}
},
"required": ["user_question"],
},
"output_schema": {
"type": "string",
"description": "The retrieved semantically similar answers to user question",
},
},
}
]
I understand that LLMs are non deterministic in nature, but we can programmatically make consuming LLMs more deterministic.
This kind of functionality exists already when using the JSON mode, even the prompt is scrutinized here.
'messages' must contain the word 'json' in some form, to use 'response_format' of type 'json_object'
Thus, it is even more critical to verify the function specification to ensure some kind of predictable behavior and helpful error messages.
