Let’s say I have a set of tools, and GPT can ask me to invoke one tool, see the result, and then ask to invoke another tool, where the input for the second tool is the output from the first one.
For example:
1. A first tool generates some JSON data.
2. A second tool uploads this JSON data to the cloud.
Currently, this often requires two separate calls to the LLM:
1. GPT asks to invoke create_data() and receives the result.
2. GPT processes the result and then asks to invoke upload_to_cloud(created_data).
The second call to the LLM seems redundant because it should be possible to chain the tools in a single step:
create_data() → upload_to_cloud(created_data)
Is there a good way to achieve this kind of efficient chaining of tools directly within OpenAI’s function calling system, without requiring multiple LLM calls? Are there any best practices for this?
If you ALWAYS want the second function to happen after the first why not change the first function to do both? (Or create a third function that does both)
Definitely.
But this is not the case (I just provided a simple example). In practice, there is a bunch of tools, and I can not know how LLM will decide to chain them.
In that case it is really a matter of prompting. I have assistants with 20 different functions. An example like yours is before using the ‘add new account’ function always use the find account function to make sure it does not exist.
Happy to help further.
Also know that different models are better than others. 3.5 in my experience is not good enough to do this reliable.
One approaches that I can think about is providing first function name as an argument to the seconds function. So that the second function will be defined like that
{
"type": "function",
"function": {
"name": "upload_to_s3",
"description": "Upload data to s3. You can use this function in chain, that is, the output of the previous function will be the input of this function. Provide the name of the previous function as input to this function.",
"strict": true,
"parameters": {
"type": "object",
"properties": {
"prev_func": {
"type": "string",
"description": "Function name of the function that generated the data."
}
},
"required": [
"prev_func"
],
"additionalProperties": false
}
}
}
While your suggestions seem interesting, they do not offer the required flexibility.
For example, I need the model to query available parts for a given vehicle.
The function uses a vehicle_id and a text query as parameters in order to do so.
But, sometimes I have more than 1 vehicle_id for a given combination of make+model+year, e.g ‘Ford Mustang 1999’ will have an id for V6 and another for V8.
It will first retrieve the vehicle_ids with another function call.
What I need is for the model to be able to respond with another tool_call for a different vehicle_id –instead of a text response– in case it does not find parts with the first input vehicle_id.
Any thoughts?
In more general terms, I think the behavior we are looking for is for the model to be able to respond to a tool_call with another tool_call, while using as params data obtained from previous tool_calls.
If you are trying to define a function that will accept call_id (from previous function) as a parameter, then I can tell that that this will not work (I tried it).
Seems that LLM is not aware of call_id and it’s generated on some outer layer.
Another approach could be to define one more tool - orchestrator. That will accept function names, invoke it and pass the result to next function. Didn’t try it, but I think it should work
For example, I ask it to paint a picture based on a linked article, it first crawls the article and then sends a reasonable prompt to Dall-E to summarise the article pictorially, both using independent functions.
This required no special programming or enhanced relationships between the functions beyond the chatbot framework id already created.
That is certainly something that promptable. 'If the vehicle search returns more than one possibilty ask the user to select the vehicle before the request the parts… etc
If you are telling the AI that only one second function is needed, and there is no AI transformation, it sounds like you have two functions that should be one.
If there is a transformation of the output to be done to make it an input to the second part, then that can be a specialized AI API call with less or no chat context and a custom prompt for one task.
Thanks for sharing! ALways fun to see ‘real world stuff’ !
What model are you using? Some are better ‘out of the box’ at re-trying than others. But in my experience you do have to be super explicit with what you want. That also goes for the instruction you put in IN the function call template. The description for each field, especially the mandatory ones. Happy to help further if you want to share more details about the actual prompt.
