The tools in the Chat completion API can be called in parallel. I do get multiple calls and execute in a different thread. I understand they can be run in parallel but I see there are calls generated that depend on each other.
For example, I asked to write a hello program in C and compile it. This required a write and compile tool call. However, clearly the write must finish before the compile should start.
I can’t find any information about this ordering, how do people handle this? It seems a pity to run all the given commands in one thread?
The model’s sole responsibility is to produce the function and its parameters.
If Function A depends on the output of Function B, managing that dependency is your task.
For independent functions, such as Function A and Function B, you can execute them in parallel and return their results to the model for synthesis.
If the model calls Function A and Function B simultaneously despite sequential instructions, you can still execute them in order yourself and return whatever should be the result.
So, all of this “ordering” nonsense is ultimately handled by you, not the model. The mode should be able to provide the correct information, but you should have proper procedures in place to handle any edge-cases.
I faced the same issue when I was building a GPT. One way is to let the model know if one function call depends on the context obtained from another. You can also ensure that one function is called at a time by disabling parallel tool calls by setting parallel_tool_calls to false.
The ai knows that my compile function depends on the completed write.
How am I supposed to know that these functions depend on each other? It might write 50 files and then compile all. It might write 2 files while only one is used in the compile.
Ie there can be a semantic dependency between A and B of which I have zero knowledge.
I can mark my function implementations to have dependencies , eg compile only after all writes are done, but that is a guess resulting in being overly cautious and thus inefficient. And an example of unnecessary complexity maintenance.
And this seems awfully wrong because so far the model always submits the calls in the correct order. It seems to ‘know’ the order, it just has no API to tell me which calls belong together and should be executed in order and which calls can be truly called in parallel.
You created the functions in the first place. How wouldn’t you know that they depend on eachother?
Are you confusing function calling with code interpreter?
Are you using the model to create lines of code via Function Calling and then compile the program? So… Are we talking about functions… inside of a script that is being automatically generated?
In any case you would be better off having the model define a
Because regardless of what’s going on, it seems like you are not having control over the model and it’s decisions, which is not a good place to be in.
This is your responsibility when you execute the code. Again, the first, and most important fact to remember is:
I’m not doubting you, but I’m not going to assume anything.
In both situations you need to process the function calling and therefore should know how it should be executed. You have clearly outlined the dependencies to me, so you can program this as well.
You can ask the model to revise it’s function calling parameters in the case it’s forgetting something.
Then you indicate to the model, as posted by @sps . You should ALSO have safety measures in place in the event that the model makes a mistake.
If what you saying is that you don’t know if the output of whatever is sufficient and may require another function based on what the model thinks, then you can simply include this in the tool call output. Simple as that. The model is capable of iterating function calls
Give the model a parameter callId and requires[] for each fc. then either build a dependency graph or simply try to resolve all functions whose requirements have been resolved until no functions remain or you’re stuck in a loop
“Has been provided information” and “properly uses” are two different things.
The best way to use functions is by not discussing them anywhere except in the function specification description fields themselves.
Thus, one would use that main description field for a function to provide the guidance needed. Something like:
description: f"{what the function does and returns}.\nThis dependent function B must never be called unless a result from function A has been first obtained. It cannot be used in parallel with other tools. It cannot be wrapped within multi_tool_use."
People seem to misunderstand the issue. There is no dependency between the functions, it is the model that assumes the side effects of these functions are ordered.
The keyword to use in the function then, is “stateless environment”. Or explain “parallel”.
“Commands issued to this function will all run independently in parallel - all at the same time. Thus, multi_tool_use cannot be used to chain a series of commands. You must await a return response from an initial call, and call again iteratively for dependent actions.”
The key is to make the AI understand, as there is little guidance in the internal parallel wrapper.
When I build a code evaluation software I also had a problem like this. You need to discover the programming language first before we could start a set of specialized agents to analyze the code.
So I gave each splitted task (in that case code parts analyzer) a status and each agent had to check the status and when it wasn’t for it then had to wait a certain time before trying again.
Well, that was before I introduced rabbitmq and a solid agent chain where each service would listen to a specific message type e.g. the IncomingQueue that gets a file:put message from S3 storage when a new file comes in will be listened to by a service that consumes file:put messages which then finds let’s a say a folder with code - or a pdf and creates a code:analyze or pdf:split message where another service will read a workflow for…
So by having consumers listening to a Queue permanentely you can add multiple new containers with the same service and got no problems with the joke
A programmer had a problem. He thought to himself, “I know, I’ll solve it with threads!”
has Now problems. two he
btw… this is how it looked like a month or so ago…
Like any complex issue, concurrency requires skill. Quite a lot of skills since our minds are awful in reasoning about concurrent problems. Interestingly, the LLMs seem just as bad in this. If it is not a standard problem they come up with strange solutions that can clearly fail.
However, the benefits are huge and there are numerous patterns that can make your code still readable & simple and concentrate the hard parts in a few places.
