# Parallel tool calling where there is an ordering dependency **URL:** \ **Category:** API\ **Tags:** chat-completion, tool, function-calling\ **Created:** [January 10, 2025, 4:38pm UTC](https://community.openai.com/t/parallel-tool-calling-where-there-is-an-ordering-dependency/1086995 "2025-01-10T16:38:45Z")\ **Posts on this page:** 15\ **Page:** 1
**Author:** ![pkriens](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/pkriens/32/511710_2.png) [@pkriens](https://community.openai.com/u/pkriens)\ **Post date:** [January 10, 2025, 4:38pm UTC](https://community.openai.com/t/parallel-tool-calling-where-there-is-an-ordering-dependency/1086995/1 "2025-01-10T16:38:45Z")
The tools in the Chat completion API can be called in parallel. I do get multiple calls and execute in a different thread. I understand they can be run in parallel but I see there are calls generated that depend on each other. For example, I asked to write a hello program in C and compile it. This required a write and compile tool call. However, clearly the write must finish before the compile should start. I can’t find any information about this ordering, how do people handle this? It seems a pity to run all the given commands in one thread? ---
**Author:** ![anon10827405](https://avatars.discourse-cdn.com/v4/letter/a/ed8c4c/32.png) [@anon10827405](https://community.openai.com/u/anon10827405)\ **Post date:** [January 10, 2025, 4:51pm UTC](https://community.openai.com/t/parallel-tool-calling-where-there-is-an-ordering-dependency/1086995/2 "2025-01-10T16:51:57Z")
The model’s sole responsibility is to produce the function and its parameters. If `Function A` depends on the output of `Function B`, managing that dependency is your task. For independent functions, such as `Function A` and `Function B`, you can execute them in parallel and return their results to the model for synthesis. If the model calls `Function A` and `Function B` simultaneously despite sequential instructions, you can still execute them in order yourself and return whatever should be the result. So, all of this “ordering” nonsense is **ultimately handled by you** , not the model. The mode should be able to provide the correct information, but you should have proper procedures in place to handle any edge-cases. ---
**Author:** ![sps](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/sps/32/7736_2.png) [@sps](https://community.openai.com/u/sps)\ **Post date:** [January 10, 2025, 5:04pm UTC](https://community.openai.com/t/parallel-tool-calling-where-there-is-an-ordering-dependency/1086995/3 "2025-01-10T17:04:09Z")
I faced the same issue when I was building a GPT. One way is to let the model know if one function call depends on the context obtained from another. You can also ensure that one function is called at a time by disabling parallel tool calls by setting [`parallel_tool_calls`](https://platform.openai.com/docs/api-reference/chat/create#chat-create-parallel_tool_calls) to `false`. > [@Function Calling - How Does It Modify The Prompt?](https://community.openai.com/t/function-calling-how-does-it-modify-the-prompt/856567/13): > > That’s exactly what I did in a project where I had to implement the NASA Open API for getting imagery. For context: You can’t simply query the API for imagery on a particular date. The process requires two steps: Request data from the Earth Assets API to find the closest satellite pass for a given location and date. Use that data to query the Earth Imagery API with the exact date-time and location. Initially, when users asked for imagery, the model would use their exact date-time to call th… ---
**Author:** ![pkriens](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/pkriens/32/511710_2.png) [@pkriens](https://community.openai.com/u/pkriens)\ **Post date:** [January 10, 2025, 5:14pm UTC](https://community.openai.com/t/parallel-tool-calling-where-there-is-an-ordering-dependency/1086995/4 "2025-01-10T17:14:05Z")
The ai knows that my compile function depends on the completed write. How am I supposed to know that these functions depend on each other? It might write 50 files and then compile all. It might write 2 files while only one is used in the compile. Ie there can be a semantic dependency between A and B of which I have zero knowledge. I can mark my function implementations to have dependencies , eg compile only after all writes are done, but that is a guess resulting in being overly cautious and thus inefficient. And an example of unnecessary complexity maintenance. And this seems awfully wrong because so far the model always submits the calls in the correct order. It seems to ‘know’ the order, it just has no API to tell me which calls belong together and should be executed in order and which calls can be truly called in parallel. ---
**Author:** ![anon10827405](https://avatars.discourse-cdn.com/v4/letter/a/ed8c4c/32.png) [@anon10827405](https://community.openai.com/u/anon10827405)\ **Post date:** [January 10, 2025, 5:26pm UTC](https://community.openai.com/t/parallel-tool-calling-where-there-is-an-ordering-dependency/1086995/5 "2025-01-10T17:26:09Z")
> [@pkriens](#): > > How am I supposed to know that these functions depend on each other? You created the functions in the first place. How _wouldn’t_ you know that they depend on eachother? Are you confusing function calling with code interpreter? Are you using the model to create lines of code via Function Calling and then compile the program? So… Are we talking about functions… inside of a script that is being automatically generated? In any case you would be better off having the model define a > **[Dependency graph](https://en.wikipedia.org/wiki/Dependency_graph)** > > In mathematics, computer science and digital electronics, a dependency graph is a directed graph representing dependencies of several objects towards each other. It is possible to derive an evaluation order or the absence of an evaluation order that respects the given dependencies from the dependency graph. > Given a set of objects > > > > S > > > {\\displaystyle S} > > and a transitive relation > > > > R > ⊆ > S > × > S > > > ... Because regardless of what’s going on, it seems like you are not having control over the model and it’s decisions, which is not a good place to be in. > [@pkriens](#): > > It seems to ‘know’ the order, it just has no API to tell me which calls belong together and should be executed in order and which calls can be truly called in parallel. This is your responsibility when you execute the code. Again, the first, and most important fact to remember is: > [@anon10827405](#): > > The model’s sole responsibility is to produce the function and its parameters. ---
**Author:** ![pkriens](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/pkriens/32/511710_2.png) [@pkriens](https://community.openai.com/u/pkriens)\ **Post date:** [January 10, 2025, 9:00pm UTC](https://community.openai.com/t/parallel-tool-calling-where-there-is-an-ordering-dependency/1086995/6 "2025-01-10T21:00:07Z")
I happen to be a world class expert in java dependencies … so you can assume I know what I am talking about. You forget that a function has parameters and an output. The dependency can depend on the parameters and the output. 1. AI calls [A(foo)-\>bar,B(bar)] 2. AI calls [A(foo)-\>bar,A(xyz)-\>xxx, B(bar)] In situation 1), A must run before B. In situation 2) A(foo) must run before B but A(xyz) can run anytime. There is no way for me to know how to order A and B. However, the model knows so it could indicate it. ---
**Author:** ![anon10827405](https://avatars.discourse-cdn.com/v4/letter/a/ed8c4c/32.png) [@anon10827405](https://community.openai.com/u/anon10827405)\ **Post date:** [January 10, 2025, 9:12pm UTC](https://community.openai.com/t/parallel-tool-calling-where-there-is-an-ordering-dependency/1086995/7 "2025-01-10T21:12:10Z")
> [@pkriens](#): > > I happen to be a world class expert in java dependencies I’m not even sure what this means. > [@pkriens](#): > > so you can assume I know what I am talking about. I’m not doubting you, but I’m not going to assume anything. * * * > [@pkriens](#): > > In situation 1), A must run before B. In situation 2) A(foo) must run before B but A(xyz) can run anytime. In both situations you need to process the function calling and therefore should know how it should be executed. You have clearly outlined the dependencies to me, so you can program this as well. You can ask the model to revise it’s function calling parameters in the case it’s forgetting something. > [@pkriens](#): > > However, the model knows so it could indicate it. Then you indicate to the model, as posted by @sps . You should ALSO have safety measures in place in the event that the model makes a mistake. If what you saying is that you don’t know if the output of whatever is sufficient and may require another function based on what the model thinks, then you can simply include this in the tool call output. Simple as that. The model is capable of iterating function calls ---
**Author:** ![Diet](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/diet/32/562749_2.png) [@Diet](https://community.openai.com/u/Diet)\ **Post date:** [January 10, 2025, 9:14pm UTC](https://community.openai.com/t/parallel-tool-calling-where-there-is-an-ordering-dependency/1086995/8 "2025-01-10T21:14:02Z")
So, what’s your question? 🤔 Give the model a parameter `callId` and `requires[]` for each fc. then either build a dependency graph or simply try to resolve all functions whose requirements have been resolved until no functions remain or you’re stuck in a loop 🤔 ---
**Author:** ![arata](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/arata/32/523049_2.png) [@arata](https://community.openai.com/u/arata)\ **Post date:** [January 10, 2025, 11:15pm UTC](https://community.openai.com/t/parallel-tool-calling-where-there-is-an-ordering-dependency/1086995/9 "2025-01-10T23:15:16Z")
> [@pkriens](#): > > The ai knows that my compile function depends on the completed write. “Has been provided information” and “properly uses” are two different things. The best way to use functions is by not discussing them anywhere except in the function specification description fields themselves. Thus, one would use that main description field for a function to provide the guidance needed. Something like: `description: f"{what the function does and returns}.\nThis dependent function B must never be called unless a result from function A has been first obtained. It cannot be used in parallel with other tools. It cannot be wrapped within multi_tool_use."` ---
**Author:** ![pkriens](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/pkriens/32/511710_2.png) [@pkriens](https://community.openai.com/u/pkriens)\ **Post date:** [January 11, 2025, 11:16am UTC](https://community.openai.com/t/parallel-tool-calling-where-there-is-an-ordering-dependency/1086995/10 "2025-01-11T11:16:50Z")
People seem to misunderstand the issue. There is no dependency between the functions, it is the model that assumes the side effects of these functions are ordered. ``` { id="1", name="write", arguments= { path="foo.c", "content": "#include ... return 0;\n}" }}, { id="2", name="bash", arguments= { command="gcc foo.c -o foo" }} { id="3", name="bash", arguments= { command="./foo" }} ``` I will add the dependent call ids see if the model can provide that information so I can order them. ---
**Author:** ![pkriens](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/pkriens/32/511710_2.png) [@pkriens](https://community.openai.com/u/pkriens)\ **Post date:** [January 11, 2025, 11:30am UTC](https://community.openai.com/t/parallel-tool-calling-where-there-is-an-ordering-dependency/1086995/11 "2025-01-11T11:30:30Z")
When I tried to give me the index or call id of dependencies it no longer ran them in parallel! It got mistaken once, it listed itself as dependency. ---
**Author:** ![arata](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/arata/32/523049_2.png) [@arata](https://community.openai.com/u/arata)\ **Post date:** [January 12, 2025, 11:25pm UTC](https://community.openai.com/t/parallel-tool-calling-where-there-is-an-ordering-dependency/1086995/12 "2025-01-12T23:25:21Z")
> [@pkriens](#): > > There is no dependency between the functions, it is the model that assumes the side effects of these functions are ordered The keyword to use in the function then, is “stateless environment”. Or explain “parallel”. _“Commands issued to this function will all run independently in parallel - all at the same time. Thus, multi\_tool\_use cannot be used to chain a series of commands. You must await a return response from an initial call, and call again iteratively for dependent actions.”_ The key is to make the AI understand, as there is little guidance in the internal parallel wrapper. ---
**Author:** ![jochenschultz](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/jochenschultz/32/491030_2.png) [@jochenschultz](https://community.openai.com/u/jochenschultz)\ **Post date:** [January 13, 2025, 12:15am UTC](https://community.openai.com/t/parallel-tool-calling-where-there-is-an-ordering-dependency/1086995/13 "2025-01-13T00:15:49Z")
When I build a code evaluation software I also had a problem like this. You need to discover the programming language first before we could start a set of specialized agents to analyze the code. So I gave each splitted task (in that case code parts analyzer) a status and each agent had to check the status and when it wasn’t for it then had to wait a certain time before trying again. Well, that was before I introduced rabbitmq and a solid agent chain where each service would listen to a specific message type e.g. the IncomingQueue that gets a file:put message from S3 storage when a new file comes in will be listened to by a service that consumes file:put messages which then finds let’s a say a folder with code - or a pdf and creates a code:analyze or pdf:split message where another service will read a workflow for… So by having consumers listening to a Queue permanentely you can add multiple new containers with the same service and got no problems with the joke # A programmer had a problem. He thought to himself, “I know, I’ll solve it with threads!” has Now problems. two he * * * btw.. this is how it looked like a month or so ago.. [![](https://us1.discourse-cdn.com/openai1/original/4X/c/8/2/c8248b5675abdc540fa8aca0d2b389d00dd907f3.jpeg "DevOpsPorn") ](https://www.youtube.com/watch?v=VpLVetfYn1Y) ---
**Author:** ![pkriens](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/pkriens/32/511710_2.png) [@pkriens](https://community.openai.com/u/pkriens)\ **Post date:** [January 13, 2025, 7:57am UTC](https://community.openai.com/t/parallel-tool-calling-where-there-is-an-ordering-dependency/1086995/14 "2025-01-13T07:57:54Z")
Somehow since this discussion it runs them with sequential invocations … it is probably listening in on our discussion? 🕶 ---
**Author:** ![pkriens](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/pkriens/32/511710_2.png) [@pkriens](https://community.openai.com/u/pkriens)\ **Post date:** [January 13, 2025, 8:04am UTC](https://community.openai.com/t/parallel-tool-calling-where-there-is-an-ordering-dependency/1086995/15 "2025-01-13T08:04:20Z")
Like any complex issue, concurrency requires skill. Quite a lot of skills since our minds are awful in reasoning about concurrent problems. Interestingly, the LLMs seem just as bad in this. If it is not a standard problem they come up with strange solutions that can clearly fail. However, the benefits are huge and there are numerous patterns that can make your code still readable & simple and concentrate the hard parts in a few places.