Optimize prompt for parallel function calling

I am building an agent that needs to make a potentially large number of parallel function calls.

Expected behavior:

  • User: Here is my grocery shopping list: … (20 items)
  • Agent: Calls function search_product(description) 20 times

This typically works well with GPT4-Turbo, but with GPT3.5, I would say it works half the time (it typically calls search_product() once the first time, and then keeps going sequentially in the rest of the conversation).

I wonder if you would have some recommendation to make this work more consistently by adding instructions to the system prompt. For example I tried adding “Call functions simultaneously rather than sequentially when possible”, but hard to say if it makes a difference.


Here are some suggestions I can think of:

  1. Did you try defining one function call that accepts an array of products? (it doesn’t matter if your api only accepts one product, you can later adjust in your agent code). This might make it simpler for the model than doing 20 parallel functions.
  2. You can try not using function calling at all, but just instruct the model in the system prompt to extract all products and return them in an array (again, agent code can handle later on). In my experience this can be a bit more reliable.

I also want to invite you to try out Promptotype (full disclosure: I’m the creator of it), which is a dev platform focusing specifically on your use case (whether you choose function calling or system message). It has a special playground supporting function calling, and enables dedicated testing for structured outputs.