Azure Openai GPT 3.5 and 4 not calling multiple tools in parallel

[adhishsekar14]

Hi,i’am facing a bug when trying to call multiple tools at parallel.The model is calling only a single tool everytime. See below code.

@tool
def add(a: int, b: int) → int:
“”"Adds a and b.

Args:
    a: first int
    b: second int
"""
return a + b

@tool
def multiply(a: int, b: int) → int:
“”"Multiplies a and b.

Args:
    a: first int
    b: second int
"""
return a * b

query = “what is 2 multiplied to 3 and added to 6”
tools = [add, multiply]
llm_with_tools = llm.bind_tools(tools)
messages = [HumanMessage(query)]
ai_msg = llm_with_tools.invoke(messages)
print(ai_msg)

output:
tool_calls=[{‘name’: ‘multiply’, ‘args’: {‘a’: 2, ‘b’: 3}, ‘id’: ‘call_iL2w2GayR28vbVGqTnafJGD9’}

This code above need to call both addition and multiplication but instead calling only multiplication.I also tried with various tools and functions.But only one tool is getting invoked.

[Diet]

Welcome to the community!

how would you call (2 * 3) + 4 in parallel, and with what arguments?

are you maybe asking about nested tool calling?

[adhishsekar14]

Thanks for replying.By parallel i mean in the “tool_calls” there should by 2 tool calls (multiply and add).

Refer to : Langchain Function/Tool calling

[Diet]

I understand, my question is, what would you do, given the task?

query = “what is 2 multiplied to 3 and added to 6”

can you manually finish this completion?

output:
tool_calls=[{‘name’: ‘multiply’, ‘args’: {‘a’: 2, ‘b’: 3}, ‘id’: ‘1’}, {{‘name’: ‘

what comes next?

[adhishsekar14]

This is the whole output :
content=‘’ additional_kwargs={‘tool_calls’: [{‘index’: 0, ‘id’: ‘call_iL2w2GayR28vbVGqTnafJGD9’, ‘function’: {‘arguments’: ‘{\n “a”: 2,\n “b”: 3\n}’, ‘name’: ‘multiply’}, ‘type’: ‘function’}]} response_metadata={‘finish_reason’: ‘tool_calls’, ‘model_name’: ‘gpt-4-32k’} id=‘run-08548196-acfb-4537-9b96-c3ec8cda12ae-0’ tool_calls=[{‘name’: ‘multiply’, ‘args’: {‘a’: 2, ‘b’: 3}, ‘id’: ‘call_iL2w2GayR28vbVGqTnafJGD9’}]

[Diet]

no, how would you, a human, Adhish Sekar, complete the following text

output:
tool_calls=[{‘name’: ‘multiply’, ‘args’: {‘a’: 2, ‘b’: 3}, ‘id’: ‘1’}, {{‘name’: ‘…?

[adhishsekar14]

Ok now i get.Its not possible for the llm to make the multiple tool call because of the query contains only 3 params and it will not calculate the result of first part and apply it in second part

But consider query = “what is 2 multiplied to 3 and what is 5 added to 4”

iam getting
output : content=‘’ additional_kwargs={‘tool_calls’: [{‘index’: 0, ‘id’: ‘call_1rgHWaMaNQfgwp1ifxnK5vg7’, ‘function’: {‘arguments’: ‘{\n “a”: 2,\n “b”: 3\n}’, ‘name’: ‘multiply’}, ‘type’: ‘function’}]} response_metadata={‘finish_reason’: ‘tool_calls’, ‘model_name’: ‘gpt-4-32k’} id=‘run-670f2b6d-e47c-463e-975a-690c310c0ff1-0’ tool_calls=[{‘name’: ‘multiply’, ‘args’: {‘a’: 2, ‘b’: 3}, ‘id’: ‘call_1rgHWaMaNQfgwp1ifxnK5vg7’}]

still the problem exists?

[Diet]

next step: consider adding a CoT (chain of thought) step, asking the model to think about what tool calls can be sent in parallel/at the same time

[adhishsekar14]

Now iam getting the below output :

content=‘The tool calls for these operations can be made in parallel as follows:\n\njson\n[\n {\n "function": "functions.multiply",\n "args": {\n "a": 2,\n "b": 3\n },\n "instance": "1"\n },\n {\n "function": "functions.add",\n "args": {\n "a": 5,\n "b": 4\n },\n "instance": "2"\n }\n]\n\n\nIn this case, the “multiply” tool call is used to multiply 2 and 3, and the “add” tool call is used to add 5 and 4. These calls are made in parallel, indicated by the different instance numbers.’ response_metadata={‘finish_reason’: ‘stop’, ‘model_name’: ‘gpt-4-32k’} id=‘run-f80b3be2-7be1-429f-8c79-64cc8473a982-0’

[Diet]

[
  {
    "function": "functions.multiply",
    "args": {
      "a": 2,
      "b": 3
    },
    "instance": "1"
  },
  {
    "function": "functions.add",
    "args": {
      "a": 5,
      "b": 4
    },
    "instance": "2"
  }
]

well, you could just parse that, no? but you may need to specifically instruct the model to actually call the tools/functions at the end of the thinking if you’re set on using functions.

[adhishsekar14]

thank you for your suggestion but i just wanted to try the code on langchain but still not able to get the output as it is given in the documentation in langchain.
from langchain_core.tools import tool

@tool
def add(a: int, b: int) → int:
“”"Adds a and b.

Args:
    a: first int
    b: second int
"""
return a + b

@tool
def multiply(a: int, b: int) → int:
“”"Multiplies a and b.

Args:
    a: first int
    b: second int
"""
return a * b

tools = [add, multiply]
llm_with_tools = llm.bind_tools(tools)

[Diet]

langchain probably isn’t making your life easier

you can try your luck here: https://www.reddit.com/r/LangChain/

[merefield]

A couple of notes:

1. Yeah frameworks like langchain can actually make tasks more complicated - stick to first principles and write stuff yourself with just a bare bones client library?
2. Consider using a (safe) language interpreter instead, e.g a local process that can safely evaluate JavaScript, python or Ruby expressions in a sandbox, giving the LLM more flexibility to perform this in one shot.

My Chatbot does this with a Ruby sandbox.

[adhishsekar14]

Thank you i will try there

[adhishsekar14]

yeah i tried without langchain but getting the same results

code :

add = {
“name”: “add”,
“description”: “Add ‘x’ and ‘y’.”,
“parameters”: {
“type”: “object”,
“properties”: {
“x”: {“type”: “number”, “description”: “First number to add”},
“y”: {“type”: “number”, “description”: “Second number to add”}
},
“required”: [“x”, “y”]
}
}
multiply = {
“name”: “multiply”,
“description”: “Multiply ‘x’ and ‘y’.”,
“parameters”: {
“type”: “object”,
“properties”: {
“x”: {“type”: “number”, “description”: “First number to multiply”},
“y”: {“type”: “number”, “description”: “Second number to multiply”}
},
“required”: [“x”, “y”]
}
}
messages =
llm_with_tools = llm.bind_tools([multiply,add])
print(llm_with_tools.invoke([
(“system”, “You’re a helpful assistant who can use tools.Use multiple tools if necessary”),
(“human”, “what’s 1 added to 1000 and 5 multiplied to 6”),
]) )

output:
content=‘’ additional_kwargs={‘tool_calls’: [{‘index’: 0, ‘id’: ‘call_cMXg3JIlWq7aURYVyPVSe5KT’, ‘function’: {‘arguments’: ‘{\n"x": 1,\n"y": 1000\n}’, ‘name’: ‘add’}, ‘type’: ‘function’}]} response_metadata={‘finish_reason’: ‘tool_calls’, ‘model_name’: ‘gpt-4-32k’} id=‘run-13f02b2a-d9fa-4a6c-b7b2-a2008dac5927-0’ tool_calls=[{‘name’: ‘add’, ‘args’: {‘x’: 1, ‘y’: 1000}, ‘id’: ‘call_cMXg3JIlWq7aURYVyPVSe5KT’}]

[merefield]

It can be done:

Parallel call with last example:

Your original example

Code here:

GitHub - merefield/discourse-chatbot: An AI bot with RAG capability for...

[adhishsekar14]

I understand that it can be done using the single tool “Calculate”.But i want it to be done using multi-tools(add and multiply) .Iam trying this to learn about multi-tool calling.

[merefield]

See my first example above, it has a parallel call.

[adhishsekar14]

Yes I saw that but still it doesnt work on mine.I want to know why the same code it works in documentation but not for me

[merefield]

as per @Diet I’d ask on Langchain reddit?