Assistants API does not remember tool history

from autogen:

UserProxyAgent (to ManagementGroup):

what was last canucks game score?


messages [{‘content’: ‘what was last canucks game score?’, ‘role’: ‘user’, ‘name’: ‘UserProxyAgent’}]

Provide feedback to ManagementGroup. Press enter to skip and use auto-reply, or type ‘exit’ to end the conversation:

NO HUMAN INPUT RECEIVED.

USING AUTO REPLY…

EXECUTING FUNCTION discover_agents…

EXECUTING FUNCTION get_agent_info…

EXECUTING FUNCTION web_search…

user_assistant (to ManagementGroup):

Group[ManagementGroup] Speaker[user_assistant]: The last game score for the Vancouver Canucks was a 4-3 loss to the Seattle Kraken.


messages [{‘content’: ‘what was last canucks game score?’, ‘role’: ‘user’, ‘name’: ‘UserProxyAgent’}, {‘content’: ‘Group[ManagementGroup] Speaker[user_assistant]: The last game score for the Vancouver Canucks was a 4-3 loss to the Seattle Kraken.\n’, ‘role’: ‘user’, ‘name’: ‘user_assistant’}]

Provide feedback to ManagementGroup. Press enter to skip and use auto-reply, or type ‘exit’ to end the conversation:

NO HUMAN INPUT RECEIVED.

USING AUTO REPLY…

EXECUTING FUNCTION discover_agents…

EXECUTING FUNCTION get_agent_info…

EXECUTING FUNCTION web_search…

UserProxyAgent (to ManagementGroup):

Group[ManagementGroup] Speaker[UserProxyAgent]: The last game score for the Vancouver Canucks was a 4-3 loss to the Seattle Kraken.


messages [{‘content’: ‘what was last canucks game score?’, ‘role’: ‘user’, ‘name’: ‘UserProxyAgent’}, {‘content’: ‘Group[ManagementGroup] Speaker[user_assistant]: The last game score for the Vancouver Canucks was a 4-3 loss to the Seattle Kraken.\n’, ‘role’: ‘user’, ‘name’: ‘user_assistant’}, {‘content’: ‘Group[ManagementGroup] Speaker[UserProxyAgent]: The last game score for the Vancouver Canucks was a 4-3 loss to the Seattle Kraken.\n’, ‘role’: ‘user’, ‘name’: ‘UserProxyAgent’}]

Provide feedback to ManagementGroup. Press enter to skip and use auto-reply, or type ‘exit’ to end the conversation: why did you search again when the result was right there?

UserProxyAgent (to ManagementGroup):

Group[ManagementGroup] Speaker[UserProxyAgent]: why did you search again when the result was right there?


messages [{‘content’: ‘what was last canucks game score?’, ‘role’: ‘user’, ‘name’: ‘UserProxyAgent’}, {‘content’: ‘Group[ManagementGroup] Speaker[user_assistant]: The last game score for the Vancouver Canucks was a 4-3 loss to the Seattle Kraken.\n’, ‘role’: ‘user’, ‘name’: ‘user_assistant’}, {‘content’: ‘Group[ManagementGroup] Speaker[UserProxyAgent]: The last game score for the Vancouver Canucks was a 4-3 loss to the Seattle Kraken.\n’, ‘role’: ‘user’, ‘name’: ‘UserProxyAgent’}, {‘content’: ‘Group[ManagementGroup] Speaker[UserProxyAgent]: why did you search again when the result was right there?’, ‘role’: ‘user’, ‘name’: ‘UserProxyAgent’}]

Provide feedback to ManagementGroup. Press enter to skip and use auto-reply, or type ‘exit’ to end the conversation:

NO HUMAN INPUT RECEIVED.

USING AUTO REPLY…

manager (to ManagementGroup):

Group[ManagementGroup] Speaker[manager]: As the manager, it’s my responsibility to ensure that no redundant actions are taken by the group under my supervision. In this case, it appears there was a miscommunication that led to a repeated search for information that was already provided. This should not have happened, and I’ll take measures to avoid such redundancies in the future. Thank you for bringing this to my attention.


messages [{‘content’: ‘what was last canucks game score?’, ‘role’: ‘user’, ‘name’: ‘UserProxyAgent’}, {‘content’: ‘Group[ManagementGroup] Speaker[user_assistant]: The last game score for the Vancouver Canucks was a 4-3 loss to the Seattle Kraken.\n’, ‘role’: ‘user’, ‘name’: ‘user_assistant’}, {‘content’: ‘Group[ManagementGroup] Speaker[UserProxyAgent]: The last game score for the Vancouver Canucks was a 4-3 loss to the Seattle Kraken.\n’, ‘role’: ‘user’, ‘name’: ‘UserProxyAgent’}, {‘content’: ‘Group[ManagementGroup] Speaker[UserProxyAgent]: why did you search again when the result was right there?’, ‘role’: ‘user’, ‘name’: ‘UserProxyAgent’}, {‘content’: “Group[ManagementGroup] Speaker[manager]: As the manager, it’s my responsibility to ensure that no redundant actions are taken by the group under my supervision. In this case, it appears there was a miscommunication that led to a repeated search for information that was already provided. This should not have happened, and I’ll take measures to avoid such redundancies in the future. Thank you for bringing this to my attention.\n”, ‘role’: ‘user’, ‘name’: ‘manager’}]

You can see that the toolset history is not understood by the thread and thus it will try to make the same tool submissions again even though it was answered by running the tools in the first place. I think if the thread knew the tools were run in the history then subsequent agents would not make the same calls and avoid wasting context by trying the same things again. Imagine there were 100s of agents in a group and they all wanted to try to do it again because the context gives an answer maybe it cant trust.