Optimizing Performance in a Multi-Agent Assistant System
Hello everyone,
We are using assistants in a complex and collaborative scenario where tools hierarchically invoke other assistants, and the “team” works together to find answers to the initial query by querying various databases and data sources.
Our main issue is performance. An interaction between 2-3 agents, involving in_progress and queued statuses, along with multiple tool invocations, takes several minutes—even for simple questions. In reality, only 2 or 3 database queries are executed, but the entire process still takes a significant amount of time.
Beyond parallel tool calls, are there any best practices to speed up runs, or do we have to accept the current technological limitations?
Here too for two days. I went on full debugging because I did not expected the API to be this slow. Normally it’s like a second. That’s what i’m used to. But now, already two days unusable where for i use it. It’s a bot that can manage a website and spam filter and is part of a group chat. It’s also used a lot to explore during discussions. Now, that became impossible. I sent three messages to OpenAI but only get basic stuff back. Also, their own support bot is facing the same performance issues.
Here too for two days. I went on full debugging because I did not expected the API to be this slow. Normally it’s like a second. That’s what i’m used to. But now, already two days unusable where for i use it. It’s a bot that can manage a website and spam filter and is part of a group chat. It’s also used a lot to explore during discussions. Now, that became impossible. I sent three messages to OpenAI but only get basic stuff back. Also, their own support bot is facing the same performance issues.
Edit: i have three agents active, a few hundred inactive.