Forgive the lack of knowledge, but would it be possible to set up an assistant “team” with different roles, and add each one of them individually as separate function calls to the master assistant? This would run off of the function description to choose the use case of when to use that particular function/assistant.
Thanks!
Yes. I have an application that uses a main “agent” to decide what “sub-agent” to use. The system message is something like “You are a helpful assistant that decides which agent to use for a specified task. The list of agents include [list of agent names with descriptions of what they do]”. Based on the response from the main agent (which is using function calling with a single function ‘choose_agent’), I then pass the user prompt to the appropriate sub-agent which has it’s own function descriptions.
I don’t see why not.
When you’re using the API you can do just about anything you want.
An assistant isn’t going to know or care what produces the response to their function call.
My main concern would be possibly timing out if you had a long chain of expert assistants.
Once you have more than 3 or 4 you’d probably want some kind of mechanical for keeping track of your depth.
But, it’s a very interesting idea to get around things like assistant file limits. Having several narrowly focused assistants will likely yield better results too.
You could even have your assistant reviewing call itself too.
Awesome, just to be clear, you’re saying this could be executed in the playground, using the new assistants feature, right?
I think, how to org-chart the assistants will not be so straightforeward. My naieve approach is to drill down vertically. But rigid top-down stuff is outperformed by more organic or emergent pathways (Yahoo vs. Google search)
I wonder if the next startup space will be assistant routing design and in six months OpenAI will announce their new feature and pull everybodys rug 
Theoretically you could in the playground each time call a different assistant manually based on the function call. Not super fun, but doable.
Would it be possible to ask for implementation details for this method?
Thanks.
My implementation is a Flask app where the user can drive the UI through natural language. My agents are the top-level “decider” agent, and sub-agents that are meant to handle different UI interactions (think panning and zooming vs. editing a feature in the view in some kind of graphics program, for example).
All prompts from a user are sent to an /ask route, which calls the listen() function of the decider agent. This agent class holds a reference to an OpenAI client and has tools and a system_message defined as I mentioned in my earlier post. This agent sends a request to OpenAI to choose the agent by returning the agent name.
For each possible agent response, I have other “agent” Flask routes defined. Client side (i.e. in JavaScript), I take the agent name response from the /ask route and based on what it returns, send it to the appropriate agent route. These routes call their respective listen() functions of the agent classes, which have thier own reference to the OpenAI client, system messages, and tools defined that take the appropriate action the user requested. I then take the JSON response (every agent defined uses function calling) and act on it in the UI.
Here’s an example decider agent class as described:
import json
import logging
logger = logging.getLogger(__name__)
class DeciderAgent:
"""A Decider agent that has function descriptions for choosing the appropriate agent for a specified task."""
def __init__(self, client, model_version):
self.model_version = model_version
self.client = client
self.tools = [
{
"type": "function",
"function": {
"name": "choose_agent",
"description": """Chooses an appropriate agent for a given task.""",
"parameters": {
"type": "object",
"properties": {
"agent_name": {
"type": "string",
"description": "The name of the agent to choose. One of 'NavigationAgent', 'EditAgent'.",
},
},
"required": ["agent_name"],
},
},
},
]
self.system_message = """You are a helpful assistant that decides which agent to use for a specified task related to navigating a graphics program and editing objects in the program.
For tasks related to navigating the program, such as panning, zooming, or rotating the view, you will use the NavigationAgent.
Example NavigationAgent prompts include 'pan left some', 'zoom in', 'pan up 5 units', 'zoom in two times', 'rotate the view 23 degrees clockwise', and 'tilt the view 45 degrees'
For tasks that edit an object, such as changing the opacity, color, or size of an object, you will use the EditAgent.
Example EditAgent prompts include 'make the object harder to see', 'change color to green', 'opacity 45%', and 'move the selected object 4 units to the left'.
"""
#initialize the messages queue with the system message
self.messages = [{"role": "system", "content": self.system_message}]
self.available_functions = {
"choose_agent": self.choose_agent,
}
def choose_agent(self, agent_name):
return {"name": "choose_agent", "agent_name": agent_name}
def listen(self, message):
logger.info(f"In DeciderAgent.listen()...message is: {message}")
"""Listen to a message from the user."""
self.messages.append({
"role": "user",
"content": message,
})
# this will be the function the model will choose if it
# determines that the user wants to call a function
function_response = None
try:
response = self.client.chat.completions.create(
model=self.model_version,
messages=self.messages,
tools=self.tools,
tool_choice={"type": "function", "function": {"name": "choose_agent"}},
)
response_message = response.choices[0].message
tool_calls = response_message.tool_calls
if tool_calls:
available_functions = self.available_functions
self.messages.append(response_message)
for tool_call in tool_calls:
function_name = tool_call.function.name
function_to_call = available_functions[function_name]
function_args = json.loads(tool_call.function.arguments)
function_response = function_to_call(**function_args)
self.messages.append(
{
"tool_call_id": tool_call.id,
"role": "tool",
"name": function_name,
"content": json.dumps(function_response),
}
)
logger.info(f"Sucessful DeciderAgent task completion: {function_response}")
return {"response": function_response}
except Exception as e:
return {"error": "Failed to get response from OpenAI in DeciderAgent: " + str(e)}, 500
return {"response": function_response}
Let me know if you have any questions. I’ll do my best to answer with the time I have.
The problem I am having with this approach is that I am trying to use an assistant to decide which secondary assistant to call, but the response from the decider assistant isn’t always clean, i.e. it doesn’t always just pass back the name of the next assistant which would be best suited to handle the user query.
How does choose_agent accomplish that?
Thank you for getting back to me.
Edward