I am seeking some guidance on how to structure my agentic system to have a certain functionality.
Let me give some context about my agents:
Master agent (main orchestrator, calls other agents with “.as_tools”, not handoff)
SQL agent (writes and executes sql queries + has tool to save them as csv files and upload to openai)
Data analysis agent (receives the csv files and uses code interpreter to perform analysis, create charts, and perhaps even output its own summary csv files)
The flow should go something like this:
User asks a question
Master agent instructs sql agent to fetch xyz data from the db.
Sql agent fetches the data, saves to csv, uploads to openai storage and hands the file-id back to master agent.
Master agent hands the file-id to data analysis agent with instructions on what information/analysis is required.
Data analysis agent uses code interpreter with that file to perform the analysis and responds to master agent which relays the answer to the user.
There are two main issues with this, other than there being lots of back and forth between agents:
When the master agent hands the file-id to the data analysis agent, how does it use its code interpreter tool on that file ID if the Data analysis agent was not initialized with that Id in its tool config already (file was created and uploaded in the same run by the sql agent):
If the data analysis agent actually creates a file, how does the master agent retrieve its container id and file id if they are in the annotations part of the response? Do agents have access to that or do they only see the raw string response of the agent its invoking with .as_tool?
If you guys have any guidance for me it would be immensely appreciated!
Not an answer to your questions but, one challenge I’ve encountered when working on something similar is that once the code_interpreter container is defined with its initial set of files, there doesn’t seem to be a straightforward way to update it with new file IDs. As a result, files uploaded to OpenAI during a conversation with users don’t appear to be accessible to an already-initialized code_interpreter.
I did come across the update method for the vector store (as shown in this guide), but I haven’t seen anything equivalent for updating the code_interpreter with new files.
It’s possible I may have missed something in the documentation, so I’d really appreciate any insights you have—or anything you’ve heard from others—on how to handle this.
And gave it to all the agents in my flow who need it. Then before doing Runner.run on my main orchestrator agent, I replace the container id with one from my flask session:
So if any of the agents create a file using code interpreter, it will be available to all the others in the same run because they are all sharing the same container.
The main challenge I have right now is extracting the file-id that the tool creates from RunResult. Been stuck on that for 2 days and am starting to think that the file annotations are not available in the run like they are in the responses api.