Assistants API Feature Request: support for files as tool outputs

Basically, I’d like to be able to create assistants that can ingest files by using tools rather than by making users upload them directly or by uploading them at assistant-creation-time.

A simple example would be an assistant with access to the file system that would have ls, cd, read_file, etc. tools, with the read_file tool adding a new file to the thread when it is run.

9 Likes

I know I could add files to the assistant when executing the tool, but I’d like to scope the files to the specific thread (as if the user uploaded it in a message).

Agreed, I don’t want to pass back a huge JSON string when it just needs to be loaded into a CSV and used in Code Interpreter

1 Like

Totally agree, was just about to make the exact same post but saw yours. Another thing that I also would find useful is being able to somehow update the tools parameter when required_action.type is “submit_tool_outputs”. Would make it so much easier to implement custom tool retrieval for assistants and guide the workflow.

The solution I’ve found is create a function that relieves the file and when called uploads the file to the client, it responds with the file_id then needs to be passed back and update the assistant with the file_id and then continue the run

2 Likes

@bmorrow35 how did you make the assistant understand that the “tools output” was successful and that he should expect the results as file(s) in following messages to resume his processing?

I can’t figure out which kind of output, function definition or instructions I should put in place for this to work. The assistant always gets confused in one way or another.

I initialize the assistant with an empty file_id parameter , then use the modify assistant function after the tool output and pass the file_id that was just generated. Probably not the most ideal solution, but I have no coding background and this is what I found works. I create a new assistant each time I start a thread. I believe there’s a way to update the file_ids in the run but I already had this coded in when I came across that, so I’ve stuck to it. Will probably try to move to updating the run instead in the future

from openai import OpenAI
client = OpenAI()

my_updated_assistant = client.beta.assistants.update(
“asst_abc123”,
instructions=“You are an HR bot, and you have access to files to answer employee questions about company policies. Always response with info from either of the files.”,
name=“HR Helper”,
tools=[{“type”: “retrieval”}],
model=“gpt-4”,
file_ids=[“file-abc123”, “file-abc456”],
)

print(my_updated_assistant)

1 Like

Ah okay, thanks for the reply @bmorrow35. Restarting the thread or re-creating the assistant is not a viable option for what I’d like to achieve.

It is indeed possible tu upload new files and make reference to them in following runs but I can’t figure out a way to properly set the instructions for the assistant to make the leap of waiting for a following run with new files references to continue his initial processing.

AFAICT this is a serious shortcoming for the “functions-calling” tool, it might be by design or not – only OpenAI could answer that.

Are you deleting the assistant at the end of the action, or just accumulating a bunch of unused assistants? Are essentially ‘containerized assistants’ a thing?

How I enabled files as tool outputs without modifying the assistant:

1- Cancel the run client.beta.threads.runs.cancel when tool is invoked.
2- Introduce a new message to indicate that the tool has been activated, specifying its name and parameters.
3- Follow up with an additional message containing the file_ids , confirming successful execution. Include a description of the file and its schema.
4- Initiate a new run within the thread, incorporating these updates.

5 Likes

been deleting them. have a trigger to delete all assistants and files when i ‘terminate’