Assistants/File Retrieval for a Production App

API
1

Hello,

We just crossed 90,000 users and I’ve been developing night and day to implement some pretty cool features with the help of assistants.

I have a question about best practices on a production app when it comes to assistants and file retrieval. I am still building my understanding of the pricing model for assistants/file retrieval and would love to know more about whether or not the scenario below would cause issues from a billing standpoint.

We expect each new user to upload at least 1 form of a PDF that is relevant to their use case. Over the course of 12 months, this would lead to 100,000s of assistants being created through my OpenAI account.

Is this a problem? Or should we store the files and save the details of an assistant API call in our database and then create + delete the assistant each and every time the user wants to use it?

I am sorry if my question does not make sense, but I suppose it is because I am struggling to understand things myself as stated. I know that each assistant will have a $.20/day/GB fee. I expect most of the assistants my users create to only have 5-50MB worth of data, but perhaps this will still cost thousands of dollars per day which is not feasible of course.

At the end of the day, I’m confident that we’ll find a solution on our end to take advantage of this awesome software you’re giving the world. However any clarity is much appreciated.

Thank you and keep up the great work OpenAI team (and app/plugin devs :slight_smile: )!

Sincerely

2

Hi Paul

The best practice on a production app is NOT to use Assistants. They are in beta and highly unstable and uncontrollable so far. They are great as… assistants, but not for production quality apps.

3

You would certainly not want to create assistant-per-user. A single file that is common to all, such as your own service’s knowledge, would balloon out of control in costs with attached documents, let alone individual documents accessed again never but needing to be retained for quality.

Instead, you can look at the process of attaching files that have been uploaded to particular messages in a thread. https://platform.openai.com/docs/api-reference/messages/createMessage

The costs in general are impenetrable and not reported to you. Saying “hi” when carrying retrieval or long chat history can cost $1+.

4

Iirc, the current openai documentation for assistants says they are not yet ready for production.

1 Like