Retrieval pricing for API assistant is $0.20/GB/day therefore an assistant with 10 files each files with a size less than 1 GB pays 0.20 x 10 x 30 = $60 per month. Is my calculation correct?
It seems to me that something is missing in the information given, I cannot believe that maintaining an assistant is so expensive. Or may I misunderstood.
More or less.
The pricing is on total storage not on some per-file rounded up.
So, if you have 10GB of files that would be $2/day.
You need to remember though, you’re not just paying to store the files. OpenAI is using them for retrieval, so they’re also (presumably) doing embeddings with them and also needing to do semantic search to perform the retrieval, and possibly other things as well.
You’re correct in thinking that would be incredibly expensive for just raw data storage, but that’s not really what it is.
Your calculation regarding the maintenance cost of an API assistant seems correct based on the information you have provided: $0.20 per GB per day for 10 files of less than 1 GB each would result in $60 per month.
However, it’s important to consider the value and scale of the stored data. Storing 10 GB of textual information is indeed a significant amount. To put it into perspective, it’s approximately equivalent to 1/20 of the total volume of Wikipedia or about 400,000 pages of text. This volume of information represents a vast reserve of knowledge and data.
Thanks for your answers. I understand that what is paid is the use of files not the storage. It means that if the bot assistant does not use the attached files therefore there is no expense. However I noted that each time I send a question to the bot assistant it tries always to get the answer by reading the attached files. I have to explicitly ask to look for the answer elsewhere when the question is not related with the attached files. Are you experiencing the same behaviour?
You read the replies and the documentation wrong. When files are simply attached to an assistant definition, you are billed daily for storage by file size. And billed again for each unique assistant ID.
Retrieval always injects a large amount of documentation into the AI context, all that will fit, regardless of question relevance, and regardless of how much you want to pay to have irrelevant documents loaded.
The AI also must perform searches to discover if files have anything relevant - even if it is the same file that was completely loaded into context. Retrieval function of assistants is…poor. Retrieval documents can warp the mind of the AI and affect its performance in other tasks.
1 Like
You are charged for each file held by an Assistant per day.
You are also charged for files held in a Message per day (excluding the Assistant-held files).
The storage fee wasn’t expensive a couple weeks ago, but with recent improvements and optimizations it’s now very expensive.
So for example I have a Triage Assistant, a Evaluation Assistant, and a Conversation Assistant. All 3 use the same knowledge files, different functions. I have to pay 3x the storage fees. On the other hand I can use a single DB.
Even without the storage fee retrieval is expensive and insufficient for production unless you are looking for very simple and small knowledge look-up.
In the sense of improving, changing, and optimizing I would consider this even more limiting than LangChain. Atleast with LangChain it’s open source and offers the potential to understand the underlying mechanics.
One can easily self-host a Vector DB, or even take advantage of free storage offers: pay nothing, have lots of opportunity to tinker, and have a much deeper understanding of the underlying process.
To be fair: It seems like their focus is/has been on embeddings (hell yeah) & retrieval may see some badass updates soon.
1 Like
Finally, the worst scenario is the one I was suspecting and fearing. Assistants are definitively a risky adventure. In any case I read in other threads that this service is free up to the end of the year 2024. Do you all agree or there are other hidden costs?
The latest pricing page mentions as $0.20 / GB / assistant / day (free until 04/01/2024), however I am still not seeing any Retrieval cost under our account.
Is it in effect?