100GB File Storage Limitation is Problematic

Hello. I’ve spent the last 2 or 3 weeks diving into the Assistants API with the idea of creating a multi user document assistant app.

I am now realizing that the 100GB file storage limit is a showstopper for such an app because hundreds if not thousands of users will all be creating many assistants for their documents which range in size from kilobytes up to a gig. If the product is successful, the storage limit would quickly be exceeded and the app would cease to function which leads to irate customers, etc.

Perhaps instead of a hard 100GB limit, we could pay for any amount beyond that? The hard limit of 100GB is making me hesitant to proceed with development. Any chance of a future pay-as-you-go model for file storage?

3 Likes

Has anyone figured out the correct way to handle this? A 100GB raw file limit doesn’t seem reasonable for a scalable RAG system. This constraint wouldn’t be practical even for a small-scale production app.

From the Files API Reference page, referring to organization about storage:

Individual files can be up to 512 MB, and the size of all files uploaded by one organization can be up to 100 GB. Please contact us if you need to increase these storage limits.

From the Assistants documentation page, referring to both vector store usage and vision on the files endpoint:

By default, the size of all the files uploaded in your project cannot exceed 100 GB, but you can reach out to our support team to increase this limit.

Reaching out is sending a message through “help” on the platform site. The correct way. Glad to have figured it out for you.

Paying $5 into credits gets you 100GB. So does $1000+, months or years, and tier 5…so you do have an interesting point about scaling. The typical production app is chatbots, though, the only way you could talk to a vector store a week ago, and that gives you 100MB dynamically per each of your thousand customers to service them with 200 million tokens. And a vector storage bill for resulting 200GB = $20 a day. They’d probably up your limit when you are buying a new 1TB SSD a week in billings of idle data.