I wanted to know how vendors manage cost, rate limits and monetization using OpenAI as part of their service offerings.
I’m working on building a SaaS that will provide some AI via OpenAI on the back end, but wondering about this topic.
One example that comes to mind is Supabase. They offer an AI agent that helpss create database artifacts like tables, queries, etc. They use OpenAI for this and yet they offer this with their free tier. Aren’t they still paying OpenAI? And would potentially millions of people on their free tier bankrupt them in OpenAI fees and clobber their allowed rate limits?
I’m curious how this works. I would assume that some vendors either pass on the cost or require customer to bring their own OpenAI key, but it seems that a lot don’t do this.
Any information or links on this topic would be appreciated.
Long answer: Oftentimes, the price of a monthly subscription can balance out costs associated with free tier use. Many free uses of API calls also set their own rate limits.
Finally, businesses can just bill to lines of business credit.
Aside from that, this is typically what business plans are for. Each business is unique and has its own business model for making money.
Are the limits either generous enough to allow a SaaS to comfortably serve say 1,000s or customer queries at a time or are there “premium” plans available for this use case?
The “Venture Capital” part of your answer makes sense as far as cost is concerned but doesn’t address the rate limit part. It would not be a good end user experience for some of our customers to encounter rate limit errors.