Usage Limits for SaaS - Go Above Tier 5

API
1

Hi,

We are developing a SaaS and it’s looking like it might take off quite quickly. We have wrapped Open AI models around it and deployed heavy guns like GPT 4 Turbo.

Our concern? $10,000 monthly usage limit (tier 5). I think we are going to smash this within the first month and then what? How do we scale with Open AI? Is it even possible or shall we look to establish custom models?

If anyone here managed to go above that limit or has contacts with someone who could get us much higher usage limits, I would appreciate it.

2 Likes
3

Yes, we did consumer tests. We will surpass the $10k very quickly and are not sure what to do next to scale.

Is using multiple APIs even an option? Is it within Open AI TOS?

1 Like
4

I’m interested to know what the idea is?

Do you know where your traffic is coming from, have you analyzed that at all?

5

We control the traffic and consumption. Our price plans are based on that.

Let’s say you have 1,000 Subscribers and each of them uses an average of $10 a month of our Open AI Limit. How can we scale that above 1,000 users if Tier 5 is capped at $10k?

6

You can get an Azure account to trigger if the OpenAI one hits the limit, that is one way.

Edit: if you have money it might be easier to talk to an Azure rep as if you’ve got something to bring to the table they’re usually easier to get on the phone. They have vendors that can give you better deals and stuff.

The other way is be very persistent / make the right friends to get an Enterprise account with OpenAI.

If it will be as successful as you think, than maybe even hosting a Llama model which can get as good as between 3.5 and 4 quality. I think chaining the options I’ve suggested definitely increases your bandwidth and ability to compensate for the limits. Hosting a llama model can be expensive so maybe its possible to use it as an absolute fall back?

Without more information on what your app actually does, its hard to suggest the correct solutions. I asked about your traffic because sometimes apps can get a big spike on like TheresAnAIForThat but it just causes a big spike in traffic because they tackle a broad market of people interested in AI, but not necessarily willing to pay for it.

1 Like
7

That is kind of what we are exploring now.

I was also wondering if opening multiple accounts and getting them all to Tier 5 would be against Open AI TOS?

We could potentially handle this with multiple APIs.

8

Yes it is against the TOS.

edit: at least I know you’re not supposed to

10

I am building a vertical SaaS app using the Assistant API and I am planning to launch in the coming weeks (if API speed improves). I am also concerned about hitting the Tier 5 limit. Does anyone know if the latest Assistant API is available through Azure? I just can’t find the information…

1 Like