Rate_limit_exceeded error when we only do one transaction at a time

API
,
1

We have an API call that has worked flawlessly for months. Today it failed. We make this call one at a time during demos, and I might make a total of 10 per week. But today, as I was in a demo, I received this message:

Request too large for gpt-4o in organization org-xxx on tokens per min (TPM): Limit 30000, Requested 33785.

There is simply no way that I executed 33785 transactions. And checking my balance, where we have deposited $25 over a space of 6 months or so, there is over $10 left. If there had indeed been 33785 transactions, I imagine that the $10 would have been exhausted. So, I think there is a platform glitch? Or is there something else?

2

Hi,

This looks like it’s for an org that has a Tier 1 account twhich has a Tokens Per Min limit of 30k, not transactions per min, to you can only make use of 30000 tokens worth of data per min. So it look slike you tried to use a large prompt and recived that message.

1 Like
3

Thanks for the feedback. maybe I don’t understand what a token is in this regard. I do have a sizable instructions string, and a modest content string, and the return is typically medium in size. We do these exact requests quite often. Maybe if I understood what the 33785 was measuring I could make some adjustments. Any suggestions?

1 Like
4

So, with typical english words 30,000 tokens is roughly 22,500 words. One token is very roughly 0.75 English words.

The next Tier up (Tier 2) has a limit of 450,000 tokens per min, so quite a jump up. YOu achive Tier 2 by buying a total of $50 of API credits and have made your first payment at least 7 days ago.

1 Like
5

Oh! Then there is a problem. One other question… is the limit for what I ask, or for what I ask and what it is about to return to me? My sizable instruction string is only 180 words or so. And everything else is less than that. If it doesn’t include what is returned in the limit, there MUST be a platform bug if they think I am sending in 22k words.

6

Are you using Vision at all maybe?

7

Good point, Paul, also are you making use of assistants with stored documents?

9

Thanks Paul and Foxalabs. I do have stored documents for the assistant. I didn’t realize they would be counted in each transaction’s egress/ingress accounting. That isn’t really intuitive in how this service is explained. I don’t know what “Vision” is, so I am likely not using it. Just vanilla API calls. I increased my deposit to $50 to qualify for tier 2 and the issue went away. But it’s still interesting that it worked flawlessly for months and then all of a sudden hit this limit for ALL calls. It’s like openai changed their rate structure or how they do their accounting or something.