Big problem with usage limits while using API

Hello,

Our company has a business account on the OpenAI platform, and we use the ChatGPT API to generate relevant keywords related to a list of services we offer. Our account is currently in “Tier-2” status. According to the official OpenAI documentation (https://platform.openai.com/docs/guides/rate-limits/usage-tiers?context=tier-two), a “Tier-2” account can make 5000 requests per minute and use 2,000,000 tokens per minute with the 3.5-turbo model. However, this is not our experience.

We have code that iterates over our list of services, generates a prompt no larger than 3000 tokens for each service, and makes a query through the ChatGPT API. Despite this, every time we make 10 requests (far from the 5000 allowed per minute), the API stops working for a certain period and returns no results. The tokens used in one minute are below 30,000, which is also far from the 2,000,000 limit.

We also tried the 4о and 4о-mini models, but the result was the same. Why does this happen? The official documentation’s limits seem misleading. We pay a lot for this service, but it is completely useless in practice!

Welcome to the Forum!

Does the API request return an actual error? If so, what is the error code? If there is no error code, what do you mean by “no response”?

Have you verified/validated the actual token consumption via the API response?

Does the API request return an actual error? If so, what is the error code? If there is no error code, what do you mean by “no response”?

Yes, the response is:

status code: 400, reason phrase: Bad Request

Have you verified/validated the actual token consumption via the API response?

Yes.

Ok, that sounds like the problem is not related to exceeding your rate limits.

For those API requests where this error is returned, have you ensured that the request is properly formatted and contains all the right parameters?

1 Like

Yes. Our code iterates a list with services and generate a prompt for each service and send it to the API. Everything is okay with the first 10 request, but after that the API start responding with bad request.

Normally you should see a more detailed error message, indicating what has led to the bad request. In any case, I would investigate the specific calls that fail.

You can also take a look at the more detailed error code guidance on the developer platform: https://platform.openai.com/docs/guides/error-codes/api-errors

There is no problem with the requests.
I can confirm that if we execute the first 10 request (with the first 10 services from our list) and wait for example for ~10-15 minutes and start the code again form 11th to 20th services from the same list - it works. For some unknown reason the API block us for ~10-15 minutes per 10 request. Not 3, 5 or 7 request. Always on 10th.

As suggested, try to locate the specific error message you are getting for the failed requests as this would be the most straightforward way to narrow down the issue.

You may also try to test a different API key and see if the issue persists or perhaps add a short break after the 10th request before proceeding.

Welcome @kostadin

This looks like it has something to do with how you’re consuming the API rather than the API itself.

It’ll be much helpful if you share more details like the code that’s consuming the API.

1 Like

Thanks for all the replays!
After deep code investigation I found that for unknown for me reason the “chat history” in our AI assistant was enabled. This makes our prompt to grow exponentially per each request…Problem solved!

2 Likes

Glad to hear it’s sorted!

1 Like