Our company has a business account on the OpenAI platform, and we use the ChatGPT API to generate relevant keywords related to a list of services we offer. Our account is currently in “Tier-2” status. According to the official OpenAI documentation (https://platform.openai.com/docs/guides/rate-limits/usage-tiers?context=tier-two), a “Tier-2” account can make 5000 requests per minute and use 2,000,000 tokens per minute with the 3.5-turbo model. However, this is not our experience.
We have code that iterates over our list of services, generates a prompt no larger than 3000 tokens for each service, and makes a query through the ChatGPT API. Despite this, every time we make 10 requests (far from the 5000 allowed per minute), the API stops working for a certain period and returns no results. The tokens used in one minute are below 30,000, which is also far from the 2,000,000 limit.
We also tried the 4о and 4о-mini models, but the result was the same. Why does this happen? The official documentation’s limits seem misleading. We pay a lot for this service, but it is completely useless in practice!
Yes. Our code iterates a list with services and generate a prompt for each service and send it to the API. Everything is okay with the first 10 request, but after that the API start responding with bad request.
Normally you should see a more detailed error message, indicating what has led to the bad request. In any case, I would investigate the specific calls that fail.
There is no problem with the requests.
I can confirm that if we execute the first 10 request (with the first 10 services from our list) and wait for example for ~10-15 minutes and start the code again form 11th to 20th services from the same list - it works. For some unknown reason the API block us for ~10-15 minutes per 10 request. Not 3, 5 or 7 request. Always on 10th.
As suggested, try to locate the specific error message you are getting for the failed requests as this would be the most straightforward way to narrow down the issue.
You may also try to test a different API key and see if the issue persists or perhaps add a short break after the 10th request before proceeding.
Thanks for all the replays!
After deep code investigation I found that for unknown for me reason the “chat history” in our AI assistant was enabled. This makes our prompt to grow exponentially per each request…Problem solved!