Throttling / Slow Response Times for Users

API
1

Apologies if this post seems redundant, but just wanted to show how users are being individually throttled with A/B testing:

  • Friend lives in same geographic region, same settings, same paid subscription
  • Max tokens set at 750, response is typically <300 tokens
  • Running a request with my API key takes >60 seconds. Running the same request with friend’s API key takes <10 seconds
  • Another funny bug, going to openai.com/api-keys, “Last Used” has not been updated since October 5 (>1 month ago) even when the keys are used. The current key that I’ve tested hundreds of times still says it has “never” been used.
  • Reaching out to OpenAI, they pretty much said to go pound sand
  • THIS IS TOTALLY HYPOTHETICAL. I DID NOT USE SOMEBODY ELSE’S API KEY BECAUSE YOU’RE NOT SUPPOSED TO DO THAT

But if I did use my friend’s API key, I would have pretty solid proof that some users are just getting throttled and there’s nothing you can do about it. If anyone has a solution, I’m all ears because I’d like to actually utilize what I’m paying for.

1 Like
2

The same thing is happening to me with the keys – it’s saying my key has never been used even though I’ve been using it all month.

I’m getting very long 10-minute timeout errors very often. Would like to know if anyone else has a solution as well…

This is the error we’re getting (we are running a series of async calls to OPENAI using langchain as a wrapper):

langchain.llms.base - Retrying langchain.chat_models.openai.acompletion_with_retry.<locals>._completion_with_retry in 4.0 seconds as it raised Timeout: Request timed out.
1 Like
3

I have the same problem. API response time is consistenly more than 30 seconds for even small chats