What's the maximum number of concurrent requests allowed?

API
1

I’m using OpenAI official python library and utilizing the rest APi, not a local model.

what’s the maximum number of concurrent requests for

  • chat.completions.create: gpt-4-turbo-preview
  • audio.transcriptions.create: whisperer-1
  • Any code examples?

I can’t use the batch API, I can’t wait for the next day, I’m trying to process data as fast as possible

2

Hi and welcome to the community!

The maximum number of requests are defined in the documentation.

https://platform.openai.com/docs/guides/rate-limits#usage-tiers

Note that there are several limits you might be hitting, RPM and TPM for different reasons.

You can also take a look at the error mitigation part of the same docs to manage cases when the limits are hit.

1 Like