Best practices for retrying requests

What are the best practices for retrying completion requests? When I occasionally encounter “That model is currently overloaded with other request” I would like to back off and try the same request again in a few seconds. Does OpenAI python library use a standard requests library that includes a retry function?

1 Like

At the moment we haven’t implemented this, but it’s a good idea, and I’ll add it to our backlog.

When using the API for processing batch jobs which shouldn’t fail I normally implement a retry logic which waits 1 second on first failure, and then a time doubling on each subsequent failure up to some maximum.