What are the best practices for retrying completion requests? When I occasionally encounter “That model is currently overloaded with other request” I would like to back off and try the same request again in a few seconds. Does OpenAI python library use a standard requests library that includes a retry function?
1 Like
boris
2
At the moment we haven’t implemented this, but it’s a good idea, and I’ll add it to our backlog.
When using the API for processing batch jobs which shouldn’t fail I normally implement a retry logic which waits 1 second on first failure, and then a time doubling on each subsequent failure up to some maximum.
I have developed a simple Python package which provides retry logic around the OpenAI API. For some reason I am not allowed to provide a link, but the package is called openai-pygenerator and can be installed using pip install openai-pygenerator. Search for it on github to view the README.
