@daveshapautomator I’m firing off 100 requests in parallel, 99 of them come back within a few seconds, 1 takes over 30 seconds. I want my app to gracefully kill long-running requests so i can return the 99 successful responses without waiting for the long-running thread.
doh just realized that there is a timeout parameter in the Python binding
Signature: openai.Completion.create(*args, timeout=None, **kwargs)
Create a new instance of this model.
timeout (float): the number of seconds to wait on the promise returned by the API, where 0 means wait forever.