We are experimenting with GPT-4 and are really impressed by the results. They are so much better for our usecase.
But there is one issue: We keep getting ReadTimeouts from the API. The Timeout is already set to 15 seconds, we retry 2 more times, but still, more than 50% (probably more like 75%) of the requests (not counting retries) fail with a ReadTimeout.
Anyone else has the same problem? Anything we can do?
It is kinda similar on 3.5-turbo, but there the increased timeout and the retries mitigate the problem somewhat…
We are using python with httpx