This is a report of behavior I’ve noticed with the API which might also explain some of the issues others have seen where there is no response from the API.
I’ve noticed that there are times when I get no response to queries via the API. A short ‘hello’ test might get a response but a longer message returns nothing.
By no response, I mean nothing: no error code and nothing is returned by the model, not even an empty result.
I’ve seen this with both 3.5-turbo and 4-turbo.
In most cases, not long after this I get a ‘server busy’ response. So my assumption is that there is a period immediately prior to the ‘server busy’ notification where the API isn’t able to respond to a query but the ‘server busy’ message hasn’t kicked in.
There might always be this lag between overload and the server returning the ‘busy’ message but it might explain the issues some people are experiencing.