I have been using the OpenAI API for a while but as I am now scaling and improving my application it leads to me using more API calls and more complex JSON function_calling structures.
This resulted in me getting continuous 502 and 503 errors.
Models I am using: gpt-4-0613 or gpt-3.5-turbo-16k-0613
My responses are quite big as I am generating a big and complex JSON structure with it, but still lower than the 8k context for gpt4 or 16k context for the gpt3 turbo API call.
So technically, that should work. I have been trying out everything, from using different models, to splitting my one API call into a batch of 3 shorter API calls but I still keep getting 502 and 503 errors.
The issue is it is very unpredictable as sometimes it works fine but other times it just does not work at all… I am a paid API user and have been for a little while now so that should not be the issue.
Has anyone else had to deal with something like this? I found some other posts on here but nothing that was really applicable to my use case.
Yesterday around night everything seemed to be working fine using gpt3.5-turbo-16k, but today morning again I keep getting 503 errors constantly.
When using GPT4 these errors mostly go away. But of course this costs me 10x more to generate basically the same thing. Anyone has an idea of why this is, a workaround or anything I can do to get this fixed?
Also noticed GPT4 response time is much faster than GPT3.5…