I’ve noticed that the performance and reliability of the
code-davinci-002 model has deteriorated quite a bit lately. API calls seem to be much slower and I’m quite often getting these two errors:
openai.error.RateLimitError: That model is currently overloaded with other requests
openai.error.APIError: The server experienced an error while processing your request
Both with the python API. I also see similar errors using the HTTP API directly from C#.
Granted I’m using a lot of tokens in my queries (since fine-tuning is not available on codex yet) and am right up against the limit, so it could be related and maybe why I haven’t found this reported much lately on here. However, using the same queries even a few weeks ago was much more reliable and performant, so it feels like something has changed (since ChatGPT release maybe?).
Are other people having this problem? Is there maybe something on my end I can try?