Best practices to handle API outage

API
1

After recent outages of API, I was wondering how to handle it, in order to give our users advance warning that the service is not working or that there could be delays.

I am now thinking in the direction of making repeated calls to API, for example once every 3 minutes, and then write the status somewhere on my side. If it is ok, do nothing, if it is non-responsive for few attempts, show the notice.

Is this good direction to go in? It is there a smarter solution?

2

We use Azure OpenAI as fallback :+1:

1 Like
3

How are you checking if the OpenAI is available in order to switch to Azure? Or do you do it for every call?

4

There is already a unofficial status page.
You can use it as reference, or maybe even “as is”.

Edit: https://openai-status.llm-utils.org/

1 Like