For 3.5 development (Not sure about 4), the Azure backend is very fast and responsive. I imagine they have dedicated resources around it that are not being as heavily used than OpenAI direct API.
They do have dedicated cluster(s) which follows cloud security compliance. These are not trivial to attain and it’s why large enterprises and military trust Azure with their data.
They don’t provide a response time SLA.
What are the SLAs for API responses in Azure OpenAI?
We don’t have a defined API response time Service Level Agreement (SLA) at this time. The overall SLA for Azure OpenAI Service is the same as for other Azure Cognitive Services. For more information, see the Cognitive Services section of the Service Level Agreements (SLA) for Online Services page.
That said, these folks live and breathe reliability. Cloud ops are in their DNA. It wouldn’t surprise me if their services were more resilient.
1 Like