Best GPT response time for real-time applications

christophe.gouet · 2023-09-22T12:50:28.996Z

Hi,

My team measured Davinci and Curie response time during the day to understand the occasional long response times we get in our application (tested with 1 token prompt):

image2317×651 106 KB

We need an answer time under 2 seconds, so here are my questions:

Are these peaks due to queuing?
Is it possible to buy like “exclusive endpoints” or something to get a better response time and stability?

Thanks!

Foxabilo · 2023-09-22T13:35:23.994Z

Hi and welcome to the Developer Forum!

You can indeed get a dedicated instance, they start to make commercial sense if you are using ~450M tokens per day, you can reach out to get more information here

openai.com

Contact sales

We’re happy to answer questions and get you acquainted with OpenAI, including connecting you with helpful resources, exploring use cases for your team, and discussing packaging options.