Response speed for Answers endpoint

Hey there. I’m using OpenAI’s Answers endpoint and I’d love to understand what are the most important factors influencing the time it takes me to get a response. Currently it’s around 15s for me and I’d love to try reducing it.

Hi @stan!

Welcome to the forum! Can you provide any details on the # of documents in your file + average size of the documents?

Thanks for your quick response, @tabarak !

For now it’s just around 30 documents, ~100-300 words each.

@stan, The response speed depends on a variety of reasons including which engine you’re using, max_tokens and the size of each document. Using ada for search_model and curie/babbage for model can also help with speed.

1 Like