Hey there. I’m using OpenAI’s Answers endpoint and I’d love to understand what are the most important factors influencing the time it takes me to get a response. Currently it’s around 15s for me and I’d love to try reducing it.
Hi @stan!
Welcome to the forum! Can you provide any details on the # of documents in your file + average size of the documents?
Thanks for your quick response, @tabarak !
For now it’s just around 30 documents, ~100-300 words each.
@stan, The response speed depends on a variety of reasons including which engine you’re using, max_tokens and the size of each document. Using ada for search_model and curie/babbage for model can also help with speed.
1 Like