Tips for Speeding Up Assistant Responses with Assistants API

Tips for Speeding Up Assistant Responses with Assistants API
Hi everyone,

I’m currently developing an AI buddy for my task management application, Flagis. The AI assistant utilizes several API calls provided by OpenAI as function calls. However, I’m facing an issue with response times. The assistant takes too long to reply, making it less usable for our application.

From the logs, it seems that the methods openai.beta.threads.runs.createAndPoll and openai.beta.threads.runs.submitToolOutputsAndPoll are the slowest, each taking about 6 seconds. Below is the log for reference:

[11:59:25.087] INFO (@flagis/api/94185): Send chatbot message init
[11:59:25.813] INFO (@flagis/api/94185): Check the existing assistant. (method: openai.beta.assistants.retrieve)
[11:59:26.081] INFO (@flagis/api/94185): Create or use the existing thread. (method: openai.beta.threads.create)
[11:59:26.386] INFO (@flagis/api/94185): Add the message to the thread. (method: openai.beta.threads.messages.create)
[11:59:32.379] INFO (@flagis/api/94185): Create and poll run. (method: openai.beta.threads.runs.createAndPoll)
[11:59:32.380] INFO (@flagis/api/94185): Init required action on BE
[11:59:32.380] INFO (@flagis/api/94185): Creating task
[11:59:32.389] INFO (@flagis/api/94185): Complete required action on BE
[11:59:38.228] INFO (@flagis/api/94185): Tool outputs submitted successfully to openAI. (method: openai.beta.threads.runs.submitToolOutputsAndPoll)
[11:59:38.493] INFO (@flagis/api/94185): Get the message from openAI and send it to the FE.

Has anyone encountered similar issues or have any tips on how to optimize these calls for better performance? Any help would be greatly appreciated!

Thanks in advance!