Tips for Speeding Up Assistant Responses with Assistants API

API
1

Tips for Speeding Up Assistant Responses with Assistants API
Hi everyone,

I’m currently developing an AI buddy for my task management application, Flagis. The AI assistant utilizes several API calls provided by OpenAI as function calls. However, I’m facing an issue with response times. The assistant takes too long to reply, making it less usable for our application.

From the logs, it seems that the methods openai.beta.threads.runs.createAndPoll and openai.beta.threads.runs.submitToolOutputsAndPoll are the slowest, each taking about 6 seconds. Below is the log for reference:

[11:59:25.087] INFO (@flagis/api/94185): Send chatbot message init
[11:59:25.813] INFO (@flagis/api/94185): Check the existing assistant. (method: openai.beta.assistants.retrieve)
[11:59:26.081] INFO (@flagis/api/94185): Create or use the existing thread. (method: openai.beta.threads.create)
[11:59:26.386] INFO (@flagis/api/94185): Add the message to the thread. (method: openai.beta.threads.messages.create)
[11:59:32.379] INFO (@flagis/api/94185): Create and poll run. (method: openai.beta.threads.runs.createAndPoll)
[11:59:32.380] INFO (@flagis/api/94185): Init required action on BE
[11:59:32.380] INFO (@flagis/api/94185): Creating task
[11:59:32.389] INFO (@flagis/api/94185): Complete required action on BE
[11:59:38.228] INFO (@flagis/api/94185): Tool outputs submitted successfully to openAI. (method: openai.beta.threads.runs.submitToolOutputsAndPoll)
[11:59:38.493] INFO (@flagis/api/94185): Get the message from openAI and send it to the FE.

Has anyone encountered similar issues or have any tips on how to optimize these calls for better performance? Any help would be greatly appreciated!

Thanks in advance!

Jiri

1 Like
2

Have the same issue with the submitToolOutputsAndPoll() function. Wondering if there’s anything to speed it up.

3

Honestly, I’m happy with anything under 60 seconds all things considered! :wink:

Seriously, though, six seconds is really good, and I don’t see it improving that much with Assistants. What you could look into is writing your own code to do the same things… it’s harder to do but gives you more control.