8-12 Seconds Response Delay with OpenAI API Using Node.js and WhatsApp API

Hi community,

I’m working on an integration that uses the OpenAI Assistants API along with the WhatsApp API. The flow is as follows:

  1. A user sends a query via WhatsApp.

  2. The message is processed using Node.js, sent to the OpenAI API, and the response is returned to the user via WhatsApp.

The issue is that I’m experiencing significant delays during the “run” phase of the OpenAI API, with response times ranging between 8 and 12 seconds, which is negatively impacting the user experience.

Technical Details:

Backend: Node.js

Integration: WhatsApp API

Problem: The majority of the delay appears to happen during interaction with the OpenAI API.

Questions:

  1. Is this response time normal for the “run” option in the OpenAI API?

  2. Are there any configurations or best practices to reduce this delay?

  3. Could this be related to server load on OpenAI’s side?

Any guidance or shared experiences would be greatly appreciated. Thanks for your help!

Normal, unfortunately. See my post I made 2 days ago sharing graphical results of stress testing I did on assistants/thread model. Run time varied between 10-40 seconds… Not at all viable for a production app.

1 Like

And it was kind of worse some 1-2 years ago but hasn’t really improved in terms of response time. Like I wrote in that same thread. Azure deployment of the same (openAI) models has way better performance. It might be the downside of the cooperation between openAI and microsoft or you need to shift to enterprise of openAI to experience the real speed. I mean I see applications that answer instantly and they clearly use OpenAI models.

Has nothing to do with the consuming programming language / environment though… If you experience differences between python or nodejs or even PHP implementations and they are your problem than congratulation on having great API response time…

1 Like