My Chatbot Conversation API hangs/Freezes Randomly with no error

API
, ,
1

We haave a chatbot that works on a custom knowledge base. It uses embeddings and vector database as a core.

Model: gpt3.5-turbo

Issue: While conversing with the chatbot OpenAI stops responding and it freezes the conversation.
Please suggest a possible solution for the same.

Thanks

1 Like
2

The root of the problem could have countless causes. A few pointers.

  • put in a time.sleep(.5) so the client connection is not rapidly called in succession.
  • use a timeout watchdog so that a non-response throws an error and can be retried. This is better with streaming and a timer that can be reset by every chunk.
1 Like
3

Thanks for the response.
Let me try this out and check.