We haave a chatbot that works on a custom knowledge base. It uses embeddings and vector database as a core.
Issue: While conversing with the chatbot OpenAI stops responding and it freezes the conversation.
Please suggest a possible solution for the same.
The root of the problem could have countless causes. A few pointers.
- put in a time.sleep(.5) so the client connection is not rapidly called in succession.
- use a timeout watchdog so that a non-response throws an error and can be retried. This is better with streaming and a timer that can be reset by every chunk.
Thanks for the response.
Let me try this out and check.