Building the Ultimate Chatbot: What Do You Think of My Strategy?

[jonah54]

@linus

I think with the current specifications your project will be difficult. There is (still) no possibility to train GPT 3.5 Turbo. Accordingly, you have to transfer all the questions you want to match with your answer to the model with every request. I have made the mistake in the past to not include this in my conceptual planning, so I hope I can help you by sharing my prior mistakes

If you find a low-token solution I would suggest a backend iterative procedure (added advantage that you do not have to build seperate solutions outside of the API):

1. you train a low token model to perform the matching between user question and stored answer. As a result of this process step you get your answer. This answer is NOT displayed to the user.
2. You transmit the user’s query and the determined answer to a model with higher costs. You submit the information with a request to answer the user’s question with the information provided from 1. You address any unanswered components of the question with a reference to your service desk.

I hope this helps you in your journey and all the luck in your undertaking,
Linus

Hi @linus ,

Thank you for sharing your insights and for the suggestions on how to make the project work. I appreciate your advice on the low-token solution and backend iterative procedure.

I was also wondering if we could potentially use embedding with ada to search the database for the most matching question/answer pairing and make the answer part of the prompt that is sent to GPT 3.5 API as context. This way, we could avoid transferring all the questions we want to match with our answer to the model with every request.

Something similar to this video towards the end: https://www.youtube.com/watch?v=EYRWYwbWrTc&t=87s

Do you think this could be a viable solution? I would love to hear your thoughts on this. Also, I am not sure if having a huge database of 500+ pairings to search through using embedding will increase token usage - in theory not because only inputs and outputs use tokens right?

Thanks again for your help and support!