We have created a solution where users can upload their PDFs and ask questions. We have used NodeJS, Langchain, and OpenAI. Currently, the app flow is we save all the content of PDF in our vector database as embeddings and upon the question asked we use these embeddings and openAI to generate a response for the user. Now I want to retrain my LLM based on questions asked by users and their responses so my model on Openai gets more mature.
How can we achieve that?
Thanks in advance.
If you’re using GPT-3.5, you can finetune the model in the API by providing enough examples formatted in JSON in a way that matches the example in the documentation. It’ll probably cost you a few dollars.
However, finetuning for GPT-4 models is currently limited to very few users (you get access after doing a lot of GPT-3.5 finetuning and applying through a form explaining your use-case).
I understand the flow, let me share the flow of our app.
The user created a group and added some learning material using a pdf. On the backend, we extract the content from the uploaded PDF, make its embeddings and save it to our vector-based database. Now the learners of this group can ask questions from that learning material. Once user asks a question we get all the embeddings of that learning material from our database and pass it to openAI, with the question asked. To get the response.
Now my question is How we will finetune our model in this flow. Since we pass the embeddings on run time to the openai model