Embeddings vs finetunes

The project is an “expert” bot. I’ve got a guideline document that the bot is supposed to answer questions about. I’ve created embeddings for the document, and I embed new questions, and compare and then when the comparison is done, I return the section of the document to davinci (or more precisely, to text-davinci-002) with the question. This works pretty well. The concern is the number of tokens that this uses, feeding entire sections of the document to answer a single question. Is it possible, via finetunes, to get the model to remember the document? I did some testing and my results, again, were not great. But what I’d like is to put the sections into finetunes, and then say “based on section 2, answer this question.” where ‘section 2’ is arrived at by comparing question embeddings against my locally saved document embeddings, without having to send section 2 to the API every time a relevant question is asked. Will this work?

1 Like

Sounds like you have created embeddings for each section of your doc, right?
That’s a vector search and you might have better results at no API costs by using sBERT or something.

I’m not aware of an effective way to teach GPT-3 the content of a particular document (or a set of documents) through fine tuning. At least my experiments to do so failed and I ended up with a vector search using sBerts sentence embeddings.
Check out Semantic Search — Sentence-Transformers documentation

This is exactly what I’m doing. Thanks for the link!