Fine-Tuning plus Embedding

General API discussion
1

I have watched this very excellent video on Fine-Tuning vs. Embedding OpenAI Q&A: Finetuning GPT-3 vs Semantic Search - which to use, when, and why? - YouTube, as well as a lot of other posts and videos, and believe I have a good understanding of why, in my case, I definitely need to use embeddings.

However, one aspect of this subject I have not seen addressed is: Is there any advantage to using both?

Let’s say you have a dataset that you both embed and additionally use to fine-tune a model. A user inputs a query. You run a vector search against the embedded dataset and return the top_k hits. You then submit the query, along with the vector search text results as context, to your fine-tuned model as a prompt. Would the completion in this scenario be better than submitting the same prompt to a non fine-tuned model?

2

That’s a good video, and good YouTube channel, too.

And there are reasons to use both.

You may not realize it, but if you are using text-davinci-003 or the ChatGPT API, then you are using both. Both of these models were fine-tuned by OpenAI to be more conversational.