I have watched this very excellent video on Fine-Tuning vs. Embedding OpenAI Q&A: Finetuning GPT-3 vs Semantic Search - which to use, when, and why? - YouTube, as well as a lot of other posts and videos, and believe I have a good understanding of why, in my case, I definitely need to use embeddings.
However, one aspect of this subject I have not seen addressed is: Is there any advantage to using both?
Let’s say you have a dataset that you both embed and additionally use to fine-tune a model. A user inputs a query. You run a vector search against the embedded dataset and return the top_k hits. You then submit the query, along with the vector search text results as context, to your fine-tuned model as a prompt. Would the completion in this scenario be better than submitting the same prompt to a non fine-tuned model?