Exploring the Intricacies of Fine-Tuning Through Loss Metrics

Hi there!

Taking a step back here: From what it sounds like in your post, you are trying to use fine-tuning as a means to inject knowledge. Is this the correct understanding? If so, then please be mindful that this is not what fine-tuning is intended for. Fine-tuning is commonly used to get the model to behave in a certain way or produce output in a certain style or format.

For your use case, you want to look at a RAG solution, i.e. converting your knowledge base into vector embeddings and then using these to perform semantic search using a distance metric such as cosine similarity to find best matches for a given user query.

With that in mind, here a few OpenAI resources you want to have a look at for additional context and some initial guidance on embeddings-based RAG:

  1. Overview of common use cases for fine-tuning

  2. A new OpenAI guide on “Accuracy Optimization” which brings the different concepts of fine-tuning, RAG and other mechanisms to achieve optimal outcomes together and discusses which mechanism to use for which purposes

  3. Some initial guidance on embeddings including a link to a OpenAI cookbook with a worked example on how to use embeddings for Q&A

1 Like