💬 Training an embedding adapter: adapt embeddings to new context, and boost the performance of RAGs,

API
,
1

Hi all,

I’ve put together a simple package to train an adapter matrix to fine-tune your embeddings to a new context. This includes OpenAI’s embedding models.

You’ll need the embeddings of the query-document pairs, and a label on whether the document is relevant to the query or not.

The idea is to have a simple and familiar api.

from embedding_adapter import EmbeddingAdapter
adapter = EmbeddingAdapter()
adapter.fit(query_embeddings, document_embeddings, labels)
adapter.transform(new_embeddings)

More details in the repo’s README.

Any feedback would be most appreciated :slight_smile:

2

This is awesome! I’ve considered training a small adapter layer after reading a Tweet about doing this and in my search to find that and related research I found your project.

Do you have any data on how this compares with cosine similarity or links to research supporting this technique?