I want to embed a data file using a external embedding software… Is it possible? if yes, can I use other models like davinci, instead of ada with it
Calculating embeddings can be done with whatever software you want.
Using the OpenAI embedding support and retriever requires using their model, but if you do your own embedding and retrieving, you can use whatever database you want (pgvector, pinecone, faiss, …)
So, either you use the built-in support in OpenAI, and let them choose the models, or you roll your own, and then you can use whatever you want!
Once you have the retrieved documents, you can formulate them into a prompt, and use the OpenAI completion models based on that context, no matter where it comes from.
1 Like
im fairly new to this context so… here are some things I need clarification on…
“retrieved documents”: what does this mean…? the result of the embedding process?
How to formulate them into a prompt?
Sry for all these questions… I searched far and wide for this but still couldn’t find anything helpful
Any thing I got was basically re-embedding every restart… and yea…
“embedding process” means “calculate embedding vectors for each document (or fragments of documents) and put them in an index, mapping vector to document reference.” That index is typically some kind of persistent database – Postgres with pgvector, Pinecone, FAISS, …
“retrieval” means “calculate embedding of the topic you want to answer, then find the N nearest document-vectors in the index, and load the documents referenced by the references keyed by those vectors.”
Prompt construction is then done by simply concatenating each of the documents, with some prefix such as “\nDocument {title}:\n{text}”
“Document” here typically means something small, and text format. With the ADA model, you can jam up to 8192 tokens into the embedder; other models may have smaller context. In general, though, you don’t want documents that large, instead you want to create chunks that are maybe 1kB to 3 kB in size (sections of documents, if the documents are larger) so that only the most relevant chunks are provided in the prompt.
Google for “retrieval augmented generation” and you get several hits for tutorials and think pieces.
1 Like