I’m building a Retrieval-Augmented Generation (RAG) system and I’m trying to choose the right embedding model.
My generation models will likely be one of these:
GPT-5.4 mini / nano
GPT-4.1 mini / nano
I understand that the embedding model and the LLM are separate components in a RAG pipeline, but I would like to know what embedding models provide the best retrieval quality when paired with these models.
Some questions:
Embedding model selection
Which embedding models are currently recommended for high-quality RAG?
Is using OpenAI embedding models the best choice when the generation model is also OpenAI?
Are there open-source embedding models that can outperform hosted embedding models for specific use cases?
Embedding dimension and performance
How much does embedding dimension affect retrieval quality?
Is a larger embedding model always better, or can smaller embeddings provide similar results with lower cost and latency?
Compatibility with GPT models
Does the choice of embedding model significantly impact the performance of GPT-5.4 mini/nano or GPT-4.1 mini/nano?
Are there recommended combinations of embedding model + GPT model for RAG applications?
Practical RAG considerations
For a production RAG system with many documents and frequent queries, what should be prioritized:
Better semantic retrieval quality?
Lower embedding cost?
Faster retrieval latency?
Smaller vector storage size?
Example use case:
Multi-tenant SaaS knowledge base
Documents: FAQs, product information, support documents, policies
Vector database: Pinecone / Chroma / FAISS
LLM: GPT-5.4 mini/nano or GPT-4.1 mini/nano
I would appreciate recommendations based on real-world RAG deployments and benchmarks.
Nice to have a proper AI question for a change (instead of a token rate-limit grumble )
It shouldn’t matter.
They are usually entirely independent - you do your searches as part of a tool call usually on the side in a separate stage invoked by the tool call.
You get the embedding of your query string and search against the pre-calculated embedding vectors of your chunks and then concatenate the close results and send those back to the LLM as the return text from the tool call.
To this day I’m still using text-embedding-ada-002 for this purpose - my Chatbot switches between a lot of different models but I almost never change this setting.
It’s very fast and cost effective and provides, for me at least, a very good level of discretion.
If you want to see the full code of the embedding and retrieval process, feel free to look at the repo it’s open source:
Embeddings is OpenAI’s oldest model. It is almost like they are out of that business.
At the same time, you want proof of a long model lifespan for the cost of building your vector database, and might want 2000 embeddings answered in one call quickly.
You can take a look at this newer benchmark for retrieval RTEB English - and find OpenAI’s first entry below open models which can run on a 2GB video card and are yours permanently.
What is not heard about much any more is HyDE, expensive hypothetical document embeddings with input generated by LLM. Nor pointed out is Codex making dozens of manual file searches it writes and greps and loads in range blocks, and trials with context loading, a 100x paradigm in costs.
Don’t remember the name exactly, but I remember some Russian speaking guy showing even a lighter and faster version of that triples based index. Need to find the name of it.
ADDED: got it!
and another candidate is:
On MacOS and Windows FFF may be a bit faster in some cases. Also that one has the “special sauce MCP”…
)