Hey! 
I’m currently working on a RAG system using OpenAI’s text-embedding-ada-002 model. Initially, it provided excellent answers by extracting the right preprocessed chunks when users responded to questions. However, after migrating the embedding model to OpenAI’s text-embedding-3-large, which has 1536 dimensions, my RAG system didn’t perform as well as before. Any insights or suggestions would be greatly appreciated! 
“Migrating” refers to re-embedding your entire search corpus on the new model at a specific choice of dimensions. The output is not backwards-compatible.
3-large can have its dimensionality reduced by truncation to 1536 through an API parameter, but its native dimensionality is 3072. If you are paying, you might as well utilize the full quality until you encounter RAM or computation time constraints.
Thresholds will also need to be adjusted. Previously, a relevance score above 0.80 might have been a good cutoff point. Now, a score of 0.50 would be an appropriate scaling to accommodate the dot products returned by comparisons.
1 Like