Hello, community. I am trying to compare Add 002’s embeddings against other common vectorizers, and while Ada 002 does seem to be better than the other metrics against which I’ve tested it (e.g., TF-IDF, Euclidean Distance, Jaccard Distance, and various BERT embedding models), the scale seems odd t…

The normalizing back out to unit vectors was in the context of me trying to “fix” the issue that ada-002 isn’t isotropic. What this means is that no matter what data you put into ada-002, your cosine similarity won’t go much below 0.7. So in that post referenced above, I used PCA to make the embed…

Ah, OK. Looks like I got the wrong end of the stick then. But maybe found the right person to help :smile:

I am also seeing cosine similarity values consistently in the 0.7 range and was shocked at the contrast with BERT models. I also tried to sanity check by comparing my start term ‘politics’ to random letters and it was also 0.77 The following strings had these cosine similarities with ‘politics’ 0.…

Maybe this is a pedantic question, but if we agree that the true range for the cosine similarities based on Ada’s embedding vectors is 0.7 - 1.0, why can’t we just normalize the range whereby 0.7 = 0, .85 = .5 and 1.0 = 1.0? It’s not perfect, but it makes comparison with BERT much easier.

Could be the normalization techniques. If the embeddings are normalized there might be a disparity between negative and positive similarities, with negative being less common, if so, this ends up with a restricted range of values. You could use a linear transform and normalise them yourself I guess. …

The minimum cosine similarity of 0.7 from ada-002 is a “feature”. :face_with_monocle: You’d think it should have a min of -1, right? Well it doesn’t. The model isn’t isotropic, and has all correlations between 0.7 and 1. (Instead of ranging from -1 to 1). You could batch process a set of embedd…

Embedding Results Scale Seems Off

API

Foxalabs June 25, 2023, 9:26pm 2

If you are seeing values from +/- 0.1 then you’re not renormalising back to unit vectors, see this post Some questions about text-embedding-ada-002’s embedding - #41 by curt.kennedy

Topic		Replies	Views
Embeddings and Cosine Similarity API	19	15488	July 24, 2022
Question on text-embedding-ada-002 API	11	6661	July 27, 2023
Poor embedding performance using ada for portuguse Use cases and examples embeddings	6	1464	October 18, 2023
Why cosine_similarity between embedding vectors is always above .68 API embeddings	6	5005	March 1, 2024
Quality of embeddings using davinci-001 embeddings model vs. ada-002 model API embeddings	14	4646	December 11, 2023

Embedding Results Scale Seems Off

Related topics