Is it possible to achieve embeddings cosine similarity approaching -1?

[curt.kennedy]

I made a post a while ago here:

Some questions about text-embedding-ada-002’s embedding

Hey @ruby_coder @debreuil Here is the code I wrote to do this. Hope it helps. import numpy as np import sklearn.decomposition import pickle import time # Apply 'Algorithm 1' to the ada-002 embeddings to make them isotropic, taken from the paper: # ALL-BUT-THE-TOP: SIMPLE AND EFFECTIVE POST- PROCESSING FOR WORD REPRESENTATIONS # Jiaqi Mu, Pramod Viswanath # This uses Principal Component Analysis (PCA) to 'evenly distribute' the embedding vectors (make them isotropic) # For more information o…

This will make your embeddings more isotropic (more spread out) and get you more toward -1. But it requires post processing a large batch of previous embeddings.

But the what/how/why of what causing the bias in the embeddings get’s into potential biases in the hidden layers that are creating the embedding vectors.

So ultimately, the most practical solution is to adjust your thresholds for each embedding model you encounter, as they all seem pretty different, and do no conform to normal vector geometry expectations.