Open AI embedding with 3072 dimensions

[Diet]

@curt.kennedy

For example, in general, you should get values ranging from -1 to +1.

But yes, here it looks like 0 to 1 for the latest models. Haven’t fully tested it myself.

Also, in general, any two random uncorrelated things will have a dot product of 0 (orthogonal).

@_j

I haven’t seen any below 0. OpenAI seems to have ensured a dynamic range 0-1. Use max(0, cosine_similarity(x,y)), and nothing is lost.

it’s probably just because they use some variation of cross entropy for training, maybe with a scaling factor, no?

I can’t think of a good simple loss function that allows for negatives in this context