I am using ada large and noticed that the model is not able to tell that “Ending an address…” and “Removing an address..” are same. Did someone else notice this pattern and anyone has any solution for this. Wanted to add that it is scoring “Updating an address..” more in this case
1 Like
You have ada-002, or 3-large. The two do not meet!
The embeddings models are not question-answerers in the same way that language models are. They provide a vector that directs to some linguistic and semantic space. The meanings being compared have unfathomable depth, such as “seems to be UK patterns of English”.
On a word basis, “Ending” (as in finish, complete) and “Removing” (deleting) to me can be quite different when viewed from a distance.
For curiosity sake, although I don’t know your full text or application, here I run a local embeddings model, one that is query-prompt-driven AI for retrieval.
query:
-“Ending an address”
Texts:
-The ending of the book was unexpected
-The removing of the book was unexpected
-Updating an address
-Removing an address (your preference)
Search Results (Threshold: 0.40)
Rank 1: Score 0.5781
Text: Removing an address
Rank 2: Score 0.5581
Text: Updating an address
Rank 3: Score 0.4081
Text: The ending of the book was unexpected
1 Like