Is it possible that this is caused by selective availability, so that you can’t determine precisely when the running model is tweaked?
Birthday problem done on 10 string outputs from embeddings (no math)
text-similarity-ada-001: All outputs match
text-similarity-babbage-001: All outputs match
text-similarity-curie-001: All outputs match
text-similarity-davinci-001: All outputs match
text-search-ada-doc-001: All outputs match
text-search-ada-query-001: All outputs match
text-search-babbage-doc-001: All outputs match
text-search-babbage-query-001: All outputs match
text-search-curie-doc-001: All outputs match
text-search-curie-query-001: All outputs match
text-search-davinci-doc-001: All outputs match
text-search-davinci-query-001: All outputs match
code-search-ada-text-001: All outputs match
code-search-ada-code-001: All outputs match
code-search-babbage-code-001: All outputs match
code-search-babbage-text-001: All outputs match
and then…
text-embedding-ada-002: Mismatch found between 0 and 1
Comparison mismatch found at line number 8
Line in string 0: 0.013175653293728828,
Line in string 1: 0.013211685232818127,
text-embedding-ada-002: Mismatch found between 0 and 2
Comparison mismatch found at line number 8
Line in string 0: 0.013175653293728828,
Line in string 2: 0.013218246400356293,
text-embedding-ada-002: Mismatch found between 0 and 3
Comparison mismatch found at line number 8
Line in string 0: 0.013175653293728828,
Line in string 3: 0.013211685232818127,
Not even past the first value…
(expand for dozens more, and the only surprise is that a few match..)
text-embedding-ada-002: Mismatch found between 0 and 4
Comparison mismatch found at line number 8
Line in string 0: 0.013175653293728828,
Line in string 4: 0.013232099823653698,
text-embedding-ada-002: Mismatch found between 0 and 6
Comparison mismatch found at line number 8
Line in string 0: 0.013175653293728828,
Line in string 6: 0.013218246400356293,
text-embedding-ada-002: Mismatch found between 0 and 7
Comparison mismatch found at line number 8
Line in string 0: 0.013175653293728828,
Line in string 7: 0.013192304410040379,
text-embedding-ada-002: Mismatch found between 0 and 8
Comparison mismatch found at line number 8
Line in string 0: 0.013175653293728828,
Line in string 8: 0.013218246400356293,
text-embedding-ada-002: Mismatch found between 0 and 9
Comparison mismatch found at line number 8
Line in string 0: 0.013175653293728828,
Line in string 9: 0.013232099823653698,
text-embedding-ada-002: Mismatch found between 1 and 2
Comparison mismatch found at line number 8
Line in string 1: 0.013211685232818127,
Line in string 2: 0.013218246400356293,
text-embedding-ada-002: Mismatch found between 1 and 4
Comparison mismatch found at line number 8
Line in string 1: 0.013211685232818127,
Line in string 4: 0.013232099823653698,
text-embedding-ada-002: Mismatch found between 1 and 5
Comparison mismatch found at line number 8
Line in string 1: 0.013211685232818127,
Line in string 5: 0.013175653293728828,
text-embedding-ada-002: Mismatch found between 1 and 6
Comparison mismatch found at line number 8
Line in string 1: 0.013211685232818127,
Line in string 6: 0.013218246400356293,
text-embedding-ada-002: Mismatch found between 1 and 7
Comparison mismatch found at line number 8
Line in string 1: 0.013211685232818127,
Line in string 7: 0.013192304410040379,
text-embedding-ada-002: Mismatch found between 1 and 8
Comparison mismatch found at line number 8
Line in string 1: 0.013211685232818127,
Line in string 8: 0.013218246400356293,
text-embedding-ada-002: Mismatch found between 1 and 9
Comparison mismatch found at line number 8
Line in string 1: 0.013211685232818127,
Line in string 9: 0.013232099823653698,
text-embedding-ada-002: Mismatch found between 2 and 3
Comparison mismatch found at line number 8
Line in string 2: 0.013218246400356293,
Line in string 3: 0.013211685232818127,
text-embedding-ada-002: Mismatch found between 2 and 4
Comparison mismatch found at line number 8
Line in string 2: 0.013218246400356293,
Line in string 4: 0.013232099823653698,
text-embedding-ada-002: Mismatch found between 2 and 5
Comparison mismatch found at line number 8
Line in string 2: 0.013218246400356293,
Line in string 5: 0.013175653293728828,
text-embedding-ada-002: Mismatch found between 2 and 7
Comparison mismatch found at line number 8
Line in string 2: 0.013218246400356293,
Line in string 7: 0.013192304410040379,
text-embedding-ada-002: Mismatch found between 2 and 9
Comparison mismatch found at line number 8
Line in string 2: 0.013218246400356293,
Line in string 9: 0.013232099823653698,
text-embedding-ada-002: Mismatch found between 3 and 4
Comparison mismatch found at line number 8
Line in string 3: 0.013211685232818127,
Line in string 4: 0.013232099823653698,
text-embedding-ada-002: Mismatch found between 3 and 5
Comparison mismatch found at line number 8
Line in string 3: 0.013211685232818127,
Line in string 5: 0.013175653293728828,
text-embedding-ada-002: Mismatch found between 3 and 6
Comparison mismatch found at line number 8
Line in string 3: 0.013211685232818127,
Line in string 6: 0.013218246400356293,
text-embedding-ada-002: Mismatch found between 3 and 7
Comparison mismatch found at line number 8
Line in string 3: 0.013211685232818127,
Line in string 7: 0.013192304410040379,
text-embedding-ada-002: Mismatch found between 3 and 8
Comparison mismatch found at line number 8
Line in string 3: 0.013211685232818127,
Line in string 8: 0.013218246400356293,
text-embedding-ada-002: Mismatch found between 3 and 9
Comparison mismatch found at line number 8
Line in string 3: 0.013211685232818127,
Line in string 9: 0.013232099823653698,
text-embedding-ada-002: Mismatch found between 4 and 5
Comparison mismatch found at line number 8
Line in string 4: 0.013232099823653698,
Line in string 5: 0.013175653293728828,
text-embedding-ada-002: Mismatch found between 4 and 6
Comparison mismatch found at line number 8
Line in string 4: 0.013232099823653698,
Line in string 6: 0.013218246400356293,
text-embedding-ada-002: Mismatch found between 4 and 7
Comparison mismatch found at line number 8
Line in string 4: 0.013232099823653698,
Line in string 7: 0.013192304410040379,
text-embedding-ada-002: Mismatch found between 4 and 8
Comparison mismatch found at line number 8
Line in string 4: 0.013232099823653698,
Line in string 8: 0.013218246400356293,
text-embedding-ada-002: Mismatch found between 5 and 6
Comparison mismatch found at line number 8
Line in string 5: 0.013175653293728828,
Line in string 6: 0.013218246400356293,
text-embedding-ada-002: Mismatch found between 5 and 7
Comparison mismatch found at line number 8
Line in string 5: 0.013175653293728828,
Line in string 7: 0.013192304410040379,
text-embedding-ada-002: Mismatch found between 5 and 8
Comparison mismatch found at line number 8
Line in string 5: 0.013175653293728828,
Line in string 8: 0.013218246400356293,
text-embedding-ada-002: Mismatch found between 5 and 9
Comparison mismatch found at line number 8
Line in string 5: 0.013175653293728828,
Line in string 9: 0.013232099823653698,
text-embedding-ada-002: Mismatch found between 6 and 7
Comparison mismatch found at line number 8
Line in string 6: 0.013218246400356293,
Line in string 7: 0.013192304410040379,
text-embedding-ada-002: Mismatch found between 6 and 9
Comparison mismatch found at line number 8
Line in string 6: 0.013218246400356293,
Line in string 9: 0.013232099823653698,
text-embedding-ada-002: Mismatch found between 7 and 8
Comparison mismatch found at line number 8
Line in string 7: 0.013192304410040379,
Line in string 8: 0.013218246400356293,
text-embedding-ada-002: Mismatch found between 7 and 9
Comparison mismatch found at line number 8
Line in string 7: 0.013192304410040379,
Line in string 9: 0.013232099823653698,
text-embedding-ada-002: Mismatch found between 8 and 9
Comparison mismatch found at line number 8
Line in string 8: 0.013218246400356293,
Line in string 9: 0.013232099823653698,
So sure, it can be made deterministic… round(dimension, 3) if you’re optimistic.