MTEB benchmark for v3 small embedding model with 256 dimensions

The official post shows MTEB score under 512 and 1536 dimensions, does anyone know how’s the performance under 256 dimensions?

Hi and welcome to the Developer Forum!

Going by this table, 3-large at 256 dimensions (62.0) is better than ada-002 at 1536 (61.0)

As you can see above, there is no 256 dimension output from 3-small.

If you made your own custom truncation or dimension reduction algorithm, I would expect the results to fall below ada-002 levels.

The performance of the benchmark may not relate to the similarity task you are actually performing, so you can see if reducing a 2k byte embedding to 1k is really worth it on your particular application by trials. You don’t have to re-embed to compare quality or speed of dot-512 vs dot-256.