# Embeddings advice

**URL:** <https://community.openai.com/t/embeddings-advice/19192>\
**Category:** API\
**Created:** [July 7, 2022, 5:28pm UTC](https://community.openai.com/t/embeddings-advice/19192 "2022-07-07T17:28:26Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![alexandru](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/alexandru/32/10196_2.png) [@alexandru](https://community.openai.com/u/alexandru)\
**Post date:** [July 7, 2022, 5:28pm UTC](https://community.openai.com/t/embeddings-advice/19192/1 "2022-07-07T17:28:26Z")

</div>

Hello,  
I have a two data sets, one rather small 500 units, each around 850 characters, and a larger one, 2500, each around 750 chars each.  
I created indexed the embeddings for both data sets using text-search-ada-doc-001 and embedding the query with text-search-ada-query-001. These are 1024 dimensions embeddings.  
The results are good in both cases, however for the smaller set is rather fast 5-7 seconds, for the larger one around 15-20. I guess it goes up linearly as i am doing the dot product using memory (in a ruby on rails app )  
I would like to ask if anyone has experience with this and would like to share some advice improving this in any way.  
One option i am considering is storing them in a [milvus](https://milvus.io/) database and hopefully get better timings.  
Second option would be to find a faster way of calculating the dot products for each query.

I would also be curios to know if for this sizes of my texts would i benefit from using babage or curie for larger dimensions 2000 or 4000, or is it not worth the extra overhead, considering the timings would also increase doing this, monetary part aside ?

---

<div class="post-metadata">

**Author:** ![daveshapautomator](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/daveshapautomator/32/8779_2.png) [@daveshapautomator](https://community.openai.com/u/daveshapautomator)\
**Post date:** [July 7, 2022, 5:57pm UTC](https://community.openai.com/t/embeddings-advice/19192/2 "2022-07-07T17:57:51Z")

</div>

There’s also FAISS which is open source. I’ll be experimenting with FAISS soon.

---

<div class="post-metadata">

**Author:** ![alexandru](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/alexandru/32/10196_2.png) [@alexandru](https://community.openai.com/u/alexandru)\
**Post date:** [July 7, 2022, 6:28pm UTC](https://community.openai.com/t/embeddings-advice/19192/3 "2022-07-07T18:28:54Z")

</div>

have read a bit about it but don’t know exactly the flow, will keep an eye on your youtube 😃

---

<div class="post-metadata">

**Author:** ![daveshapautomator](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/daveshapautomator/32/8779_2.png) [@daveshapautomator](https://community.openai.com/u/daveshapautomator)\
**Post date:** [July 7, 2022, 9:05pm UTC](https://community.openai.com/t/embeddings-advice/19192/4 "2022-07-07T21:05:16Z")

</div>

Okay I wrote a thing that should help. It is not optimized but it works!

> **[vdblite](https://pypi.org/project/vdblite/)**
>
> Vector Database Lite

[![](https://us1.discourse-cdn.com/openai1/original/2X/1/113c4483bf1da426d1fb96e13a6646a148230724.jpeg "I needed SQLITE but for vectors so I wrote it myself. Now it's on PyPI - introducing VDBLITE") ](https://www.youtube.com/watch?v=gHrIrWyUq8U)

---

<div class="post-metadata">

**Author:** ![joyasree78](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/joyasree78/32/35263_2.png) [@joyasree78](https://community.openai.com/u/joyasree78)\
**Post date:** [May 5, 2023, 7:58pm UTC](https://community.openai.com/t/embeddings-advice/19192/5 "2023-05-05T19:58:12Z")

</div>

The home page says file not found and does not show the github