Calculating embeddings costs

API
1

Hi, can someone help me understand the costs for Semantic text search using embeddings?
Let’s assume a csv file with 100 lines having 100 tokens each and a term of 50 tokens for search. What’s the calculation per requst?

3

Thanks, but it doesn’t answer the question. I’m interested in knowing if there are multipliers, such as when using the old semantic search method.

4

Unlike search, you only need to run each piece of text through an embedding engine once and then can do the rest on your own machine. So it should be close to:

100 lines * 100 tokens/line (for the document corpus) + 50 tokens (the search query) = Roughly 10050 tokens. And then multiply that by the price of the model you’re using.

And if you did more searches on the same corpus, just 50 tokens per search.

3 Likes
5

Thanks for the clarification!

6

Hello!

Is there anyway to store the results of embedding into the openai model so that the embeddings don’t need to sent with each query?

For instance using this openai example is great:

Question_answering_using_embeddings

But as the number of embeddings grows this is going to get pretty expensive pretty fast.

So is there any way to store the embeddings so they don’t need to be included with each query? But rather they are in sort of a long term memory that chat gpt can reference?

I couldn’t find anything online regarding this…

7

Store them in your own database. Hash the chunk getting embedded, and check for this hash for future embeddings so you don’t have to re-embed anything ever again! (Until you switch models of course).

8

Ah, ok so following up on my own question. It seems maybe there isn’t a way to store the embeddings on openai.

However the example I mentioned (which is from the openai cookbook), uses a search then ask technique where it firsts queries the local data and only includes the most relevant embeddings as prompts…

Is it fair to say that this is the best approach and there isn’t a way to actually store the embeddings purely in openai?

It seems the example I linked should help reduce costs but figured I would check.

Thanks

9

Thanks!
Yeah seems similar to what they are doing in the notebook they linked.

Appreciate the help

10

OAI doesn’t provide storage for you. It’s on the developer. Also, you need your own compute to do the correlations. OAI is just the AI “dumb pipe”, not much else, but everything at the same time. :sunglasses:

2 Likes