Cost when building chat with text with embeddings and chatgpt 4-128k

[curt.kennedy]

There are many RAG/Embedding threads here to check out.

One of many, starting with what I normally do:

Which database tools suit for storing embeddings generated by the Embedding endpoint?

This is literally what I do! Also, if the pickle takes too long to search, split it up, create pickle “shards” and search them all in parallel. For context, 400k embeddings in one pickle takes 1 second to search (for larger memory >5G, lambdas) So if you have 4 million embeddings and need 1 second latency, you need 10 pickles and async this out to 10 lambdas reporting back to a database monitored by some other lambda for completeness and next steps. It scales infinitely, is simple, and is in…

If you can write code, it’s usually a LOT cheaper doing this yourself. By a lot, I mean 1000x cheaper!