Did assistant api kill manual RAG with vector databases?

[merefield]

More in this conversation:

Error while using Assistants api

Basically yes. Unlike ChatGPT where the conversation is clipped to where people complain it doesn’t remember anything, assistants will run up the conversation to the maximum of the model when you continue to chat. 128k. Also, when running your own vector database, for example with 1MB of your company’s tech support knowledge base and product offerings, you might have a threshold where only the top 5 chunks are fed to the AI, and only if they meet a semantic similarity threshold. Not the case wi…

Having now implemented a similarity threshold for my search results, my RAG search is even more efficient and the gap is even wider.

Consider Assistants API to be a beta product with some way to go I suspect … (but packaging up functionality in that way is a totally reasonable goal).