Assistant Retrieval method and RAG (are they doing same?)

API
, , , ,
1

While working over my project i came across to a point where Assistants retrieval seems more consistent than RAG with chroma DB, is it just mere luck or Assistant retrieval is better than RAG from scratch ?

2 Likes
2

It should be at least as good if not better, but it is a beta test so there will be refinements and changes to improve it, if you are getting good results, that’s awesome!

1 Like
3

Here is an OpenAI YouTube video that came from one of the Dev Day sessions where they explain their process with getting the retrieval to the level it’s at in the Assistants API.

2 Likes
4

Where they explain the process of developing a custom solution for a top-tier European customer that gets OpenAIs developer attention, tuned specifically to the customer’s dataset.

assistants is not that.

RAG needs intelligence applied to such a custom solution, such as metadata, chunking and annotation techniques. Assistants removes the ability to do such.

2 Likes
5

Ah okay. My apologies. I must have misremembered the context of the example. Serves me right for linking a video and not double-checking.

6

What is your suggested path to leveraging a quality ready for use RAG solution within your Open AI Assistant? Can you use function calling to an external RAG system reliably?

7

I’m just in the process of trying this. It looks like using a tool to RAG and then feeding it with the prompt should work and give the model “focus.” I’ve done this using lanchain but I think going direct to the OpenAI Assistants is cleaner for my use case

1 Like
8

Assistants do not use RAG and vector similarity, but they work in a similar fashion by stuffing “relevant” context into the window. First, RAG’s traditional VDB vector similarity is very expensive (CPU) esp at scale; so that might not be a situation they want to wrestle with at this point. From what I can determine, they use a tool called myfiles_browser to get a ballpark context. RAG gives you a much more granular control over your context relevancy. Assistants plus RAG seems to be the direction for serious company use.