Retrievals from files are cool , but I want to use retrieval from databases any ideas?

I used vector databases before , assistant api came as a surprise , but I have no idea how can I connect my assistant to a normal db , like files are okay but I want to get request from my normal db to openai directly without having to transform them to files then later deleting this files to keep It updated.

I don’t know if I understood your question right. But if I did, you’re looking for a way to implement RAG (Retrieval Augmented Generation) ‘manually’, instead of having to use the assistant API.

Well, in principle you can do that. If I understood correctly, the Assistants API picks the documents to be provided to the bot using some retrieval algorithm, and then those documents are sent in as context (so a standard api message with a context tag) for the bot to use them.

You can do it manually! Implement a retrieval algorithm and then a way to load those documents straight in the prompt (context or user message, experiment!). As an example, you could use sentence embeddings to understand which document is related to the user message, and then select top-n and pass them to the bot dynamically.

Now, what I told you is speculation from a learner about how stuff works under the hood. If someone could confirm, I’d be happy :slight_smile:

1 Like

I was using function calling while testing database interaction. I would programmatically insert a list of table names and their associated column names into the user’s prompt. My function description was just a “query” parameter. GPT-3.5 and GPT-4 were ok at generating valid queries but it was far from bullet-proof and expensive with tokens. It works better for simple schemas obviously. If it had panned out for me, I would have had to think about security, either safety-checking the query or (more ideal) adjusting the function descriptions to facilitate sql parameters in my backend.

As for the assistants API, it’s so expensive I can’t see anyone using it in production with any sizeable documents as it stands now.

1 Like