Creating a Chatbot using the data stored in my huge database

I’m not entirely sure if I understand your question, but it appears to be related to the difference between a keyword search (sql) and semantic search (AI).

They are two different animals, designed for two different results. I can tell you that trying to use AI (gpt-4, gpt3.5, etc…) to search for items by keyword is doomed to failure. These systems are designed to look for ideas, concepts, summarizations, interpretations which come as close to your query as they can get. They are not designed to locate information based upon specific words in a document.

In fact, I just recently posted an issue here where I had this exact same problem: How to format context documents to allow model to recognize specific fields within documents

So, if your video tags are titles, locations, authors, you know, specific information, you may be better off going with SQL. If your video tags, on the other hand, are free flowing descriptions, critiques, summarizations, then AI is the tool for the job.

Hope this helps.

1 Like