I am trying to understand use-cases for vector search on structured, business/enterprise data. An interesting extract from the below post -
By building a simple string of key field values in a table row and using that to determine its word vectors within a model such as OpenAI’s text-embedding-ada-002 LLM
Assume there is a relational table of doctors with the usual columns - name, age, address, discipline, education, work experience, visit hours and many more columns.
And a user query is - Find all orthopedics who have greater than 10 years of experience and speak French and studied in Europe …
How does the application handle above query?
- In the early days, application would have just some input boxes and check boxes for basic filters and then compose/execute a SQL :- SELECT * FROM doctors WHERE …
- Now today we can do text-to-SQL using specific trained LLMs - pass the above English query and hopefully the model returns a correct SQL.
- My question really is if vector search fits here? First step, generate vector embeddings for each table row after text’ifying the columns as key-value pairs. Next, generate embedding for the search query and do a vector similarity search.
The doctors table is just for illustration. I am trying to understand usecases for vector search on structured enterprise data - transactions, invoices, orders, inventory, customers etc. I am comfortable with vector search on documents/images/social media etc.
Thanks!