Why has the software market not filled this vacuum?

[ruby_coder]

@rkaplan

But my understanding is that vectorized searches are accepted as the best way to do a semantic search.

Vectorized searches are great for NLP searches, no doubt.

@rkaplan

Can that be done in a full-text search without lots of additional manual semantic tagging?

Basic text substring matching and full-text DB searches powers almost all of the web-sites on the Internet. Is the Internet “falling apart” or “broken” from a lack of good search tools?

Simple question. Yes? No?

I think you would agree that the answer is “No”.

So, what is the “amazing use case” that every major DB vendor out there has missed over the past few decades? Vector-based searches are not actually “super cutting edge, SOTA”.

Take at look at this simple vectorize search for “Hello World” using the OpenAI API in this post here in our community:

A valid, working PHP library to help compare vectors for embeddings

OK… here is a simple example of both: Vector DB Search “Hello World?” [Screenshot 2023-02-08 at 11.35.12 AM] Basic Wildcard “LIKE” DB Search “Hello World” [Screenshot 2023-02-08 at 11.34.26 AM] Which is better? For CMS search engines, there will be a lot of “bad matches” for short phrases because vectorized text must be around 300-500 tokens (or words, need to confirm) to provide a vector which is useful. Most people doing a CMS or forum search will use much smaller search terms and the i…