I’ve been facing a bit of a challenge with embeddings made with text-embedding-ada-002 and pgvectors
The issue I’m encountering is related to the user input. Specifically, when he asks for a service with two specific requests simultaneously, pgvector matching sometimes match resources that respect only one request at a time. Ideally, I’d like the matching to strictly adhere to both requests to narrow down the results effectively.
For instance, if I set the city filter to ‘Madrid and the type filter to ‘Villa,’ the results should only display results that meet both criteria - Villa in Madrid. However, what’s happening is that it returns mixed results.
- Around 33% sources that are perfects
- 33% are in Madrid but not villa
- 33% are villa but not in Madrid.
When GPT-4 generates its answers, it filters them perfectly, resulting in correct final answers. However, this poses a problem as some correct sources in my database are not selected in the matching process, despite their accuracy. This situation might lead the final customer to think that we are intentionally promoting certain offers over others.
Has anyone else encountered a similar situation or knows of a workaround to ensure that both requirements are strictly respected in the search?
Any help or suggestions would be greatly appreciated! Thanks in advance for your assistance.