How to make a tiny version of a ““perplexity””-like tool.
Say I have 1000 press article previews: ID, Title, Abstract, URL. This is my knowledge… a total of about 50-80 thousand tokens on average
Then I have some user preferences: keywords, language, geo, age… (1000 tokens max)
Question for the pros: to the best of your knowledge, what is the best way to obtain the following result with AI:
select among the 1000 articles the top-20 relevant articles given the user preferences (and return me a JSON array with those 20 IDs.
Putting 1000 articles in the prompt can be an option, dumping them as JSON or CSV, many models accept more than that. It can be expensive but I don’t care much for now. I’m open to spending $1 for this task
What about using “RAG” instead? Embedding my knowledge in a vector db and asking the LLM to select for relevance? Will it work? Is a RAG able to return the exact ID of the source? this last part is something I’ve never done
Thanks