Frontend That Searches 750 Products with 100K Tokens Each - Usecase help

API
,
1

Hello,

I am building with the API and need some help. A nudge in the right direction will be immensely helpful to me.

This is my usecase:

I have a catalog of 750 products on my website, and each product has between 50,000 and 100,000 tokens of detailed information stored in my database.

Now when users arrive on my site, I want them to be able to visit the site and interact with a LLM from OpenAI and specify their requirements.

For example, a user might type, “I’m looking for a shirt with pandas and penguins on it.” The LLM needs to understand the request, search through the full content associated with each product—including long-form descriptions, specifications, and metadata—and identify which items match the criteria. The assistant should then return a set of relevant products that best fit what the user is asking for, all in a conversational, real-time experience.

What’s the best way to approach this currently? RAG for each of the 750 products and searching against each product at every user query or something else?

Thanks

2

Yes basic rag should be pretty simple. Each article is a record in your RAG and you get back a list of articles ranked by relevance that the model can use to
Talk about

1 Like