Hello, I’m thinking to build Q&A bot where the LLM could handle question about related to deep business context. So far I use system message but got stucked with token limit.
What would you suggest to master for my need? Fine tuned, embedding search, or semantic search method ?
Embeddings are a way of doing semantic search, and that’s the direction you want to go.
This blog post might help you understand the process a little better Q&A with ChatGPT + Embeddings - Basic & HyDE Examples - WFH Brian
Fine tuned model might not be a bad shout but embeddings would be the way to go. Also, alongside embeddings, givings sample will allow you more control over the type of output as well
Hi,
I had almost same situation, but another task todo: I need to search for products in a list of 100k products like “coca cola light 10ml red”. The problem was that input from user could be “ccola lght 10.0ml RED dose, fresh drink” or something like that. And embeddings had problems with it (not exact same example).
I was thinking I could somehow tune the embeddings and there is a openai cookbook on github: “Customizing embeddings”
but that was to complicated for me and my language is c# not python. So I solved it a bit pragmatically like this:
By adding these user inputs “ccola lght 10.0ml RED dose, fresh drink” parts or relevant additions to the texts, you ensure that the search for similar vectors or text detection works better. My text would look like this: “coca cola light 10ml red. ccola lght 10.0ml RED dose, fresh drink”. This need to updates vector data by time, but this is not a problem for our use case.
In essence, you are “tuning” the vector by enriching it with more context and relevant data. This approach can be quite useful, especially when the model struggles to capture certain patterns or information in the texts. It’s a pragmatic solution that can work well in many cases, especially when fine-tuning the model is not possible or practical.