Working on a question-answering application that will give everyone on our team the ability to answer proposal questions. How can I separate primary company documentation(sources of truth) from supporting knowledge like articles when uploading files?
@tabarak mentioned using fine-tuning, “Perhaps including company documentation for high-priority prompts, and supplementing with articles for info not available in the documentation.”
I don’t think I fully understand Fine-tuning yet to see this solution.
Thanks in advance!
This is a good question and perhaps does not have a simple answer. First, you will want to include metadata with your records. Then you can use a combination of Search to find relevant files and Answers to specify which documents to search.
I personally prefer to do this on my side since it can be faster with other search technologies like SOLR or ElasticSearch. So what I do is find all relevant documents with my own search logic and then use the Answers endpoints and just send up what I need (small subset of all files). There are pros and cons to either method, but I basically did not want to worry about managing files/records in multiple locations.
Thanks for info! This brings clarity!