I am developing the chat with PDF solution to our department. When I ask question chatGPT gives me set of paragraphs relevant to my question. As a developer I would like the most relevant passages for the obtained answer to be shown first. Hence we need to rerank the passages after receiving the generated answer. What we have tried so far, is
- to split the passages in sentences, compute the embeddings of the sentences and the generated answer and get the most relevant sentences by just computing vector similarities
- another thing that I just tried is to prompt this analysis by asking gpt to score the passages for their relevance for the respective question-answer pairs (attached)
Seems like these ideas giving me the expected output, hence I would like to ask the community for the much needed inputs.