I am developing a file search system where data is stored in Markdown format within a vector store and follows a specific structure. To improve efficiency, I have implemented a pre-filtering mechanism to narrow down the search scope.
For example, out of 1000 documents stored, the pre-filtering process selects the 30 most relevant documents to run the search on. From this subset, the system retrieves the top 5 results closest to the search query.
While the system achieves 100% precision in ensuring that all final search results come from the pre-filtered list, it suffers from low consistency. Running the same query multiple times produces different results from the same pre-filtered set of 30 documents.
How can I address this issue to ensure more consistent search output?
My current setup
const run = await this.openai.beta.threads.runs.createAndPoll(thread.id, {
assistant_id: searchAssistantId,
additional_instructions:
'My prompt here',
max_prompt_tokens: 20000,
max_completion_tokens: 2000,
tool_choice: { type: 'file_search' },
tools:[
{
type: 'file_search',
file_search: {
max_num_results:limit,
ranking_options:{
score_threshold:0.5,
ranker:"default_2024_08_21",
}
}
}
],
include:["step_details.tool_calls[*].file_search.results[*].content"]
});