Knowledge Retrieval - How to optimize the process

Hi there,

To get a better understanding of how the knowledge retrieval process works behind the scenes, I conducted a series of experiments, each requiring a different approach of the retrieval process. Initially I just aimed to figure out, if the file size and the location of the information within the file has an impact on runtime and accuracy. But the focus shifted towards are more general question on how to best utilize the knowledge retrieval tool. Here are my key findings:

  • :exclamation:For trivial search tasks, the file size or the location of the information within the file does not matter.
  • :exclamation:For complex search tasks it does.
  • :exclamation:Some level of “abstraction” is performed if neccesary. E.g. it searches for synonyms or changes the threshold of the vector search.
  • :exclamation:Formatting matters!
  • :exclamation:There was an undocumented limit of 2M tokens for knowledge files (thanks @logankilpatrick for adding it!).
  • :exclamation:Everything is quite inconsistent and beta and sometimes the assistants just goes rogue and you might end up with $0.50 API calls without a decent answer.
  • :exclamation:A lot feels like a black box and it would be really helpful to at least get to see what exactly the knowledge retrieval process is adding to the context!! @OpenAI: Please populate the retrieval field :pray:

I wrote a detailed article about the experiments and my findings. You can find it over here

I am curious to hear your thoughts on this and experiences you made with the knowledge retrieval process!

9 Likes

Very amazing article, thank you! I jumped here looking for something for my topic: MyGPT - Knowledge Behavior

It would be great to have your suggestions for it

1 Like

Thanks @luona.dev for the article, very interesting! I’m also doing some experiments, it’s crazy how many tokens the retrieval tool consumes, I hope they improve it soon.

2 Likes