Knowledge Retrieval - How to optimize the process

Documentation
, , ,
1

Hi there,

To get a better understanding of how the knowledge retrieval process works behind the scenes, I conducted a series of experiments, each requiring a different approach of the retrieval process. Initially I just aimed to figure out, if the file size and the location of the information within the file has an impact on runtime and accuracy. But the focus shifted towards are more general question on how to best utilize the knowledge retrieval tool. Here are my key findings:

  • :exclamation:For trivial search tasks, the file size or the location of the information within the file does not matter.
  • :exclamation:For complex search tasks it does.
  • :exclamation:Some level of “abstraction” is performed if neccesary. E.g. it searches for synonyms or changes the threshold of the vector search.
  • :exclamation:Formatting matters!
  • :exclamation:There was an undocumented limit of 2M tokens for knowledge files (thanks @logankilpatrick for adding it!).
  • :exclamation:Everything is quite inconsistent and beta and sometimes the assistants just goes rogue and you might end up with $0.50 API calls without a decent answer.
  • :exclamation:A lot feels like a black box and it would be really helpful to at least get to see what exactly the knowledge retrieval process is adding to the context!! @OpenAI: Please populate the retrieval field :pray:

I wrote a detailed article about the experiments and my findings. You can find it over here

I am curious to hear your thoughts on this and experiences you made with the knowledge retrieval process!

10 Likes
How to Optimize data for Knowledge Retrieval with Assistant API
How does the knowledge of custom GPT actually work
2

Very amazing article, thank you! I jumped here looking for something for my topic: MyGPT - Knowledge Behavior

It would be great to have your suggestions for it

1 Like
3

Thanks @luona.dev for the article, very interesting! I’m also doing some experiments, it’s crazy how many tokens the retrieval tool consumes, I hope they improve it soon.

2 Likes
4

Sounds very interesting and thanks for the brief summary here!
Unfortunately the site link:


It’s likely the web site’s certificate is expired, which prevents Firefox from connecting securely.

What can you do about it?

kon.foo has a security policy called HTTP Strict Transport Security (HSTS), which means that Firefox can only connect to it securely. You can’t add an exception to visit this site.```