Custom GPT Knowledge Versus External Actions

GPT builders
, ,
3

Thanks PaulBellow for your suggestion!
I agree that part of the issue may be related to attention as I’m not running into this issue when I configure the Custom GPT with only the smaller file. I plan to try breaking down the knowledge files into smaller chunks as well in the near future and see if it works better.

However, I suspect there may be something else at play here. The Custom GPT can in fact answer questions on the larger knowledge file correctly, but only when I see the loading spinner saying “Searching Knowledge”. That never seems to happen on the first prompt to the Custom GPT. Hence, my hypothesis is that Custom GPTs work as follows:

  1. OpenAI summarizes the content of the knowledge base into a compact form that it feeds to the model as context when it is prompted for the first time (no true RAG)
  2. if the user asks explicitly for more information, then it uses a RAG pipeline to extract more detailed knowledge from the files and provide more accurate answers
    This may be a good compromise for most users, but it is a poor user experience for what we are trying to do.

What I’m wondering is: if I provided my own retrieval function through an API as an action will the Custom GPT use it consistently every time or will I run into the same type of issue?

3 Likes