ChatGPT halluciating with very long plugin response

GPT builders Plugins / Actions builders
, ,
1

We built a plugin that can fetch information for an entity from internet. So sometimes, a user asks “can you summarize A?” on ChatGPT. And we generated a very long JSON response to ChatGPT. What we notice is that ChatGPT can generate relevant and coherent texts for the first few sentences. And somewhere in the middle, it starts to hallucinate with completely unrelated and quite often wrong information. We hypothesized that it is due to the limitation of the context window?

I understand that we can always cut down the content we return to ChatGPT. But there are many cases where a long JSON response is needed (think a manual of doing something). Is there some way for us to still send the texts back to ChatGPT and help it avoid hallucination?

Is it possible for us to add prompts such as “read first 1000 words and summarize them into 200 words; and then read the next 1000 words and summarize them into 200 words; …”? Or something along this line?

Your help or experience sharing will be greatly appreciated. Thanks a lot!

3 Likes
2

Yeah, this is a common issue with LLM’s when the input is too large it starts to hallucinate.

There is a similar question on stack overflow openai api - How to send longer text inputs to ChatGPT API? - Stack Overflow

1 Like
3

We also noticed that hallucinations increase suddenly when crossing the 6000 tokens barrier. Not sure where exactly the barrier is - somewhere between 6 and 7 k. Our plugin’s openai.yaml file already has 800 tokens. I’m assuming there is some system message (among other things?) with a fixed length.
Which parts get thrown out if the prompt is too long? What is the exact budget that is available before stuff gets deleted?

4

Have not found a super general solution for this. But let’s all hope OpenAI lifts the context window to 100k for everyone soon!