From my experience using other plugins, ChatGPT does “hallucinate” that its knowledge has a cutoff date of Sep 2021, and thinks it does not have access to any plugins, even in the very early rounds of chats.
I’m not sure whether this is the type of “hallucination” you have encountered. My situation definitely strikes into the dilemma where pre-training and fine-tuning conflict with newly introduced functionalities brought by plugins (at least occasionally and depends on your plugin prompt instruction). I wonder very much how OpenAI will approach to solve this.
I’m having this same issue with the plugin I’m developing.
When the plugin queries our API and gets a response that exceeds a certain size, it seems to respond with made up data. I’m sure this is related to context length limitations, but I’m not sure how to proceed considering the response size will be based on the amount of data needed to fulfill the query. Is there a recommended way to handle this?
I’m not sure that the suggestions in this thread related to plugin development specifically. @speedplane were you able to find a workaround?