I was thinking this Knowledge Cutoff is hallucination because there were not enough information around community yet.
In my case, gpt-4 gave me exact date and explanation for events. I tested about asking events and when it happened. It certainly know about events after Jan 2022.
For example, I asked about the 2022 FIFA World Cup, Queen Elizabeth, and Python 3.11 features. GPT-4 gave detail explanation with accurate date.
I’ve had GPT in various forms report “as of my latest update in March 2022” (and now back to Sept 2021 from also January 2022 being pretrained answers), and the ability to answer the same question magically appear and disappear from API, so it’s clear there’s some retraining and model monkeying going on.
The AI is so over-trained on denying answers and inserting cutoff dates without them now even being placed in a system prompt, that it takes GPT-4 and excessive prompting to even make your augmentation injection be acknowledged.
Any less than both system and user prompting and you still get “Sept 2021, denied, search the web.” At least today.
