Yes, and that’s exactly what I was after with the initial bug report – additional model anchoring is required in the system prompt at the moment.
See the code snippet I posted: above: Gpt-4-turbo-2024-04-09 does not anchor to most current cutoff date by default - #9 by FlyingFathead
Try it out for yourself as an A/B test, or just use the API Playground with the same kind of additional model steering. Again, specifically stating i.e. “You are gpt-4-turbo-2024-04-09” in the system prompt does all the difference in the world when it comes to the model being able to access the latest cutoff dates.
I think the problem you described about worsening output quality might revolve around that. It’s misaligned and doesn’t attach to the correct cutoff date, however you want to put it.
Since gpt-4-turbo now points to gpt-4-turbo-2024-04-09 as well, be aware that without further specification and model steering in the system message, it’s unable to access the later cutoff date (December 2023) properly.
That is more than likely contributing to the sub-optimal performance, even if you’re not actively asking about recent events between Apr-Dec 2023. It’s likely still tossing the coin towards its earlier checkpoints/cutoff dates.
In my view, that’s symptomatic of a larger issue and also might point to a larger model structure problem and the perceived diminishing performance on GPT-4’s newer checkpoint variants.
Also, even if someone contends that the cutoff date should be pointed in the system instructions (a.k.a. “deal with it”), I still maintain that the model should not require that kind of pinpoint steering separately in the system message for the model to actually utilize its latest cutoff data that it’s supposed to. It should not be a system prompt add-on requirement. It’s also not like that with other AI API providers (Claude, Perplexity, …)
Rather, it’s a rudimentary workaround at best – OpenAI is passing their internal version anchoring problems to their API clients or end user with the current implementation. I see this is as a bug, not as a feature, especially when the model variant is already being called from the API with the model selection.