Anyone else seeing performance issues with GPT-04?


I have been seeing sudden performance issues with my GPT-04 (06/13 snapshot) prompt starting Feb 27th 2024. This includes things like repetitiveness, high bias on the one-shots and injecting those examples in inappropriate places, and just overall not following instructions as well as it was.

I have not updated my prompt since Feb 20th, and even tried reverting to a prompt I used at the beginning of the month, but am still seeing this issues.

Has this snapshot been deprecated? Even using the most current GPT-04 endpoint is giving me these same issues.

Is anyone else seeing things like this? Any ideas on what can help?


The model has been deprecated, in that its shutoff in June is announced, and you’re not supposed to be able to fine-tune it according to the deprecations page.

I don’t have a particular edge case to retry, but this model has been a dumping ground for performance reductions since it was introduced. One could have hoped that they’d leave it alone after there were more models to pay attention to, but obviously not. I get a sense that -0301 is also not acting the same, even more than the damage remaining after the model was corrupted with bad ChatML for several days.

1 Like