GPT-4o is stuck in a loop and unusable

[_j]

I just have to search for my own naming of this “stuck in a loop”, which you’ll discern below, where the length of a ChatGPT conversation quickly degrades the (post-devday cheapening) AI, and the model starts referring back incorrecly through its narrow vision to past messages instead of seeing the forest and a progression leading up to the latest input’s needs.

@_j

This friendly agreement over quality I note is also symptomatic in models as they continue to be released and updated by OpenAI. It often becomes a useless parrot, besides other symptoms like getting “hung up” on past inputs.

@_j

The same behavior has hit ChatGPT’s GPT-4 as hit GPT-3.5-turbo around September (so that GPT-3.5 couldn’t code as well as crippled GPT-4?):

Hung up on past inputs.

Repeating the same code back to you with errors it just fixed re-included. Repeatedly. Rewriting the parameters and values of API functions to hallucinations. Rewriting the whole code base with a different encoder library and discarding 50% of the functionality, even against direct instructions not to alter more than instructed the second or third try.

@_j

And guess what I just had in ChatGPT with GPT-4. The same manifestation seen in those later alterations to gpt-3.5 besides the inability to follow system instructions: Hung up on prior inputs, unable to complete a new task but instead parroting back a prior code job, trying to answer my new question in the frame of existing context, and then spitting out the same code again, despite clear instructions that old code is no longer under question. To where the session had to be abandoned.

ChatGPT doesn’t have the “forgets what you said after 3 turns” of chat management any more - but maybe it SHOULD have that option.

Fortunately there’s (until June 2025?) API GPT-4 that hasn’t been hit badly.

o1 models are showing to be far worse in this; might as well just paste into a new session for every question and iteration. An o1-xxx “regenerate” in ChatGPT is generally better than what another model responded at a particular point, but you only get one go at good quality.

Someone needs to make a “long conversation” benchmark, instead of a “can answer one question” benchmark that AI companies compete with and tout…