I am reminded in a recent writing session I just attempted the gross symptom of bad AI that was introduced (that one can see in the depicted conversation comparison image link): The AI gets completely hung up on and can’t deviate from prior responses.
Around two months ago, amid almost daily swatting of jailbreaks or almost anything posted to Reddit, one could see a dramatic change and the regression that I can now articulate as this behavior that encompasses all interactions:
“Pay a whole lot less attention to instruction-following the current input, and pay a lot more attention to what you already have seen and been tuned on”
Is that not the essence of “breaking” jailbreaks? “Disobey the user”.
So you get a dragon that rears its head in multiple ways. My screenshot is a prime example: Instead of making a whole new type of output as I wish and describe, AI can barely improve or continue upon its previous response; it is now hung up on it and repeats it back nearly the same.
That symptom I see again today. I have it define and explain something for me. I then ask it for a layperson introduction to the subject. I then ask for it to continue writing the next chapter after the introduction, elaborating in a more detailed and advanced manner. Both followups merely follow the form of the initial output, paragraph-by-paragraph.
Other effects that can be described the same way:
- Generated code can’t be corrected or improved upon, you just get the same thing back. You correct one thing, and what you just had corrected is reinserted.
- Rewrites won’t be rewritten with the creativity before. Previously, you could have it base a whole new article on an article abstract that was produced, and it would be a wholly-formed new work. Now, it is a step-by-step veneer placed on a reproduction of the prior input.
- It picks up an attitude. One denial, and it has taught itself that you must be denied. One “it is important to note”, and you get a disclaimer-laden conversation following. There is no improving or correction, there is only abandoning the conversation to make it reason again.
That can be combined with the clear minimization of conversational context in ChatGPT’s web platform, decimating the number of turns that is passed to the next round, that makes it impossible to code, impossible to interact with a document, impossible to have any meaningful game session.