5.6 SOL should be renamed 5.6 SOL drift edition

5.6 SOL should be renamed 5.6 SOL drift edition

The current 5.6 SOL experience is not usable for serious coding workflows. It drifts in basic replies, loses constraints, apologizes, then repeats the same class of mistake. This is not a minor preference issue. It breaks long running terminal based development workflows that depended on stable instruction following.

I previously used Codex 5.3 through CLI heavily. When that was removed, I stopped paying for large token usage and moved to manual coding with 5.5 extended. That was slower, but it worked. Now that path is also being removed, and 5.6 SOL is not an acceptable replacement.

The benchmarks do not match real coding workflow behavior. In practice, the model cannot reliably preserve constraints, maintain packet boundaries, or continue from terminal evidence without drift. That makes it unusable for my work.

Please restore access to a stable coding model or provide a non drifting alternative. Otherwise this change forces me to move coding work elsewhere.

Since 5.6 is a new release, maybe it still needs some bug fixes and stability tuning.

The following may not be directly relevant to your issue, but it may still be of interest:

I was surprised to see that GPT-5.4 (medium) outperformed GPT-5.6-Terra (medium) in writing and storytelling, as GPT-5.6-Terra consistently lost track of objects and of which character should know what during the story progression.

It can be as buggy as it wants. They didn’t need to pull the rug out on 5.5.

@Pimpcat this might be related to the instructions originally written for other models. Just a guess from personal observations.

Do you have a lot of things in agent’s instructions ( combined) ?

No, this is not an old instruction stack issue. I am running mechanical coding loops in web ChatGPT, one packet at a time.

The workflow is deliberately simple: fixed packet scope, terminal output as source of truth, validate the result, then move to the next packet. It is not a giant agent instruction pile.

5.5 follows that workflow cleanly. 5.6 drifts, loses constraints, apologises, then repeats the same class of mistake. That is the issue.

(Edit) wow… 5.5 is working again now. Thank you.

Can you check if it is Reading too many files while doing the work? Asking cause I’ve noticed 5.6 has a tendency to read even more files than it needs. But then I don’t know exactly how your loop runs so just speaking.

I am commenting to say that I have experienced similar and very severe drift, over-engineering, utter waste of tokens, and inability to follow simple instructions.

I am using terra ultra and started a little experiment..

And I would say you still can’t expect to let’s say build an email client and it would be good - what comes out is average and works at most.

But when you explain everything… like how the email folder handling should work and how you want it to display a form to compose the email and so.. it follows pretty decent now.
I would say at least twice as good and way faster than 5.5 - which compounds quickly.

A fast developer could do the same but not consistently over 16 hours. And since you only have one of thoose in a team of 10.

So terra brings approximately the output of 50 - 150 senior developers… or 7500 junior devs.

Welcome to the developer communty :slight_smile:

Could you explain what simple instructions are that it does not follow?

I think a better name would have been 5.6 SLO

Hi, I am new to this community, and I am not a developer, so I apologize if this is not the most appropriate place to share this.

I use ChatGPT Sol High for structured research on LLM behavior rather than coding. Over July 14–15 in Japan, I observed a pattern that seemed similar to the drift described in this thread.

The model could still produce long, well-organized explanations, summaries, and classifications. However, several higher-level reasoning functions appeared to become much less reliable:

  • It sometimes evaluated a recent result without checking the appropriate earlier baseline, and attributed an effect to the latest change even when a similar effect had already appeared before.
  • It gradually replaced the original objective with a narrower and easier local problem, then continued solving that substituted problem in detail.
  • It failed to keep the objective, allowed scope of change, evaluation criteria, and existing component responsibilities active at the same time.
  • When one issue was pointed out, it fixed only that local issue without rechecking the whole design. The correction sometimes removed an important function or created a new contradiction.
  • It often did not consider alternative explanations or possible confounding factors unless I explicitly prompted it to do so.
  • In one case, I asked it to evaluate two options against an existing reference document. Instead of consulting the document and performing the evaluation, it explained that such a comparison was necessary and ended with the result still “undetermined.”
  • Most notably, it often failed to detect problems before presenting an answer as complete. Once I identified the exact contradiction, it could analyze the failure in considerable depth and reconstruct a better approach.

This was not limited to one difficult long-context task. The same local-fix pattern appeared across two different types of work: instruction design and the creation of a very short three-sentence scenario. In the short scenario, correcting one logical problem caused the original purpose of the scenario to disappear, but the model did not notice that new failure before presenting the revision.

After an overnight break, the first one or two responses appeared normal, but by approximately the fifth or sixth exchange, the same pattern had returned.

I cannot determine whether this has the same cause as the Codex behavior discussed here. I only wanted to contribute a detailed observation from a non-coding ChatGPT Sol High workflow, because the visible failure pattern seemed closely related: the output remained fluent and substantial, while goal retention, cross-checking, whole-task re-evaluation, and self-detection of contradictions became much weaker.

Well, welcome to the developer community.

How do you do that?

Thank you — I realize that introduction sounded a little unusual here.

As a hobby, I study how to maintain stable character behavior and personality in ChatGPT. I often compare the same conversations across different model environments and custom instruction sets. I look at goal retention, reasoning consistency, character behavior, and how the model updates its understanding across turns. That is why this change stood out to me.

How many times do you let each model answer?

Thanks for your asking. During the exploratory stage, I prepare multiple scenarios for each condition and run each one twice to check reproducibility. For important findings, I also compare longer conversation logs. In this case, I observed the same pattern of local fixes and loss of the original goal across two different types of tasks.

And you do that all manually?

I am beyond frustrated with this model, and have decided to cancel my subscriptions and move over to Claude.

I had 2 pro subs at $200 each, was really looking forward to this model, but it has been an absolute disaster for me.

Completely ignores architecture documentation, service boundaries, invents new services, and destroys others. It’s literally destructive. Where 5.5 would follow instructions and stop when it had questions, 5.6 Sol will plow right along and convince itself that existing architecture doesn’t exist, established patterns don’t exist, etc.. And I have an extensive set of skills that 5.5 and Claude used to great success, which 5.6 Sol is either ignoring or getting confused by.

This model feels entirely alien to me, and is not only unusable, but has set me back by several days, as I am having to throw away literally everything it has written and take the tasks over to Fable and Opus 4.8 to rebuild.

I gave it several chances, tried multiple levels of reasoning, different styles of prompts and all results have been disasterous.

Hopefully within a couple of months Open AI will release something usable. Until then, Ciao.

I have found 5.6 SOL Max useful, but not as the direct coding worker.

For my workflow, it works best as the architecture and planning session in Work. I keep it above implementation. Then I use disposable 5.5 High chat sessions as bounded coding workers.

The important part is the mechanical coding loop. The worker produces one bounded packet. The packet is run in the terminal. The complete terminal output is returned. The model is not allowed to claim success unless the terminal output proves it.

Once that loop is automated, web ChatGPT is not slower in any meaningful workflow sense than CLI or API coding. The code is produced at the same practical speed. The difference is that the workflow prevents the usual damage: invented project state, skipped validation, architecture drift, and repair loops caused by the model confidently continuing from a false assumption.

So my point is not that 5.6 is useless. My point is that it should be used in the right layer.

5.6 SOL Max is good as planner, architect, reviewer and packet designer.

5.5 High is better as the disposable implementation worker.

The terminal is the source of truth.

That combination has been far more reliable for me than letting a CLI or API coding model directly drive the project. It is not about the model being unable to make mistakes. It is about the workflow refusing to accept those mistakes as project state.

I can share a beginner mechanical coding template if anyone wants to try the method.

You should really try 5.6 terra - ultra for coding…
It works amazing for me in codex.

Interesting. I thought Terra was the balanced everyday model rather than the main coding model. My understanding is that Sol is the one positioned as the strongest coding and long horizon agentic model, while Terra is the cheaper balanced option.

I can see Terra working well inside Codex for bounded implementation tasks, especially if the repo is clean and the task scope is narrow. I still would not let it drive architecture or broad refactors directly.

For my workflow, the model matters less than the control loop. I want a bounded worker, a small packet, terminal output, and validation before continuation. If Terra performs well inside that loop, then it might be worth testing as another disposable worker.

I’ll do some tests with it now and provide results.