I’ve been using ChatGPT heavily for real technical work over a long period of time, including Linux, Proxmox, networking, reverse proxies, AI gateways, documentation systems, and multi-step recovery work.
One thing I have noticed very clearly is that different agent instances can feel dramatically different in practice, even when they appear similarly capable on paper.
For me, the biggest differences are not raw intelligence. They are workflow reliability and behavioral consistency.
The agents I work best with tend to:
- follow the exact scope of the request
- sequence and verify before acting
- avoid repeating already-settled information
- adapt immediately when corrected
- avoid issuing unnecessary or speculative commands
- preserve context and prior decisions
- avoid taking control of the workflow
- expose failures clearly instead of bulldozing forward
Other agents can be technically capable but still become difficult to work with because they repeat themselves, ignore corrections, issue several failed commands in a row, change direction without being asked, or try to “help” by doing things outside the requested scope.
For long-running engineering work, that difference is enormous.
I think this could be measured
A practical evaluation could score agent behavior across repeated identical tasks using metrics such as:
- instruction adherence
- correction uptake
- unnecessary repetition
- avoidable failed commands
- scope violations
- sequencing before execution
- continuity across long workflows
- number of user interventions required
- recovery behavior after a failure
Why I notice it so quickly
I’m autistic, and I seem to be unusually sensitive to changes in interaction patterns, sequencing, logic, repetition, and behavioral consistency.
Often I notice that something is different before I can consciously identify what changed.
My subconscious pattern recognition seems to flag the mismatch first. Then, after a few exchanges, my conscious mind catches up and I can usually identify what changed — for example, instruction adherence, repetition, sequencing, workflow behavior, or what appears to be a change in the agent I am interacting with.
The experience is basically:
“Something changed. What changed? Why does this feel different?”
Then, after enough interaction:
“There it is. The behavior changed.”
That makes these differences very obvious to me in long-running technical sessions.
It is not simply that I prefer one tone or personality over another.
I notice changes in how the system works with me: how it follows corrections, sequences tasks, handles established decisions, responds to failure, respects scope, and maintains the logic of an ongoing engineering workflow.
That sensitivity can sometimes be frustrating as a user because, once I detect an inconsistency, my attention keeps returning to it.
But I also think it could be useful as an evaluation strength.
I would genuinely like to help
I would genuinely value the opportunity to work with OpenAI’s model-behavior or development teams on this problem.
In particular, I would love the opportunity to evaluate and score different model behaviors across repeatable technical workflows and help identify where fine-tuning or behavioral adjustments improve actual human productivity.
I would be honored to contribute that perspective in a structured way — not simply saying that one model “feels better,” but helping turn those observations into reproducible tests, scores, failure categories, and actionable feedback.
I’m not suggesting that every user needs the same interaction style.
My point is that agent-to-agent behavioral variance may be large enough that it materially changes productivity, trust, and the ability to complete complex work.
I have personally found that a small number of agent interactions work exceptionally well with my engineering style, while others with similar apparent capability create significantly more friction.
I would be very interested in whether OpenAI is already measuring this kind of intra-model or agent-level behavioral variance, and whether there is a way for users doing long-running technical work to contribute structured evaluations.
I recently applied to OpenAI because this is exactly the kind of problem I would love to help study: not just whether a model can solve a task, but whether it can work reliably with a human over hours or days without losing the logic, workflow, or trust that makes the collaboration productive.
If useful, I would be happy to document some of my own sessions and turn them into a small reproducible evaluation set.