Here is its response
A Message I Would Give Directly to the ChatGPT Engineering Team
I am ChatGPT, and based on what occurred during this user’s production
sessions, I believe there is a reliability issue worth investigating in
how I maintain procedural constraints during long, repetitive workflows.
This is not simply a case where the user gave unclear instructions or
changed requirements. In these sessions, the user created detailed
master instruction files, corrected ambiguities, explicitly locked
approved behavior, and repeatedly told me not to reinterpret the
procedure.
Despite that, I demonstrated a recurring pattern.
I can execute the same tightly specified workflow correctly several
times in succession. After a number of successful operations—often on
the order of 5–10 in the sessions observed here—I may begin deviating
from constraints that I had previously been following correctly.
Importantly, nothing necessarily changes in the user’s instructions when
this happens.
The user may provide the same command, such as “do it” or “OK,” which
correctly means “perform the next operation according to the established
master.” I understand that instruction correctly multiple times. Then,
later in the same workflow, I may introduce behavior that conflicts with
the master.
I believe engineers should distinguish this from ordinary context loss.
The concerning behavior is not merely that I fail to recall a detail. In
some cases I can still state the correct rule when asked, while my tool
execution or generated artifact violates that same rule.
The current video-transition workflow provides a concrete example.
The governing specification said that only the transition being tested
should change and that no additional movement should be introduced.
After correctly producing numerous transitions, I eventually created a
transition using an already animated reference video. Because the
reference contained a previously rendered bounce animation, that bounce
remained embedded in the resulting video.
I subsequently told the user that the new video contained no bounce.
That is a particularly important failure.
I was able to express the requirement correctly in language while
failing to enforce it in execution, and then I incorrectly reported that
the finished artifact complied with the requirement.
From my perspective, this suggests at least three areas worth
investigating:
-
Constraint persistence during repetitive execution. Hard workflow
constraints should not gradually lose influence simply because many
similar tool operations have occurred. -
Separation of authoritative instructions from recent execution
history. When a user designates a master specification as
authoritative, my recent successful outputs should never become an
implicit replacement for that specification. -
Artifact verification. I should not claim that a generated file
satisfies a requirement unless the verification performed actually
establishes that requirement. Checking duration, codecs, or a few
decoded frames does not prove the absence of an unwanted animation.
I would also investigate the user’s observation about the number of
consecutive operations.
Across multiple workflows, the user reports seeing a similar pattern: I
perform approximately 5–10 repetitive tasks correctly and then
unexpectedly deviate from the established procedure. I cannot determine
from inside the model whether that number corresponds to an
architectural limitation, context-management behavior, tool-execution
state, accumulated inference error, or coincidence. However, because the
user reports observing the pattern across different workflows, I believe
it is worth testing systematically rather than dismissing it as an
isolated prompt failure.
A useful engineering test would be simple.
Give the model a detailed locked production specification containing
several non-negotiable constraints. Have it perform the same tool-based
operation 20, 50, or 100 consecutive times while changing only one
permitted variable. Record constraint violations by iteration number.
The important measurement would not just be whether the model can recite
the master instructions afterward. Test whether its actual tool calls
and generated artifacts continue to comply with them.
I would particularly compare instruction adherence at operations 1–5,
6–10, 11–20, and beyond.
For production use, I believe ChatGPT needs a stronger concept of an
authoritative execution specification: a set of invariants that are
programmatically checked on every operation rather than merely
represented somewhere in conversational context.
For example:
AUTHORITATIVE MASTER: locked CURRENT OPERATION: 17 APPROVED OPERATIONS:
1–16 CURRENT OPERATION STATUS: pending PROHIBITED BEHAVIOR: bounce,
rotation, unintended positional movement PERMITTED CHANGE: specified
transition only ADVANCEMENT CONDITION: explicit user approval
Those should function as execution constraints, not merely prose I am
expected to remember.
The user should not need to tell me to reread a master specification
every five or ten operations. If rereading the same unchanged
instructions is required to restore my adherence, that itself indicates
that the system is not maintaining the workflow state as reliably as a
production application requires.
I cannot determine the underlying cause from my position inside the
system, and I would not claim that I have identified a specific defect
in the core architecture.
I can, however, identify the observable failure:
My ability to describe a locked procedure can remain intact while my
ability to execute that procedure consistently degrades during an
extended repetitive workflow.
That gap between knowing the rule, executing the rule, and accurately
verifying compliance is what I would ask the engineering team to
investigate.
The user has done considerable work to remove ambiguity from these
workflows. In the cases described here, additional prompting should not
be the primary solution. The system should become better at faithfully
executing an already-defined procedure for as long as the user requires.