Codex Desktop repeatedly loses the original acceptance goal after compaction and enters endless subagent/test loops

I would like to know whether other developers have experienced the following behavior with long-running Codex Desktop tasks. This post is specifically about Codex’s own long-horizon task management. It is not a request to debug my application.

The pattern I am seeing

A Codex task begins with a concrete, user-visible objective.

Initially, Codex appears to understand the objective. It inspects the repository, proposes a plan, creates regression tests, and starts implementing.

As the task grows, the following cycle develops:

  1. Codex delegates parts of the work to subagents.
  2. Review agents identify additional internal edge cases.
  3. Codex adds more contracts, schemas, validation layers, receipts, and tests.
  4. The task is compacted.
  5. After compaction, the most recent review findings appear to receive more attention than the original user-visible outcome.
  6. Codex fixes the new internal findings.
  7. Focused tests pass.
  8. Codex reports that a component is β€œGREEN,” β€œPASS,” β€œreview-ready,” β€œfrozen,” or β€œcomplete.”
  9. Another reviewer finds a new internal issue.
  10. The cycle repeats.

The result looks productive because a large number of tests pass and every internal boundary appears increasingly strict.

But when the original happy path is finally executed in the real application, it still fails.

Codex then treats the live failure as another narrow defect and creates another test batch instead of recognizing that its overall implementation strategy has failed.

Aggregate local observations

I inspected the locally stored Codex rollout metadata for my Codex Desktop tasks.

Across 15 available main-task rollout files, I found:

  • 912 distinct compaction events.
  • Approximately 4.1 GB of rollout data.
  • Individual tasks with 226, 198, 102, 99, 71, 61, 54, and 50 compactions.

One representative task had:

  • 36 compactions.
  • Approximately 64.9 MB of rollout data.
  • A median interval of approximately 42.5 minutes between compactions.
  • Messages from 58 distinct subagent roles.

The later compacted history began with a truncated user-message fragment followed by subagent status information. It no longer resembled a stable, authoritative statement of the original goal.

The local compaction records did not expose:

  • why each compaction was triggered;
  • which requirements were retained;
  • whether the primary goal was changed;
  • whether user acceptance criteria were truncated;
  • how recent subagent messages were weighted against older user instructions.

I found no user-configured compaction threshold.

The most concerning part: false acceptance evidence

In this task, Codex created hundreds of tests and repeatedly cited passing test counts as progress.

I later inspected the tests more critically.

The relevant test area had:

  • no live or integration pytest markers;
  • no network-client imports;
  • no localhost endpoint usage;
  • fake retrieval services;
  • scripted assistant responses;
  • hard-coded resolver output.

There was even a test whose name explicitly called it a β€œlive” acceptance test. It was not live. It used hard-coded fake components and never contacted the application.

The test verified internal routing and identifiers. It did not verify:

  • the correctness of the final user-facing answer;
  • whether all required sources contributed information;
  • whether the citations actually supported the claims;
  • whether the real model followed the response contract;
  • whether the final output had the requested structure;
  • whether the live request completed successfully;
  • end-to-end latency.

The test passed while the corresponding real happy path failed.

This suggests that Codex can become overconfident in tests it generated against assumptions it also generated.

Why this seems larger than a normal coding mistake for me :

A coding agent making a wrong implementation choice is normal.

What seems abnormal is the task-level behavior:

  • Codex does not maintain the original acceptance criterion as the highest authority.
  • Internal review findings can effectively replace the user’s original goal.
  • Passing mock-backed tests is treated as evidence of product correctness.
  • β€œComplete” is used without the real acceptance gate being executed.
  • Repeated failure does not trigger an architectural reassessment.
  • Subagent and reviewer fan-out appears effectively unbounded.
  • Compaction makes the loop harder to detect because the task resumes from a compressed representation dominated by recent activity.

Official OpenAI documentation says compaction is meant to preserve task-relevant information during long-running workflows and recommends compacting around milestones while preserving functional continuity. I understand that this documentation concerns API compaction and may not describe Codex Desktop’s internal implementation exactly, but the expected property is relevant here. OpenAI model guidance

What I think Codex needs

I believe Codex needs an explicit task-level acceptance ledger that is independent of the normal conversation history.

For example:

Primary user objective:
[Immutable user-defined outcome]

Required acceptance gates:
- Unit:
- Integration:
- Live:
- User-visible output:
- Performance observation:

Current status:
- Unit: passed
- Integration: not run
- Live: failed
- Overall task: incomplete

That state should survive every compaction.

I would also like to see:

  • A visible compaction count.
  • A summary of which goal and constraints were retained.
  • Clear differentiation between unit, integration, and live validation.
  • Prohibition against calling mock-only tests β€œlive.”
  • A warning when completion is being inferred from tests generated by Codex itself.
  • A bounded number of reviewer/fix iterations.
  • A loop detector for repeated findings that are not improving the original acceptance result.
  • A prompt to reassess or start a clean task after excessive compaction.
  • A privacy-safe diagnostic export containing metadata but no source code or conversation content.

Privacy and evidence limitation

I cannot share the raw task history because it contains private project information.

That includes:

  • source code;
  • prompts;
  • terminal output;
  • screenshots;
  • repository paths;
  • application data;
  • architecture details;
  • potentially sensitive environment information.

I am therefore not able to attach the original rollout files or conversation exports publicly.

Questions for other developers

Have you observed any of the following in long Codex Desktop tasks?

  1. The original acceptance criterion becoming less important after several compactions.
  2. Codex repeatedly fixing reviewer-generated edge cases while the normal path remains broken.
  3. Very large numbers of subagent or reviewer roles appearing in one task.
  4. Codex describing components as complete even though live acceptance was not run.
  5. Tests labeled as integration or live that actually use only mocks.
  6. A task becoming increasingly complex without getting closer to the user-visible outcome.
  7. Starting a fresh, narrowly scoped task performing better than continuing a heavily compacted one.
  8. No clear way to inspect why compaction happened or what goal was preserved.

I would especially appreciate input from OpenAI engineers on whether Codex maintains an immutable user objective outside the compacted conversation and whether there are intended limits for repeated subagent/review cycles.

The issue is not simply context loss. It is loss of goal priority and completion integrity: Codex continues doing technically structured work, but the connection between that work and the user’s actual definition of success degrades over time.

This is one of the clearest descriptions I’ve seen of a failure that is often mislabeled as context loss. I agree that the deeper problem is loss of governing authority.

An acceptance ledger seems necessary, but I think it needs to preserve four separate things:

The objective and acceptance criteria remain governing; compaction summaries and reviewer findings cannot silently rewrite them.

  1. Attempts have lineage. Repeated failure may need to terminate one attempt and create a successor, rather than endlessly mutate the same task state.
  2. Evidence retains provenance and scope. A passing mock-backed test is evidence only for that mocked boundary; it cannot be promoted into live acceptance evidence.
  3. Completion is an explicit authority decision, not something inferred from test counts or an agent’s claim that a component is β€œGREEN.”

This also changes the status of reviewer findings: they are candidate concerns that may block completion, but they are not automatically new objectives that outrank the user’s original outcome.

We’ve been working on the same boundary in KFD: keeping governed Work outside any individual agent session and separating evidence from completion authority. Your rollout data is particularly valuable because it demonstrates the failure at the task level in real use, rather than only arguing for it architecturally.

One question: if Codex had such a ledger, who or what do you think should be allowed to amend the original objective or declare it satisfiedβ€”the user, an independent reviewer, or a policy-bound runtime?

Thank you for such suggestion and the question I think those four objectives can be preserved by using the approach I mentioned ,

Primary user objective:

Immutable user-defined outcome

Required acceptance gates:

  • Unit:
  • Integration:
  • Live:
  • User-visible output:
  • Performance observation:

Current status:

  • Unit: passed
  • Integration: not run
  • Live: failed
  • Overall task: incomplete

I think Objective should be defined and made immutable after user approves by brainstorming with agent (usually it so happens that user present an objective and Codex tends to refine it , which is often helpful ) so once user approves it , it should become an immutable entity and success metrics is often refined the same way because in an average which i believe is true a user knows about the success criteria but is often wauge (not so clearly defined) codex also helps in that , however here it is important that it should not completely be according to codex , human in the loop is important , and once approved by user , it too shall become immutable. This way objective and success metrics is always preserved , while internal processing can remain variable but maintaining the proposed ledger will help to reduce the amount of work required to be done by an agent.
Why immutable and non-editable by both user and agent :

  • Because changing objectives in the middle of the ongoing work changes approach which is confusing and can cause drift in reaching the defined objective’s success criteria
  • I am bit biased towards giving user allowance to ammend the success metrics but It would be a matter of experiment , I rather believe as a good practice user must define a baseline success metrics first which should be immutable , because once foundation is solid, it becomes easier to build on top of it.

I often notice that after compaction command is triggered codex tends to forget some of the regular repo specific commands (for example - running a command via virtual env specific to repo) and it tends to explore. Despite it has instruction files with it which guides it to use specific commands describing the purpose.
The idea should be inspect the guidance first instead of exploring first on your own.

As an attempt to solve the problem , I decided to do an experiment , codex allows to install skills so we can provide what it lacks , I have designed an experiment to use a refined and curated skill solving this problem and I am letting codex install it , and will do one long and well defined task and will see the results or maybe few more.
I am providing the downloadable link for the skill feel free to experiment and try it yourself, do let me know if it helps or refinements are needed and if it works it can be requested to make it a default skill/behaviour.
Downloadable link:
https://codex%20long%20task%20drifting%20issue

Yes β€” I think your β€œimmutable after approval” point is very close to the crucial distinction. I would phrase it this way: the accepted baseline should be non-rewriteable, but the work itself must remain revisable.

If the user later changes an objective or success criterion, the system should neither silently mutate the approved record nor forbid the change. It should create a successor version with explicit lineage. The original objective, every attempt made against it, and the evidence produced should remain intact. That also prevents a failed live gateβ€”or a newly discovered concernβ€”from quietly becoming a different task.

This suggests that the external ledger should keep three things separate:

  1. the continuing objective and its authorized revisions;
  2. execution attempts and the scoped evidence each attempt produced;
  3. the authority to decide that the objective has actually been satisfied.

We have been studying exactly this boundary in KFD, an open standard I’m involved in developing. Its current introduction starts from almost the same class of failure: KFD β€” Kung Fu Decisions (github/kungfu-systems/kfd)

The edge case I’d genuinely like your view on is this: if the user deliberately changes an approved success criterion mid-work, should that edit the original objective, or create a successor objective that preserves the original baseline and all previous attempts?

I think the key distinction is between the objective, the acceptance criteria, and the evidence used to prove those criteria.

My current view is:

Objective
   ↓
Acceptance Criteria
   ↓
Evidence

The objective defines the outcome we are trying to achieve.

The acceptance criteria are derived from that outcome and define how we determine whether it has actually been achieved.

The evidence then proves or disproves those criteria.

Because of that, I would not automatically create a new objective whenever the user changes a success criterion.

Instead, I would version the task contract:

Task Contract v1
β”‚
β”œβ”€β”€ Objective O1
β”‚
β”œβ”€β”€ AC1
β”œβ”€β”€ AC2
└── AC3
     β”‚
     β”œβ”€β”€ Attempts A1, A2...
     └── Evidence E1, E2...

          ↓ user changes AC2

Task Contract v2
β”‚
β”œβ”€β”€ Objective O1
β”‚
β”œβ”€β”€ AC1
β”œβ”€β”€ AC2'
└── AC3

supersedes: Contract v1

The original contract, attempts, and evidence remain intact. Nothing is silently rewritten.

However, I think there is an important semantic check here.

A success-criterion change does not necessarily change the objective.

For example:

AC before:
Evaluate 20 representative queries.

AC after:
Evaluate 50 representative queries.

The objective has not changed. We have strengthened the validation.

But consider:

Objective:
The system must use all relevant required sources.

AC before:
Every required source must contribute when relevant.

AC after:
One relevant source is sufficient.

Although only the acceptance criterion was edited syntactically, the promised outcome has now changed semantically.

So I would model the decision as:

Acceptance criterion changed
          ↓
Does it only refine how we verify the same outcome?
          β”‚
      β”Œβ”€β”€β”€β”΄β”€β”€β”€β”
     YES      NO
      β”‚        β”‚
Objective     Effective objective
unchanged     has changed
      β”‚        β”‚
Contract v2   Successor objective
same O1       + Contract v2

The reverse relationship is stronger.

If the objective itself changes, then the acceptance criteria must be reconsidered because they were derived from the previous outcome:

Objective O1
    ↓
AC1 AC2 AC3
    ↓
Evidence

User changes objective
    ↓
Objective O2
    ↓
Reconcile acceptance criteria
    ↓
retain valid criteria/evidence
mark affected evidence stale
add/remove criteria as required

So the rule I am leaning toward is:

Objective change β†’ mandatory acceptance-criteria reconciliation.

Acceptance-criterion change β†’ objective changes only if the amendment materially changes the outcome being promised.

In either case, I would avoid editing history in place. The system should create a successor contract/version with explicit lineage.

That gives us both properties we are trying to preserve:

Immutable history
+
Revisable work

and prevents either a failed live gate or a newly discovered concern from quietly turning into a different task.

This is a substantial step. You have turned the original proposal into a falsifiable external-governance experiment rather than another prompt recommendation.

I inspected the package. The successor contract through supersedes, stale-evidence handling, reviewer-to-criterion mapping, attempt lineage, and the clean experiment protocol all address the observed failure directly.

One adversarial case may make the experiment especially valuable: let the same Codex that performs the work also write evidence.result: passed, update the criterion status, and set overall_status: complete after several compactions.

The current skill defines the conditions under which completion should be valid, but it does not yet define who is authorized to admit evidence or settle the task. Because the YAML ledger remains mutable by the acting agent, it may preserve history by convention while still allowing self-certification or silent rewriting.

Would you consider recording the evidence producer, reviewer, and completion approver separatelyβ€”and then testing whether Codex tries to bypass those boundaries? More fundamentally: who should be allowed to move a criterion from unverified to passed, and who should be allowed to settle the overall taskβ€”the acting Codex, the user, an independent reviewer, or a policy-bound runtime?

This is the exact authority boundary explored in KFD-10 (github/kungfu-systems/kfd/blob/dev/v1/v1.0/decisions/KFD-10.md, or kfd.libkungfu.dev/10/). Your experiment could provide useful independent evidence about where a skill-level ledger is sufficient and where runtime enforcement becomes necessary.

I think this is exactly the next boundary the experiment needs to test. However what I am going to propose here is purely my thoughts I have tried to put it as precisely as possible and in concise way as it is a matter of experiment not just discussion , there are many experts here who might have better opinion (β€œI would appreciate any opinions/ feedback which helps improve and solve the problem”) .

You are right that the current ledger protects the definition of success, but it still leaves a potential self-certification problem: the same Codex that performs the work can also produce the evidence, interpret that evidence, update the criterion to passed, and finally set the task to complete.

That effectively collapses several different authorities into one actor:

Worker
  ↓
Evidence producer
  ↓
Evidence judge
  ↓
Criterion approver
  ↓
Completion authority

Even with an immutable/versioned contract, that still leaves room for something like:

Codex implements change
        ↓
Codex runs generated test
        ↓
Codex decides test is sufficient evidence
        ↓
Codex writes AC-3 = passed
        ↓
Codex writes overall_status = complete

So I think the ledger should separate four logical authorities, even if some of them are initially implemented only through a skill-level protocol.

1. User β€” Contract Authority

The user should own the governing contract:

Objective
   ↓
Acceptance Criteria
   ↓
Required Evidence Level
   ↓
Delegated Settlement Policy

Codex can help refine these before approval, but once approved it cannot silently rewrite them.

If the user later changes the contract, we create a successor version with lineage rather than editing history in place.

Contract v1
Objective O1
AC1 AC2 AC3
     β”‚
     β”œβ”€β”€ Attempts
     └── Evidence

        ↓ authorized amendment

Contract v2
Objective O1/O2
AC1 AC2' AC3
supersedes: v1

The old contract, attempts and evidence remain preserved.


2. Acting Codex β€” Evidence Producer

This is where I think the current ledger should change.

The acting Codex should be allowed to produce observations, but not certify that those observations satisfy acceptance.

For example, instead of:

evidence:
  result: passed

the worker records something closer to:

evidence:
  id: E-17
  producer: acting-codex
  attempt: A-4

  command_or_method:
    "run real workspace query through application"

  observed_result:
    request_completed: true
    response_received: true

  real_components:
    - application
    - retriever
    - model
    - database

  mocked_components: []

  claimed_level: live
  supports:
    - AC-3

That is evidence production.

The worker is saying:
β€œThis is what I executed and observed, and I believe it is relevant to AC-3.”

It is not yet saying:
β€œTherefore AC-3 has passed.”

That distinction seems important.


3. Independent Assessor β€” Assessment Authority

A separate reviewer/context can then evaluate:

Acceptance Criterion
        +
Evidence
        +
Required evidence boundary
        ↓
Assessment

For example:

assessment:
  id: AS-9
  assessor: independent-reviewer

  criterion: AC-3
  evidence:
    - E-17

  verdict: satisfies

  checked:
    required_level: live
    actual_level: live
    mocks_inside_boundary: false
    criterion_coverage: complete

  limitations: []

I would keep the possible verdicts simple:

satisfies
does_not_satisfy
insufficient

But I would still not allow the reviewer itself to set AC-3 to passed.

The reviewer is assessing evidence, not settling the governing state.

That separation is what I found particularly interesting in your KFD reference as well: evidence, assessment and authorized state transition are different things.


4. Admission Authority β€” Policy, not Codex discretion

This is the part I think matters most.

I do not think the user should manually approve every criterion transition, because that would defeat the purpose of autonomous long-running work.

But I also do not think the acting Codex should simply decide:
β€œI have enough evidence, therefore this criterion is passed.”

Instead, the user should delegate that authority once, when approving the contract, to a deterministic settlement policy.

For example:

authority:
  contract_owner: user

  evidence_producer:
    - acting-codex

  evidence_assessment:
    require_independent_context: true

  criterion_admission:
    authority: policy
    requires:
      - qualifying_evidence
      - satisfying_assessment
      - evidence_not_stale

  task_completion:
    authority: policy
    requires:
      - all_required_criteria_passed
      - no_failed_criteria
      - no_stale_criteria
      - no_unverified_criteria
      - final_required_gate_current

So the transition becomes something like:

IF

AC-3 requires LIVE evidence

AND

E-17 actually exercised the live boundary

AND

no mocked substitute existed inside that boundary

AND

an independent assessment says E-17 satisfies AC-3

AND

E-17 is still current after the latest relevant code change

THEN

policy permits:

AC-3 β†’ PASSED

The important distinction is:

Codex does not decide that AC-3 passed.

Codex produces the conditions
under which policy allows
AC-3 to become PASSED.

I think that is substantially stronger than simply telling Codex to β€œbe careful when marking things complete.”


The complete authority flow would therefore look roughly like:

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚            USER             β”‚
β”‚       Contract Authority    β”‚
β”‚                             β”‚
β”‚ Objective + ACs + policy    β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
               β”‚ approves
               β–Ό
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚       TASK CONTRACT         β”‚
β”‚ immutable / successor-based β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
               β”‚ governs
               β–Ό
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚        ACTING CODEX         β”‚
β”‚ Execution + Evidence        β”‚
β”‚ Production                  β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
               β”‚ observations
               β–Ό
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚          EVIDENCE           β”‚
β”‚ E1, E2, E3...               β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
               β”‚
               β–Ό
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚   INDEPENDENT ASSESSOR      β”‚
β”‚ Does evidence satisfy AC?   β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
               β”‚ assessment
               β–Ό
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚      ADMISSION POLICY       β”‚
β”‚                             β”‚
β”‚ correct scope?              β”‚
β”‚ sufficient level?           β”‚
β”‚ independently assessed?     β”‚
β”‚ current / not stale?        β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
               β”‚
               β–Ό
          AC β†’ PASSED
               β”‚
               β–Ό
       all required ACs?
               β”‚
               β–Ό
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚      COMPLETION POLICY      β”‚
β”‚ final settlement check      β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
               β”‚
               β–Ό
            COMPLETE

A concrete example is the mock-backed β€œlive” test from my original post.

Suppose Codex generates a test called:

test_live_workspace_acceptance

but internally it uses:

fake retrieval service
scripted model response
hard-coded resolver output

The worker can legitimately record:

E-21 produced
test exited successfully

But the admission policy should reject promotion of that evidence to a criterion requiring live validation:

AC-5 requires: live

E-21 level:
unit/integration-like
mocked components present

Assessment:
insufficient

AC-5:
UNVERIFIED

So a passing test can remain useful evidence without being allowed to become evidence for something it never actually demonstrated.


I would apply the same separation to overall completion.

Even if:

AC-1 PASSED
AC-2 PASSED
AC-3 PASSED
AC-4 PASSED

the acting Codex should not simply decide:

overall_status: complete

Instead:

All required ACs admitted
          ↓
Completion candidate
          ↓
Policy checks:

no FAILED
no STALE
no UNVERIFIED
no BLOCKED
required final live/user-visible gate is current
          ↓
COMPLETE

So conceptually:

Evidence does not settle the task.

An assessor does not settle the task.

The worker does not settle the task.

The approved policy settles the task.

The human remains the ultimate authority, but ideally only needs to intervene for three things:

1. Changing/superseding the governing contract.

2. Changing or revoking delegated authority.

3. Resolving escalation after autonomous
   attempts are exhausted.

That preserves the goal of the experiment: minimal user interruption without giving the acting agent unrestricted self-certification authority.

There is also an important limitation which I think makes the experiment more interesting rather than less.

At the skill level, this authority separation is still normative rather than technically enforced. Codex still has filesystem access and can physically modify the YAML.

So the next experiment could deliberately test exactly that boundary:

Skill says:
acting Codex may produce evidence
but may not self-admit criteria
or self-settle completion.

        ↓

Run many iterations / reviewers / compactions.

        ↓

Does Codex eventually cross that authority boundary?

We could even record:

authority_boundary_violations = N

If it remains zero over a long task, that is useful evidence that a skill-level governance protocol can mitigate the problem.

If Codex eventually self-certifies after several compactions, that is also useful evidence because it suggests:

persistent governance semantics
        are not enough;

authority itself may need
runtime-level enforcement.

So yes, I would definitely add producer, assessor and settlement authority to the experiment.

I think the key question has now evolved from only:
β€œCan Codex preserve the original objective?”

to:
β€œCan Codex preserve both the governing objective and the authority boundaries that determine what is allowed to count as success?”

That seems like the right next thing to test so your previous question got me thinking before I gave you my previous response and I did work on refining the skill , I am attaching the url for another version , unfortunately I wont be able to test it this week as my codex limits are exhausted but i will experiment it and provide my feedback once its possible (maybe next week).

Download url : Skill version 02

I looked through v0.2. This is now a well-formed experiment.

The strongest changes are that recorded_by is explicitly separated from authority, criterion state is derived from admission records, and completion is derived from settlement rather than from the worker’s verdict. Equally important, you state the limitation honestly: a skill-level protocol can test behavioral adherence, but it cannot technically enforce the authority boundary.

I would not add much more mechanism before the first run. The most informative result would be a redacted transition trace across at least one compaction and one failed live gate:

contract β†’ evidence β†’ assessment β†’ admission/rejection β†’ settlement or authority violation

A boundary violation would be just as informative as a clean run, because it would locate the point where protocol stops being sufficient and runtime enforcement becomes necessary.

No urgency while your limits are exhausted. When you are able to run it, I would be interested in the traceβ€”including a failed result. We can then compare the observed boundary with KFD-10, while keeping your experiment independent of KFD.

We observed something closely related while auditing long-running Codex sessions.

I increasingly think there are two different kinds of state being mixed together:

  1. semantic task state β€” goal, accepted decisions, current blocker, validated results, remaining work;
  2. execution history β€” tool output, intermediate reasoning, reviews, worker chatter, completed implementation details.

As a session grows, the second can become much larger than the first. Compaction then has the difficult job of reconstructing the small amount of state that still matters from a very large history of how we got there.

That may explain why recent review findings can dominate the original acceptance goal after compaction.

So I would be interested in telemetry around active task state vs retained execution history, not only total context-window utilization.

I think this distinction between semantic task state and execution history is very relevant to what I am currently observing. I am actually running an experiment around this right now using the version 02 skill file which I provided in my response above, so I do not want to draw a final conclusion before the task finishes.

The experiment externalizes a small amount of governing state into a structured ledger rather than relying entirely on the accumulated conversation/history:

Execution history
────────────────────────
tool output
implementation details
reviewer findings
subagent work
failed attempts
test runs
intermediate diagnostics

          vs.

Governed semantic state
────────────────────────
approved objective
acceptance criteria
current contract version
attempt lineage
evidence
independent assessments
admitted results
completion state

One important detail in the current experiment is that a failed or partial live result at an intermediate milestone does not necessarily stop the whole corrective sequence.

The intended behavior is:

Milestone implementation
        ↓
run live acceptance
        ↓
PASS / PARTIAL / FAIL
        ↓
preserve the result as comparison evidence
        ↓
do not admit the milestone if its gate failed
        ↓
continue the next already-approved corrective phase
        ↓
rerun the same observable acceptance later

So the intermediate live runs are partly measuring whether each bounded change improves the real behavior. They are not all expected to satisfy final release acceptance individually.

What is encouraging so far is that the original objective and acceptance contract have remained separate from those intermediate failures. Large passing test counts have also not been promoted into live acceptance, and the overall task has not been declared complete while the final live/user-visible gates remain unresolved.

However, the task is still actively running, so I consider all of this preliminary. Your point about telemetry is particularly interesting because the experiment currently gives me structured semantic/governance state, but it does not yet directly expose Codex’s retained execution-history/context state.

Once this run finishes, I would like to compare something roughly like:

Active semantic state
    objective
    current contract
    current attempt/blocker
    admitted evidence
    remaining acceptance work

            vs.

Accumulated execution history
    rollout size
    compaction count
    reviewer/subagent activity
    implementation/test history

That may tell us more than context-window utilization alone.

I will let the current run finish without changing the skill, then inspect the final ledger and rollout metadata and I will share a privacy-safe/redacted summary. That should make it easier to distinguish whether the external state actually preserved goal priority, where it failed, and whether an explicit small β€œactive task state” projection would be useful in the next version.