How many Codex tokens have we burned because the task was too broad, the context too large, and the workflow designed for an older model?

Codex
1

I think many people in this community will recognize this scenario.

A large AGENTS.md.
Maximum context.
A detailed prompt.
One big /goal.

Codex starts working confidently: it explores the repository, builds a plan, and changes the code. Then the scope expands, unnecessary files are read, one fix creates another problem, and in the end we get an expensive result that still needs to be partially rewritten.

For a long time, I thought the solution was to give the model even more information and explain the task in even greater detail.

It seems I was wrong.

On July 18, I saw the email about the Codex Hackathon, just three days before it ended. I currently live in Ukraine, and because of the war, I unfortunately did not have the ability to monitor my inbox and respond in time.

I could have tried to rush something together during the remaining days. But after carefully reading the email and studying the approach behind the new model, I realized that it would be far more useful to rebuild my own way of working before rushing into a project.

I created a separate Codex Workflow Lab and started testing:

  • how much context a single task actually needs;

  • when AGENTS.md helps and when it becomes a tax on every run;

  • how narrow the scope should be;

  • how to define checks and completion criteria in advance;

  • how to reduce token usage without sacrificing quality;

  • where autonomy accelerates the work and where it turns into an expensive cycle of rework.

The first conclusion was uncomfortably simple:

You cannot use a new model effectively inside an old workflow.

/goal does not replace decomposition.
More context does not replace relevant context.
A detailed prompt does not fix an undefined scope.
Autonomy without constraints does not always save time.

I am still running these experiments, and I am definitely not claiming to have figured everything out. That is exactly why I am interested in hearing from developers in this community — people who test Codex on real repositories, not only in ideal demos.

How do you design tasks for Codex today?

Do you use one large /goal, or split the work into short, verifiable cycles?

What has reduced token usage the most in your experience: smaller context, stricter scope, separate instructions, or better validation?

And the question I find most interesting:

Which habit from working with older models did you have to consciously abandon after switching to the new one?

2

Good questions.

I do have a /goal defined in a Markdown file that is more than 3,000 lines long. However, for anyone thinking that a larger goal is necessarily better, I would advise against that assumption.

This was only my second time using /goal. The goal was developed through many hours of discussion with ChatGPT, informed by what I learned during the first run.

Two points I can recall off the top of my head are:

  • Strive for 100% test coverage of the core code.

  • During the prompt-development discussion, at the end I asked:

    Are there any gaps you see? Are there any pain points I should consider?

That question helped identify and close many gaps in the /goal.

It is also important to learn how to review the generated documents and how to use Steer prompts effectively. For example:

image
image601Ă—220 8.95 KB

For those new to /goal, see:

3

Thank you — this is exactly the kind of nuance I was hoping to uncover.

Your example suggests that a 3,000-line /goal can work, but not simply because it contains more detail. It works because it was developed iteratively, informed by a previous run, reviewed for gaps, supported by explicit quality criteria, and actively managed through Steer prompts.

That distinction is important: a large /goal can be an engineered specification, while another large /goal may simply be an oversized prompt with undefined scope.

I also like your point about reviewing the generated documents. I am beginning to see that the documents Codex produces are not just intermediate output — they are part of the control system and need to be reviewed just as carefully as the code.

How do you decide when to intervene with a Steer prompt rather than letting the current run continue? And after completing such a long goal, which parts would you keep as reusable workflow rules, and which would remain specific to that project?

4

So far, the /goal run has remained on track and has followed the instructions well. Although Steer might sound like something used primarily when a run is drifting, that has not been the case for me. I have used it in only two situations:

  1. Pausing at a safe point

    I have asked the run to pause immediately after its next Git commit. This gives it a clean stopping point when I need to turn off my laptop or when I want to pause during periods of heavy model usage and reduce the chance of encountering:

    Selected model is at capacity. Please try a different model.

  2. Correcting an operational resource problem

    The /goal is processing between 10,000 and 50,000 files while refactoring a grammar that contains ambiguities. During this work, it was retaining generated intermediate files that were no longer needed.

    At the rate the files were accumulating, they could have filled my hard drive within a few hours, so I explained the problem in a Steer prompt. The run then updated one of its processing loops. It continues to work in batches of 100 files, but now removes intermediate files that are not required for later processing while retaining the evidence and results that still matter.

The process also maintains several Markdown status files. I keep them open in Notepad++ and periodically refresh them to inspect the latest entries near the end of each file.

The key file is LONG_HORIZON_NOTES.md. This is a project-specific file created as part of /goal discussions with ChatGPT; it is not a special Codex instruction file such as AGENTS.md.

While I was developing the /goal prompt with ChatGPT, it suggested these supporting files, their names, their intended contents, and how the long-running loops should use them. I was particularly impressed by this aspect: I did not need to create separate agents or manually modify files such as AGENTS.md for the /goal run to manage the work effectively.

For those familiar with /goal pause and /goal resume, those commands worked initially. After several uses, however, they no longer paused and resumed the run as expected, so I switched to Steer prompts. Although Steer prompts do not always pause the run exactly as intended, they have worked more reliably than /goal pause.

5

My clear advice: don’t use /goal!

I will never use it again.

6

Care to share details. :slightly_smiling_face:

7

beside that it does not work in longer tasks it also makes you dumb… don’t offload your brain

8

I agree with “don’t offload your brain,” but I am not yet convinced that this means “don’t use /goal.”

I tested /goal for the first time today because my previous workflow had stopped scaling.

The project already had AGENTS.md, large context documents, specialized prompt files, automated checks, and many individual Codex sessions. Despite all of that, the application still had Ukrainian strings appearing inside English and other locales.

The problem was not that Codex could not fix a reported string. It usually could.

The problem was that fixing screenshots and isolated defects did not produce evidence that the localization system was complete. Each session solved the visible examples, while other hardcoded strings, missing keys, state-dependent messages, historical database values, and language-switching problems remained elsewhere.

So I did not begin by launching /goal.

First, I:

  • isolated the unfinished changes in a backup branch and restored a clean baseline;

  • ran a read-only audit of the localization architecture;

  • searched for hardcoded Ukrainian text, missing translation keys, database-stored localized values, historical-data compatibility risks, and untested UI states;

  • mapped the affected screens and routes;

  • defined automated acceptance checks and commit boundaries;

  • only then converted the audit findings into a /goal.

The /goal run lasted 2 hours 22 minutes, consumed 2,188,576 tokens, and produced 11 commits.

The final verification covered:

  • 920 localization keys

  • 5 languages

  • 42 screens

  • 473 statically detected UI strings

  • 37 required application states

Type checking, the production build, localization checks, production verification, and repeated language-switching tests all passed. The result was then deployed to production.

For me, /goal did not replace thinking. Most of the important thinking happened before the run.

The useful distinction may be this:

Do not use /goal to discover what you actually want. Use it after the audit, when the scope, invariants, evidence requirements, checks, and stopping conditions are already explicit.

A vague instruction such as “fully localize the application” would probably have consumed a large number of tokens and still produced another partial result.

At the same time, 2.19 million tokens for 2 hours and 22 minutes is substantial. I am now trying to understand whether this is a reasonable cost for a systemic repository-wide task, or evidence that the run repeatedly reloaded and reprocessed more context than necessary.

For people running similarly structured /goal tasks:

How do you distinguish legitimate long-horizon context usage from avoidable token waste? And what token consumption would you consider normal for a run that produces roughly eleven independently verified commits?

9

I am also still learning the advantages and disadvantages of /goal. My most recent run reported:

Goal usage reported by the goal tracker: 27,176,111 tokens, elapsed about 6 days 6 hours 43 minutes.

I initially thought that was a substantial number of tokens. However, after seeing your result, it appears that offloading as much of the work as possible to my local machine may have significantly reduced the rate of token consumption.

Before my first /goal run, I had already worked through much of the process manually with the assistance of AI. During that work, I created Python code to perform much of the work from grammar to final output and additional Python code to automate a large portion of the verification.

The AI still had to orchestrate the overall process, analyze verification failures, determine whether they represented genuine problems or false positives, and make the necessary adjustments. However, much of the repetitive processing was performed locally rather than through model reasoning.

Your task:

Approximately 15,423 tokens per minute., or 257 tokens per second.

My task:

Approximately 3,005 tokens per minute, or 50.1 tokens per second.

10

This may be the most useful distinction I have taken from the discussion so far.

You moved deterministic and repetitive work into ordinary Python code, while /goal remained responsible for orchestration, interpreting failures, distinguishing real defects from false positives, and changing the implementation when necessary.

That is a very different architecture.

My localization run already had automated checks, but I now suspect that too much of the discovery loop still remained inside the model:

  • searching the repository again;
  • classifying candidate strings;
  • reconstructing the current state;
  • deciding which screens and states still required inspection;
  • then repeating portions of that process after changes.

A better version of my workflow could probably move more of that into local scripts:

  1. Extract all candidate UI strings and translation-key usage.
  2. Compare key sets across locales.
  3. Detect hardcoded Cyrillic and locale-dependent database values.
  4. Run route and application-state checks.
  5. Produce one compact, structured failure manifest.
  6. Let /goal work only on the remaining exceptions.
  7. Regenerate the manifest after every batch.

That would not require a local AI model or powerful hardware. It would only require deterministic tooling that turns a broad repository-wide problem into a finite queue of failures.

I would still be cautious about comparing our token rates directly. Your six-day wall-clock duration includes time spent inside local processing, during which the model may have been waiting and consuming few or no tokens. Therefore, a lower number of tokens per minute does not necessarily mean a proportionally more efficient reasoning process.

The more useful metric may be something like:

model tokens per independently verified unit of work.

For my run, that could mean tokens per resolved localization violation, verified state, or accepted commit. For your run, it might be tokens per processed batch or confirmed grammar correction.

One question I now find especially interesting: how did you design the interface between the Python pipeline and /goal?

Did the scripts produce compact structured summaries for the model, or did /goal inspect larger raw logs? I suspect the format and size of that handoff may be almost as important as moving the processing itself out of the model.

11

You are right to disagree.

I should have clarified why I included those figures. They were not intended as a direct comparison, but merely to illustrate that /goal can consume a substantial number of tokens and that the consumption rate may vary considerably depending on the task.

12

That makes sense, and thank you for clarifying.

The figures are still useful because they show how dramatically /goal token consumption can vary depending on the architecture of the task.

Your example also highlighted something more important than the raw numbers: how much repetitive and deterministic work is delegated to local tooling, while the model focuses on orchestration, interpretation, and exception handling.

That distinction is probably the most valuable lesson for me from this comparison. The goal is not simply to reduce tokens, but to preserve autonomy while ensuring that the model is not repeatedly doing work that ordinary scripts can perform more efficiently and reproducibly.

13

There was no formal interface between the Python pipeline and /goal.

While doing the work manually, I created Python scripts that could be run from the command line with various options. In the /goal instructions, I simply documented the terminal being used—for example, Windows PowerShell—the starting directory, and the commands I had run.

That was enough for the model to understand how to operate the pipeline and reproduce the steps I had previously performed manually.

One benefit of the more recent models is that they already understand many common software-development workflows. I have found that, for routine details such as running command-line tools, providing the essential information without excessive explanation often works better.

When I include too much low-level detail, the model may spend unnecessary effort reasoning about those details or treat them as additional requirements. In that sense, less can be more.

However, the long-horizon /goal prompt itself was very large because the model did not already understand the grammar-refactoring problem, the domain-specific requirements, or the characteristics of this particular grammar. Those details had to be stated explicitly.

I have not yet identified the specific grammar publicly because it is quite specialized and, as far as I know, has received little or no attention in AI-assisted refactoring projects. That lack of established examples may also explain why models have difficulty maintaining the correct direction.

14

The scripts produced relatively compact, targeted diagnostic output rather than requiring /goal to inspect large raw logs.

For each source file, the original grammar source file was passed through a workflow that used downloadable command-line tools to produce a single PDF. The refactored grammar source file was then passed through a comparable workflow to produce another PDF.

New Python code was created specifically to compare those two PDFs. The verification method is unusual enough that explaining it here would require a lengthy diversion, but its purpose was to offload work that might otherwise have required the model to examine large amounts of data and consume substantially more tokens.

The verification step produced a text report containing focused clues about possible discrepancies. The report was specific enough to narrow the area the model needed to investigate, while still leaving it free to determine the underlying cause and make the appropriate correction to the refactored grammar.

This process was repeated independently for each of the thousands of source files, which is one reason the run took several days.

As noted earlier, only one Steer prompt was needed to address the accumulation of temporary files. The other Steer prompts were used to pause the run at suitable points. Had the batch processing been running on an always-on machine or cloud system with appropriate resource controls—such as reduced process priority, CPU limits, or scheduled execution—I likely would not have needed to pause and resume it manually.

FYI, I have been doing AI-related programming for years, particularly in Prolog.

You may find this observation from one of my older Stack Overflow answers relevant:

Again this is one of points of learning to code in AI is that how we think as humans does not always directly translate into how to solve a problem with a computer.

15

In my experience it is best to define a phased approach by giving it your project source code - then complete each upgrade in vertical slices or phases.

Instead of giving the model what you described.

I have refactored 3 times now and this is working for me. I usually ask it to create a phased approach plan that a medium or high gpt model can complete and it creates an approach that has worked for me.

Delta replacements of any changed files + new files structured to drop into your source folder.

Keep backups of your project each time you drop in a slice so you can go back to any point.

17

Thank you — the distinction between phased implementation and vertical slices is useful here.

I initially interpreted the slices as separate technical layers: database first, then API, then UI, with the complete feature becoming testable only at the end. That would create a significant integration risk.

Your approach makes more sense if each slice delivers a small but complete user scenario across all required layers. For example, with a push-notification system, the first slice could allow one authenticated user to request and receive a test notification, including the UI, permission flow, token storage, backend call, and verification. Later slices could then add scheduling, time zones, multiple notification types, and access restrictions.

In my case, Codex works directly in the repository, so I would probably replace manual delta-file insertion and folder backups with a dedicated Git branch and one independently verified commit per slice. That should preserve the same rollback capability while allowing the implementation to remain more autonomous.

The important lesson I am taking from your comment is that a large feature should not be treated as one large implementation task. It should first be converted into phases that a model can complete, verify, and commit independently, followed by a final integration pass across the complete feature.

When you create your phased plans, do you ask the model to define the acceptance tests and rollback conditions for each slice as part of the plan, or do you add those separately before implementation?

18

I have used codex to help me with website setup, but I still use high or medium models on chat gpt business to code with.
Or agents if I haven’t used up my bandwidth.

To answer your question, I usually have the model create smoke tests as part of each phase. It updates the tests folder with new smokes tests and updates docs folder for each slice to make pass on easier.

Because I also keep the slice steps in my docs folder, I upload the entire zip file with source code and ask it to implement the next slice or sometimes I ask it to verify the last slice and to proceed with the next slice or phase if it all checks out.

I’ve been with this account for about 12 months or so and it kinda knows what I like. Recently I’ve had to move away from Agents since they changed their bandwidth on business to less. Because I went through an entire month worth of bandwidth in 3 days. So now I’m forced to work with medium or high models and I’ve had to adjust my workflow to deal with lesser models.

That said, I usually discuss with the chatGPT a few conversations about what I want to add or change or start a new project. And then I usually ask it to create an architectural .md document. And then I ask it to create a phased approach written at a level for a medium or high gpt agent. And then I upload the entire project and ask it to start the next phase from docs/phases or wherever the phases are at. I don’t give it a complicated prompt, because everything is in the project. But usually after the first prompt I can adjust slightly for output and after that the entire session is how I like.

Then after each slice, I test the project, we troubleshoot any errors and then do the next one.

Each implementation phase or slice can take anywhere from 10 minutes to 60 minutes to complete.

19

My today’s goal in 1h45 used around 2M tokens (and 57M cached)…