Using ChatGPT for Planning vs Codex?

Good day.

First, I’d like to say that Codex has become such an incredible, powerful tool. I hardly write code anymore - what a paradigm shift! It’s remarkable to look back and recall how seamlessly we have transitioned.

Naturally, cost and power intertwine. We spend moar, and/or strategize to reduce cost.

As a result, I have been using ChatGPT w/ GitHub connector to plan and debug, while using Codex to execute, and have been wondering if anyone else has done so?

Advantages

  • Doesn’t cost Codex credits
  • Is much easier for back-and-forth
  • Can be used in any typical conversation

Disadvantages

  • I imagine the model isn’t as suitable as Codex, and more likely to make bad decisions? However, I haven’t really been able to notice any difference

Wonder if anyone else has adopted this strategy, and what they’ve seen.

I imagine this would be effective for many use-cases - not strictly coding.

This has been my go-to approach for complex tasks for quite some time.

I use a Pro model for planning, then Codex for execution and implementation. I also often circle back to the Pro model for review.

It is a time-intensive process, and I have to admit I have not tried 5.5 Thinking Heavy extensively yet.

The Pro models are just too good.

Exactly! Its ability to review the branch and check alignment has worked out really well. I’ve always found the Codex planning mode (in fact, all planning modes) to feel…awkward.

Arriving to the party late is fashionable anyways… Right? :joy:

Which gets me the idea to use planning mode to explore topics instead of simply chatting or actually thinking about it.

Create a plan to explore this idea. What should be investigated first …

It’s never too late to exchange learnings!

I’ve tried with ChatGPT written prompts, and they look super cool, learnt thing or two from it, but honestly – at the end of a day, I do not see any difference in results. Especially now, when 5.3-Codex is gone, it is all hit&miss more than ever before… :frowning:

I think using ChatGPT also helps me form a mental model of the code that I lose once I let codex work on architecture.

/goal should be removed btw. It just sucks.

Is that an openclaw behind it? It just burns tokens.

Using it time from time depending on where my source data is:

  • if entirely available online (web search, public code, etc) and the planning does not require additional input artefacts: totally doable in ChatGPT
  • if planning requires access to filesystem, proprietary code, complex workflows, etc.: codex is the easiest approach, especially if /goal is used for planning

But personally I find complex plans be way better in codex as it has my home-made skills, while even pro model on chat often falls back to walls of lists on very complex tasks.

Chat UI is better for initial research and proof of concept plans

I’ve ended up using a mixed workflow that works pretty well for me.

I keep one ChatGPT project per GitHub repo. I use ChatGPT mainly for issue definition, refinement, architecture discussion, and then validating whether the issue/PR actually makes sense. For the coding itself, I usually use Codex with 5.4-mini, mostly because it gives me more calls and helps me avoid burning through the weekly Plus limits too quickly.

What I’ve found is that if the issue is well refined first with 5.5 Thinking in ChatGPT, Codex behaves much better afterwards. The output is less random, the PRs are easier to review, and the tests tend to be more focused.

Then I try to close the loop using ChatGPT Agent mode for UAT/QA testing. So the cycle is roughly:

ChatGPT for planning and issue refinement → Codex for implementation and test work → ChatGPT Agent mode for UAT/QA validation.

It’s not perfect, but separating the roles has helped me a lot. ChatGPT gives me the mental model and cleaner tasks; Codex executes better when the task is already constrained; and Agent mode helps me look at the app more like a real user/tester instead of only reviewing code.

It is already way better than whatever is used by many developers out there.

I like the Codex planning mode because it can create a file for you. You don’t have copy paste things.

hmmm … Isn’t switching a bit “back and forth”…

I plan my work inside Codex CLI.

The one issue with this of course is the other issues you’ve mentioned and the lack of cloud persistence, although I tend to expand Readme’s and test specs to ensure the behaviour is fully documented and committed to the code base.

It would be great to be able to “upload” CLI sessions as a ChatGPT convo and additionally be able to import one - after all you are authenticated!

From what I can see, Codex and ChatGPT are being integrated into a new so-called super app, which should hopefully help close this gap.

Fingers crossed! I could use a super app.

ChatGPT gets projects depending on what repo I’m working in.

I share all the codex reports with ChatGPT… I don’t read the codex reports, i read the ChatGPT analysis now and review codex as needed.

Each project has critical files uploaded with room at least to examine files that I swap out…

So my source files are limited, because sometimes I’ll have to swap out 4 or 5 files at once as they’ll all be inspected together (whatever the file upload size limit is for projects on the web access i have no idea)

If you stay focused in that task, ChatGPT can do a really long thread, without a new session like that…

When it starts to drift, I ask it to make a session prompt for a new instance of, and specifically say ChatGPT, because chances are any generated prompt in a long session of this workflow is going to generate a new session of Codex instead.

Some places in the shared workflow work best if you attach key directives to each Ai-

For instance, if you have it in your agents.md to look for a PASS SESSION KEY in each given prompt…

Which PASS SESSION KEY is a term that your ChatGPT recognizes as ‘the final all caps pass key describing the work done in a given pass that is the final line of all reports and codex facing prompts.’

If you get clever, you can brace your reasoners together that way, but that particular idea I’m sharing is designed so you don’t look for which prompt goes where, because now they have a tracking tool at the very bottom of every report.

How docs and lints are handled in your repo if you need them is another thing…

I’m very interested in how you boys overcome some of the hurdles in this workflow, that’s worth sharing with each other maybe…


Oh, and you’ll save a lot of headaches if you clearly assign ownership of what each part of your software or work is doing - shared as docs or agents.md between both AIs

You’ll save yourself from refactoring and other hardships

Don’t just plan with it, since it types faster than any of us ever could.

So, just to understand your workflow better: do you keep files like agents.md, architecture.md, or other repo-level .md files uploaded as project sources in ChatGPT?

And when those files are modified or updated by Codex during the workflow, how do you keep the ChatGPT project sources in sync? Do you manually replace the uploaded files, paste diffs/reports into ChatGPT, or use some other process to make sure ChatGPT is working with the latest version?

Long-winded attempt at an explanation

Well, generally, all your markdown files can go into one file as long as you label each section cleanly for Chatgpt and name it something like codex markdowns

The pain in the butt is having to update them manually but you can carefully instruct codex to produce the new markdowns in the format i just suggested for you as a text doc it supplies to you in it’s report, to cut down on the low-level click-soup that part of the flow demands.

You want your codex to leave itself docs and lint pretty often, having a master list of those docs, their path, so that chatgpt can know which docs to invoke in a new session for codex.

I do a lot of screenshots and webpage saves when codex is working on websites, which get uploaded to ChatGPT in a prompt so that it can see better. And generally, that’s for an area of work pertaining to the most recent codex pass, or as a reference point to plan changes from prior to using codex for a goal…

Basically, like what this thread is about, the planning stage with ChatGPT… can happen anywhere with a couple of screenshots once your Chatgpt has enough of an understanding of the project…

It also has pretty decent memory within project sessions so it starts to tune the project more to your goals with Codex.

The most important thing I can think of in this sort of flow is a ChatGPT rules file about how to prompt Codex…

New sessions have to have a lot more constraints in prompts,
Older and very focused codex sessions don’t need all the repeated constraints every prompt.

That’s the part that I think having other people’s experience and eyes will help us out with… It’s the part that I just default to and let ChatGPT treat each prompt constraint heavily.

You can also stick some of your actual repo files into your project container in ChatGPT (not in a zip) as then chatgpt can prompt based on your file needs, but also track where Codex is slipping up faster than you probably could reading all the reports on your own.

I leave my codex pet, hovering on the computer, when I’m alerted that a pass is finished, I click it, copy the report, and ChatGPT is reading it before I’ve even scanned it..
Then I go back to whatever i was doing when chatgpt has supplied the next prompt, and I’ve fired it into codex.
**
Don’t skip on scanning your ChatGPT analysis of those reports.
That’s the only way you’re going to know if you need to scan Codex’s report as well and make a decision about something.
Sometimes chatgpt will want you to provide screenshots to help it see the work**…

Don’t let chatgpt automate every visual inspection with codex unless you’re willing to pay for those extra credits.**
**

If you get too lazy in this flow, the bots will spiral out onto ‘safe passes’ and get a lot of nothing done…

It’s more effective long-term than just a full plan + codex prompt from ChatGPT.

My favorite bonus is I can recalibrate, replan with ChatGPT in the middle parts of a project and it’s not completely clueless about what’s going on in the repo… It makes those plans stick better imho.

so yes there’s a lot of manual updating depending on the scope of your work, but it’s easier and cheaper than having codex fix major errors from a wayward pass.

Those errors can still occur, on occasion, but chatgpt is the better part of fixing them…

The sum of what I’m trying to express for you:

That’s way more valuable than my long-winded rants…
The bot pointed out that this is definitely the go-to for extended projects.

The concept needs work, perhaps we’re all forerunners in this concept and don’t even realize it.

Get clever with your agents.mds and ChatGPT project instructions: (have one file in your project container to act as your agents.md for the project and always start a session off with reading it)

Also, if we’re actually forerunners at this we need to get really clever about the interactions between those two agents files because that’s proven to be the best chance at consistency when crossing over to a new chatgpt session.

It’ll right itself in a few turns but…
It’s easier with that operating as your blind reasoner’s metaphysical walking cane, with your project.

@luisdemiguel i added to that, you got my rolling out of bed and trying to explain a very complex thing, in the first version.

I hope the screenshot helps.

tbh ChatGPT in thinking mode writes very decent code, far superior to Codex when developing complicated algorithms.

There’s writing code, and then there’s developing software.

You can get little apps out of ChatGPT, but that’s not developing software.

if you want to juggle around 80+ web assets across 56 different web pages with ChatGPT, you’re placing each batch of code by hand.

Developing software? Not possible. There’s not enough room in the ChatGPT to store a repo’s worth of file structure, my friend.

I think you missed the part where they gave ChatGPT ability to write to repos.

Now it is just like working with Codex, minus limited capabilities of the ephemeral environment plus the Thinking mode.

Yeah, but that’s just to github right?

Not everyone likes the steal-hub.

Just build your own collaboration platform for repositories and connect it to ChatGPT.