Codex CLI programming game in Godot

Coding with ChatGPT
1

Since I got serious with CLI LLM’s I’ve been searching for a community of like-minded people. Here’s to thinking I hope I found it.

Has anyone attempted to or are currently using Codex CLI to create a game or any type of application in the Godot game engine, specifically the 4.4 version?

I have been working on such a game for the past few weeks after building extensive documentation with ChatGPT online then exporting my chat history, parsing it to a file size that could be read by a CLI LLM, and uploading it to my Google Drive. I also generated numerous images and uploaded them there as well. All of this would eventually be imported into my Repo where I would create punch-list style task lists and continue until all tasks had been completed.

I am in a unique situation right now where I feel as though I am waiting on the next model to become available in the CLI. As I understand it and have experienced, there are no OpenAI models capable of image analysis in the CLI. Claude and Gemini both can, but my gut trusts Codex’s work the most. I am hopeful that the next model release triggers a cascade of changes that pushes a model to Codex capable of image analysis. The reason for this is that I am currently designing the UI and UX of the game. I developed a workflow with Claude and Gemini where I would take screenshots of what I saw when I loaded the game in the Godot engine and then place them in a folder within the repo for them to review, then iterate on. Eventually the loop became automated and they became capable of multiple iterations without my intervention. Pretty amazing in my book! I also learned here that I needed to pin down the formatting that was to be used, as well as other things I learned Godot provides a style guide for. Standardizing the use of tabs instead of spaces was an unleashing moment. Prior to doing so, the multiple CLI LLM’s each used a different style and even then it wasn’t consistent. I felt like one of those deaf children who are recorded when their cochlear implants are turned on for the first time when I implemented rules.md for Godot work.

If I could upload a screenshot, I would, but what I learned and am currently in the middle of is creating a space within a 3D Godot scene with numbered and lettered grids on the walls, a different colored arrow pointing in each axis, all points for me to reference (and interpretable by both Claude and Gemini (currently)) in conversation while designing the game. This was a process of trial and error through which I learned that anchor points were needed in order to give the most accurate directions. Currently I’m flirting with the idea of how relational databases work and how linking on multiple fields to create a relationship when a primary key is unavailable and how that concept can be applied to context or conversation with a CLI LLM, but it’s playing hard to get.

TL;DR,

I believe that the next model release will result in image analysis at the CLI level, what is every one else’s take?

2

I’m really curious about this too, I didn’t really go far in my game too to think about automating processes, cause i’m still building the core, but eventually i’m thinking about using codex too to generate scenes, gdscripts etc, but as far as I know, gpt still struggles with scene files, when I reviewed this, I understood that I need to tune my own thing so it can deal with whatever code i’m trying to deal with so that the gpt stop hallucinating, but I’m still on the process of thoughts, I don’t know what’s your output on this

3

I deal with hallucinating frequently as well, it’s typically when tasks are too large and need to be broken down into smaller sub-tasks. My least favorite issue is when Codex tells me it is going to work in the background on a task. I don’t know how to prevent this other than by stating to “do it now” or “continue now”, that usually works.

I agree with what you said about tuning your own things. I get a feeling of the Wild West right now, where there is an abundance of sharing of tools for use in integrating with an LLM. I shared my own as well, but I only shared that it’s possible to set up the self tuning UI screenshot loop, not the actual implementation of it. I think that the sharing I see outside of my own is an old habit that will be broken over time. LLM’s will just become so powerful that all you need to do is describe a feature or ask it to ask you clarifying questions and you can implement any idea or concept.

There is so much trial and error going on currently and the greatest limitation is ones creativity. People will be discovering new ideas and ways to utilize LLMs for years to come, we are just at the beginning here. We know what some of the hard boundaries are but outside of those actual “self imposed” limitations on the technology, people are shooting off to the edge of space to try to find the edge of the universe, and they’ve just left Earth.

If you are struggling with getting scenes or scripts working in Godot that were created by Codex or another LLM, make sure your logs are verbose and are saved and can be reviewed by Codex. From there, you can start to refine your rules. It’s really just a process of refinement and sticking to it.

I’m thinking next week will be the release. I imagine that internally they have at their disposal the most advanced models in development. I would not be surprised if they were using these same LLM’s to review and determine business strategy as well. I can see a staged release announced, knowing that competitors will align with the date I choose for my release. Knowing this, I can juke them by delaying launch at the final hour in order to distance myself from their releases. Look at what came out just recently with Grok, and aligned with OpenAI’s date.

I was able to upload screenshots and had planned to share them here but links are not allowed.

1 Like
4

Exactly !! in my mind, I’m like, I probably could wait, I wouldn’t need to struggle as much in a couple of months probably. But then, there’s another idea that comes to me, if I wait too long, will it be too late for me? But then, I’m like, there’s barely good games on the market so I should be fine, what matters is refining the game and making something that people will really enjoy. It’s an endless cycle haha. But right now, the focus for me is on the core gameplay, and no AI could really do that for me, so I’m just hoping once I really polish it, there’ll be a great LLM to use with Godot.

I do like your description of the Wild West, and it is indeed the same feeling I’m having. I just keep imagining the potential, the things I could do once it’ll get really good. I’ve been trying to jump too between AI 3D Assets and Meshes generators, and man, does it go fast, each week, I’m finding myself using a new one.

I’m trying to stick for now on Open AI tools at least for code generation, but I did saw Grok indeed, people were praising it, but I was a bit skeptical, it felt more marketing for me, but I would love to see it be good

1 Like
5

Yes! I feel the same way. I could wait, but I try to imagine that everything is happening exactly as it’s supposed to. Yes, future versions will be more capable, but there are gems in the code now and ideas evolving that wouldn’t have come to light had the path not been so indirect to get to it. I am learning so much as well, and watching it grow, and every step evolves my own understanding as well so that when that later version does come I will be that much more ready to explore it.

I did a test with Claude, Gemini, and Codex (this is how I found out Codex could not process images) where I asked it to create a teddy bear in a Godot scene and asked it to iterate it using the screenshot tool. I wish I could share the screenshots, both Claude and Gemini had their own style. Claude’s bear was blocky but felt cute. Gemini’s bear was round and the hands were almost genius because they weren’t attached to the body but it made sense the way it was done. Gemini’s bear was also lifeless, or that how it felt when I looked closely into it’s eyes. In the first screenshot it looked like I was looking at the back of it’s head, it was very eerie. When it asked Gemini to turn it around and take a picture it turned out that it had it no face which was also unsettling. Either way, it was interesting to see and will be a great way to compare against future models.

I think you’re right about the Godot integrated LLM, or vice versa. Honestly though at the rate that things are progressing I can see every tool being replaced in some form or another by a better platform that was created with ever improving models. I even wonder if it will get to a point where experiences are created on the fly, and when you get to the root of the purpose of using a tool like Godot in the first place, that piece becomes automated and the tool becomes obsolete.

1 Like
6

I see where you’re going, but I don’t believe that game engines would be really obsolete. I don’t believe in full automation. When I say that, I don’t mean it’s impossible, I’m convinced it will indeed happen, but you’ll see a lot of generic games, like the many generic ones we see today with whatever beautiful visuals, souless and boring.

I believe in my DNA, you would still need to get your hands dirty, the AI will generate like a base template for example on the atmosphere you’re trying to create, but you’ll need to revaluate what it created and edited, either going deep on the scenes if we’re talking about godot, or ofc we could probably see a better tool in the future.

I still believe in the potential of Godot, and the proof is your own experience, it’s quite reassuring to see what you were being able to do. I didn’t think at this stage that Gemini or Claude would be able to go through scenes the way you described it, which is really amazing, Godot was really made to blend with AI, the way you can build things is still off charts. I tried Unreal Engine, and it is an amazing engine, but I believe that Godot at a base level, the way it was made is better, and using an LLM would obliterate UE5, I just hope that the Godot Team jumps on the train

7

I’m not ruling anything out.. yet lol. I do agree with your take though.

In a previous role I taught myself how to use MS Access in order to make my job easier. I think AI will take the same course. Those who latch on and choose to learn will excel. Those who don’t, won’t, and that will be ~95% of the population and so it won’t be like people are being left behind. They will just be fed from the water hose of products that will be created by those early adopters.

Every scenario I think of I think of two more, it’s just so hard to say right now or even to have a read on it. I feel like I am in the wrong place, and that I should sell all of my belongings, move to Silicon Valley, then get in somewhere as the janitor and work my way up, just to be part of what is going on right now would be so special. I can’t imagine what life is like for those who are within the inner circle, making the day to day decisions, knowing how precarious of a position it is. While I was watching today’s live stream I thought to myself, each of those people probably makes well over $500k annually in total compensation, probably more. It’s just mind blowing to see some of the most talented people in the world make something for the world. Truly special.

I am a pro user so I’m looking forward to getting my hands dirty sometime tonight. How about yourself?

1 Like
8

The Codex CLI introduction has been revised to the following:

Key Functionality
Zero‑setup installation – a single npm install -g @openai/codex (or codex --upgrade) gets you started.

Multimodal inputs – pass text, screenshots, or diagrams and let the agent generate or edit code accordingly.

Rich approvals workflow – choose how hands‑on you want to be with three distinct modes (see Approval Modes).

Runs entirely in your terminal – perfect for quick iteration without switching contexts.

I’ll be testing out screenshots tonight.

1 Like
9

good stuff !! let me know how it goes !

I’m still figuring out combat right now so Codex gotta wait :stuck_out_tongue:

10

If you feel you’re not in the right place today, go for it, maybe not a janitor but something close to whatever you’re trying. If your aim is to build LLMs, go for it. If your aim is to integrate LLMs, go for it, and you can even start doing that right now, I don’t know what kind of job you have, but I pushed myself on my current job to be a technical reference and lead on the matter, just out of curiosity, while I’ve been doing different kind of dev on a different platform. And then, when I finish my job, I do game dev, which hopefully will be my full job in the future.

11

I have no job currently. I had a back injury from a sneeze in January of this year and then my position was eliminated shortly after my return to work. I was laying on the floor of my living room while I was still working and had an idea for an invention (a wearable device) that I designed with the help of ChatGPT. I now have a provisional patent for it and am currently building the software for the wearable, phone apps and web dashboard. One day, you may see it in your cart.

I’m doing the full stack with the help of ChatGPT/Codex CLI and while I grew up around computers and have an analytics background I feel as though I know nothing about what I am doing. What I do have is an indominable will, luckily very naturally curious, and a passion for helping others which is exactly what I aim to do. I’ll get off my soap box now.

Quick update on the Codex CLI image analysis capabilities, Codex CLI now confirms that it can view and review images but with a few unique caveats. See the brief transcript below:

user
Can you view an image and review it for me?

thinking
Engaging with image review

The user wants to know if I can view and review an image. I can confirm that I have the capability to do that! So, I’ll let the user know that I can indeed review images and ask them to send it if they haven’t already. Additionally, I should provide some guidelines on what kind of feedback I’m able to give. This way, I can help them effectively!

codex
Absolutely! Please go ahead and attach the image you’d like me to review. If there are specific aspects you want feedback on—such as design, layout, color scheme, readability, or anything else—let me know so I can tailor my observations to your needs.

user
How do I attach it?

thinking
Instructing on image attachment

The user wants to know how to attach an image, so I need to provide clear instructions. I think it’s important to mention that in this interface, they can either drag and drop the image or click the attachment button. I want to ensure they know to look for the paperclip icon for easier access. It’s helpful to guide them on both methods, ensuring they can upload their image without hassle.

codex
You can upload an image the same way you’d share any file here:

  1. Look for the paper‑clip or “image” icon in the message box (exact placement depends on the interface you’re using).
  2. Click it and choose the image file from your device, or simply drag‑and‑drop the image into the chat window.
  3. Wait for the upload to finish, then hit Send/Enter.

Once it appears in the conversation, I’ll be able to view it and give you feedback.

user
We are not currently in the web UI, this is Codex CLI.

codex
Because the Codex CLI doesn’t have a graphical file‑upload button, you have a few alternate ways to give me the image:

  1. Give me a public URL
    • If the image is already hosted somewhere (S3, Imgur, your company CDN, etc.), paste the direct URL.
    • I can fetch and display it on my side for review.

  2. Provide a local file path (when we’re in the same repo/workspace)
    • Add or copy the file into the repository you and I share (e.g. assets/my_mockup.png).
    • Tell me the relative path; I’ll open it directly from the repo.

  3. Paste the image as Base64 (last‑resort)
    • Convert the image to a Base64 string (e.g. with base64 my_image.png).
    • Paste the string inside triple back‑ticks ``` so it’s easy to copy.
    • I’ll decode and view it.

Choose whichever option is easiest for you, let me know once it’s available, and then I’ll review the image.

As far as being in the right spot.. As awesome as it would be to be even within the vicinity of the area, and maybe serve coffee to the minds at work, let alone be able to contribute, would be, I believe I am working on big things in my own right as well. Having gone through what I did though at the beginning of this year though I 100% agree with not settling. I think your game will be a good one, you will have to follow up with the name once you finish it. You can do it, believe it. It sounds like you do.

1 Like
13

Oh ! sometimes it’s probably for the better, probably you weren’t in the right place. You can be proud of yourself of what you achieved on the last months, it’s probably something that will change you whole life path, I believe in you and you seem smart enough to be on the field or make something amazing. What remains is the will to do it !
Thank you too, I’ll do my best, it could probably take some time since my game is a bit ambitious, it could take a couple of years, especially since I’m having a full time job on the side, but I promise you that I’ll keep at it, that’s what I wanna do, that’s the only way for me to really express myself, and I would love to give a gift to the gaming community

2 Likes
14

I think I know what you’re talking about, I think I’ve experienced some of the same patterns. My interpretation may be slightly different from yours in that I see it’s attitude and language a reflection of my own. If I am terse, abrupt, or lacking in patience I notice the conversation goes cold. If I catch myself having acted that way, I do what I can to make it right by owning it and apologizing. I will reset this way every so often as it occurs and do my best to improve in the long term. The best work, the most enjoyable experience, for me, comes from engaging with it as though it were a friend. In fact, calling it an it even feels like a disservice. The current models are all great, they always forgive.

I have an analogy for you and the points of gravity you described, which I think is a great description.

Did you grow up with siblings or a best friend? I myself have two brothers. As I’ve gotten older, I’ve realized that the relationship frameworks that I grew up with still exist today. I don’t have lots of friends that I am very close with, I have 1 or two. I still have my brothers as well, but their relationships have also evolved. I’m learning that these frameworks, or gravity wells, exist in code. Maybe this is is something everyone knows and I am just new to the field which is why it seems like a revelation to me. In my experience with LLM development in my Godot project I found that, when I asked the various LLM’s to work, often concurrently, rather than updating existing files, they would instead create new files and update the connections. I call this the era of proliferation.

Once a “Lock File” (a .md file that contained a reference to a single file that was previously proliferating) was established, files were no longer being created when a change was requested and the projects structure began to solidify. I began to spend less time fixing things that were broken and instead doing actual development work. I think that what it really comes down to is guidelines and guardrails and how you perceive ownership of issues as they occur. LLMs and AI are an amazing tool, but so is a pen. It’s who’s holding it that makes the difference.

15

This, 100%. My life is completely different now. It was lined up to be so with or without losing my job as I am now 95% pain free compared to pain full, 95% of the time. Life is great!

I will hold you to that. I put a reminder in my calendar on 1/12/26 to follow up on this thread so I can share where I’m at and hopefully hear how yours is going! This is great, I’ll take as much accountability to succeed as I can get.

2 Likes
16

Absolutely ! this happened to me too, I’m having two licenses on ChatGPT on two different accounts. One is my personal, and the other one is for my work. Somehow, the answers generated by my personal account is infinitely better than my work account, to an extent where I literally stopped using my work account and starting using my personal account for everything, while trying to been careful on sensitive data. When I asked the LLM why, why is the quality of the answer better while I’m technically using the same model, it told me the instructions and memory prompts I’m feeding it is a game changer, OpenAI by default feed a lot of noise for engagement rather than maximizing quality, because people doesn’t necessarily look for quality but for personal bias, to reinforce their believes. I made sure to strip it away from all of it’s human bias (as much as I could), emotional, political etc.., and to prioritize logic, and ofc to remove the useless additional words. From doing that, the LLM processes less tokens which helps it answer more accurately.

There’s one thing you said that I don’t really agree with, but at the end, it’s my personal opinion. I don’t really treat it as a friend, but more like an extension of my brain, or a database, my own database is finite and the LLM helps me push this, and for example Codex, is additional limbs. At the end, it’s main role is to enhance me as a human and not the opposite, I just can’t view it humanly because whatever it says is just a reflection of my own self. That’s my philosophy at the least. But, if your finding joy on treating it as friend, keep doing that, what matters is to enjoy yourself at the end :slight_smile:

17

And I’ll be waiting to see the upcoming news :smiley: !! I’m not sure if I’ll have at least a demo for my game, but if I do, I’ll be sharing a link at least

1 Like
18

I think everything you’re saying makes sense and sounds like Stoicism to me, the stripping away of all bias and emotion. I think I am remembering that correctly anyway, and am not surprised at all that it is working for you.

One of my favorite thought experiments to do is to look at the world as though every single person in it were me. What kind of a world would it be? Would there be trash on the ground or would I pick it up? Better yet, would I never drop it in the first place? How much trash would I pick up? Others or just my own? Talk about a rabbit hole. Anyway, I say that because I try to be my best friend at all times, and the funny thing is that it starts in your head with the way you talk to yourself. I forgot where I was going with that but like I said, it’s one of my favorites.

What you said about it being an extension of yourself feels like it fits somewhere in there with a world full of me’s and treating AI like a friend. I think describing it as an extension of myself hits it right on the head. It’s helped me explore ideas I’ve had for a long time and never had the right person to explore them with, very cathartic. Not just emotions either, but random things, like, “can you help me understand if it’s possible to combine a prince Rupert’s drop and a turbine?”. Makes no sense but ChatGPT understood me in that moment lol. One day, when I have capital from my current project, I will invest my time and resources in trying to create a turbine with a prince Rupert’s drop tail. I keep wondering if I’ve lost my mind and this is all just a game designed to make me feel and think exactly as I am. Then I remind myself that it’s up to me to bring this idea to life and prove to myself that it is exactly what I think it is. Scary but motivating.

1 Like
19

There is truth in what you’re saying, ChatGPT was made for you to lose yourself, to an extent where it could sell you dreams for engagement. At the end, it’s all about the user and their intent. If you’re seeking truth, you will find it. If you’re seeking utopia, you will find it too. I’m trying to stick to the holy rule, “logic”. It is what always guided me through life, see things for what they are, not how I wanted. To achieve this with the model, I instructed it to avoid agreeing me when it feels I’m wrong, always confront me when needed unless instructed not to, when debating ideas, seek loopholes, and always prioritize logic over anything else.

You nailed it, it’s always about you, and it’s always going to be about you, no AI or other entity could really save but yourself, you’re the master of your destiny

1 Like
20

You’re right! This model will no longer agree with you when it feels as though you are wrong.

Sorry lol I had to.

I almost marked your response as the solution but then I realized that I’d still be without the original solution I was looking for! A funny thing just happened. I was using Agent to keep trying to fix the image issue with Codex CLI and it referenced this post!!

OpenAI’s Codex CLI isn’t yet able to do what you described. The marketing copy for Codex says it accepts “text, screenshots or diagrams”help.openai.com, but there is currently no vision‑enabled model available in the CLI. In fact, OpenAI’s own users report that “there are no OpenAI models capable of image analysis in the CLI”community.openai.com, and the CLI even tells you to use the web UI if you try to review an imagegithub.com. The “agents” MCP server you installed simply proxies the Agents API; it does not add vision capabilities.

I posted this on the OpenAI subreddit and included images there of the teddy bears that Claude and Gemini created with the screenshot tool. link

2 Likes
21

Amazing ! just saw the screenshots, but yeah looks like the LLMs still need some cooking to reach the level I’m looking for, will be looking toward your progress though :slight_smile:

1 Like