nmurrell011:
Since I got serious with CLI LLM’s I’ve been searching for a community of like-minded people. Here’s to thinking I hope I found it.
Has anyone attempted to or are currently using Codex CLI to create a game or any type of application in the Godot game engine, specifically the 4.4 version?
I have been working on such a game for the past few weeks after building extensive documentation with ChatGPT online then exporting my chat history, parsing it to a file size that could be read by a CLI LLM, and uploading it to my Google Drive. I also generated numerous images and uploaded them there as well. All of this would eventually be imported into my Repo where I would create punch-list style task lists and continue until all tasks had been completed.
I am in a unique situation right now where I feel as though I am waiting on the next model to become available in the CLI. As I understand it and have experienced, there are no OpenAI models capable of image analysis in the CLI. Claude and Gemini both can, but my gut trusts Codex’s work the most. I am hopeful that the next model release triggers a cascade of changes that pushes a model to Codex capable of image analysis. The reason for this is that I am currently designing the UI and UX of the game. I developed a workflow with Claude and Gemini where I would take screenshots of what I saw when I loaded the game in the Godot engine and then place them in a folder within the repo for them to review, then iterate on. Eventually the loop became automated and they became capable of multiple iterations without my intervention. Pretty amazing in my book! I also learned here that I needed to pin down the formatting that was to be used, as well as other things I learned Godot provides a style guide for. Standardizing the use of tabs instead of spaces was an unleashing moment. Prior to doing so, the multiple CLI LLM’s each used a different style and even then it wasn’t consistent. I felt like one of those deaf children who are recorded when their cochlear implants are turned on for the first time when I implemented rules.md for Godot work.
If I could upload a screenshot, I would, but what I learned and am currently in the middle of is creating a space within a 3D Godot scene with numbered and lettered grids on the walls, a different colored arrow pointing in each axis, all points for me to reference (and interpretable by both Claude and Gemini (currently)) in conversation while designing the game. This was a process of trial and error through which I learned that anchor points were needed in order to give the most accurate directions. Currently I’m flirting with the idea of how relational databases work and how linking on multiple fields to create a relationship when a primary key is unavailable and how that concept can be applied to context or conversation with a CLI LLM, but it’s playing hard to get.
TL;DR,
I believe that the next model release will result in image analysis at the CLI level, what is every one else’s take?
How you’re handling the bridge between CLI output and Godot’s scene system—are you converting commands into nodes or actions dynamically? The performance and responsiveness side seems like it could be tricky, but if done right it’s super flexible. Anyway, great work and excited to see how it evolves.