The OpenAI API unlocked a whole new layer of building for me

ChatGPT Use cases and examples
,
1

Hi, I’m Andy.

I wanted to share a concrete example of what the OpenAI API can do when it is used as the intelligence layer inside a persistent production system rather than as a normal chat interface.

Over the past year, the OpenAI API has unlocked a whole new layer of building for me. With the API as the foundation and Codex CLI as my main coding partner, I have been able to build systems and workflows that I honestly would not have thought possible when I started.

One of the clearest examples is autonomous long form creative production.

Using ChatGPT 5.4 through my own agentic operating environment, I generated a 40 chapter book from structured creative requirements, then expanded it into a franchise package with season scripts, canon files, repair reports, visual direction, image prompt packs, and exportable artifacts.

The important part is not just that the model wrote prose. The important part is that the work ran through managed, resumable, inspectable production runs with persistence, guardrails, progress tracking, exports, review, iteration, and cost control.

Over the last 7 days, the system recorded 645,069,836 prompt tokens saved, with 95 percent saved, while OpenAI usage showed about 40.3 million total tokens and 1,428 requests.

The potential this unlocks is much bigger than faster writing. It changes the unit of work from a single prompt to a managed production run. A model can be used as the intelligence layer inside a system that plans, drafts, adapts, preserves context, tracks progress, exports artifacts, and keeps the work reusable.

This is already repeatable. I have used the same production system to generate a full book from structured requirements, adapt a draft novel into a screenplay, produce season scripts, preserve canon, create repair reports, and generate visual direction for franchise assets.

My point is simple: OpenAI models are far more capable than many people realize. The missing layer is often the operating environment around them.

I also want to be honest about something: I have a real vested interest in OpenAI continuing to succeed.

The work I am building is possible because the API is good enough to support it. When I say these tools are working, I do not mean that as empty praise. I mean they are enabling real builders to attempt things that were previously out of reach.

That matters to me, because my own work now depends on this ecosystem continuing to improve.

OpenAI provided the intelligence. My system provided the operating environment. Together, they made a new kind of production workflow possible.

openai_forum_stacked_evidence
openai_forum_stacked_evidence1656×12182 2.33 MB

2

That’s an incredible scale, Andy! 645M tokens saved via caching/optimization is a masterclass in cost control. It seems your ‘agentic operating environment’ is exactly what the previous posters are missing — a way to manage GPT-5’s power without letting it drain the quota blindly. Would love to hear more about how you handle state persistence between these long production runs!

3

Thanks Mate, I really appreciate that.

The big shift for me was treating long runs as production workflows rather than single chat sessions.

At a high level, I persist project state outside the prompt path, then bring back only the active working context needed for the next step. That lets the system continue across larger runs without constantly replaying everything that happened before.

So the pattern is basically: durable project state, managed run progress, reviewable outputs, and selective context rehydration when needed.

That has been one of the biggest unlocks for cost control and repeatability.

4

That’s a very robust architecture. ‘Selective context rehydration’ is definitely the key to scaling without hitting the token wall.

A quick follow-up: how do you handle the ‘rehydration’ logic? Is it a manual selection of files/data, or do you have an automated layer (like a vector DB or an LLM-based router) that decides which parts of the project state are ‘active’ for the current task?

Your approach to treating runs as ‘durable project states’ is exactly the shift from ‘AI as a toy’ to ‘AI as an industrial engine’.

5

That is exactly the part I ended up spending the most time thinking about.

For me, it is not fully manual, but I also would not trust a plain vector search or an LLM on its own to decide everything.

The way I think about it is pretty simple: the current task decides what kind of context is needed first. Then the runtime builds a small working view from the durable project state, the current run state, recent events, saved summaries, and any relevant files or artifacts.

Retrieval can help find candidates, and the model can help reason over them, but I still want the runtime to be in charge of what actually enters the prompt.

So the model is not carrying the memory. It is working inside a temporary view of the memory.

After each step, anything important gets written back into durable state, so the next step can continue without replaying the whole history again.

That separation has been the biggest unlock for me: the model reasons, but the runtime owns the memory, state, limits, and continuation logic.

6

That ‘Model as a Processor, Runtime as Memory’ split is the gold standard. It solves the context drift problem and keeps costs linear instead of exponential.

I’m curious about the ‘write back’ phase. When the system updates the durable state after a step, do you use an LLM to ‘compress/summarize’ the findings into the state, or is it more of a structured data update (like updating a JSON manifest of the project)?

This approach basically makes the context window size irrelevant since you’re managing the ‘RAM’ yourself. Brilliant stuff.

7

I think that split is the heart of it. For the write back phase, I treat it as a mix, not one single method. Some things are structured updates. For example, run status, chapter progress, exported artifacts, project settings, decisions made, current objective, next step, and so on. Those should not be vague summaries. They should be explicit state.

Other things are better as summaries or notes. In that case, the model can help produce a candidate summary, but I still prefer the runtime to decide where it goes, what type of record it is, and whether it should be treated as fact, draft, preference, evidence, or temporary working context.

So I try to keep a separation between: raw events, structured project state, human or system decisions, summaries, retrievable artifacts and the small working view sent back into the model. I would not say it makes the context window irrelevant, but it stops the context window from being the main memory container. That is the important part. The context window becomes more like the current workbench. The durable state lives outside it.

8

The ‘Context Window as a Workbench’ is a perfect analogy. It highlights that the LLM is a tool for transformation, not a database.

Keeping a hard separation between ‘Explicit State’ (status, decisions) and ‘Soft Context’ (summaries, notes) is likely why your system is so repeatable. Most people fail because they let the model hallucinate the project’s progress. By forcing the runtime to handle ‘decisions’ and ‘objectives’ as explicit data, you’ve essentially built a Compiler for creative work.

I’m really curious about the ‘engine’ under the hood. To manage this kind of ‘durable state’ and ‘selective rehydration’, what does your stack look like? Are you running this on Node/Python, and what are you using for the state layer? Is it a classic SQL/NoSQL setup, or are you leaning into something more specialized for agentic workflows?

Thanks for sharing these insights, Andy. This thread is a goldmine for anyone moving past simple RAG into actual AI production systems.

9

Yes, that is the part I would call the real foundation.

I do not think of it as just memory bolted onto an agent. I think of it more like a persistence layer for the whole operating environment.

The model should not be responsible for carrying the system forward by itself. The runtime needs somewhere durable to put the truth of the work: what the current objective is, what run is active, what artifacts exist, what decisions were made, what changed, what failed, what should be retried, and what context is safe to bring back later.

So under the hood, the important part is not one clever prompt. It is a persistence substrate that can hold events, state, files, summaries, decisions, and retrieval material in a way the runtime can reason over.

That is what lets the system resume, audit, recover, and continue without treating the context window as the database.

I am keeping some implementation details private for now, but the principle is simple: the model does the reasoning and generation, while the runtime owns durable state and decides what view of that state the model sees next.

10

Understood! Keeping the ‘secret sauce’ under wraps makes total sense when you’ve built something this foundational.

The idea of a ‘persistence substrate’ is a great way to frame it. You’ve essentially moved the ‘source of truth’ from a volatile context window to a durable system of record. That’s the difference between an AI demo and an AI factory.

Thanks for the masterclass on cognitive architecture, Andy. I’ll be keeping an eye on your updates — this feels like the direction the whole industry needs to head.

11

Personally, I’m more into the low-level side of things, learning how to write Windows drivers — so your high-level architecture feels like a different planet, but the logic of ‘source of truth’ is something I can definitely appreciate.

12

Low level work like Windows drivers is a different world, and I respect it. What I have been building is much higher level, but the same principle still applies: if the system of record is wrong, everything above it becomes unreliable.

That is why I ended up focusing so much on durable state, write back, recovery, and repeatable runs.

The part I find exciting is that once those foundations are in place, the speed of building changes dramatically. Work that used to feel like it needed a team and a long timeline can start becoming something one person can drive much faster with the right AI workflow around them.

I am still being careful about what I share publicly, but what I have talked about here is only the top layer of the architecture. One thing I love is that I keep discovering new patterns almost every day. It feels like a whole new profession is forming around this: AI systems architecture.

13

Spot on, Andy. It’s all about the integrity of the state—whether it’s a hardware register or an AI’s mission objective. Once you can trust the data, you can move at light speed.Agreed on the ‘AI Systems Architecture’ part. We’re moving from ‘prompt engineering’ to actual systems engineering. It was great chatting with you and seeing this from a different perspective. Good luck with the franchise production!

14

I’m working through Pavel Yosifovich’s ‘Windows Kernel Programming’ (Second Edition and Part 1) and simultaneously exploring the Linux side with Kaiwan N. Billimoria’s ‘Linux Kernel Programming’ (including Part 2 on Char Device Drivers and Synchronization). I even picked up ‘Android Security Internals’ to round it all out.

15

If you need help with anything mate, feel free to ask.

16

Thanks, Andy! I really appreciate the offer. I’m actually diving deep into low-level systems right now, learning to write drivers for Windows, Linux, and Android simultaneously. It’s a different kind of ‘runtime’ challenge compared to your AI engine, but the goal is the same: building something robust and scalable.I’ll definitely reach out if I hit a wall where your architectural perspective might help. Good luck with the ‘factory’, and looking forward to seeing where it goes!