Rollover Before Compaction Request

Codex Feature Requests
1

Dear Codex team.

Please expose the context window so the codex session can check it from within the session by itself.
Currently /status can only be used by the user and not the Ai.
Im trying to get it to automatically Rollover to a new session before Compaction, because compaction cost tokens.
I have the Rollover working, but without the Ai being able to see the actual % context left, it cant check if the work “its about to do” will lead to compaction, while instead it should then have rolled over to a new session.
Im setting this up as a skill, and your help on this last bit would be appreciated.

Hope im making sense.
Kind regards.

2

I’m not sure I get your proposal?

Any summarisation/information extraction of a prior session is going to cost tokens.

How do you propose to bring information over from the prior session?

Yes, at the moment we are relying on the system to decide when to force a compaction to create context space. In what ways can you do better? And whatever you do, it’s likely to cost tokens.

Btw, there is an experimental mode that might interest you.

Amongst other things, it provides the agent the ability to search prior context, including tool results that occurred before rollover.

To enable it, put this in your Codex config.toml:

[features.context_management]
experimental_mode = true

if you are running the app-server as a daemon, you may have to restart the app-server first:

codex app-server daemon restart

then start a new task … which should pick up the new config

Note, this was introduced to the forum here

3

I will check what that mode offers thanks for the tip.

To answer your question: How do you propose to bring information over from the prior session?
Im using the codex app on windows and always provide a new project with a Rules.md and any Md file it needs as a blueprint for what i want to develop.
When a couple of implementations are done, i have it write a consise Handoff.md
Which is a small rundown of the current application state.
Then i create a new session in the project and tell codex to read the rules.md and handoff.md

That has been my proven method of how i build applications and it works like a charm.
My new RollOver skill does this handoff.md update and new session creation automatically.

Currently i use /status to keep an eye on the context window to see how much % i have left before compaction.
When its at about 75-80% and i know the next update to the application is substantial enough that it will reach the point of compaction, i tell codex to RollOver.
It automatically updates the handoff.md creates the new session, reads both files there, and continues the work.

This way i save the token on the compaction.
I have control over the amount of data codex needs to start with again.
And my sessions dont need to compact.
Meaning no information on errors during development are stuck in context and such.
Its a very clean way (for me) to do development work.

Yes i do understand this method also costs tokens.
The only thing currently missing for me is to automate this process.

The best version of this skill would be that codex can check by itself before and after a big implementation phase what the % context window still available is, so it can auto roll over using my method, before compaction would happen.

Hope im making sense.

4

I was watching this topic and am glad you replied. What you described is almost exactly the same as a technique I use at times.

I also run into the same issue of having to manually run /status and paste the result back into the conversation for the AI.

For me, the extra effort is worth it if it helps preserve important information in the context. Automatic context compaction is something I actively try to avoid when possible.

5

Out of interest have you compared the output handoff.md with the compaction result and what aspects do you prefer about your handoff.md and how does the compaction fall short?

Wouldn’t a better solution be to actually hook into compaction and or replace its prompt so that process (which is automated) can produce your handoff?

Then you don’t have an interrupted workflow.

6

The handoff never contains “mistakes” made , its just a checklist of what has succesfully been implemented, and what the next implementation is about.
Plus the basic “check these things before starting any work” statements.
That by itself is a main difference between compaction and what i prefer.
I rather have a new session start with a clean current version, then the sum of all things leading to it which can be poluted easilly.

Another reason i dont want to hook into compaction is because codex updates all the time, and i do not want to overwrite open ai’s methods.
They have reasons to make the system as they do, eventho i might use some things differently.
If i start overwriting those methods to much, at some point i would not notice improvements they made along the way.

These are just (me) things ofcourse.
We all have our ways.

7

so you want to throw away the automated summary but instead manually create a Handoff.md?

why not let the AI write the Handoff.md? I find Codex is really at writing lists and remembering what it’s built and what’s left to do.

Granted it sometimes makes mistakes over rollover boundaries.

That makes me think what you really want is the ability to write your own rollover context prompt which is acted upon automatically.

Right at the end of your context, I’m pretty sure the LLM is very capable of summarising the things you want to put in your Handoff.

Then all we need is the hooks and configuration settings to allow you to do a custom rollover?

(btw, very useful discussion - and glad to see someone else using the excellent CLI tool!)

8

I did an AI analysis of this to quicken research.

Looks like there is some support already:

There is a custom compaction prompt setting, but it doesn’t guarantee exact control over what survives.

  In ~/.codex/config.toml, you can specify:

  compact_prompt = """
  Write a handover for the next context window. Preserve:
  - The original objective and all subsequent user corrections.
  - Constraints, decisions, and their rationale.
  - Completed changes, relevant files, and test results.
  - Unresolved issues and the next concrete steps.
  Omit repetitive tool output and superseded plans.
  """

  Or load the prompt from a file:

  experimental_compact_prompt_file = "/absolute/path/compaction.md"

So maybe that’s something to explore?

I would not be surprised if, with a good model, that will be able to get close to or beat your manual version and the speed will be far better.

The big plus is you won’t have to worry when the rollover occurs, because your rollover will bring the information you’ve prioritised over automatically.

The danger with a custom prompt (and your manual one) is that useful details outside of the Book of Work might be lost.

Anyway, I’ve learnt something here, great to know that’s customisable!

9

It is writing the handoff.md during the rolloff.

To make things easyer for this topic, here is the entire Rollover skill.
Codex made the skill after i explained what i needed from it.
(And you can see in there it needs my explicite mention to do a rollover, because i cant yet make it access the context window itself, needed to do all this automatic).

Checkpoint and continue in a fresh Codex task

Manual Rollover

Skill

Checkpoint and continue in a fresh Codex task
Perform the proven project rollover workflow only when the user explicitly invokes [$rollover] or unambiguously asks to use the rollover skill. Never activate it merely because a task is substantial or the context might be low. Do not ask for, estimate, track, or act on a Context Left percentage.

Stop at a coherent boundary

Treat invocation as a request to move the current project work into a fresh Codex task. If one atomic tool call is already running, let it finish, then stop normal implementation. Do not start another implementation, test, build, dependency, or Git operation in the old task.

For a post-implementation rollover, preserve completed work and verification as completed state. Tell the replacement task not to repeat them. When no next priority is established, set the next action to await the user’s instruction.

Capture the exact project identity

Before editing the handoff or creating the replacement task, establish the canonical project root:

  • For a Git project, resolve both the current working directory and git rev-parse --show-toplevel.
  • Otherwise, resolve the primary workspace root supplied by the current project.
  • Record the absolute canonical root, current branch when applicable, repository identity, and current execution environment.
  • Never infer a fallback root from the user profile, a temporary directory, the Codex configuration directory, or a drive letter.

Inspect the working-tree status and preserve all user-authored, untracked, and intentional existing changes. Read mandatory project rules and the existing root handoff.md when present.

Write the handoff

Create or update handoff.md in the canonical project root. Preserve still-relevant information and user-authored constraints while replacing stale progress. Never claim a check passed unless it ran successfully.

Use this structure:

# Project Handoff

## Rollover metadata
- Timestamp
- Canonical project root
- Execution environment
- Repository and branch, if applicable

## Current objective

## Completed work

## Important decisions and reasoning

## Constraints and protected resources

## Repository state
- Relevant tracked modifications
- Relevant untracked files
- Verification already completed

## Relevant files and symbols

## Unresolved issues and risks

## Exact next action

Keep the handoff concrete and compact but sufficient to continue without replaying the conversation. Include exact commands only when their exact form matters.

Create one replacement task

After the handoff is fully written:

  1. Resolve the saved project whose canonical primary root equals the recorded root.
  2. Create exactly one Codex task with that saved project’s identifier and the Local environment so it uses the same direct checkout.
  3. Never choose Worktree, Cloud, projectless, a new clone, or an invented project.
  4. Omit model and reasoning overrides so the user’s configured defaults apply.
  5. Seed the new task with the canonical root and instructions to read handoff.md plus all mandatory project rules, verify location and repository state, and continue from the exact next action.
  6. Once creation succeeds, stop project work in the old task. Never let both tasks edit or build concurrently.

If task creation is unavailable, finish the handoff and clearly tell the user automatic rollover could not be dispatched. Do not substitute context compaction or create a projectless task.

Verification required in the replacement task

Before any edit, dependency operation, Git mutation, test, or build, the replacement task must:

  • Resolve its working directory.
  • Resolve the Git top-level when applicable.
  • Confirm both match the canonical root recorded in handoff.md using the platform’s canonical path semantics.
  • Inspect its branch and working-tree status and reconcile them with the handoff.

If the root differs, stop immediately and report the mismatch. Do not create directories, clone, fetch, restore dependencies, build, or repair the location automatically.

For every shell command, set the working directory explicitly to the verified root or a deliberate descendant. Keep project files, generated artifacts, dependencies, tests, and builds inside the verified project tree unless the project already has an explicit user-approved external location.

10

But wouldn’t this skill become redundant?

If you can write a custom rollover prompt you can get it to deliver whatever structure of context you want it to begin with in newly rolled session?

I can see having a file might be useful for audit purposes, but that also risks racking up a load of wasted space over time with things you never look at again?

btw, I am a big fan of having a maintained Roadmap in the README or a supporting doc file in the checked in repo - but I digress!

11

The skill is solely to automate me doing this.

Hey codex.
Update the handoff.md so it reflects the current state of the application.

Codex now does work, and finishes.
I create a new session, and tell codex.

Please read the handoff.md and rules.md and any other md file that is required.
After that we talk more.

So, Codex reads everything, and i can begin.

This is very manual, and i have been doing it like this for months now.
So the skill now saves me time as i can simply check the context window with /status.
Then, when i think its time for a rollover i tell it to use that skill and it does all that for me.

I then jump into the new session and its there ready for me to continue.
It understands the project, where we are in it, and whats next.
Importantly, there is no leftover context from the previous session.
Its all fresh.

The purpose of my topic here is not to find a better way of doing this skill.
Its to ask open ai if we can get the context window exposed to the model in the session.
And the reason for that is, that i would like this skill to happen before compaction automatically.

Compaction is useful if you want/need the context of a very long session, which my workflow does not require.
I use many sessions instead, with my handoff.md as the controled bridge.
Is it better?
At the very least of this im not poluting the context in a very long session anymore, which i was doing in the past.
And that lead to many moments of: why does it not understand me/the project anymore.
Why is suddenly a feature gone that was already built by it.
Turned out the poluted context made its judgement foggy because there were moments of troubleshooting etc in there and feature requests that were already resolved.

Hence i switched to this method, and never had those issues again.

Now im simply trying to automate it, and the only missing link is codex not being able to access the context window because its simply not exposed to it.

12

ok so where I think your workflow is fundamentally different is this:

If I’m working on the same product - I tend to return to the (ever-lengthening[1]) session that I used to work on it last time.

To me your workflow would have the same outcome as if I wrote a custom compaction prompt to write out the things you most cared about for resuming after a rollover and I continued to use the same session, rolled-over, but with my custom compaction that included only the things you cared about - this would also be “clean” and left only with the things you wanted.

You can do this, apparently, today.

  1. of course it isn’t really lengthening at all - it’s only the size of the starting prompts, the compaction result and the current back and forth. ↩︎

13

And i think that demonstrates perfectly the difficulty but also beauty openai has to work with.
Many different ways of working with it.

I think all of the methods, will lead to a better usecase for us users.
Thanks for sharing yours, i have also learned from that.

14

I agree. Exposing the current context usage or remaining percentage to the AI would make automatic rollover much more reliable and help avoid unnecessary compaction costs.