Exactly that issue, I’ve put a reverse engineering skill for my agents so that they try to approach the problem from the final result back to the inputs
I use IDE tools and lints for that so agent has to run the command line tool to get the report and then it is presented that as a good signal for refactoring so all code before being committed goes through this procedure which basically skips me this issue.
Do you think it would be helpful to you to have a program that connects with Git and tracks not only what changed (Git) but why it changed. The purpose, the logic, the test that failed, etc. It would track all the reasons why the changes are being made or the plans for new code. It would also be able to generate a handoff document when the chat begin to lose functionality from being too long with too many compression cycles. It would do other things as well. I am working on it now, does that sound like something you might use if it were well executed?
@OjaiAlchemy that’s already the case for the most part.
My docs are structured by what they apply to. The repo subject is maybe a quarter of that with other 3 quarters describing exactly what you’re talking about + related researches and recommendations.
Including the tools and agent “layer” to maintain that.
Yes this happens to me as well. I have been trying to figure out how to solve this, but it seems that the issue is that they want it to make mistakes so it uses more of your CPU time and credits so that you have to buy more after codex breaks half of your codebase, then it needs lots and lots of credits to get it working again – i watched it go through the full 5 hours of hourly alloted time in less than 2 minutes trying to fix its own self-inflicted bugs.
Its either that or its caused by the new hooks they added where you have to hook into tool calls and such if you want codex to run custom instructions like it used to do with AGENTS.md
It sounds like a problem with AGENTS.md. Also, you might want to add a layer of more detailed planning documents so it would be like this: ChatGPT internal planning tool > detailed external template-driven detailed plan/TODOs > execution of the detailed plan/TODOs.
Make a rule in your agents.md to suggest fixes before implementing them
reinforce with
Only implement approved solutions to problems
and then, however you want your audits handled
Try this
backup the file i just do a copy and paste. then I tell codex to do a refactor. Also if the code is trying to do to much, I tell it to reduce the file and make helper files, or assistance to break up the roles. Your one code just might need to be broken down into smaller domain specific roles, and have one code to rule them all.
yeah that could be, maybe they’re trying to save tokens, I don’t know.
I talk to codex in Chinese before, today I changed to English and it suddenly remembered those rules in AGENTS.md…
What language are your rules written in?
English, and all docs are in English, As_Is, notes.md, spec of modules.
Try doing that in polish .. polish works better.. which makes a lot of sense
That link is arXiv 2503.01996, titled “One ruler to measure them all: Benchmarking multilingual long-context language models” by Yekyung Kim, Jenna Russell, Marzena Karpinska, and Mohit Iyyer. It introduces OneRuler, a benchmark for testing long-context LLMs across 26 languages.
The big finding: long-context ability is not evenly multilingual. As context grows from 8K to 128K tokens, the gap between high-resource and low-resource languages gets worse. English is not even the top performer in their results. It ranks 6th, while Polish comes out on top.
The spicy bit: adding a “maybe there is no answer” option breaks a lot of models. Their modified needle-in-a-haystack task lets the correct answer be none, and many systems start wrongly saying no answer exists even when the needle is present. They specifically call out OpenAI o3-mini-high as struggling with this, especially at longer contexts.
Why this matters: it means “long context” is not one ability. A model can look good in English, look good on easy needle tests, and still fail when the task requires multilingual retrieval, aggregation, or knowing when absence is real versus imagined. OneRuler tests seven synthetic tasks: five needle-style retrieval variants plus two aggregation tasks.
For your stuff, the metaphor is tasty: the model’s lens changes depending on language, context length, and instruction language. Same garden, different lens, different truth. The paper’s subtext is basically: “A longer window is not the same as better seeing.”
Heh. Sorry it was a thread with images and it tried to make it fit lol