yeah, it automatically creates spatial zones and regex and changes the scoring for accepted extractions.. the due date alone has ~12k regex - mostly because it supports 250 languages
Wow… So the labeler is not just correcting the current invoice. It is creating reusable extraction knowledge spatial zones, regex rules, and scoring changes for future accepted extractions. The thing I would be watching is rule explosion and over fitting. If the due date alone has around 12k regex rules because of language support, how are you scoping and ranking them? Are those rules global, per language, per vendor, per document type, or per field? And when a user corrects something, does it only adjust that vendor/template path, or can it affect scoring globally? That is the part I would want to get right, because reusable correction memory is powerful, but only if it does not turn into a giant pile of rules fighting each other.
Sorry if I am asking too many questions.
let me keep this 6 years of research result for myself 
Sorry, totally understood.
But maybe one more thing. There is alot of data in databases e.g. for adresses .. this way a damerau levensthein function can help to identify an address even when ocr made a mistake in a couple letters and automatically play around with kontrast and denoising and retry the ocr if needed.. this part is not completely finished yet..
and if you want to compare bounding boxes .. I did this here with aws textract analyze expense and analyze text…
That was hand written SQL and it was done under alot of pressure years ago when analyze expense came out and was really bad.. had multiple of such SQL in that pipeline.
It was probably at least a thousand times faster than our python apporach
Yeah, that old SQL looks like the kind of glue code people only write after fighting Textract for real. The database anchor idea makes sense. OCR on its own is fragile, but if you already know likely addresses, creditors, invoice fields or customer records, fuzzy matching gives you a recovery path instead of trusting raw OCR text. Damerau Levenshtein also fits OCR errors well, especially swapped letters, missing characters and small substitutions.
No need to share the private rule logic. One safe high level question when you compare bounding boxes, are you mainly checking field location stability, or using the comparison to decide which OCR or preprocessing path worked best?
it really depends.. obviously the line item table (if one exists) doiesn’t have the same bounding boxes on two different invoices. And for the rest this is also something I want to keep for myself.
It’s certainly interesting. Thanks for sharing with me.
It seems the problem has inflated now. Every model no matter what I use, CLI, API and the web models. All of them are producing output with assumptions. I literally have to audit every out put now before running it. Every one of their models is now doing it. Something is broken. Even when I use ironclad rules the models are all still using assumptions that seed drift or the code straight up doesn’t work. The amount of token wastage occurring now is staggering.
For me, 5.6 SOL does exactly what it should be doing, and better than 5.5 did.
Two friends of mine are working on a codebase, me with Codex, them with Claude.
What keeps my current and fresh agents, and any external (Claude) agent aligned is something I’ve implemented dubbed the Agents Control-tower.
Essentially, the AGENTS.md file is the starting point for an agent, written such that any agent with whatever context of the project from whatever AI provider quickly and token-efficiently understands how to work with the codebase.
AGENTS.md points to specific folders like instructions/, audit/, change-log/, and anything else relevant to your project. Given the scope and intensity of the current task, the agent is pointed to the respective folders.
Agents are instructed to keep the phase of the implementation in mind and use specific instruction folders for different phases.
Let’s say you’re developing a web-app (like we are), instructions/ contains subfolders like frontend/, backend/, and database/. Each of these folders hold various .md files like frontend/design-standards.md. Also, each folder has its own README.md explaining the contents of the folder, which .md to use for which task, etc.
All .md files have a standardised set-up:
- Content summary
- General instructions like which tools to use, which checks to do after implementation, etc.
- More specific instructions that are not always relevant (kept in the middle of the .md, because agents are best at reading top and bottom of text files)
- More specific instructions at the bottom of the .md
There’s also an instructions/workflows/ folder that contains our codebase-specific skills. Like /delegation.md, which instructs when and how an agent should deploy sub-agents. This .md file, for example, is pointed to through AGENTS.md which specifies that any implementation should check delegation.md. Other .md files can be more specific and only prompted to be read for specific tasks.
If set up appropriately for your use-case, this setup ensures each agent only has to start with reading AGENTS.md to get a full overview of the codebase. Any implementation will operate mostly within the bounds set by the control tower. Any new agent from whichever AI developer will be up-to-date of your entire codebase within one prompt. All without needing a massive context window to internalise the codebase, continuously reexplaining how the codebase works, staying within what is already implemented, and ultimately drastically reducing token usage.
Any implementation should prompt the agent to read AGENTS.md and follow it’s pointers. And, most importantly for this to work, the control tower should force agents to update it themselves or at least ask us (the users) whether a new implementation requires an update of the control tower. Additionally, the control tower should be checked regularly for whether it’s still all-encompassing as your codebase grows.
I’m not saying this is exactly how you, or anyone reading this, should implement this. I’m merely trying to inspire you using my setup as an example.
This would directly address the issue in this thread by more-or-less setting guardrails for the agents, clearly listing what parts of your code do what, and limiting the guess-work necessitated for agents for larger-scoped implementation.
Hope this helps anyone 
Always open to discuss or explain
PS we’ve also added an interactive codemap, showing all routes in our codebase, mainly to detect dead code, duplicated functionality, and to manage maintainability of the codebase. This codemap is entirely readable by our agents as well (pointed to through the control tower), making the agent again not having to guess or reevaluate on each prompt that spans multiple routes in our codebase. Instead, they have a graph they can read to directly know which paths are involved in the implementation and which changes affect other and which other dependencies.
I return to gpt 5.5 because SOL is too crazy intelligent. Its too much perfectionist. found always something wrong and project get stuck. It’s not balanced to reach the main goal
Yeah, I saw that too. Asked for a small refactoring and it did it correctly like a real dev instead of quickfixing it. I guess that’s a good thing - although it is very bad to meet deadlines.
Use Terra instead on x-high. It works like 5.5 but is faster. Waaay faster! I did an experiment where i checked out an old hash in git and told it to do something that 5.5 needed 2 days for.. it finished it in minutes.
Also you may want to tell Sol to use spark subagents. That’s a workhorse.
The more “advanced” these AI models become, the less reliable they sometimes seem at following clear instructions, respecting constraints, or even understanding basic language.
ChatGPT keeps making serious mistakes despite explicit prompts and repeated corrections, and Claude Code is no better. The problem is no longer just hallucination. It is inconsistent instruction-following, forgotten context, unjustified confidence, and claiming that work has been verified when it has not.
More capability means very little without reliability.
Hi and welcome to the developer community,
could you elaborate on that?
Maybe a peek into what you are working on or which stack?
Did you test the output quality ?
Like what exactly do you mean?
Scroll up .. I posted some screenshots of what I did so far with terra.
And today I did a huge refactoring on that.. Since yesterday I did this:
I’ve implemented a videocall system that has a sdk for so called call apps…
For example this:
And then I made a call app that allows to create and deploy other apps:
with theming:
The call app is like a person inside the video call.
Well, the thing is I don’t want to give the application builder to users. So the deployment of apps (immutable core + versioned modules + descriptor logic and core ui) should be without the application builder.
Well, I asked sol to split it up. And boy did it do a refactoring. It introduced so much stuff (and I am ok with that since it has to be done - I just wanted to do it later)…
I saw a few drifts so it is not ready to leave it unattented .. but it wasn’t much and when you keep track of the changes you can see it early…
The goal is to open callapps that fit to the context of the videocall.. the avatar in the call will suggest call apps…
e.g. if you are plannnig to open a company it should open a wizzard where you can do that step by step…
etc.. always depending on what you do it should load an individual call app - or create one
Thanks. I cannot share the engine or the architecture in detail, but I can give some context.
I have been building a fairly complex system for about six months using both ChatGPT and Claude, including Claude Code and Codex. Recently, we were working on a specific algorithmic problem. We identified the correct fix, but implementing it reliably became much harder than it should have been.
The main issue was not the complexity of the prompt. It did not matter whether the instructions were short or long, simple or detailed. GPT-5.6 repeatedly failed to follow constraints that had just been stated, forgot basic rules from the immediately preceding context, expanded the scope, and sometimes claimed that work had been checked when it had not been properly verified.
Claude Code showed similar problems. Even at its strongest settings, I have not found it close to Codex for this type of work.
Codex is not perfect, but when used on its own, it has been the most reliable for coding, audits, repository analysis, and following strict implementation instructions. My concern is that newer models may be becoming more capable in general, while becoming less dependable in the areas that matter most for serious engineering work: consistency, instruction-following, context retention, and verification.
You should try to let sol write a doctoral thesis about what you want to build and use a couple research agents before it does that. Then you need to tell it that you unhappy with that - no matter if you are or not… and then read it! In detail!
And then let it create an EPIC from that with checkboxes and then you can ask terra to create a sprint with ~50-100 checkboxes that it can do in one go and then let it run…
The thing is that reading the thesis will give you hints if it would do it correctly.
ah and from time to time use this prompt:
“check the folders and do an uncle bob run - it’s messy”
I like fable, I use it for most of my technical stuff.
