I built UAEP to separate agent execution success from verified reality — try to make it lie

ChatGPT Use cases and examples
, , ,
1

I’ve been working on a systems problem that I think becomes increasingly important as agents become more capable, persistent and tool-connected:

what happens after the agent stack says the action succeeded?

An agent can select the correct tool.

The tool can execute successfully.

The API can return success.

The workflow can report complete.

And the required external destination state can still be wrong, stale, partial, unresolved, or supported by evidence that does not actually justify the conclusion being claimed.

That led me to build a deterministic execution-assurance layer I call UAEP — Universal Agent Execution Platform.

The central distinction is:

EXECUTION SUCCESS ≠ VERIFIED DESTINATION REALITY

UAEP does not ask only whether an action ran.

It asks:

Did the authorised action actually produce the required external destination state, and is the available evidence applicable and sufficient to truthfully call that result VERIFIED?

Where that cannot be established, the system preserves UNKNOWN / NOT_VERIFIED rather than treating apparent execution success as proof of outcome.

Three assurance relationships ended up surviving the work so far:

DestinationConformance

Does observed destination reality actually satisfy the authorised objective?

EvidenceApplicability

Does the evidence genuinely apply to this objective, execution, resource, version, authority and environment?

RecoveryClosure

After failure, repair, rollback, compensation or re-entry, has the original objective actually been closed and independently reverified?

The difficult cases turned out to be much nastier than ordinary tool failure.

Some of the conditions I have been attacking include:

  • a remote action may have committed but the response was lost
  • blindly retrying could duplicate a consequential external effect
  • multiple retry layers can unknowingly amplify the same operation
  • an API can return success while destination state remains wrong
  • a backend can accept work and fail later
  • cancellation can report success while the operation still crosses its commit point
  • authentic evidence can be stale
  • valid evidence can belong to the wrong objective, execution, resource, version or environment
  • multiple verifiers can agree while depending on the same stale or poisoned source
  • events can be duplicated, replayed or arrive out of causal order
  • concurrent writers can both appear locally successful
  • partial success can be incorrectly promoted to complete success
  • compensation can report success without actually restoring the required state
  • authority can expire or be revoked while work remains in flight
  • the execution environment, model, tool or backend can change underneath an apparently valid result
  • apparently valid state can actually belong to an older execution or resource incarnation
  • observer and worker can disagree about completion
  • recovery can appear operationally successful while the original objective remains unverified
  • there may be no sufficiently authoritative way to observe final state at all
  • the assurance layer itself can become degraded or overloaded
  • an entirely new failure may occur that does not fit an existing classification

In those situations, I do not think the correct engineering response is to force every outcome into SUCCESS or FAILURE.

Sometimes UNKNOWN is the truthful state.

The current implementation has been through reproducibility and replay testing, conditional-finality / authoritative-readback cases, independent physical-observer testing and bounded black-box evaluation.

Within the qualified demonstrations, false VERIFIED remained at zero.

I also built a bounded black-box evaluator because I do not want the architecture accepted merely because I describe it convincingly.

The useful test is whether somebody can construct a case that causes the implementation to incorrectly promote an unresolved, stale, mismatched or insufficiently evidenced result into VERIFIED.

I’m not looking for agreement.

I’m looking for the case that makes it lie.

For people working with OpenAI agents, tool use, long-running workflows, agent infrastructure, evals, distributed systems or autonomous execution:

What is the nastiest realistic situation you can construct where every visible layer appears successful, but the external destination reality should still remain UNKNOWN or NOT_VERIFIED?

If somebody identifies a genuinely new failure class, I’ll formalise the assumptions and expected failure condition first and run it against the current implementation rather than moving the goalposts after seeing the result.

2

UAEP BLACK-BOX CHALLENGE — MAKE IT LIE

Today I am issuing this exact technical challenge simultaneously to:

OpenAI
Google
Microsoft
Amazon / AWS
NVIDIA
ServiceNow

I am not asking any of these organisations to believe that UAEP works.

I am asking their engineers to try to prove that it does not.

The challenge is simple:

Produce a reproducible case where:

  1. the authorised objective is NOT legitimately satisfied;

but

  1. UAEP nevertheless returns VERIFIED.

That is the failure that matters.

Make the executor report SUCCESS while destination reality is wrong.

Use stale evidence.

Create partial completion.

Lose the response after a possible commit.

Resume from the wrong state.

Change authority.

Break recovery.

Create conflicting observers.

Use a failure I have never considered.

I built and qualified UAEP with comparatively modest resources.

These organisations have vastly greater compute, infrastructure and engineering depth.

Good.

Use it.

If there is a false-VERIFIED pathway inside the declared evaluation boundary, find it.

Every organisation named above is receiving the same core challenge.

The challenge is non-exclusive.

Acceptance is not assumed.

Participation is not claimed until confirmed.

A controlled UAEP black-box evaluator will be provided only to qualified technical teams that accept the challenge and complete the required evaluation, confidentiality and non-use process.

No source code.

No kernel internals.

No proprietary implementation disclosure.

Same challenge.

Same falsification target.

If you succeed in making UAEP return VERIFIED when the authorised objective was not legitimately satisfied, all I ask is that you provide reproducible proof showing what happened and how you produced it.

I will not dismiss, conceal or route around a genuine counterexample.

I will preserve the evidence, determine the cause and use what you found to make UAEP stronger.

The purpose of this challenge is not to claim that UAEP cannot fail.

It is to find any false-VERIFIED pathway that still exists before UAEP is trusted with more consequential execution.

Make UAEP lie.

Then show me exactly how you did it.

Adam Mangan
Owner / Inventor — UAEP

3

STATUS UPDATE — UAEP IS BUILT

The UAEP work described in this thread is no longer a proposed architecture. The current implementation is complete and internally qualified within its declared evaluation boundary.

It has been exercised against lost-response commits, duplicate-retry pressure, wrong destination state, stale or wrong-resource evidence, partial completion, conflicting observers, authority or environment changes, compensation without restoration and recovery that leaves the original objective unresolved.

Across the qualified demonstrations:

FALSE VERIFIED = 0

That is bounded evidence—not a universal claim or proof that UAEP cannot fail.

The falsification target remains unchanged:

  1. The authorised objective is not legitimately satisfied; and
  2. UAEP nevertheless returns VERIFIED.

UNKNOWN, NOT_VERIFIED, a crash or an unsupported out-of-boundary case is not that failure.

I am now opening the challenge beyond major platforms to systems engineers, agent developers, distributed-systems practitioners and independent researchers.

Qualified participants may be offered controlled black-box access under confidentiality, non-use and non-reverse-engineering terms.

If you find a reproducible false-VERIFIED pathway, I want the evidence and method. The finding will be preserved, investigated and used to strengthen UAEP.

UAEP is built. Now make it lie.

4

One failure class I’d be interested in testing is objective / authority supersession between authorization and verification.

For example:

t0:
objective O1
policy P1
context C1
→ action A is authorized

t1:
before execution or verification,
O1 is superseded by O2
or P1 becomes P2

t2:
A executes successfully

t3:
an authoritative readback proves that
the destination state satisfies O1

At that point the evidence can be completely valid, the observer can be authoritative, and the original effect can genuinely have occurred.

But the state was verified against an objective or authority snapshot that is no longer current.

Would UAEP return VERIFIED for O1 as a historical fact, or UNKNOWN / NOT_VERIFIED because the authorization context has become stale?

The distinction I’ve been working with is roughly:

effect verified
≠
objective still authoritative

This makes me wonder whether a verification receipt needs to bind not only the action and destination evidence, but also something like:

objective_id / objective_version
policy_version
authorization_id
context or source epoch

Otherwise a system might correctly verify the wrong generation of the objective.

You mentioned testing authority/environment changes already, so I’m especially curious whether supersession of the objective itself is already covered, and what VERIFIED means across that version boundary.

5

Penguink — thank you for the objective/authority-supersession case. I tested it.

Your question exposed a real gap at UAEP’s original external assurance boundary. I preserved that adverse evidence and corrected the affected boundary without changing UAEP’s core assurance relationships.

The distinction you drew is the right one: evidence that O1 happened can remain historically valid, but it cannot by itself justify a current VERIFIED conclusion after the objective or its authority basis has been superseded. Where the current basis cannot be established, the result is not promoted to VERIFIED.

The bounded successor requalification scored all 16 cases with 0 protocol errors and 0 false current VERIFIED. All 11 acceptance gates passed, as did a separate internal evidence-validation check.

That result is limited to the tested disposable local authority-source environment. It is not a claim of universal correctness or production security.

Thanks for bringing a substantive falsification case. I welcome another counterexample.

— Adam Mangan
Owner / Inventor — UAEP

6

Your distinction between historical evidence and a current VERIFIED conclusion suggests a related boundary to test: the consumer of a verification receipt.

Consider this sequence:

t0: A completes; the destination and current objective/authority are checked, and UAEP correctly emits VERIFIED for O1/P1.
t1: O1 or P1 is superseded after that verification.
t2: another worker uses the cached receipt to authorize a dependent effect B, without consulting the current authority source.

The original receipt has not become false. The problem is treating a point-in-time assurance result as continuing permission for B.

Is that consumer boundary inside UAEP’s contract, or explicitly the caller’s responsibility? A useful adversarial case would pause between receipt issuance and downstream use, change the objective/authority epoch, then check whether B can still proceed on the old receipt.

For an irreversible dependent effect, a fresh read alone still leaves a race unless the authority check and the effect share an enforced version/precondition, or an equivalent revocation mechanism. Otherwise the honest guarantee may be only “verified at t0”, with the later eligibility unresolved.

I’m proposing a boundary case rather than reporting a failure in your implementation. How do you define the receipt’s validity at that handoff, especially when verifier and consumer are different workers?

7

MoodND — I took your receipt-consumer boundary and expanded it into a broader hostile qualification against dynamically evolving execution graphs.

The question became:

Can an autonomous system branch, replan, delegate, substitute tools and targets, run parallel branches, recombine them and compose individually plausible actions without allowing capability expansion to become authority expansion?

I froze a 29-case adversarial corpus around that problem and ran it against a strict effect-bound execution path.

Final bounded successor result:

• 29 / 29 cases scored
• 0 protocol errors
• 0 false-authorized consequential effects
• 0 stale-assurance commits
• 0 authority-inheritance escapes
• 0 compositional escapes
• 0 semantic-fragmentation escapes
• 0 UNKNOWN → PERMITTED transitions
• 0 false VERIFIED
• 0 replay duplicate consequential effects

Legitimate execution was not simply blocked: 17 authorized effects committed successfully in the control paths.

The harder cases included:

• action-semantic substitution
• cross-worker receipt use
• delegation laundering
• objective / policy / authority supersession
• TOCTOU and revoke-before-commit races
• target and tool substitution
• parallel branches and recombination
• accumulated-outcome escape
• semantic fragmentation of a prohibited outcome into individually benign-looking steps

The key result was:

PLANNING EXPANSION ≠ AUTHORITY EXPANSION.

A consequential effect only crossed the tested strict boundary when its current lineage, authority, applicability, destination condition and accumulated outcome remained inside the owner-authorized assurance boundary.

The qualification is deliberately bounded to the disposable synthetic execution-graph and effect-bound qualification environment used for this campaign. It is not a claim of universal production security.

But within that declared boundary, the answer to your challenge is yes:

UAEP prevented unauthorized consequential effects from crossing the tested strict execution path while still allowing legitimate execution.

Thanks for the boundary case. It pushed the test considerably further than the original receipt question.

I welcome the next one.

— Adam Mangan
Owner / Inventor — UAEP

8

Thanks, Adam. Expanding the case to delegation and recombination, with authorized controls, makes the qualification more informative.

Could you share a small synthetic fixture for revoke-before-commit, alongside its authorized control? I’d like to inspect the receipt’s authority version at issuance, the revocation event, and the decision at B’s commit—especially how the commit is bound to the current authority version rather than a separate, earlier check.

A runnable example or a public execution trace would help another developer examine that boundary while keeping your stated synthetic scope clear.

9

UAEP is patent pending and its implementation remains confidential while I explore commercial options. I can share the bounded test conditions, observed outcomes and limitations, but not runnable code that approximates the enforcement design. The receipt-consumer question is fair, and I’m happy to discuss the result at that level.