Capability ≠ Permission: A Verification-First Approach to Governed Autonomy

ChatGPT Use cases and examples
, ,
1

Governed Autonomy: separating AI capability from authority

I’ve been working on an R&D project called KAIROSEED, focused on a question I think becomes increasingly important as agentic AI systems become more capable:

«If an AI system can do something, what determines whether it is actually authorized to do it?»

My current working principle is:

Capability ↑ does not imply Authority ↑

I’m exploring this through a verification-first architecture I call Governed Autonomy.

The model is:

AI Capability → Agentic Capability → Consequential Action → Authorization → Execution → Verification → Evidence

The key boundary is between capability and consequential execution.

An agent may be able to reason, plan, call tools, coordinate workflows, interact with APIs, generate media, or eventually control physical systems. None of those capabilities should, by themselves, become permission to perform every consequential action available to the agent.

My KAIROSEED research therefore focuses on making that transition explicit:

Capability → Authorization → Enforcement → Evidence

Authorization should be a first-class architectural boundary for agentic systems, enforced outside the model at the point of consequential execution.

The engineering question becomes:

«Can we place an independently enforced, deterministic reference-monitor boundary around consequential tool execution so that an action is allowed only when an authorization artifact is valid for that exact invocation—and can we produce inspectable evidence of what happened?»

The core security claim I am testing is intentionally narrow:

«No protected tool invocation can execute unless an independently verifiable authorization artifact is valid for that exact invocation and is atomically consumed or otherwise made replay-safe.»

This requires treating capability, authorization, enforcement, and evidence as distinct. A model may propose an action, but it should not be the final authority or the sole enforcement mechanism.

I’m currently testing ideas around:

  • complete mediation across direct, indirect, nested, and distributed tool calls
  • explicit authorization evidence
  • policy-bound execution
  • binding authorization to canonicalized parameters, principal, tool, resource, scope, time, policy version, and execution context
  • replay prevention under concurrency, retries, crashes, and distributed deployment
  • time-of-check/time-of-use resistance
  • delegation and confused-deputy risks across agents, tools, services, and MCP servers
  • revocation and expiry of short-lived grants
  • fail-closed behavior and defined failure semantics
  • runtime containment, including quotas, sandboxing, monitoring, and emergency stops
  • audit evidence, including independently trustworthy roots, signing keys, timestamps, storage, and verification procedures
  • semantic equivalence between differently encoded actions with the same consequence
  • verification of blocked and permitted paths
  • separating runtime viability from permission creation
  • separating action proposals, authorization decisions, execution receipts, and observed outcomes

That last separation is especially important:

Action Proposal → Authorization Decision → Execution Receipt → Observed Outcome

An approval is not proof of execution, and an execution log is not proof of authorization.

I’m also documenting the assumptions and adversary model explicitly, including which components issue authorization, enforce decisions, protect keys, and record evidence.

One important boundary in the research:

I am not claiming formal proof, universal security, or production certification.

The current implementation and tests provide condition-specific evidence for selected behaviors. Independent reproduction, formal verification, concurrency safety, complete deployment controls, and production certification remain separate evidence gates.

I’m deliberately treating:

Tested ≠ Proven
Evidence ≠ Authorization
Capability ≠ Permission
Unknown ≠ False

as first-class engineering constraints.

Why I’m sharing this with the developer community

As agents become more capable, I’m interested in whether authorization should become a first-class architectural boundary rather than something implicitly delegated to the model or application layer.

The core proposition is:

«Autonomy can expand without authority expanding automatically—but only when authority is represented explicitly, narrowly scoped, independently enforced, revocable, and auditable.»

I’d particularly value criticism from developers building:

  • tool-using agents
  • multi-agent systems
  • agent orchestration
  • MCP integrations
  • production AI workflows
  • robotics / embodied AI
  • authorization and policy systems
  • AI evaluation and security infrastructure

The question I’m trying to answer is simple:

«Can we let AI autonomy expand without allowing authority to expand automatically?»

I’m publishing the R&D direction openly because I want the boundary tested, challenged, reproduced, and—where necessary—broken.

Capability is not permission.

GitHub / implementation artifacts:

2

Good framing, and the narrow claim is the right one to test. Some criticism of building this kind of boundary:

  1. Single-use artifacts vs. retries. If the artifact is consumed before execution and the tool call crashes halfway, the retry needs a new authorization, and that gets annoying fast. I feel you need an idempotency key bound into the artifact, so the same invocation can retry, but a different one can’t reuse it.

  2. Canonicalization is where it breaks. The artifact binds to parameters, but the model can send extra fields, string “500” vs. number 500, or a path that resolves to the same file. Reject unknown args and use strict types before you hash anything. Otherwise, two different encodings of the same action get two different decisions.

  3. TOCTOU on args. The executor should run with the exact canonical args that were authorized, not re-read them from the model output after the check.

  4. Receipt vs outcome. Your separation is right. One addition: put a hash of the authorized args in the execution receipt, so an auditor can match receipt to decision without trusting the log writer.

We enforce the deterministic part of this in agent-airlock (strict Pydantic validation, unknown args stripped or rejected, deny wins over allow), but not the single-use artifact part. Would be interested to see how you handle consumption under concurrency.

3

Thank you for the detailed feedback. Your points identify several important conditions that KAIROSEED needs to account for if its verification-first governance claims are to be meaningful.

I’m incorporating the feedback at the specification and validation level rather than treating the architecture itself as proof.

In particular, the current KAIROSEED model is being extended to explicitly account for:

  • Authorization lifecycle: authorization must have defined validity, scope, and consumption semantics so that retries or repeated requests do not implicitly expand the original authorization.
  • Action binding: authorization must correspond to a well-defined representation of the intended action, rather than relying on an ambiguous or mutable request representation.
  • Concurrency and state: authorization and execution need a defined relationship to the relevant state and execution context so that changes between decision and execution are observable and testable.
  • Revocation and expiry: authorization validity must remain bounded by explicit policy and lifecycle conditions.
  • Execution mediation: governance claims are meaningful only to the extent that relevant execution paths are actually covered by the governed boundary.
  • Receipt versus outcome: an execution record should not automatically be treated as proof of the resulting state. Outcome verification remains a separate evidentiary step.
  • Failure and recovery: retries, interruptions, partial execution, and recovery behavior need to be included in the validation model rather than treated as exceptional cases.

This fits the direction I am formalizing as:

CAN → MAY → BIND → ENFORCE → EXECUTE → OBSERVE → VERIFY → EVIDENCE

The important distinction is that these are currently specification and validation requirements, not claims that every requirement has already been demonstrated in every implementation.

The appropriate next step is therefore to convert each requirement into a bounded, reproducible test with explicit pass/fail criteria and preserved evidence. A passing result would establish the relevant property only for the tested configuration, threat model, and observation scope; a failure would identify the boundary requiring further refinement.

I appreciate the feedback because it helps turn KAIROSEED from an architectural proposition into a progressively testable governance specification.