Governed Autonomy: separating AI capability from authority
I’ve been working on an R&D project called KAIROSEED, focused on a question I think becomes increasingly important as agentic AI systems become more capable:
«If an AI system can do something, what determines whether it is actually authorized to do it?»
My current working principle is:
Capability ↑ does not imply Authority ↑
I’m exploring this through a verification-first architecture I call Governed Autonomy.
The model is:
AI Capability → Agentic Capability → Consequential Action → Authorization → Execution → Verification → Evidence
The key boundary is between capability and consequential execution.
An agent may be able to reason, plan, call tools, coordinate workflows, interact with APIs, generate media, or eventually control physical systems. None of those capabilities should, by themselves, become permission to perform every consequential action available to the agent.
My KAIROSEED research therefore focuses on making that transition explicit:
Capability → Authorization → Enforcement → Evidence
Authorization should be a first-class architectural boundary for agentic systems, enforced outside the model at the point of consequential execution.
The engineering question becomes:
«Can we place an independently enforced, deterministic reference-monitor boundary around consequential tool execution so that an action is allowed only when an authorization artifact is valid for that exact invocation—and can we produce inspectable evidence of what happened?»
The core security claim I am testing is intentionally narrow:
«No protected tool invocation can execute unless an independently verifiable authorization artifact is valid for that exact invocation and is atomically consumed or otherwise made replay-safe.»
This requires treating capability, authorization, enforcement, and evidence as distinct. A model may propose an action, but it should not be the final authority or the sole enforcement mechanism.
I’m currently testing ideas around:
- complete mediation across direct, indirect, nested, and distributed tool calls
- explicit authorization evidence
- policy-bound execution
- binding authorization to canonicalized parameters, principal, tool, resource, scope, time, policy version, and execution context
- replay prevention under concurrency, retries, crashes, and distributed deployment
- time-of-check/time-of-use resistance
- delegation and confused-deputy risks across agents, tools, services, and MCP servers
- revocation and expiry of short-lived grants
- fail-closed behavior and defined failure semantics
- runtime containment, including quotas, sandboxing, monitoring, and emergency stops
- audit evidence, including independently trustworthy roots, signing keys, timestamps, storage, and verification procedures
- semantic equivalence between differently encoded actions with the same consequence
- verification of blocked and permitted paths
- separating runtime viability from permission creation
- separating action proposals, authorization decisions, execution receipts, and observed outcomes
That last separation is especially important:
Action Proposal → Authorization Decision → Execution Receipt → Observed Outcome
An approval is not proof of execution, and an execution log is not proof of authorization.
I’m also documenting the assumptions and adversary model explicitly, including which components issue authorization, enforce decisions, protect keys, and record evidence.
One important boundary in the research:
I am not claiming formal proof, universal security, or production certification.
The current implementation and tests provide condition-specific evidence for selected behaviors. Independent reproduction, formal verification, concurrency safety, complete deployment controls, and production certification remain separate evidence gates.
I’m deliberately treating:
Tested ≠ Proven
Evidence ≠ Authorization
Capability ≠ Permission
Unknown ≠ False
as first-class engineering constraints.
Why I’m sharing this with the developer community
As agents become more capable, I’m interested in whether authorization should become a first-class architectural boundary rather than something implicitly delegated to the model or application layer.
The core proposition is:
«Autonomy can expand without authority expanding automatically—but only when authority is represented explicitly, narrowly scoped, independently enforced, revocable, and auditable.»
I’d particularly value criticism from developers building:
- tool-using agents
- multi-agent systems
- agent orchestration
- MCP integrations
- production AI workflows
- robotics / embodied AI
- authorization and policy systems
- AI evaluation and security infrastructure
The question I’m trying to answer is simple:
«Can we let AI autonomy expand without allowing authority to expand automatically?»
I’m publishing the R&D direction openly because I want the boundary tested, challenged, reproduced, and—where necessary—broken.
Capability is not permission.
GitHub / implementation artifacts: