The Human-AI Decision Boundary (Part 1 of 2): What Seven Governance Frameworks Get Right — and Where They Stop

A close reading of the academic and industry literature on agentic AI autonomy, and the structural question none of it has yet answered.

The question of where human authority ends and AI agency begins is no longer a philosophical problem. It is an engineering problem, a governance problem, and increasingly an accountability problem for any organization that has put an agentic system into production.

The past eighteen months have produced serious work on this question — from academic research institutions, from major AI developers, and from a government regulator. Seven frameworks warrant close reading. Together they represent the current state of thinking on how to specify, constrain, and govern AI autonomy. Reading them carefully also reveals a structural gap that none of them set out to solve. That gap is the subject of the second article in this series.

The Academic Frameworks

Levels of Autonomy for AI Agents

Kevin Feng, David W. McDonald, Amy X. Zhang — Knight First Amendment Institute at Columbia University / arXiv:2506.12469, July 2025

This framework proposes five levels of AI autonomy defined by the role the human occupies in the decision loop: Operator, Collaborator, Consultant, Approver, Observer. The progression is not about what the AI can do — it is about what the human does as AI capability increases. At Level 1, the human operates the agent directly. At Level 5, the human observes outcomes without intervening in the process that produced them.

The paper’s central argument is its most important governance contribution: autonomy is a design decision, not a capability outcome. A system technically capable of operating at Level 5 can be deliberately architected to function at Level 2. This is not a limitation on the technology — it is a deliberate governance choice. The paper argues it should be treated as such, made explicitly by developers, and communicated clearly to the humans who interact with the system.

This reframing has real consequences for how organizations think about deploying agents. If autonomy is a design decision, then shipping an agentic system without specifying the intended autonomy level is not a technical default. It is an ungoverned governance decision. The human-AI decision boundary was placed somewhere — it simply was not decided.

The framework’s contribution is conceptual clarity. It gives the field a shared vocabulary for locating human authority in AI decision processes. Its scope boundary is equally clear: the five levels classify position. They say where the human sits relative to the AI’s decisions. They do not address what the AI is deciding toward, under what organizational constraints, or on the basis of what human-approved intent.

Agent Behavioral Contracts

arXiv:2602.22302, 2026. Evaluated across 200 scenarios, 1,980 sessions, seven models from six vendors.

Agent Behavioral Contracts (ABC) is the most formally rigorous framework in this review. It introduces runtime-enforceable contracts comprising preconditions, invariants, and recovery mechanisms, organized around a foundational distinction: hard constraints, which must never be violated, and soft constraints, which admit transient violations provided recovery occurs within a bounded window.

The empirical results are significant. Contracted agents achieved 88–100% hard constraint compliance across 1,980 sessions and detected 5.2–6.8 soft violations per session that uncontracted baselines missed entirely. The framework proves a Drift Bounds Theorem: a system whose recovery rate exceeds its natural drift rate will bound behavioral drift to a calculable ceiling. This is formal governance — the kind of mathematical guarantee that turns a governance claim into a verifiable property.

The hard/soft constraint distinction is practically important. Hard constraints are the decisions that must never be made autonomously under any circumstance. Soft constraints are the decisions where autonomous action is acceptable within a recovery window. Some decisions always require human sign-off; others can be handled operationally provided they are corrected within a defined period. ABC formalizes this.

ABC’s scope is precisely where it states it is: runtime enforcement. It specifies how constraints behave once defined. It does not address who defined those constraints, under what authority, or how they connect to the intent of the organization that deployed the system. That sourcing question sits upstream of what ABC was designed to answer.

Agent Contracts

Ye and Tan — arXiv:2601.08815, January 2026. Zero conservation violations in multi-agent delegation benchmarks.

Agent Contracts governs resource consumption rather than behavior. The framework formalizes multi-dimensional budget constraints — token consumption, execution time, cost ceilings, delegation hierarchies — with a structural principle the authors call conservation laws: resource budgets delegated to a sub-agent cannot exceed the parent agent’s remaining allocation. Delegation does not create resources that do not exist at the parent level.

The ABC paper characterizes the relationship between the two frameworks accurately: one governs how much an agent may consume, the other governs how an agent must behave. These are orthogonal problems that production agentic systems face simultaneously. A system can consume resources within budget while behaving outside its behavioral constraints, and vice versa.

Agent Contracts, like ABC, operates at the enforcement layer. Both take the content of constraints as a given. Neither addresses where that content comes from.

The Trust-Based Delegation Literature

Drawing on MIT Media Lab — South et al., arXiv:2501.09674, January 2025 — and broader practitioner literature from 2025–2026. The permission ladder framing appears across multiple sources rather than a single canonical paper.

The trust-based delegation literature describes a permission structure as a ladder: read access, then action access, then decision access. AI systems begin with minimal permissions and earn expanded decision rights through demonstrated, audited reliability over time. The MIT Media Lab’s authenticated delegation work contributes the authentication and authorization layer for this concept — formal mechanisms for verifying that an agent is acting within the scope its principal granted, built on extensions to OAuth 2.0 and OpenID Connect standards.

The permission ladder concept is intuitive because it matches how authority is extended over time in any organized context. You begin with limited scope, demonstrate reliable judgment within it, and earn more. The broader practitioner literature extends this into the question of how trust accumulates — not merely how it is authenticated once granted. Delegations are micro-bounded, reversible, and accumulate through track record rather than being granted in a single event.

What the literature has not yet produced is operational precision: the specific evidence required to justify moving from action access to decision access, the governance process through which that determination is made, the criteria for reversing expanded permissions when performance degrades. The concept of earned autonomy is sound. The specification of how it is implemented remains open.

The Industry and Regulatory Frameworks

Singapore Model AI Governance Framework for Agentic AI

Infocomm Media Development Authority (IMDA), January 22, 2026. Announced at the World Economic Forum in Davos. Voluntary guidance.

Singapore’s framework is the first government-issued governance framework specifically designed for agentic AI systems. Its four dimensions — assessing and bounding risks upfront, making humans meaningfully accountable, implementing technical controls throughout the agent lifecycle, and enabling end-user responsibility — form a coherent and comprehensive governance structure. The framework requires human approval at significant decision points and places limits on agent autonomy and tool access.

Two aspects are worth noting for organizations thinking about where governance requirements are heading. First, the framework is described as a living document — what it requires today is not the floor it will require when revised. Second, it was announced at Davos, not through a domestic consultation process — its publication signals that AI governance has become a matter of international coordination rather than domestic advisory guidance issued in isolation.

The framework specifies what governed agentic AI must achieve. It does not specify how to design systems that achieve it. The distance between “humans must be meaningfully accountable” as a requirement and an operational architecture that delivers that accountability is exactly where most organizations find themselves without adequate tools.

Anthropic’s Framework for Safe and Trustworthy Agents

Distributed across Anthropic’s published model specifications and Claude Code permission architecture.

Anthropic’s approach establishes read-only defaults, requires explicit approval for system modifications, and permits persistent permissions for trusted routine tasks. The read-only-by-default posture makes the restrictive state the default — affirmative human action is required to expand what the AI can do, not to restrict it. This is a principled approach to placing the human-AI decision boundary at the access layer.

It governs what the AI can touch. It does not address what the AI is working toward, how that goal was authorized, or what organizational intent the permissions are in service of. Its scope is appropriate to what it claims to be.

OpenAI’s Agentic Governance Approach

Shavit et al., “Practices for Governing Agentic AI Systems” (OpenAI white paper); OpenAI Agentic Governance Cookbook, developers.openai.com, 2026. Model Specification, December 2025.

OpenAI’s Agentic Governance Cookbook demonstrates governance codified as versioned, automatically-enforced software packages — pip-installable Python policy packages that travel with the code, update through standard dependency management, and generate audit traces for every governance decision. The model specification states the governing principle: autonomy must be bounded by a mutually understood scope agreed between the system and the user.

Treating governance boundaries as software artifacts solves a real failure mode: governance policies that exist in documents but are never enforced because enforcement at scale requires continuous human attention that no team can sustain. The governance boundary runs because it is code.

The limitation is the same one that applies across the enforcement layer: the mechanism for enforcing a scope does not address how that scope is determined, how it connects to organizational intent, or how it should evolve as strategy changes.

What Seven Frameworks Establish — and What They Leave Open

These frameworks represent genuine progress on the human-AI decision boundary problem. They have established that autonomy is a design choice that must be made explicitly. They have formalized behavioral constraint enforcement with mathematical precision. They have defined resource conservation for multi-agent systems. They have provided authentication mechanisms for delegated authority. They have produced the first government-level governance structure for agentic AI. They have demonstrated enforcement infrastructure that runs without human intervention and generates auditable records.

Read together, they also share a structural gap.

Every framework reviewed here treats the human-AI boundary as a horizontal line — the AI operates at a specified autonomy level, within specified permissions, subject to specified constraints. What none of them addresses is the vertical dimension: the hierarchy of organizational intent within which that boundary sits. Goals inform objectives. Objectives bound strategies. Strategies constrain tactics. Tactics govern actions. An AI system is not merely operating at an autonomy level — it is operating toward something, within a chain of human decisions that authorized that direction.

The question that follows from this gap is what the second article addresses: how does the human-AI decision boundary connect to organizational strategy, and how do autonomy levels relate to the human-approved intent they are meant to serve? That question is what a framework called GOSTA — Goals, Objectives, Strategies, Tactics, Actions — was built to answer.

Sources: Feng, McDonald, Zhang — “Levels of Autonomy for AI Agents,” Knight First Amendment Institute / arXiv:2506.12469, July 2025. “Agent Behavioral Contracts,” arXiv:2602.22302, 2026. Ye and Tan — “Agent Contracts,” arXiv:2601.08815, January 2026. South et al. — “Authenticated Delegation and Authorized AI Agents,” MIT Media Lab / arXiv:2501.09674, January 2025. Singapore IMDA — “Model AI Governance Framework for Agentic AI,” January 2026. Anthropic model specifications and Claude Code permission architecture. Shavit et al. — “Practices for Governing Agentic AI Systems,” OpenAI; OpenAI Agentic Governance Cookbook, developers.openai.com, 2026.