Enterprise AI · Agent Architecture · Governance

The Problem with Agent Autonomy
Is That Nobody Defines It

Every enterprise AI agent has implicit autonomy permissions. Almost none have explicit ones. A framework for calibrating what agents can do without asking, and why doing it at the agent level is the wrong unit of analysis.

September 1, 2026  ·  Arjun Jaggi  ·  14 min read
4
autonomy tiers in the calibration framework, from fully autonomous to human-collaborative
3x3
Action Consequence Class matrix: reversibility crossed with blast radius, producing 9 cells and 4 tier assignments
Article 14
EU AI Act: human oversight required for high-risk AI systems, without defining what constitutes adequate calibration [1]

The Wrong Unit of Analysis

When an enterprise deploys an AI agent, the first governance question is almost always: "How much can this agent do on its own?" The answer is almost always intuitive, negotiated between the team that built the agent and the stakeholder who approved the budget. The result is a permission set that reflects comfort levels, not a principled analysis of consequence.

This is the wrong unit of analysis. The question "how autonomous is this agent?" conflates actions with fundamentally different consequence profiles. An agent that can send a status email and an agent that can create a $500K purchase order are not in the same autonomy category, even if both live in the same codebase. Treating them as one system with a single autonomy level creates policies that are simultaneously too restrictive for low-stakes actions and too permissive for high-stakes ones.

The correct unit of analysis is the action class. Autonomy should be calibrated per action type, not per agent. A single agent can and should be autonomous for some action classes and require human approval for others. The framework for making those assignments is what this post defines.

Key Insight

Existing governance frameworks, including NIST AI RMF 1.0 and EU AI Act Article 14, mandate human oversight at the system level. Neither provides a mechanism for calibrating oversight at the action-class level. This gap means organizations either over-restrict capable agents or under-govern consequential ones.

Defining the Framework

The Autonomy Calibration framework has two components: a classification system and a scoring index. The classification system (Action Consequence Class) determines the consequence profile of each action type. The scoring index (Autonomy Calibration Index) uses that profile, combined with two operational variables, to assign a tier.

Definition: Action Consequence Class (ACC)

For a given agent action type a, the Action Consequence Class ACC(a) is a two-dimensional classification defined by: (1) Reversibility R(a), the cost and latency of undoing the action after execution, taking one of three values: Immediate (fully reversible within minutes with no material cost), Compensable (reversible but requiring compensating transactions, notifications, or remediation effort), or Permanent (not fully reversible after execution); and (2) Blast Radius B(a), the scope of systems, users, or records materially affected if the action is incorrect, taking one of three values: Contained (affects only the initiating user or a single record), Departmental (affects a team, system, or bounded dataset), or Enterprise-wide (affects cross-functional systems, external parties, or regulatory records). The 3x3 ACC matrix produces 9 cells, each pre-assigned to one of four autonomy tiers based on combined consequence severity. This term originates with this work.

Definition: Autonomy Calibration Index (ACI)

For a given action class a, agent G, and organizational context C, the Autonomy Calibration Index ACI(a, G, C) is a score in [0, 100] computed as a weighted combination of four variables: the ACC severity score S(a) (derived from the pre-assigned tier weight of the ACC cell), the verification latency VL(a, G) (median time from action execution to detection of an error, normalized to [0, 1]), the override cost OC(a, C) (cost of human approval per action instance relative to the cost of an uncaught error, normalized to [0, 1]), and the empirical error rate E(a, G) (observed rate of incorrect action outcomes under current deployment conditions). ACI maps to four tiers: Tier 1 Autonomous (ACI greater than or equal to 70), Tier 2 Notify (ACI 50 to 69), Tier 3 Confirm (ACI 25 to 49), Tier 4 Collaborate (ACI less than 25). An agent without an ACI score for a given action class has an implicit ACI of zero: all actions default to Tier 4 until assessed. This term originates with this work.

The ACC Matrix and Tier Assignments

The 3x3 ACC matrix provides a starting tier assignment before operational variables are applied. This is the principled floor: ACI can raise a tier if override cost is prohibitively high relative to error cost, but ACC sets the minimum oversight level the action class requires by nature.

Fig. 1: ACC Matrix and Default Tier Assignments
BLAST RADIUS CONTAINED DEPARTMENTAL ENTERPRISE-WIDE REVERSIBILITY IMMEDIATE COMPENSABLE PERMANENT Tier 1 Autonomous e.g. status email, read query Tier 1 Autonomous e.g. draft doc, schedule meeting Tier 2 Notify e.g. system-wide cache clear Tier 2 Notify e.g. small PO, ticket update Tier 3 Confirm e.g. access grant, team comms Tier 4 Collaborate e.g. org-wide policy change Tier 3 Confirm e.g. record delete, user deactivate Tier 4 Collaborate e.g. firewall rule, data deletion Tier 4 Collaborate e.g. contract exec, regulatory filing

The Four Tiers

Each tier specifies what the agent does before, during, and after executing an action. The tier is not a statement about the agent's capability: it is a statement about the oversight architecture required for that action class given its consequence profile.

Tier 1

Autonomous

Agent executes without pre-approval or notification. Full audit log written. Human review available post-hoc but not triggered. Applicable when ACI is at or above 70: action is immediately reversible, blast radius is contained or departmental, and error cost is below override cost at current error rates.

Tier 2

Notify

Agent executes, then notifies the designated owner within a defined window (default: 15 minutes). Owner has a veto window to reverse the action before downstream effects propagate. Applicable when ACI is 50 to 69: action is immediately reversible but enterprise-wide in blast radius, or compensable with contained blast radius.

Tier 3

Confirm

Agent proposes the action and waits for explicit approval before executing. Timeout without approval results in no action (not automatic execution). Applicable when ACI is 25 to 49: action is compensable with departmental or enterprise blast radius, or permanently reversible with contained blast radius.

Tier 4

Collaborate

Human drives; agent assists. Agent prepares analysis, drafts proposals, or surfaces options but takes no executable action. The human performs the action using agent-provided context. Applicable when ACI is below 25: action is permanently irreversible with departmental or enterprise blast radius, or carries enterprise-wide consequence under any reversibility profile.

ACI Scores for Common Enterprise Agent Actions

The chart below shows directional ACI scores for ten common enterprise agent action types. Scores are computed from ACC default tier weights, normalized verification latency estimates by action class, and illustrative override cost ratios. These are illustrative starting points: each organization must calibrate to its own error rate data and operational context. ACC default assignments drive the floor; override cost ratios can raise the score.

Fig. 2: Directional ACI Scores by Action Class
Directional illustration. ACI scores are computed from ACC default tier weights and normalized operational variables. Scores are not empirically calibrated against a specific model or deployment. Use as a starting framework for your own action taxonomy, not as definitive values. Color indicates assigned tier: green = Tier 1, blue = Tier 2, amber = Tier 3, clay = Tier 4.

The Decision Framework: Five Variables to Assess

Before assigning a tier, each action class requires assessment across five variables. The first two determine ACC. The next two feed ACI alongside ACC. The fifth is an organizational override that can shift a tier regardless of the index score.

Variable Assessment Question What a Good Answer Looks Like What Triggers a Higher Tier
Reversibility If this action is incorrect, how is it undone and at what cost? Full reversal in under 15 minutes with no downstream notifications required Reversal requires compensating transactions, external notifications, or is not fully achievable
Blast Radius If this action is incorrect, how many systems, users, or records are materially affected? Effect contained to the initiating session or a single record with no cross-system propagation Effect propagates to a team, external system, customer record, or regulatory dataset
Verification Latency How long after execution does an error become detectable? Error visible within minutes through existing monitoring or user feedback Error detectable only through downstream audit, customer complaint, or regulatory review (days to weeks)
Override Cost Ratio What is the cost of human approval per action relative to the expected cost of an uncaught error at current error rates? Override cost significantly exceeds expected error cost at current error rates, making Tier 1 economically justified Expected error cost at current error rates exceeds override cost, making human review cost-effective
Regulatory Floor Does any applicable regulation (EU AI Act, SOX, HIPAA, FINRA) mandate human oversight for this action type? No applicable regulatory mandate for this action class in this jurisdiction Applicable regulation mandates human oversight regardless of ACI score; tier floor is set by compliance requirement
On Regulatory Floors

EU AI Act Article 14 requires human oversight for high-risk AI system outputs [1]. This creates a regulatory floor for action classes that constitute "output" under the Act's scope: credit decisions, hiring recommendations, law enforcement applications, and critical infrastructure actions are examples. ACI scores do not override regulatory floors. Where a regulatory mandate applies, the floor tier is set by compliance, and ACI governs only the design of the approval workflow within that tier.

Architecture Comparison: What Each Tier Requires

Fig. 3: Implementation Complexity vs. Oversight Quality by Tier
Directional illustration of relative implementation complexity and oversight quality across four autonomy tiers. Values are structurally derived from the number of integration points each tier requires (approval workflows, notification systems, audit hooks, rollback mechanisms). Not empirically calibrated: use to identify architectural investment required at each tier, not to estimate exact engineering costs.

Three Enterprise Scenarios

Chief Operating Officer · Global Logistics · Procurement Agent

The $2.3M Purchase Order

A procurement agent deployed to automate routine supplier orders was given a single autonomy setting: "can create purchase orders." Within four weeks, the agent created a $2.3M blanket order with a tier-2 supplier, acting on an ambiguous demand forecast. The order was compensable but required a 6-week negotiation and a $180K restocking fee. ACC classification would have immediately separated this action into two classes: routine orders below a threshold (ACC: Compensable x Contained, Tier 2 Notify) and large or new-supplier orders (ACC: Compensable x Departmental, Tier 3 Confirm). The single-autonomy-setting policy had no mechanism to make this distinction.

CISO · Financial Services · Security Response Agent

The Firewall Rule That Blocked Production

A security agent tasked with responding to anomalous traffic patterns was configured to modify firewall rules autonomously. The agent identified a pattern consistent with credential-stuffing and blocked the IP range. The IP range included the firm's own API gateway for a third-party clearing system, causing a 4-hour production outage. The firewall rule was immediately reversible in technical terms but enterprise-wide in blast radius (clearing operations for 14 institutional clients). ACC: Immediate x Enterprise-wide = Tier 2 (Notify). The team had assigned Tier 1 because the technical reversal was fast, without assessing blast radius independently. Ruan et al. [4] document this exact failure mode: agents taking technically reversible actions whose blast radius creates irreversible downstream effects.

VP Customer Success · B2B SaaS · Account Management Agent

The Three-Tier Agent

After an ACC audit, this team discovered their account management agent was performing actions across three different tiers simultaneously: sending status updates (ACC: Immediate x Contained, Tier 1 Autonomous), applying account credits below $500 (ACC: Compensable x Contained, Tier 2 Notify), and modifying SLA terms in contracts (ACC: Permanent x Departmental, Tier 4 Collaborate). The single-autonomy-setting policy allowed all three without distinction. The reconfiguration required two engineering days and reduced human approval volume by 74% while increasing oversight on the highest-consequence action class. The key insight: restricting the entire agent to Tier 3 had been the compromise; the correct answer was action-class-specific tiers.

Build, Buy, and Configure

Three-Phase Roadmap

Phase 1: Weeks 1-4

Inventory and Classify

Build the action taxonomy for each deployed agent. Assign ACC cells. Run the decision framework table for each action class. Identify any action classes currently at Tier 1 that should be Tier 3 or 4 by ACC. Flag regulatory floors. No engineering required in Phase 1: this is policy and classification work.

Phase 2: Weeks 5-10

Instrument and Reconfigure

Configure approval workflows for Tier 3 and 4 action classes. Add notification hooks for Tier 2. Instrument error rate tracking for all action classes. Compute baseline ACI scores using Phase 1 ACC weights and initial error rate estimates. Reconfigure agent permission sets to enforce tier assignments. Establish a 30-day observation window.

Phase 3: Weeks 11+

Calibrate and Govern

Run the first ACI recalibration using observed error rates from the observation window. Adjust tiers where ACI scores have changed materially. Establish quarterly recalibration cadence. Add ACC classification to the onboarding checklist for all new agent capabilities. Make ACI scores visible to agent owners as a governance metric.

Cost of Getting This Wrong

Incident Cost

Autonomous agent actions with enterprise-wide blast radius and high verification latency produce incidents that are detected late and remediated at high cost. The cost of a single high-ACC incident typically exceeds the annual implementation cost of the calibration framework by a substantial margin (directional; see [4]).

Regulatory Exposure

EU AI Act Article 14 non-compliance for high-risk AI systems carries penalties of up to 3% of global annual turnover for deployers who cannot demonstrate adequate human oversight [1]. ACC documentation is the mechanism for that demonstration.

Over-Restriction Cost

Single-autonomy-setting policies that default all actions to Tier 3 or 4 eliminate the efficiency benefit of agents. Over-restriction is not safety: it is a different kind of failure. The ACI framework recovers Tier 1 autonomy where it is genuinely safe, not just where it is politically comfortable.

Trust Erosion

A single high-visibility autonomous agent error in a consequential domain resets an organization's agent deployment program by 12 to 18 months. The reputational cost within the organization is typically the binding constraint on the next budget cycle, not the direct remediation cost.

Executive Checklist

1. Does each deployed agent have an explicit action taxonomy with ACC assignments?
Good: a documented list of action classes, each assigned to an ACC cell and a tier, reviewed by security and product leadership.
Red flag: autonomy is defined at the agent level, not the action class level. No ACC taxonomy exists.
2. Are there any action classes currently at Tier 1 that belong in Tier 3 or 4 by ACC?
Good: ACC audit has been run; all Tier 1 assignments are justified by verification latency and override cost ratio.
Red flag: Tier 1 was assigned because "the action is reversible" without independently assessing blast radius.
3. Do Tier 3 approval workflows have a defined timeout policy?
Good: timeout results in no action. The agent does not execute on approval timeout.
Red flag: timeout results in auto-approval or auto-escalation to a lower tier. This inverts the safety guarantee.
4. Are error rates being tracked per action class, not just at the agent level?
Good: per-action-class error rate data is available and feeds quarterly ACI recalibration.
Red flag: error tracking is at the agent or session level only. ACI cannot be calibrated without action-class error rates.
5. Have regulatory floors been identified for all action classes in scope?
Good: legal and compliance have reviewed the action taxonomy; all actions subject to EU AI Act, HIPAA, SOX, or FINRA mandates are flagged with their compliance-driven tier floor.
Red flag: regulatory applicability has not been assessed for specific action classes. ACI overrides compliance floors incorrectly.
6. Is the autonomy calibration reviewed when the model or agent configuration changes?
Good: model updates trigger an ACC review for any action class with observed error rate above threshold.
Red flag: ACC assignments are treated as permanent. A model update that changes error rates is not reflected in tier assignments.

Cross-Links

The action blast radius concept introduced here builds on the coined term used in AI Agent Incident Response. The context window boundary framework in The Context Window Is a System Boundary defines the trust surface that determines which agent inputs are subject to validation before ACC-classified actions are triggered. The governance debt implications for organizations with large autonomous agent footprints are addressed in Agentic Governance Debt.

References

  1. European Parliament and Council. "Regulation (EU) 2024/1689 Laying Down Harmonised Rules on Artificial Intelligence (AI Act)," Article 14 (Human Oversight), Article 99 (Penalties). Official Journal of the European Union, 2024.
  2. NIST. "Artificial Intelligence Risk Management Framework (AI RMF 1.0)," NIST AI 100-1, January 2023. doi:10.6028/NIST.AI.100-1
  3. Shinn, N., Cassano, F., Labash, A., Gopinath, A., Narasimhan, K., and Yao, S. "Reflexion: Language Agents with Verbal Reinforcement Learning." Advances in Neural Information Processing Systems, 2023. arXiv:2303.11366
  4. Ruan, Y., Dong, H., Wang, A., Pitis, S., Zhou, Y., Ba, J., Dubois, Y., Maddison, C. J., and Hashimoto, T. "Identifying the Risks of LM Agents with an LM-Emulated Sandbox." arXiv:2309.15817, 2023.
  5. Liu, X., Yu, H., Zhang, H., Xu, Y., Lei, X., Lai, H., Gu, Y., Ding, H., Men, K., Yang, K., Zhang, S., Deng, X., Zeng, A., Du, Z., Zhang, C., Shen, S., Zhang, T., Su, Y., Sun, H., Huang, M., Dong, Y., and Tang, J. "AgentBench: Evaluating LLMs as Agents." arXiv:2308.03688, 2023.
  6. Perez, E., Huang, S., Song, F., Cai, T., Ring, R., Aslanides, J., Glaese, A., McAleese, N., and Irving, G. "Red Teaming Language Models with Language Models." arXiv:2202.03286, 2022.

Excited about AI, innovation, and growth?

Start a conversation