The enterprise technology stack has, across successive generations, resolved governance crises through the introduction of a control plane. When relational databases proliferated in the 1980s and 1990s, database administration emerged to provide a governance layer for schema management, access control, query optimization, and backup. When virtual infrastructure became predominant in the 2000s, hypervisor management planes and cloud control planes provided the policy enforcement, cost attribution, and observability required to govern distributed compute at enterprise scale. In software-defined networking (SDN), the separation of the data plane from the control plane, formalized in the OpenFlow protocol, became the canonical architecture enabling centralized governance of distributed switching infrastructure [1].

Large language model deployments in enterprise environments in 2025 and 2026 are in an analogous pre-control-plane state. Organizations deploy individual model API endpoints, construct prompt templates, build retrieval pipelines, and connect models to tool execution environments without a coherent layer governing inference request routing, inference cost attribution, model input and output logging, access policy enforcement, or failure handling. The result is a consistent pattern of governance failure that prior technology transitions have trained enterprise architects to recognize.

The consequences of this absence are predictable: inference costs accumulate without attribution, making ROI measurement impossible; compliance teams cannot answer basic regulatory questions about what data was sent to which model; security teams cannot enforce least-privilege access to AI capabilities; and engineering teams independently build ad hoc retry logic in each application. The OWASP Top 10 for Large Language Model Applications identifies insecure output handling, sensitive information disclosure, and excessive agency as the leading risk categories for deployed LLM systems [7]. Each of these risks is structurally mitigated by the EACP components defined in this paper.

IBM's Cost of a Data Breach Report 2024 documents a mean breach cost of $4.88 million, with a $1.88 million differential between organizations that deploy AI and automation in security operations and those that do not [2]. This differential reflects observability and response speed benefits that a mature AI control plane enables. Organizations without this layer cannot capture this benefit systematically.

This paper makes four primary contributions:

• The Enterprise AI Control Plane (EACP) is introduced as a named, bounded architectural concept distinct from both the model layer and the application layer, with formal analogs to established control plane patterns in SDN, cloud infrastructure, and enterprise database administration.
• Five EACP components are defined with operational specifications, failure mode analysis, and component-level organizational consequences (Table I). The Retroactive Impossibility Property of the Audit and Observability component is identified and characterized as a governance property not shared by any other component.
• The AI Access Policy Matrix is introduced as a named three-dimensional control structure (payload classification × capability scope × output disposition) required for least-privilege AI governance and not addressed by existing RBAC standards.
• A formal definition of governance debt is presented, distinguishing a retrievable component (D_r, addressable through investment) from an irretrievable component (D_i, permanent evidence loss). A four-tier maturity model enabling organizations to quantify D_r is presented with a 90-day Foundation implementation roadmap (Table III).
• A return on investment framework mapping four value driver categories to verifiable evidence and organizational measurement approaches is presented (Table II), with explicit acknowledgment of the conditions under which each value driver is measurable.

The framework is technology-agnostic and vendor-neutral, designed as an open reference model for enterprise adoption.

The EACP architecture is grounded in control plane patterns from three prior technology transitions: software-defined networking, cloud infrastructure management, and enterprise database administration.

McKeown et al. formalized the separation of network control logic from packet-forwarding hardware through the OpenFlow protocol [1]. This separation enabled a centralized controller to govern distributed network infrastructure through a standardized interface, decoupling policy definition from policy enforcement. The SDN control plane manages routing tables, quality-of-service policies, security group rules, and traffic shaping, while the data plane executes forwarding at line rate. This architectural separation is the direct precedent for the EACP's position between the application layer (which declares requirements) and the model layer (which executes inference).

Major cloud providers implement multi-tier control planes governing identity and access management, resource provisioning, cost attribution, service quotas, and audit logging. These control planes evolved from ad hoc management tooling into standardized services with defined APIs, SLA guarantees, and compliance attestations. AWS CloudTrail, for example, provides immutable audit logging of all API calls across an organization's AWS footprint, a direct functional analog to the EACP's Audit and Observability Layer. The cloud control plane is now a prerequisite for enterprise cloud adoption under SOC 2 Type II, ISO 27001, and FedRAMP. The EACP occupies an equivalent position for enterprise AI adoption.

Enterprise relational database systems introduced governance layers for query optimization, connection pooling, access control, backup and recovery, and performance monitoring. These layers evolved from informal practices into formal products and specifications (JDBC, ODBC). The database control plane's role in governing access to data, determining which principals can query which tables under which conditions, is directly analogous to the EACP's Access and Policy Enforcement component governing which principals can invoke which models with which data payloads.

The EACP follows this established pattern. A new infrastructure class creates governance requirements initially met ad hoc by individual teams, then codified into a bounded control plane, and eventually standardized across the industry. The enterprise AI control plane is early in this cycle.

The Enterprise AI Control Plane is defined as the architectural layer positioned between the application layer and the model layer. Applications declare their requirements (latency SLA, quality floor, data classification, cost budget) to the EACP. The EACP resolves those requirements to specific models, providers, and parameters according to organizational policy, then manages the lifecycle of each inference request including routing, logging, cost attribution, and failure handling.

The EACP comprises five components. Fig. 1 presents the reference architecture. Table I presents the failure mode analysis for each component: the primary failure that emerges in its absence and the organizational consequence.

These failure modes are not hypothetical. They represent the consistent findings from practitioner analysis of enterprise AI deployments operating without formal control plane infrastructure. The governance debt they represent compounds nonlinearly: each new AI application deployed without the EACP adds routing decisions, cost exposure, audit gaps, access risks, and retry implementations that must eventually be retrofitted.

The Model Routing Engine directs inference requests to the appropriate model, model tier, or model provider based on four input signals: (i) estimated task complexity, inferred from prompt structure or declared by the application; (ii) latency requirements, specified by the calling application or derived from SLA configuration; (iii) cost constraints, expressed as per-request or per-period budget limits; and (iv) data classification, which determines which model providers or deployment topologies are permissible.

The routing function is formally analogous to the load-balancing and traffic-shaping functions of the SDN control plane, with the addition of a quality dimension absent from pure networking contexts. Chen et al. demonstrate that cascade-based routing, directing requests first to a lower-cost model and escalating to a more capable model only when the initial response fails a quality gate, yields substantial cost reduction while maintaining response quality at the population level [3]. The Model Routing Engine operationalizes this cascade pattern as organizational policy. Fig. 2 illustrates the cost composition effect of progressive cascade routing tiers.

A critical design constraint is that routing logic must reside in the control plane, not in application code. Applications should declare requirements; the routing engine resolves those requirements to specific models according to current policy and pricing. This constraint prevents the proliferation of model-selection logic across individual application teams and ensures routing policy can be updated centrally as model capabilities and prices evolve.

The Cost Governance Engine provides budget allocation, real-time spend monitoring, cost attribution, and anomaly detection for inference expenditure. Without this component, inference costs are visible only in aggregate provider invoices, making it impossible to attribute costs to specific teams, products, or use cases, or to detect unusual expenditure patterns before they become material budget overruns.

The Cost Governance Engine maintains a cost model mapping each model tier and provider to per-token pricing, aggregates spend by attribution dimension (team, product, use case, cost center), enforces per-period budget limits with configurable alerting thresholds, and detects anomalous spend patterns using statistical process control methods applied to token consumption time series. At the Scaling maturity tier, this component integrates with existing FinOps tooling and cost center accounting, enabling AI inference costs to be treated with the same rigor as cloud compute costs.

The Audit and Observability Layer provides immutable logging of every inference request and response, quality and latency metrics, and model behavior drift detection. This component is the primary mechanism by which the EACP enables regulatory compliance, forensic investigation of model outputs, and systematic quality management.

The NIST AI Risk Management Framework (AI RMF 1.0) identifies "Manage" as one of its four core functions, requiring organizations to track, assess, and respond to AI risks in deployment [4]. The Audit and Observability Layer operationalizes this function by creating the evidence base required for risk tracking. Without logged prompt and completion data, organizations cannot demonstrate to auditors or regulators what their AI systems have produced, nor can they reconstruct the sequence of model decisions that led to a given output.

Drift detection, a function of this layer, monitors distributional shifts in model output quality over time. Without this function, model degradation is detected only when downstream business metrics decline, typically weeks or months after the degradation begins.

The Access and Policy Enforcement component implements role-based access control for AI capabilities, enforces data classification rules governing which data may be sent to which model providers, and maintains an allowlist of approved model providers and deployment topologies.

The access control model for AI requires three control dimensions beyond traditional RBAC. Standard enterprise RBAC governs who may read or write which data. AI access control must govern three distinct questions simultaneously: (i) payload classification: whether a given data payload may transit a given model provider's network boundary; (ii) capability scope: which AI capabilities a given principal may invoke in a given context; and (iii) output disposition: whether a given model output may be returned to a given calling context or must be filtered, redacted, or blocked. These three dimensions together constitute the AI Access Policy Matrix, a control structure not addressed by existing RBAC standards (NIST SP 800-162, ANSI INCITS 359) or by existing API gateway policy models. The OWASP LLM Top 10 identifies sensitive information disclosure as the primary risk class for deployed LLM systems; the AI Access Policy Matrix is the structural control for this risk class, enforced at the inference request level rather than at the application level [7].

The Failure and Fallback Orchestration component manages circuit breakers, retry policies, fallback chains, and graceful degradation behavior when model providers are unavailable or when individual inference requests produce low-confidence or policy-violating outputs.

In the absence of this component, each application team implements independent retry logic, typically without consistent circuit-breaker behavior, leading to thundering herd effects when model providers experience degradation and inconsistent user experiences across AI-enabled products. The Failure and Fallback Orchestration component centralizes this logic, enabling consistent behavior and the ability to update fallback policies without application code changes. At the Optimizing tier, low-confidence outputs trigger automatic rerouting to more capable models, providing a structured mechanism for quality assurance at the inference level.

A four-tier maturity model is defined for EACP assessment: Ad Hoc, Defined, Managed, and Optimizing. These tiers are adapted from the Capability Maturity Model Integration (CMMI) framework's five-level progression [9], collapsed to four levels to reflect current enterprise AI governance practice. NIST AI RMF 1.0 characterizes AI risk management as a nascent discipline, noting that organizations are at early stages of developing systematic AI risk management capabilities [4]; McKinsey's 2024 State of AI survey similarly finds that a minority of organizations with active AI deployments have fully embedded risk mitigation into their AI practices [10]. The maturity heatmap (Fig. 3) reflects these assessments: the majority of enterprise deployments are estimated at Tier 1 or Tier 2 for most components. Fig. 3 presents the maturity heatmap across the five EACP components.

The four maturity tiers are defined as follows:

Three reference deployment architectures are defined for organizations at different maturity levels. These architectures are not prescriptive implementations but reference configurations illustrating the minimum viable EACP for each maturity tier.

The Foundation architecture establishes the minimum viable EACP for organizations beginning control plane implementation. The priority sequence is: Access and Policy Enforcement first (to establish data classification rules and model provider governance before expanding deployment), followed by Audit and Observability (to create the log record required for compliance attestation), followed by Cost Governance (to enable budget management before costs scale). Model Routing and Failure Orchestration are deferred to the Scaling architecture.

The Foundation architecture is appropriate for organizations with fewer than five active AI deployments and without formal AI governance requirements from regulators or enterprise risk functions. Its primary deliverable is a defensible audit trail and a documented AI access policy. Table III presents the 90-day implementation roadmap for the Foundation architecture.

The Scaling architecture adds the Model Routing Engine and Failure and Fallback Orchestration to the Foundation components. At this tier, the routing engine implements a two-level cascade: a primary model for standard-complexity requests and a frontier model for requests that fail the quality gate. The fallback orchestration component implements standardized circuit-breaker patterns across all AI-enabled applications.

The Scaling architecture integrates with existing enterprise infrastructure: the Audit and Observability Layer feeds into the organization's SIEM; the Cost Governance Engine integrates with FinOps tooling and cost center accounting; and the Access and Policy Enforcement component integrates with the enterprise identity provider. At this tier, the EACP begins generating measurable cost reduction through cascade routing.

The Optimizing architecture adds continuous evaluation pipelines, policy-as-code for routing and access rules, and feedback loops from production observability data to the model routing policy. At this tier, routing decisions are informed by real-time quality metrics from the Audit and Observability Layer, enabling dynamic adjustment of cascade thresholds based on observed model performance.

The Optimizing architecture aligns with the NIST AI RMF's "Govern" function, requiring organizations to establish and maintain policies, processes, procedures, and practices across the AI lifecycle [4]. Policy-as-code ensures that governance requirements are enforced programmatically rather than through manual review processes, and ISO/IEC 42001:2023 management system requirements are satisfiable through documented policy artifacts and audit evidence from this architecture [6].

The NIST AI RMF 1.0 (NIST.AI.100-1, 2023) organizes AI risk management around four functions: Govern, Map, Measure, and Manage [4]. The EACP provides direct operational support for each function. The Govern function, which establishes policies, processes, and accountability structures, is operationalized by the Access and Policy Enforcement component. The Map function, which identifies AI risks in context, is supported by the Audit and Observability Layer's log infrastructure. The Measure function, which analyzes and assesses AI risks, is supported by the quality and drift metrics generated by the Audit and Observability Layer. The Manage function, which responds to AI risks, is supported by the Failure and Fallback Orchestration component and the Model Routing Engine's capability to redirect traffic away from underperforming models.

The EU AI Act establishes a risk-tiered regulatory framework for AI systems deployed in the European Union, with the most stringent requirements applying to high-risk AI systems as defined in Annex III [5]. For organizations deploying AI in high-risk domains, several EACP components provide direct compliance support.

Article 9 requires high-risk AI systems to implement quality management systems. The Audit and Observability Layer's logging infrastructure directly supports this requirement. Article 12 requires high-risk AI systems to enable automatic logging of events throughout the system lifecycle. Article 13 requires sufficient transparency. The Failure and Fallback Orchestration component's explainability logging and the Model Routing Engine's decision logging contribute to this transparency requirement.

Penalties under the EU AI Act reach 7% of global annual turnover for violations involving prohibited AI practices (Article 85(3)), and 3% for violations of provider and deployer obligations including documentation and transparency requirements (Article 85(2)) [5]. The 3% tier directly covers the failure to implement logging (Article 12) and quality management (Article 9) requirements that the EACP Foundation architecture addresses.

ISO/IEC 42001:2023 defines requirements for an AI management system, analogous to ISO 27001 for information security management [6]. The standard requires organizations to establish an organizational context for AI, implement documented controls for AI risk management, and maintain evidence of management system effectiveness through audit and measurement. The EACP's Access and Policy Enforcement component maps to ISO 42001's controls for data governance and AI system access. The Audit and Observability Layer provides the measurement and monitoring infrastructure required by the standard's performance evaluation clause. Organizations pursuing ISO 42001 certification will find that Scaling-tier EACP maturity provides a substantial portion of the required control evidence.

The business case for EACP investment rests on four value driver categories: inference cost reduction, regulatory penalty avoidance, breach cost reduction, and engineering overhead reduction. Table II presents the value driver framework. Each category is mapped to the mechanism by which the EACP generates value, the evidence basis for the quantification, and the organizational measurement approach.

Organizations should sequence EACP implementation according to their primary value driver. For organizations in EU-regulated markets with high-risk AI deployments under Annex III, regulatory penalty avoidance justifies Foundation architecture investment immediately: the cost of Foundation implementation is substantially lower than a 3% annual-turnover penalty. For organizations with growing AI application portfolios and no centralized cost visibility, Cost Governance Engine implementation produces measurable ROI within the first billing cycle. For organizations with mature security operations seeking to capture AI-related breach cost reduction, Access and Policy Enforcement paired with Audit and Observability is the highest-priority investment.

The retrievable component D_r grows nonlinearly with deployment volume: each additional AI application deployed without the EACP adds routing, cost, access, and failure management debt that compounds because uncontrolled applications develop application-specific model dependencies, cost assumptions, and retry behaviors that must be untangled during retrofitting. NIST AI RMF 1.0 explicitly describes AI risk management as a nascent discipline, acknowledging that most organizations lack the frameworks required to manage AI risks systematically [4]. The EACP maturity model (Section IV) provides the organizational self-assessment tool required to quantify D_r before committing a remediation budget. Fig. 4 illustrates the divergence between a controlled deployment trajectory and an uncontrolled one, showing how governance debt accumulates as a function of application deployment count.

The following decision framework is provided for C-level executives assessing EACP investment priority. The framework maps organizational state to recommended architecture and primary justification.

The EACP framework as defined in this paper is a conceptual architecture, not an implementation specification. Organizations implementing the framework will encounter integration challenges with existing enterprise infrastructure that are outside the scope of this paper. The maturity assessment model in Section IV is based on practitioner assessment rather than systematic survey data, and should be treated as a reference framework rather than an empirical finding. The 90-day implementation roadmap in Table III is indicative; actual timelines depend on organizational complexity, existing infrastructure, and team capacity.

The framework addresses LLM-based AI systems. Enterprises deploying non-LLM AI systems, including traditional machine learning models for prediction and classification, will find that some EACP components require adaptation to address per-prediction cost structures and batch inference patterns.

The recommended implementation sequence prioritizes components by their risk reduction impact relative to implementation effort. Access and Policy Enforcement is implemented first, as its absence creates the highest regulatory and security exposure (Table I). Audit and Observability follows immediately, as its absence prevents retroactive compliance attestation and incident investigation. Cost Governance, Model Routing, and Failure Orchestration are sequenced according to the organization's primary pain point, though the sequence in Table III reflects the most common enterprise priority ordering.

Major technology vendors have published responsible AI frameworks addressing fairness, transparency, and harm avoidance at the model and product design level. These frameworks address the question of what AI systems should do. The EACP addresses a different and complementary question: how AI systems are governed once deployed in production at enterprise scale. Vendor responsible AI frameworks do not specify how inference costs are attributed across organizational units, how access policy is enforced at the request level, how audit logs are retained and queried for regulatory compliance, or how failure behavior is centralized across many deployed applications. The EACP fills this operational governance layer, which is architecturally downstream of model design decisions and upstream of application business logic. The two frameworks are not alternatives; an organization can fully implement a vendor responsible AI policy and still operate at Tier 1 EACP maturity, with all five failure modes in Table I active.

The Model Context Protocol (MCP) and similar integration specifications define how applications expose tools and resources to AI models. The EACP operates at a different abstraction level: where MCP governs the interface between model and tool, the EACP governs the interface between application and model. These specifications are complementary. The EACP's Access and Policy Enforcement component must be aware of MCP tool invocations to enforce data classification rules on tool results returned to the model context. Enterprise API gateway platforms provide partial EACP functionality and should be evaluated as candidate infrastructure for the Access and Policy Enforcement component before building from scratch.

Enterprise AI deployments in 2026 are in the pre-control-plane state that characterized cloud infrastructure in the early 2010s and network infrastructure before the SDN paradigm. The failure modes are consistent with prior transitions and predictably worsen with deployment volume. But AI governance has a property that cloud and network governance did not: part of the resulting debt is irretrievable. Cloud infrastructure deployed without access controls can be locked down retroactively. Networks deployed without audit logging can be instrumented. AI applications deployed without inference logging have destroyed the audit record permanently. The longer an organization operates without the Audit and Observability Layer, the larger its irretrievable governance debt component, and no remediation budget changes that.

This paper has introduced three original contributions not present in existing AI governance literature: the Enterprise AI Control Plane as a named architectural concept with five operationally specified components; the Retroactive Impossibility Property of the Audit and Observability component, characterizing a class of governance debt that is permanent and irreducible; and the AI Access Policy Matrix, a three-dimensional control structure (payload classification × capability scope × output disposition) required for least-privilege AI governance and not addressed by existing RBAC standards. These contributions are intended as a shared vocabulary for practitioners, regulators, and researchers working to bring AI governance to parity with the maturity of adjacent infrastructure disciplines.

The governance decision facing enterprise AI leaders is not whether to implement a control plane, but how much irretrievable debt they are willing to accumulate before they do. Every unlogged inference request is not merely a deferred cost: it is a permanently destroyed record. The EACP framework provides the vocabulary, the architectural specification, and the implementation roadmap. The window for zero irretrievable debt closed at the first uncontrolled production inference request. The window for minimizing it is open now.

© 2026 Arjun Jaggi and Aditya Karnam Gururaj Rao. Open reference model, free to use with attribution. arjunjaggi.com/papers/enterprise-ai-control-plane.html