Spend overruns are not a tooling problem. They are a governance accounting failure. When enterprises run three AI tools in parallel with no formal record of which model handled what, cost attribution is impossible by construction.
The CTO owns this problem. Not the CFO, not procurement, the CTO, because the root cause is architectural, not financial. Every quarter, organizations throttle access to AI tools, renegotiate contracts mid-cycle, and issue emergency budget requests for line items that did not exist six months ago. The explanation from finance is always cost. The explanation from the CTO's office is always growth in usage. Neither is wrong. Both miss the structural cause.
Enterprises running Claude Code, GitHub Copilot, Palantir, Salesforce AI, and a custom RAG pipeline are not running five AI products. They are running an implicit AI ensemble, multiple models, routing decisions made at the application layer, outputs arbitrated by no formal mechanism, and costs logged at the invoice level rather than the decision level. You cannot control costs you cannot attribute. And you cannot attribute costs when the ensemble that generated them has no formal specification.
This post introduces two formal constructs, the Routing Policy Gap and the Model Ensemble Specification, that define exactly why cost attribution fails and what a CTO must build before the next budget cycle. Both originate in Enterprise AI Ensembles (EAEF-2026-010), a formal framework paper published August 2026.
Before introducing the fixes, it is worth being precise about the failure modes. Enterprise AI budget overruns are not caused by one thing. They are caused by three structural conditions that co-occur in virtually every multi-tool deployment. Understanding them individually is how you diagnose which one is hitting your organization hardest.
The Routing Policy Gap is the structural condition in a multi-model deployment where the routing policy, the mechanism that determines which model handles which task, is implicit, unversioned, and unlogged. A deployment exhibits a Routing Policy Gap when an auditor cannot reconstruct, from available logs, which model processed a given request at a given point in time. This term originates in EAEF-2026-010 (arjunjaggi.com/papers/enterprise-ai-ensembles.html).
The routing decision, which model handles which task, is encoded in application logic, not in a named, versioned governance artifact. When the application routes a contract review task to GPT-4o in January and to Claude Sonnet in March because a developer updated a config file, no record exists that this change occurred. The cost profile of the deployment changed silently. Finance sees invoice variance; the CTO sees no corresponding change in the system record. Attribution is impossible because the routing policy was never a governed artifact.
Even when routing logic is explicit, it is often not logged at the transaction level. The routing rule exists in code. The individual routing decision, this request, this model, this timestamp, this token count, is not written to a queryable store. Cost can be allocated to a tool-level invoice but not to a workload, a team, a use case, or a business outcome. A CFO asking "how much did contract review cost in Q2" cannot receive an answer, because no record maps individual model calls to that workload.
Many enterprise AI deployments route requests to an alias, "production-llm," "default-model," "legal-ai", rather than a specific model version. The alias resolves to whatever model the vendor serves at the time of the call. When a vendor silently upgrades the model behind an alias, the cost, latency, and output quality of every call routed to that alias changes with no corresponding change in the organization's governance records. Spend variance attributed to "usage growth" is often alias-resolved routing responding to a vendor's model swap.
The Routing Policy Gap is not a logging failure. It is a specification failure. You cannot log what you have not defined. The fix is not "add more logging", it is "create a Model Ensemble Specification first, then log against it."
The second original construct addresses what must exist before logging can mean anything. Most enterprises treat their AI stack as a collection of vendor subscriptions. A Model Ensemble Specification treats it as a governed system, one that has a defined composition, defined routing rules, defined arbitration logic, and a defined provenance record for every output.
A Model Ensemble Specification is a formal record defining the minimum four components of an auditable multi-model deployment: (1) the model set M, every model in the ensemble, versioned and identified; (2) the routing policy R, the rules governing which model handles which input class, versioned and logged; (3) the arbitration mechanism A, how conflicting or parallel outputs are resolved; (4) the genealogy record G, the per-output provenance log mapping each output to the specific model version that generated it. MES = (M, R, A, G). This term originates in EAEF-2026-010.
Most enterprises have partial implementations of MES components without recognizing them as such. A vendor contract specifies the model (M), but not the routing policy. An application config encodes routing rules (R), but not versioned or logged. An LLM gateway may log requests (partial G), but without the arbitration record (A). The MES is the framework that makes these partial records into a complete, auditable specification.
This connects directly to the governance debt problem documented in Agentic Governance Debt: every month an enterprise operates a multi-model deployment without an MES, the cost of retroactively constructing one grows. Past spend cannot be attributed. Past outputs cannot be audited. The EU AI Act's Article 12 logging requirements apply to ensemble outputs as much as to single-model outputs, and an ensemble without an MES has no mechanism to satisfy them.
The diagram below shows the four MES components and how they sit in relation to the request flow in a typical enterprise multi-model deployment. The clay components are the ones that must exist for cost attribution to be possible. The passive components are where enterprises typically invest first, and where the gap lives.
The critical insight in this diagram: the genealogy record (G) is downstream of the arbitration step, not the routing step. This means an enterprise that logs individual model calls but does not log the arbitration decision, which model's output was selected, and why, still cannot attribute an output to a specific model. Cost per output requires genealogy. Genealogy requires arbitration logging. Arbitration logging requires a defined arbitration mechanism. Which requires a complete MES.
The EU AI Act's Article 12 requires logging of "automated logging of events" for high-risk AI systems. NIST AI RMF's GOVERN 1.7 requires documentation of AI system components. ISO 42001 clause 9.1 requires performance evaluation. None of these standards define what a multi-model deployment must log, version, or specify. They require records without defining what those records must contain for an ensemble.
This is the gap the MES fills. An enterprise pointing at its vendor contracts as its "model documentation" satisfies the letter of no governance standard and the spirit of none. The MES is the operational specification that makes regulatory compliance achievable rather than performative. For a deeper analysis of how governance theater develops in enterprise AI, see AI Governance Theater.
A large carrier deployed Claude Code for policy drafting, a vendor RAG tool for underwriting lookups, and a third model for regulatory translation. Invoices arrived from three vendors. The Chief AI Officer could not answer the CFO's question: "What did AI cost per policy issued in Q2?" Without a Routing Policy Log, the allocation of model calls to specific workflows was not reconstructable. The answer required a four-week engineering effort to approximate, and was ultimately presented as a directional range rather than an audited figure. The MES would have made this a reporting query, not a reconstruction project.
A bank's AI infrastructure team built a multi-model routing layer that dispatched compliance queries to a conservative model and customer-facing queries to a faster, cheaper model. A vendor alias upgrade changed which model sat behind the compliance route. Compliance output quality dropped for six weeks before the change was detected through a manual audit. The routing policy was not versioned. The alias change was not logged. The first signal was a regulatory examiner flagging inconsistent output quality, not an internal control. With Alias-Resolved Routing identified as a failure mode, the bank would have had an automated check for alias-to-model-version mapping changes on every deployment cycle.
A SaaS company running an AI coding assistant for 800 developers experienced a sharp cost spike in March. Finance attributed it to "increased usage." The engineering team suspected a model upgrade had changed token consumption rates. Without a genealogy record (MES component G), neither hypothesis could be tested. Both were partially true. The company had no mechanism to separate usage growth from per-call cost changes triggered by a vendor model change, because both factors appeared in the same invoice line. An MES with per-request genealogy records would have decomposed the spike into its two components in real time.
This framework helps a CTO diagnose which failure mode is present and how severe it is. Apply it to your primary multi-model deployment. If you answer "no" or "unsure" to any of the first four questions, you have a Routing Policy Gap.
| Question | If Yes | If No or Unsure |
|---|---|---|
| Is the routing policy a named, versioned artifact separate from application code? | No Implicit Routing gap | Implicit Routing failure mode present |
| Does every model call write a per-request record including model ID, version, timestamp, and token count to a queryable store? | No Unlogged Routing gap | Unlogged Routing failure mode present |
| Does your deployment route to specific model versions rather than vendor-managed aliases? | No Alias-Resolved Routing gap | Alias-Resolved Routing failure mode present |
| Can you reconstruct which model handled a specific request from 90 days ago using available logs? | MES genealogy record is functional | Retroactive Unverifiability applies, past spend cannot be attributed |
| Can you produce a report of AI spend by workload (not by vendor) for the last quarter? | Cost attribution is operational | Budget overrun conditions are structurally active |
A deployment that operates without an MES does not simply have incomplete records, it has permanently unverifiable records. The genealogy information that would allow attribution of past spend to specific models, workloads, or routing decisions cannot be reconstructed retroactively. This is Retroactive Unverifiability, formally established in EAEF-2026-010, and it means the cost of operating without an MES grows with every week of operation, not just at audit time.
The MES is not a product. No vendor sells it. It is a specification you define and then instrument against. Here is how to decompose the implementation:
| MES Component | Build | Buy / Configure | Rationale |
|---|---|---|---|
| Model Set (M) | Internal registry of versioned model IDs and deployment dates | LLM gateway (LiteLLM, Portkey, Helicone) can surface model versions; configure to log them | Vendors change aliases without notice; internal registry is the ground truth |
| Routing Policy (R) | Named YAML or policy-as-code artifact, version-controlled in git | Configure existing routing layer to read from the artifact, not hardcoded logic | Routing policy must be auditable independent of application code |
| Arbitration Mechanism (A) | Arbitration logic and selection rationale written to the log at the time of output selection | Not available off-shelf; this is always built | No vendor product logs why a specific model output was selected over alternatives |
| Genealogy Record (G) | Per-request event schema: request ID, model ID, version, routing rule applied, arbitration outcome, timestamp, token count | Observability platforms (Datadog, Splunk) can store the schema; you define and emit the events | The schema must match MES structure, not generic observability schemas |
Define the MES for your primary multi-model deployment. Name every model in the ensemble with its version. Write the routing policy as a versioned artifact. Define the genealogy event schema. Instrument one production workload to emit events against the schema. Go/no-go gate: the pilot workload can produce a per-request attribution report from the genealogy store.
Expand MES coverage to all regulated workloads. Implement alias-to-version monitoring with automated alerting on any alias resolution change. Wire the genealogy record into your existing cost reporting pipeline so workload-level AI spend is a first-class report. Go/no-go gate: finance can produce a workload-level AI cost report without engineering involvement.
Extend MES to all AI workloads across the enterprise. Establish a review cadence for routing policy changes (proposed change logged, reviewed, version-bumped). Map MES records to EU AI Act Article 12 and NIST AI RMF GOVERN 1.7 requirements. Success criteria: an external auditor can reconstruct any model routing decision from the prior 12 months from production logs alone.
When finance requests a workload-level AI spend breakdown with no MES in place, the engineering team reconstructs it manually. This is a recurring cost that compounds with ensemble complexity.
EU AI Act Article 12 logging requirements apply to high-risk AI outputs regardless of whether they come from a single model or an ensemble. No MES means no compliant log.
Without per-request genealogy data, the enterprise cannot verify whether a cost spike came from usage growth or a vendor model change. The vendor controls the information; the enterprise controls only the invoice.
Every week of operation without an MES adds to the permanently unverifiable record. Retroactive Unverifiability means past spend cannot be reconstructed after the fact, regardless of engineering investment.
A CTO can use this checklist to assess whether their organization is structurally capable of controlling AI spend. It doubles as a board-ready readiness assessment.
For a wider view of how AI governance debt accumulates beyond the cost attribution problem, see The Enterprise AI Governance Stack, the MES sits at the infrastructure layer of the governance stack, and the debt from missing it compounds upward into every layer above it.