Enterprise AI deployments have moved beyond single-model architectures. Organizations today operate multiple AI models simultaneously: a frontier model for complex reasoning, a fine-tuned domain model for specialized tasks, and a lightweight model for high-volume routing decisions. Yet no governance framework defines how these models should be composed, how routing decisions should be made and recorded, or who is accountable when the ensemble produces an output that no individual member model would have generated on its own. This paper introduces three original contributions:
Together, MES, Routing Policy Gap, and Arbitration Collapse give governance officers and engineers the vocabulary to specify, assess, and remediate the audit gap in enterprise multi-model deployments. The Ensemble Governance Maturity Model introduced in Section VI provides an indicative four-tier adoption path.
Abstract: Enterprises increasingly deploy multiple AI models simultaneously, combining frontier reasoning models, fine-tuned domain specialists, and lightweight routing classifiers in a single inference pipeline. No existing governance framework addresses this compositional architecture: routing decisions are implicit, aggregation functions are unversioned, and accountability for ensemble outputs is undefined. We introduce the Model Ensemble Specification (MES) as the formal 4-tuple (M, R, A, G) that defines a governed enterprise AI ensemble. We define Routing Policy Gap as the structural condition in which the routing function R is not treated as a versioned, logged governance artifact, and establish that this condition makes ensemble routing audits structurally impossible. We introduce Arbitration Collapse as the condition in which the aggregation function A produces an output not attributable to any single ensemble member, and prove that this extends the Retroactive Unverifiability result from Output Genealogy to multi-model settings: a collapsed output requires a sixth genealogy component beyond the five-tuple defined in prior work. We map MES to EU AI Act Article 12 and ISO/IEC 42001 clause 9.1, and introduce a four-tier Ensemble Governance Maturity Model. These contributions provide practitioners with formal vocabulary to specify, audit, and govern enterprise multi-model AI deployments.
Index Terms: Model Ensemble Specification, Routing Policy Gap, Arbitration Collapse, enterprise AI ensembles, multi-model orchestration, output attribution, ensemble governance, EU AI Act, ISO 42001, Output Genealogy.
The single-model assumption is breaking down. Enterprise AI deployments in 2026 routinely combine multiple models in a single inference pipeline: a frontier model handles open-ended reasoning, a domain-fine-tuned model handles specialized extraction, and a cheap classifier decides which to invoke. This compositional architecture is operationally sensible and cost-effective, but it creates a governance gap that no existing framework addresses.
The gap is not about model selection or cost optimization, topics that the routing literature addresses [3][5]. The gap is about accountability and auditability. When a regulated organization uses a multi-model pipeline to make or assist a consequential decision, three questions arise that no current framework can answer:
This paper answers these questions by introducing three formal contributions. The Model Ensemble Specification (MES) defines what a governed ensemble must specify. The Routing Policy Gap names the structural condition in which routing decisions are unauditable. Arbitration Collapse names the output condition that breaks existing audit frameworks and requires a formal extension of the Output Genealogy framework introduced in our companion paper [8].
The paper is organized as follows. Section II reviews the ensemble, routing, and aggregation literature and identifies the governance gap. Section III introduces MES. Section IV defines Routing Policy Gap and its consequences. Section V introduces Arbitration Collapse and its genealogy extension. Section VI presents the Ensemble Governance Maturity Model. Section VII discusses regulatory alignment and limitations. Section VIII concludes.
The self-consistency approach of Wang et al. [1] demonstrated that sampling multiple outputs from a single model and selecting by majority vote improves reasoning accuracy on chain-of-thought tasks. This established that aggregation of multiple outputs, even from the same model, can outperform any single output. LLM-Blender [2] extended this to multi-model settings, introducing pairwise ranking and generative fusion to combine outputs from distinct models. These are research contributions focused on accuracy, not on the governance of the ensemble as an enterprise artifact.
The routing literature addresses how to assign queries to models to optimize cost and quality. Shnitzer et al. [3] frame routing as a classification problem over benchmark performance data. FrugalGPT [5] introduces cascade routing, querying cheaper models first and escalating to more expensive models only when confidence is insufficient. Neither framework treats the routing policy as a versioned governance artifact or addresses the audit consequences of implicit routing.
Verga et al. [4] propose using a panel of diverse LLM judges rather than a single evaluator, demonstrating that diversity in the evaluator set reduces systematic bias. This work establishes that ensemble diversity is a governance property, not only a performance property. It does not define how to specify or audit the panel composition as a governance artifact.
Wan et al. [6] introduce model merging at the parameter level, combining the weights of multiple fine-tuned models into a single model. This is a training-time operation rather than an inference-time ensemble, and it eliminates the individual member models as distinct artifacts. The resulting merged model is a single M in our framework, not a multi-model ensemble: MES does not apply to parameter-level fusion, which produces a new atomic model rather than a governed composition.
Bommasani et al. [7] identify compositionality as a defining characteristic of foundation model deployments and flag the attribution and accountability questions it raises. No subsequent work, to our knowledge, has formalized the minimum governance specification for a composed multi-model deployment, named the structural condition in which routing is unauditable, or established the genealogy extension required when aggregation produces an output not attributable to any single member. These are the three contributions of this paper.
A governed enterprise AI ensemble is not simply a collection of models with a routing script. It is a formal artifact that can be versioned, logged, audited, and assigned accountability. The Model Ensemble Specification formalizes what such an artifact must contain.
A Model Ensemble Specification is the 4-tuple:
MES = (M, R, A, G)
where M = {m1, ..., mk} (k ≥ 2) is the model set, each mi identified by a unique, immutable version identifier; R: Q × C → 2M \ {∅} is the routing policy mapping a query q ∈ Q and context c ∈ C to a non-empty subset of M; A: ∏m∈S Om → O is the aggregation function mapping the output tuple from routed subset S = R(q,c) to a single ensemble output; and G is the governance specification defining the audit log schema, accountability assignments per member, and escalation policy. An MES is well-formed if M, R, A, and G are each formally specified, version-controlled, and logged with every inference event.
M: Model Set. Each model in M must be identified by an immutable version identifier, not an alias. The same alias resolution problem that creates M-ambiguity in single-model Output Genealogy [8] applies here: if mi is identified by an alias that resolves to different underlying versions over time, the model set M cannot be reconstructed for past inference events. In multi-model deployments this problem compounds: if any member model uses an alias, the entire ensemble is M-ambiguous.
R: Routing Policy. R must be formally specified as a function, not implemented only as application code. The function specification must define the input space Q × C (what features of the query and context are used), the output space 2M (which subsets of M are valid routing targets), and the decision criteria (thresholds, classifiers, or explicit rules). R must be versioned: when routing logic changes, the version of R in effect at each inference event must be logged alongside the routing decision.
A: Aggregation Function. A defines how member outputs are combined. Common aggregation strategies include selection (returning the output of a single designated member), voting (selecting among member outputs by majority or weighted vote), and fusion (generating a new output conditioned on all member outputs [2]). The choice of A determines whether Arbitration Collapse is structurally possible: selection never collapses; voting collapses when the selected output differs from the input it was selected from through a tie-breaking rule; fusion almost always collapses.
G: Governance Specification. G assigns accountability for the ensemble output to named principals, defines the audit log schema (what is recorded per inference event), and specifies the escalation policy for edge cases (when R returns an empty routing target, when A cannot resolve disagreement, or when a member model returns an error). G is the component most frequently absent in practitioner deployments.
A multi-model deployment D exhibits Routing Policy Gap if its routing function R is not formally specified as a versioned artifact and the routing decision for each inference event is not logged. A deployment with Routing Policy Gap cannot produce, for any past query (q, c, t), a verifiable record of which model subset S = R(q, c) was invoked and why, because the version of R in effect at time t may have changed and was not captured.
Routing Policy Gap is to multi-model deployments what Provenance Opacity [8] is to single-model deployments: it is the structural condition that makes audit impossible, and it is architectural rather than incidental. A deployment that implements routing in application code without a formal versioned specification has Routing Policy Gap by construction, regardless of how comprehensive its output logging is.
The EU AI Act Article 12 requires that high-risk AI systems maintain logs sufficient to allow post-hoc assessment of inputs. For a multi-model deployment, the routing decision is part of the input processing: it determines which models see the query and with what context. A Routing-Policy-Gap deployment cannot satisfy Article 12 for its routing layer even if it fully satisfies Output Genealogy for its individual member models.
ISO/IEC 42001 clause 9.1 requires evaluation of the AI management system, which for a multi-model deployment must include evaluation of the routing policy's effectiveness and correctness. This evaluation is impossible without a versioned routing policy specification and per-event routing logs.
Routing Policy Gap arises from three distinct failure modes in practitioner observation:
Implicit Routing. The routing decision is encoded directly in application code as conditional logic. When the code changes, no version record is created and no log entry captures the change. This is the most common failure mode.
Unlogged Routing. The routing policy is formally specified but the per-event routing decision is not logged. The specification exists but cannot be linked to individual inference events at audit time.
Alias-Resolved Routing. The routing policy specifies models by alias rather than version identifier. When a member model updates, the routing policy appears unchanged but its effective behavior has changed, creating a silent version mismatch that is undetectable at audit time without per-event version resolution logs.
Given an ensemble output o produced by aggregation function A over member outputs {o1, ..., ok} from routed subset S = R(q, c), Arbitration Collapse occurs when o ∉ {o1, ..., ok}: the aggregation function produces an output that no individual member model generated. The degree of collapse DAC(o) = 1 − maxi sim(o, oi) measures how far the ensemble output diverges from its closest member output, where sim is a semantic similarity function over the output space O.
The Output Genealogy framework G(o) = (M, P, C, Dr, Theta) defined in [8] assumes M is a single model identifier. When Arbitration Collapse occurs, G(o) for the ensemble output o cannot be reduced to any G(oi) for member i, because o was produced by A, not by any mi ∈ S directly. A complete genealogy record for a collapsed ensemble output requires a sixth component Aspec: the version-identified specification of the aggregation function A in effect at inference time. The extended genealogy is: Gens(o) = (MS, P, C, Dr, Theta, Aspec) where MS = {(mi, oi) : mi ∈ S} is the set of member models with their individual outputs. This is an existence result: the six-tuple is the minimum record that makes a collapsed ensemble output auditable.
Observation 1 has a direct operational consequence. An organization that deploys a fusion-based ensemble and implements Output Genealogy as defined in [8] will still be Provenance-Opaque for any inference event in which Arbitration Collapse occurs, because the five-tuple G(o) does not capture Aspec or the individual member outputs MS. The extended six-tuple Gens(o) is required.
The Retroactive Unverifiability result from [8], establishing that time-varying components of G(o) lost at generation time cannot be reconstructed at audit time, extends directly to Gens(o). Aspec is time-varying: aggregation logic changes as the ensemble evolves. If Aspec is not version-controlled and logged at inference time, the aggregation function in effect at time tg cannot be established at audit time ta > tg. The individual member outputs MS are also non-reproducible for stochastic aggregation functions. Retroactive Unverifiability therefore applies to both new components of Gens(o), making ensemble Provenance Opacity an irreversible condition for past inference events, consistent with the single-model result.
We define four tiers of ensemble governance maturity. Tiers are indicative, based on practitioner observation of multi-model deployment patterns, and not derived from systematic survey data.
Multiple models are called from application code without a formal MES. Routing is implicit conditional logic. Aggregation is hardcoded. No governance specification exists. This tier is Routing-Policy-Gap by construction and cannot satisfy any regulatory audit requirement for the routing or aggregation layers. In practitioner observation, the majority of enterprise multi-model pilots begin at Tier 1.
M, R, and A are formally specified as versioned documents. Routing and aggregation logic are derived from the specification. Per-event routing decisions are not yet logged. This tier closes the specification dimension of Routing Policy Gap but not the logging dimension: the routing decision for a specific past inference event still cannot be verified.
Full MES is in place. Per-event routing decisions are logged with the version of R in effect. Individual member outputs MS and Aspec are captured in a tamper-evident log, satisfying Gens(o) for all inference events. Arbitration Collapse is detected and flagged. Routing Policy Gap is eliminated. This tier satisfies the record-keeping requirements of EU AI Act Article 12 and ISO 42001 clause 9.1 for the ensemble as a composite system.
The ensemble is governed and additionally operates with continuous routing policy evaluation: routing performance metrics are tracked per query type, Aspec parameters are tuned with versioned change records, and model set M is updated through a formal model admission process with documented justification. Policy changes trigger automated genealogy audit events. The ensemble's governance posture is continuous rather than point-in-time.
| Component | Formal Role | Governance Artifact Required | Gap When Absent | Regulatory Mapping |
|---|---|---|---|---|
| Model Set M | Identifies all models in the ensemble by immutable version | Versioned model registry with immutable IDs per member | M-ambiguity: member models unidentifiable at audit time | EU AI Act Art. 12(1)(a); ISO 42001 cl. 9.1 |
| Routing Policy R | Maps query and context to active model subset | Formal R specification (versioned) + per-event routing log | Routing Policy Gap: routing audit structurally impossible | EU AI Act Art. 12(1)(b); NIST AI RMF MAP 1.5 |
| Aggregation Function A | Combines member outputs into ensemble output | Versioned A specification + per-event Aspec log | Arbitration Collapse unresolvable; Gens(o) incomplete | EU AI Act Art. 12(1)(c); ISO 42001 cl. 9.1 |
| Governance Spec G | Assigns accountability, defines audit schema, specifies escalation | Named accountability matrix; log schema document | Accountability void: no principal owns ensemble output | NIST AI RMF GOVERN 1.7; ISO 42001 cl. 5.3 |
| Member Outputs MS | Individual outputs from each routed member model (for Gens) | Per-event log of each oi alongside ensemble output | Retroactive Unverifiability for Arbitration-Collapsed outputs | EU AI Act Art. 12; companion to Output Genealogy [8] |
The Output Genealogy framework [8] defines G(o) = (M, P, C, Dr, Theta) for single-model deployments. MES and the Arbitration Collapse result extend this framework in two directions. First, M in Gens(o) is a set of (model, output) pairs rather than a single identifier, because the ensemble output is conditioned on the outputs of all routed members. Second, Aspec is a new time-varying component that does not exist in single-model deployments. Both extensions are required: neither G(o) applied independently to each member output, nor G(o) applied to the ensemble output treating the ensemble as a single model, produces a complete genealogy record for a collapsed ensemble output.
A selection-based aggregation function, in which A selects exactly one member output without modification, never produces Arbitration Collapse. For selection ensembles, the ensemble output o = oj for some j, and G(o) = G(oj) is sufficient: the ensemble genealogy reduces to the single-model genealogy of the selected member. However, the routing function R must still be governed under MES: the selection of j is a governance decision that requires logging and accountability regardless of whether A collapses.
The EU AI Act Article 12 requires logs of inputs to high-risk AI systems. For a multi-model ensemble, the input to the system includes the routing decision: which models saw the query is part of the input specification. MES closes this gap by requiring that S = R(q, c) be logged per inference event. ISO/IEC 42001 clause 9.1 requires evaluation of AI management system performance: for an ensemble, this requires evaluation of R's effectiveness and A's behavior across query types, which is only possible with a versioned R specification and per-event logs. NIST AI RMF GOVERN 1.7 requires provenance tracking: Gens(o) is the provenance record for ensemble outputs.
The MES framework addresses static ensembles in which M is fixed at deployment time. Dynamic ensembles, in which M changes at inference time based on model availability or performance signals, require a temporal extension of M: M(t) rather than a fixed set. This extension is a direction for future work.
The degree of collapse metric DAC(o) requires a semantic similarity function over the output space O. For structured outputs (classification labels, SQL queries, structured data), similarity is well-defined. For free-form text outputs, similarity measurement introduces additional uncertainty that must be disclosed in the governance specification G.
The framework assumes that member model outputs oi are independently generated: each member processes the query without access to other members' outputs. Iterative or collaborative ensemble architectures, in which member models condition on each other's outputs, require a directed graph extension of MS that tracks inter-member dependencies.
We have introduced the Model Ensemble Specification (MES) as the formal 4-tuple (M, R, A, G) that defines a governed enterprise AI ensemble, the Routing Policy Gap as the structural condition in which routing decisions are not auditable, and Arbitration Collapse as the output condition in which existing single-model audit frameworks are insufficient and require a sixth genealogy component Aspec. We have established that Arbitration Collapse extends the Retroactive Unverifiability result from Output Genealogy [8] to multi-model settings, making ensemble Provenance Opacity an irreversible condition for past inference events when Aspec and MS are not captured at inference time.
The practical implication is direct: any enterprise multi-model deployment that does not implement a well-formed MES is operating with Routing Policy Gap and is not auditable for its routing decisions, regardless of how comprehensively it logs individual model outputs. Fusion-based ensembles additionally require the extended genealogy Gens(o) to satisfy regulatory audit requirements. The Ensemble Governance Maturity Model provides an indicative path to Tier 3 compliance, which we estimate as achievable within a focused engineering effort for a team with access to the ensemble's inference pipeline.
The coined terms introduced in this paper, Model Ensemble Specification, Routing Policy Gap, and Arbitration Collapse, are intended as operational vocabulary for engineers, governance officers, and regulators working with multi-model AI deployments. They extend the vocabulary of Output Genealogy to the compositional architectures that are now the dominant pattern in enterprise AI, and provide the definitional foundation that regulatory frameworks presuppose without supplying.
© 2026 Arjun Jaggi and Aditya Karnam Gururaj Rao. All rights reserved. Academic citation permitted with attribution; commercial use and derivative frameworks require written permission.